Fossil

Check-in [ad9d418c]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge latest trunk
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | dir_history_link
Files: files | file ages | folders
SHA3-256:ad9d418cbb1dd7b8245bfe9330f1df096a8efbf839e95c6f444b4a455ae202c3
User & Date: mgagnon 2017-03-15 18:20:30
Context
2017-03-15
19:06
Add the "[history]" link on the /dir page for subdirectories. check-in: 6a01228f user: drh tags: trunk
18:20
Merge latest trunk Closed-Leaf check-in: ad9d418c user: mgagnon tags: dir_history_link
17:27
If a directory listing (from /file or /dir or /tree) contains a README file, then render the content of that README beneath the directory listing. Example: /file/skins check-in: a6993f9b user: drh tags: trunk
2017-02-08
18:28
On /dir page, add a link named "[history]" that point to the timeline affecting only the files inside the current directory. Use the "chng=dirname/*" query parameter of the /timeline page)
TODO: Implement the same in Tree-View mode (could add the link beside each directory)
check-in: c458c3f6 user: mgagnon tags: dir_history_link
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to VERSION.

1
1.37
|
1
2.1

Deleted compat/zlib/doc/algorithm.txt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
1. Compression algorithm (deflate)

The deflation algorithm used by gzip (also zip and zlib) is a variation of
LZ77 (Lempel-Ziv 1977, see reference below). It finds duplicated strings in
the input data.  The second occurrence of a string is replaced by a
pointer to the previous string, in the form of a pair (distance,
length).  Distances are limited to 32K bytes, and lengths are limited
to 258 bytes. When a string does not occur anywhere in the previous
32K bytes, it is emitted as a sequence of literal bytes.  (In this
description, `string' must be taken as an arbitrary sequence of bytes,
and is not restricted to printable characters.)

Literals or match lengths are compressed with one Huffman tree, and
match distances are compressed with another tree. The trees are stored
in a compact form at the start of each block. The blocks can have any
size (except that the compressed data for one block must fit in
available memory). A block is terminated when deflate() determines that
it would be useful to start another block with fresh trees. (This is
somewhat similar to the behavior of LZW-based _compress_.)

Duplicated strings are found using a hash table. All input strings of
length 3 are inserted in the hash table. A hash index is computed for
the next 3 bytes. If the hash chain for this index is not empty, all
strings in the chain are compared with the current input string, and
the longest match is selected.

The hash chains are searched starting with the most recent strings, to
favor small distances and thus take advantage of the Huffman encoding.
The hash chains are singly linked. There are no deletions from the
hash chains, the algorithm simply discards matches that are too old.

To avoid a worst-case situation, very long hash chains are arbitrarily
truncated at a certain length, determined by a runtime option (level
parameter of deflateInit). So deflate() does not always find the longest
possible match but generally finds a match which is long enough.

deflate() also defers the selection of matches with a lazy evaluation
mechanism. After a match of length N has been found, deflate() searches for
a longer match at the next input byte. If a longer match is found, the
previous match is truncated to a length of one (thus producing a single
literal byte) and the process of lazy evaluation begins again. Otherwise,
the original match is kept, and the next match search is attempted only N
steps later.

The lazy match evaluation is also subject to a runtime parameter. If
the current match is long enough, deflate() reduces the search for a longer
match, thus speeding up the whole process. If compression ratio is more
important than speed, deflate() attempts a complete second search even if
the first match is already long enough.

The lazy match evaluation is not performed for the fastest compression
modes (level parameter 1 to 3). For these fast modes, new strings
are inserted in the hash table only when no match was found, or
when the match is not too long. This degrades the compression ratio
but saves time since there are both fewer insertions and fewer searches.


2. Decompression algorithm (inflate)

2.1 Introduction

The key question is how to represent a Huffman code (or any prefix code) so
that you can decode fast.  The most important characteristic is that shorter
codes are much more common than longer codes, so pay attention to decoding the
short codes fast, and let the long codes take longer to decode.

inflate() sets up a first level table that covers some number of bits of
input less than the length of longest code.  It gets that many bits from the
stream, and looks it up in the table.  The table will tell if the next
code is that many bits or less and how many, and if it is, it will tell
the value, else it will point to the next level table for which inflate()
grabs more bits and tries to decode a longer code.

How many bits to make the first lookup is a tradeoff between the time it
takes to decode and the time it takes to build the table.  If building the
table took no time (and if you had infinite memory), then there would only
be a first level table to cover all the way to the longest code.  However,
building the table ends up taking a lot longer for more bits since short
codes are replicated many times in such a table.  What inflate() does is
simply to make the number of bits in the first table a variable, and  then
to set that variable for the maximum speed.

For inflate, which has 286 possible codes for the literal/length tree, the size
of the first table is nine bits.  Also the distance trees have 30 possible
values, and the size of the first table is six bits.  Note that for each of
those cases, the table ended up one bit longer than the ``average'' code
length, i.e. the code length of an approximately flat code which would be a
little more than eight bits for 286 symbols and a little less than five bits
for 30 symbols.


2.2 More details on the inflate table lookup

Ok, you want to know what this cleverly obfuscated inflate tree actually
looks like.  You are correct that it's not a Huffman tree.  It is simply a
lookup table for the first, let's say, nine bits of a Huffman symbol.  The
symbol could be as short as one bit or as long as 15 bits.  If a particular
symbol is shorter than nine bits, then that symbol's translation is duplicated
in all those entries that start with that symbol's bits.  For example, if the
symbol is four bits, then it's duplicated 32 times in a nine-bit table.  If a
symbol is nine bits long, it appears in the table once.

If the symbol is longer than nine bits, then that entry in the table points
to another similar table for the remaining bits.  Again, there are duplicated
entries as needed.  The idea is that most of the time the symbol will be short
and there will only be one table look up.  (That's whole idea behind data
compression in the first place.)  For the less frequent long symbols, there
will be two lookups.  If you had a compression method with really long
symbols, you could have as many levels of lookups as is efficient.  For
inflate, two is enough.

So a table entry either points to another table (in which case nine bits in
the above example are gobbled), or it contains the translation for the symbol
and the number of bits to gobble.  Then you start again with the next
ungobbled bit.

You may wonder: why not just have one lookup table for how ever many bits the
longest symbol is?  The reason is that if you do that, you end up spending
more time filling in duplicate symbol entries than you do actually decoding.
At least for deflate's output that generates new trees every several 10's of
kbytes.  You can imagine that filling in a 2^15 entry table for a 15-bit code
would take too long if you're only decoding several thousand symbols.  At the
other extreme, you could make a new table for every bit in the code.  In fact,
that's essentially a Huffman tree.  But then you spend too much time
traversing the tree while decoding, even for short symbols.

So the number of bits for the first lookup table is a trade of the time to
fill out the table vs. the time spent looking at the second level and above of
the table.

Here is an example, scaled down:

The code being decoded, with 10 symbols, from 1 to 6 bits long:

A: 0
B: 10
C: 1100
D: 11010
E: 11011
F: 11100
G: 11101
H: 11110
I: 111110
J: 111111

Let's make the first table three bits long (eight entries):

000: A,1
001: A,1
010: A,1
011: A,1
100: B,2
101: B,2
110: -> table X (gobble 3 bits)
111: -> table Y (gobble 3 bits)

Each entry is what the bits decode as and how many bits that is, i.e. how
many bits to gobble.  Or the entry points to another table, with the number of
bits to gobble implicit in the size of the table.

Table X is two bits long since the longest code starting with 110 is five bits
long:

00: C,1
01: C,1
10: D,2
11: E,2

Table Y is three bits long since the longest code starting with 111 is six
bits long:

000: F,2
001: F,2
010: G,2
011: G,2
100: H,2
101: H,2
110: I,3
111: J,3

So what we have here are three tables with a total of 20 entries that had to
be constructed.  That's compared to 64 entries for a single table.  Or
compared to 16 entries for a Huffman tree (six two entry tables and one four
entry table).  Assuming that the code ideally represents the probability of
the symbols, it takes on the average 1.25 lookups per symbol.  That's compared
to one lookup for the single table, or 1.66 lookups per symbol for the
Huffman tree.

There, I think that gives you a picture of what's going on.  For inflate, the
meaning of a particular symbol is often more than just a letter.  It can be a
byte (a "literal"), or it can be either a length or a distance which
indicates a base value and a number of bits to fetch after the code that is
added to the base value.  Or it might be the special end-of-block code.  The
data structures created in inftrees.c try to encode all that information
compactly in the tables.


Jean-loup Gailly        Mark Adler
jloup@gzip.org          madler@alumni.caltech.edu


References:

[LZ77] Ziv J., Lempel A., ``A Universal Algorithm for Sequential Data
Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3,
pp. 337-343.

``DEFLATE Compressed Data Format Specification'' available in
http://tools.ietf.org/html/rfc1951
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


































































































































































































































































































































































































































Deleted compat/zlib/doc/rfc1950.txt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619






Network Working Group                                         P. Deutsch
Request for Comments: 1950                           Aladdin Enterprises
Category: Informational                                      J-L. Gailly
                                                                Info-ZIP
                                                                May 1996


         ZLIB Compressed Data Format Specification version 3.3

Status of This Memo

   This memo provides information for the Internet community.  This memo
   does not specify an Internet standard of any kind.  Distribution of
   this memo is unlimited.

IESG Note:

   The IESG takes no position on the validity of any Intellectual
   Property Rights statements contained in this document.

Notices

   Copyright (c) 1996 L. Peter Deutsch and Jean-Loup Gailly

   Permission is granted to copy and distribute this document for any
   purpose and without charge, including translations into other
   languages and incorporation into compilations, provided that the
   copyright notice and this notice are preserved, and that any
   substantive changes or deletions from the original are clearly
   marked.

   A pointer to the latest version of this and related documentation in
   HTML format can be found at the URL
   <ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html>.

Abstract

   This specification defines a lossless compressed data format.  The
   data can be produced or consumed, even for an arbitrarily long
   sequentially presented input data stream, using only an a priori
   bounded amount of intermediate storage.  The format presently uses
   the DEFLATE compression method but can be easily extended to use
   other compression methods.  It can be implemented readily in a manner
   not covered by patents.  This specification also defines the ADLER-32
   checksum (an extension and improvement of the Fletcher checksum),
   used for detection of data corruption, and provides an algorithm for
   computing it.




Deutsch & Gailly             Informational                      [Page 1]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


Table of Contents

   1. Introduction ................................................... 2
      1.1. Purpose ................................................... 2
      1.2. Intended audience ......................................... 3
      1.3. Scope ..................................................... 3
      1.4. Compliance ................................................ 3
      1.5.  Definitions of terms and conventions used ................ 3
      1.6. Changes from previous versions ............................ 3
   2. Detailed specification ......................................... 3
      2.1. Overall conventions ....................................... 3
      2.2. Data format ............................................... 4
      2.3. Compliance ................................................ 7
   3. References ..................................................... 7
   4. Source code .................................................... 8
   5. Security Considerations ........................................ 8
   6. Acknowledgements ............................................... 8
   7. Authors' Addresses ............................................. 8
   8. Appendix: Rationale ............................................ 9
   9. Appendix: Sample code ..........................................10

1. Introduction

   1.1. Purpose

      The purpose of this specification is to define a lossless
      compressed data format that:

          * Is independent of CPU type, operating system, file system,
            and character set, and hence can be used for interchange;

          * Can be produced or consumed, even for an arbitrarily long
            sequentially presented input data stream, using only an a
            priori bounded amount of intermediate storage, and hence can
            be used in data communications or similar structures such as
            Unix filters;

          * Can use a number of different compression methods;

          * Can be implemented readily in a manner not covered by
            patents, and hence can be practiced freely.

      The data format defined by this specification does not attempt to
      allow random access to compressed data.







Deutsch & Gailly             Informational                      [Page 2]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


   1.2. Intended audience

      This specification is intended for use by implementors of software
      to compress data into zlib format and/or decompress data from zlib
      format.

      The text of the specification assumes a basic background in
      programming at the level of bits and other primitive data
      representations.

   1.3. Scope

      The specification specifies a compressed data format that can be
      used for in-memory compression of a sequence of arbitrary bytes.

   1.4. Compliance

      Unless otherwise indicated below, a compliant decompressor must be
      able to accept and decompress any data set that conforms to all
      the specifications presented here; a compliant compressor must
      produce data sets that conform to all the specifications presented
      here.

   1.5.  Definitions of terms and conventions used

      byte: 8 bits stored or transmitted as a unit (same as an octet).
      (For this specification, a byte is exactly 8 bits, even on
      machines which store a character on a number of bits different
      from 8.) See below, for the numbering of bits within a byte.

   1.6. Changes from previous versions

      Version 3.1 was the first public release of this specification.
      In version 3.2, some terminology was changed and the Adler-32
      sample code was rewritten for clarity.  In version 3.3, the
      support for a preset dictionary was introduced, and the
      specification was converted to RFC style.

2. Detailed specification

   2.1. Overall conventions

      In the diagrams below, a box like this:

         +---+
         |   | <-- the vertical bars might be missing
         +---+




Deutsch & Gailly             Informational                      [Page 3]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


      represents one byte; a box like this:

         +==============+
         |              |
         +==============+

      represents a variable number of bytes.

      Bytes stored within a computer do not have a "bit order", since
      they are always treated as a unit.  However, a byte considered as
      an integer between 0 and 255 does have a most- and least-
      significant bit, and since we write numbers with the most-
      significant digit on the left, we also write bytes with the most-
      significant bit on the left.  In the diagrams below, we number the
      bits of a byte so that bit 0 is the least-significant bit, i.e.,
      the bits are numbered:

         +--------+
         |76543210|
         +--------+

      Within a computer, a number may occupy multiple bytes.  All
      multi-byte numbers in the format described here are stored with
      the MOST-significant byte first (at the lower memory address).
      For example, the decimal number 520 is stored as:

             0     1
         +--------+--------+
         |00000010|00001000|
         +--------+--------+
          ^        ^
          |        |
          |        + less significant byte = 8
          + more significant byte = 2 x 256

   2.2. Data format

      A zlib stream has the following structure:

           0   1
         +---+---+
         |CMF|FLG|   (more-->)
         +---+---+








Deutsch & Gailly             Informational                      [Page 4]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


      (if FLG.FDICT set)

           0   1   2   3
         +---+---+---+---+
         |     DICTID    |   (more-->)
         +---+---+---+---+

         +=====================+---+---+---+---+
         |...compressed data...|    ADLER32    |
         +=====================+---+---+---+---+

      Any data which may appear after ADLER32 are not part of the zlib
      stream.

      CMF (Compression Method and flags)
         This byte is divided into a 4-bit compression method and a 4-
         bit information field depending on the compression method.

            bits 0 to 3  CM     Compression method
            bits 4 to 7  CINFO  Compression info

      CM (Compression method)
         This identifies the compression method used in the file. CM = 8
         denotes the "deflate" compression method with a window size up
         to 32K.  This is the method used by gzip and PNG (see
         references [1] and [2] in Chapter 3, below, for the reference
         documents).  CM = 15 is reserved.  It might be used in a future
         version of this specification to indicate the presence of an
         extra field before the compressed data.

      CINFO (Compression info)
         For CM = 8, CINFO is the base-2 logarithm of the LZ77 window
         size, minus eight (CINFO=7 indicates a 32K window size). Values
         of CINFO above 7 are not allowed in this version of the
         specification.  CINFO is not defined in this specification for
         CM not equal to 8.

      FLG (FLaGs)
         This flag byte is divided as follows:

            bits 0 to 4  FCHECK  (check bits for CMF and FLG)
            bit  5       FDICT   (preset dictionary)
            bits 6 to 7  FLEVEL  (compression level)

         The FCHECK value must be such that CMF and FLG, when viewed as
         a 16-bit unsigned integer stored in MSB order (CMF*256 + FLG),
         is a multiple of 31.




Deutsch & Gailly             Informational                      [Page 5]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


      FDICT (Preset dictionary)
         If FDICT is set, a DICT dictionary identifier is present
         immediately after the FLG byte. The dictionary is a sequence of
         bytes which are initially fed to the compressor without
         producing any compressed output. DICT is the Adler-32 checksum
         of this sequence of bytes (see the definition of ADLER32
         below).  The decompressor can use this identifier to determine
         which dictionary has been used by the compressor.

      FLEVEL (Compression level)
         These flags are available for use by specific compression
         methods.  The "deflate" method (CM = 8) sets these flags as
         follows:

            0 - compressor used fastest algorithm
            1 - compressor used fast algorithm
            2 - compressor used default algorithm
            3 - compressor used maximum compression, slowest algorithm

         The information in FLEVEL is not needed for decompression; it
         is there to indicate if recompression might be worthwhile.

      compressed data
         For compression method 8, the compressed data is stored in the
         deflate compressed data format as described in the document
         "DEFLATE Compressed Data Format Specification" by L. Peter
         Deutsch. (See reference [3] in Chapter 3, below)

         Other compressed data formats are not specified in this version
         of the zlib specification.

      ADLER32 (Adler-32 checksum)
         This contains a checksum value of the uncompressed data
         (excluding any dictionary data) computed according to Adler-32
         algorithm. This algorithm is a 32-bit extension and improvement
         of the Fletcher algorithm, used in the ITU-T X.224 / ISO 8073
         standard. See references [4] and [5] in Chapter 3, below)

         Adler-32 is composed of two sums accumulated per byte: s1 is
         the sum of all bytes, s2 is the sum of all s1 values. Both sums
         are done modulo 65521. s1 is initialized to 1, s2 to zero.  The
         Adler-32 checksum is stored as s2*65536 + s1 in most-
         significant-byte first (network) order.








Deutsch & Gailly             Informational                      [Page 6]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


   2.3. Compliance

      A compliant compressor must produce streams with correct CMF, FLG
      and ADLER32, but need not support preset dictionaries.  When the
      zlib data format is used as part of another standard data format,
      the compressor may use only preset dictionaries that are specified
      by this other data format.  If this other format does not use the
      preset dictionary feature, the compressor must not set the FDICT
      flag.

      A compliant decompressor must check CMF, FLG, and ADLER32, and
      provide an error indication if any of these have incorrect values.
      A compliant decompressor must give an error indication if CM is
      not one of the values defined in this specification (only the
      value 8 is permitted in this version), since another value could
      indicate the presence of new features that would cause subsequent
      data to be interpreted incorrectly.  A compliant decompressor must
      give an error indication if FDICT is set and DICTID is not the
      identifier of a known preset dictionary.  A decompressor may
      ignore FLEVEL and still be compliant.  When the zlib data format
      is being used as a part of another standard format, a compliant
      decompressor must support all the preset dictionaries specified by
      the other format. When the other format does not use the preset
      dictionary feature, a compliant decompressor must reject any
      stream in which the FDICT flag is set.

3. References

   [1] Deutsch, L.P.,"GZIP Compressed Data Format Specification",
       available in ftp://ftp.uu.net/pub/archiving/zip/doc/

   [2] Thomas Boutell, "PNG (Portable Network Graphics) specification",
       available in ftp://ftp.uu.net/graphics/png/documents/

   [3] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification",
       available in ftp://ftp.uu.net/pub/archiving/zip/doc/

   [4] Fletcher, J. G., "An Arithmetic Checksum for Serial
       Transmissions," IEEE Transactions on Communications, Vol. COM-30,
       No. 1, January 1982, pp. 247-252.

   [5] ITU-T Recommendation X.224, Annex D, "Checksum Algorithms,"
       November, 1993, pp. 144, 145. (Available from
       gopher://info.itu.ch). ITU-T X.244 is also the same as ISO 8073.







Deutsch & Gailly             Informational                      [Page 7]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


4. Source code

   Source code for a C language implementation of a "zlib" compliant
   library is available at ftp://ftp.uu.net/pub/archiving/zip/zlib/.

5. Security Considerations

   A decoder that fails to check the ADLER32 checksum value may be
   subject to undetected data corruption.

6. Acknowledgements

   Trademarks cited in this document are the property of their
   respective owners.

   Jean-Loup Gailly and Mark Adler designed the zlib format and wrote
   the related software described in this specification.  Glenn
   Randers-Pehrson converted this document to RFC and HTML format.

7. Authors' Addresses

   L. Peter Deutsch
   Aladdin Enterprises
   203 Santa Margarita Ave.
   Menlo Park, CA 94025

   Phone: (415) 322-0103 (AM only)
   FAX:   (415) 322-1734
   EMail: <ghost@aladdin.com>


   Jean-Loup Gailly

   EMail: <gzip@prep.ai.mit.edu>

   Questions about the technical content of this specification can be
   sent by email to

   Jean-Loup Gailly <gzip@prep.ai.mit.edu> and
   Mark Adler <madler@alumni.caltech.edu>

   Editorial comments on this specification can be sent by email to

   L. Peter Deutsch <ghost@aladdin.com> and
   Glenn Randers-Pehrson <randeg@alumni.rpi.edu>






Deutsch & Gailly             Informational                      [Page 8]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


8. Appendix: Rationale

   8.1. Preset dictionaries

      A preset dictionary is specially useful to compress short input
      sequences. The compressor can take advantage of the dictionary
      context to encode the input in a more compact manner. The
      decompressor can be initialized with the appropriate context by
      virtually decompressing a compressed version of the dictionary
      without producing any output. However for certain compression
      algorithms such as the deflate algorithm this operation can be
      achieved without actually performing any decompression.

      The compressor and the decompressor must use exactly the same
      dictionary. The dictionary may be fixed or may be chosen among a
      certain number of predefined dictionaries, according to the kind
      of input data. The decompressor can determine which dictionary has
      been chosen by the compressor by checking the dictionary
      identifier. This document does not specify the contents of
      predefined dictionaries, since the optimal dictionaries are
      application specific. Standard data formats using this feature of
      the zlib specification must precisely define the allowed
      dictionaries.

   8.2. The Adler-32 algorithm

      The Adler-32 algorithm is much faster than the CRC32 algorithm yet
      still provides an extremely low probability of undetected errors.

      The modulo on unsigned long accumulators can be delayed for 5552
      bytes, so the modulo operation time is negligible.  If the bytes
      are a, b, c, the second sum is 3a + 2b + c + 3, and so is position
      and order sensitive, unlike the first sum, which is just a
      checksum.  That 65521 is prime is important to avoid a possible
      large class of two-byte errors that leave the check unchanged.
      (The Fletcher checksum uses 255, which is not prime and which also
      makes the Fletcher check insensitive to single byte changes 0 <->
      255.)

      The sum s1 is initialized to 1 instead of zero to make the length
      of the sequence part of s2, so that the length does not have to be
      checked separately. (Any sequence of zeroes has a Fletcher
      checksum of zero.)








Deutsch & Gailly             Informational                      [Page 9]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


9. Appendix: Sample code

   The following C code computes the Adler-32 checksum of a data buffer.
   It is written for clarity, not for speed.  The sample code is in the
   ANSI C programming language. Non C users may find it easier to read
   with these hints:

      &      Bitwise AND operator.
      >>     Bitwise right shift operator. When applied to an
             unsigned quantity, as here, right shift inserts zero bit(s)
             at the left.
      <<     Bitwise left shift operator. Left shift inserts zero
             bit(s) at the right.
      ++     "n++" increments the variable n.
      %      modulo operator: a % b is the remainder of a divided by b.

      #define BASE 65521 /* largest prime smaller than 65536 */

      /*
         Update a running Adler-32 checksum with the bytes buf[0..len-1]
       and return the updated checksum. The Adler-32 checksum should be
       initialized to 1.

       Usage example:

         unsigned long adler = 1L;

         while (read_buffer(buffer, length) != EOF) {
           adler = update_adler32(adler, buffer, length);
         }
         if (adler != original_adler) error();
      */
      unsigned long update_adler32(unsigned long adler,
         unsigned char *buf, int len)
      {
        unsigned long s1 = adler & 0xffff;
        unsigned long s2 = (adler >> 16) & 0xffff;
        int n;

        for (n = 0; n < len; n++) {
          s1 = (s1 + buf[n]) % BASE;
          s2 = (s2 + s1)     % BASE;
        }
        return (s2 << 16) + s1;
      }

      /* Return the adler32 of the bytes buf[0..len-1] */




Deutsch & Gailly             Informational                     [Page 10]
 
RFC 1950       ZLIB Compressed Data Format Specification        May 1996


      unsigned long adler32(unsigned char *buf, int len)
      {
        return update_adler32(1L, buf, len);
      }















































Deutsch & Gailly             Informational                     [Page 11]
 
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






















































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































Deleted compat/zlib/doc/rfc1951.txt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955






Network Working Group                                         P. Deutsch
Request for Comments: 1951                           Aladdin Enterprises
Category: Informational                                         May 1996


        DEFLATE Compressed Data Format Specification version 1.3

Status of This Memo

   This memo provides information for the Internet community.  This memo
   does not specify an Internet standard of any kind.  Distribution of
   this memo is unlimited.

IESG Note:

   The IESG takes no position on the validity of any Intellectual
   Property Rights statements contained in this document.

Notices

   Copyright (c) 1996 L. Peter Deutsch

   Permission is granted to copy and distribute this document for any
   purpose and without charge, including translations into other
   languages and incorporation into compilations, provided that the
   copyright notice and this notice are preserved, and that any
   substantive changes or deletions from the original are clearly
   marked.

   A pointer to the latest version of this and related documentation in
   HTML format can be found at the URL
   <ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html>.

Abstract

   This specification defines a lossless compressed data format that
   compresses data using a combination of the LZ77 algorithm and Huffman
   coding, with efficiency comparable to the best currently available
   general-purpose compression methods.  The data can be produced or
   consumed, even for an arbitrarily long sequentially presented input
   data stream, using only an a priori bounded amount of intermediate
   storage.  The format can be implemented readily in a manner not
   covered by patents.








Deutsch                      Informational                      [Page 1]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


Table of Contents

   1. Introduction ................................................... 2
      1.1. Purpose ................................................... 2
      1.2. Intended audience ......................................... 3
      1.3. Scope ..................................................... 3
      1.4. Compliance ................................................ 3
      1.5.  Definitions of terms and conventions used ................ 3
      1.6. Changes from previous versions ............................ 4
   2. Compressed representation overview ............................. 4
   3. Detailed specification ......................................... 5
      3.1. Overall conventions ....................................... 5
          3.1.1. Packing into bytes .................................. 5
      3.2. Compressed block format ................................... 6
          3.2.1. Synopsis of prefix and Huffman coding ............... 6
          3.2.2. Use of Huffman coding in the "deflate" format ....... 7
          3.2.3. Details of block format ............................. 9
          3.2.4. Non-compressed blocks (BTYPE=00) ................... 11
          3.2.5. Compressed blocks (length and distance codes) ...... 11
          3.2.6. Compression with fixed Huffman codes (BTYPE=01) .... 12
          3.2.7. Compression with dynamic Huffman codes (BTYPE=10) .. 13
      3.3. Compliance ............................................... 14
   4. Compression algorithm details ................................. 14
   5. References .................................................... 16
   6. Security Considerations ....................................... 16
   7. Source code ................................................... 16
   8. Acknowledgements .............................................. 16
   9. Author's Address .............................................. 17

1. Introduction

   1.1. Purpose

      The purpose of this specification is to define a lossless
      compressed data format that:
          * Is independent of CPU type, operating system, file system,
            and character set, and hence can be used for interchange;
          * Can be produced or consumed, even for an arbitrarily long
            sequentially presented input data stream, using only an a
            priori bounded amount of intermediate storage, and hence
            can be used in data communications or similar structures
            such as Unix filters;
          * Compresses data with efficiency comparable to the best
            currently available general-purpose compression methods,
            and in particular considerably better than the "compress"
            program;
          * Can be implemented readily in a manner not covered by
            patents, and hence can be practiced freely;



Deutsch                      Informational                      [Page 2]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


          * Is compatible with the file format produced by the current
            widely used gzip utility, in that conforming decompressors
            will be able to read data produced by the existing gzip
            compressor.

      The data format defined by this specification does not attempt to:

          * Allow random access to compressed data;
          * Compress specialized data (e.g., raster graphics) as well
            as the best currently available specialized algorithms.

      A simple counting argument shows that no lossless compression
      algorithm can compress every possible input data set.  For the
      format defined here, the worst case expansion is 5 bytes per 32K-
      byte block, i.e., a size increase of 0.015% for large data sets.
      English text usually compresses by a factor of 2.5 to 3;
      executable files usually compress somewhat less; graphical data
      such as raster images may compress much more.

   1.2. Intended audience

      This specification is intended for use by implementors of software
      to compress data into "deflate" format and/or decompress data from
      "deflate" format.

      The text of the specification assumes a basic background in
      programming at the level of bits and other primitive data
      representations.  Familiarity with the technique of Huffman coding
      is helpful but not required.

   1.3. Scope

      The specification specifies a method for representing a sequence
      of bytes as a (usually shorter) sequence of bits, and a method for
      packing the latter bit sequence into bytes.

   1.4. Compliance

      Unless otherwise indicated below, a compliant decompressor must be
      able to accept and decompress any data set that conforms to all
      the specifications presented here; a compliant compressor must
      produce data sets that conform to all the specifications presented
      here.

   1.5.  Definitions of terms and conventions used

      Byte: 8 bits stored or transmitted as a unit (same as an octet).
      For this specification, a byte is exactly 8 bits, even on machines



Deutsch                      Informational                      [Page 3]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


      which store a character on a number of bits different from eight.
      See below, for the numbering of bits within a byte.

      String: a sequence of arbitrary bytes.

   1.6. Changes from previous versions

      There have been no technical changes to the deflate format since
      version 1.1 of this specification.  In version 1.2, some
      terminology was changed.  Version 1.3 is a conversion of the
      specification to RFC style.

2. Compressed representation overview

   A compressed data set consists of a series of blocks, corresponding
   to successive blocks of input data.  The block sizes are arbitrary,
   except that non-compressible blocks are limited to 65,535 bytes.

   Each block is compressed using a combination of the LZ77 algorithm
   and Huffman coding. The Huffman trees for each block are independent
   of those for previous or subsequent blocks; the LZ77 algorithm may
   use a reference to a duplicated string occurring in a previous block,
   up to 32K input bytes before.

   Each block consists of two parts: a pair of Huffman code trees that
   describe the representation of the compressed data part, and a
   compressed data part.  (The Huffman trees themselves are compressed
   using Huffman encoding.)  The compressed data consists of a series of
   elements of two types: literal bytes (of strings that have not been
   detected as duplicated within the previous 32K input bytes), and
   pointers to duplicated strings, where a pointer is represented as a
   pair <length, backward distance>.  The representation used in the
   "deflate" format limits distances to 32K bytes and lengths to 258
   bytes, but does not limit the size of a block, except for
   uncompressible blocks, which are limited as noted above.

   Each type of value (literals, distances, and lengths) in the
   compressed data is represented using a Huffman code, using one code
   tree for literals and lengths and a separate code tree for distances.
   The code trees for each block appear in a compact form just before
   the compressed data for that block.










Deutsch                      Informational                      [Page 4]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


3. Detailed specification

   3.1. Overall conventions In the diagrams below, a box like this:

         +---+
         |   | <-- the vertical bars might be missing
         +---+

      represents one byte; a box like this:

         +==============+
         |              |
         +==============+

      represents a variable number of bytes.

      Bytes stored within a computer do not have a "bit order", since
      they are always treated as a unit.  However, a byte considered as
      an integer between 0 and 255 does have a most- and least-
      significant bit, and since we write numbers with the most-
      significant digit on the left, we also write bytes with the most-
      significant bit on the left.  In the diagrams below, we number the
      bits of a byte so that bit 0 is the least-significant bit, i.e.,
      the bits are numbered:

         +--------+
         |76543210|
         +--------+

      Within a computer, a number may occupy multiple bytes.  All
      multi-byte numbers in the format described here are stored with
      the least-significant byte first (at the lower memory address).
      For example, the decimal number 520 is stored as:

             0        1
         +--------+--------+
         |00001000|00000010|
         +--------+--------+
          ^        ^
          |        |
          |        + more significant byte = 2 x 256
          + less significant byte = 8

      3.1.1. Packing into bytes

         This document does not address the issue of the order in which
         bits of a byte are transmitted on a bit-sequential medium,
         since the final data format described here is byte- rather than



Deutsch                      Informational                      [Page 5]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


         bit-oriented.  However, we describe the compressed block format
         in below, as a sequence of data elements of various bit
         lengths, not a sequence of bytes.  We must therefore specify
         how to pack these data elements into bytes to form the final
         compressed byte sequence:

             * Data elements are packed into bytes in order of
               increasing bit number within the byte, i.e., starting
               with the least-significant bit of the byte.
             * Data elements other than Huffman codes are packed
               starting with the least-significant bit of the data
               element.
             * Huffman codes are packed starting with the most-
               significant bit of the code.

         In other words, if one were to print out the compressed data as
         a sequence of bytes, starting with the first byte at the
         *right* margin and proceeding to the *left*, with the most-
         significant bit of each byte on the left as usual, one would be
         able to parse the result from right to left, with fixed-width
         elements in the correct MSB-to-LSB order and Huffman codes in
         bit-reversed order (i.e., with the first bit of the code in the
         relative LSB position).

   3.2. Compressed block format

      3.2.1. Synopsis of prefix and Huffman coding

         Prefix coding represents symbols from an a priori known
         alphabet by bit sequences (codes), one code for each symbol, in
         a manner such that different symbols may be represented by bit
         sequences of different lengths, but a parser can always parse
         an encoded string unambiguously symbol-by-symbol.

         We define a prefix code in terms of a binary tree in which the
         two edges descending from each non-leaf node are labeled 0 and
         1 and in which the leaf nodes correspond one-for-one with (are
         labeled with) the symbols of the alphabet; then the code for a
         symbol is the sequence of 0's and 1's on the edges leading from
         the root to the leaf labeled with that symbol.  For example:











Deutsch                      Informational                      [Page 6]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


                          /\              Symbol    Code
                         0  1             ------    ----
                        /    \                A      00
                       /\     B               B       1
                      0  1                    C     011
                     /    \                   D     010
                    A     /\
                         0  1
                        /    \
                       D      C

         A parser can decode the next symbol from an encoded input
         stream by walking down the tree from the root, at each step
         choosing the edge corresponding to the next input bit.

         Given an alphabet with known symbol frequencies, the Huffman
         algorithm allows the construction of an optimal prefix code
         (one which represents strings with those symbol frequencies
         using the fewest bits of any possible prefix codes for that
         alphabet).  Such a code is called a Huffman code.  (See
         reference [1] in Chapter 5, references for additional
         information on Huffman codes.)

         Note that in the "deflate" format, the Huffman codes for the
         various alphabets must not exceed certain maximum code lengths.
         This constraint complicates the algorithm for computing code
         lengths from symbol frequencies.  Again, see Chapter 5,
         references for details.

      3.2.2. Use of Huffman coding in the "deflate" format

         The Huffman codes used for each alphabet in the "deflate"
         format have two additional rules:

             * All codes of a given bit length have lexicographically
               consecutive values, in the same order as the symbols
               they represent;

             * Shorter codes lexicographically precede longer codes.












Deutsch                      Informational                      [Page 7]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


         We could recode the example above to follow this rule as
         follows, assuming that the order of the alphabet is ABCD:

            Symbol  Code
            ------  ----
            A       10
            B       0
            C       110
            D       111

         I.e., 0 precedes 10 which precedes 11x, and 110 and 111 are
         lexicographically consecutive.

         Given this rule, we can define the Huffman code for an alphabet
         just by giving the bit lengths of the codes for each symbol of
         the alphabet in order; this is sufficient to determine the
         actual codes.  In our example, the code is completely defined
         by the sequence of bit lengths (2, 1, 3, 3).  The following
         algorithm generates the codes as integers, intended to be read
         from most- to least-significant bit.  The code lengths are
         initially in tree[I].Len; the codes are produced in
         tree[I].Code.

         1)  Count the number of codes for each code length.  Let
             bl_count[N] be the number of codes of length N, N >= 1.

         2)  Find the numerical value of the smallest code for each
             code length:

                code = 0;
                bl_count[0] = 0;
                for (bits = 1; bits <= MAX_BITS; bits++) {
                    code = (code + bl_count[bits-1]) << 1;
                    next_code[bits] = code;
                }

         3)  Assign numerical values to all codes, using consecutive
             values for all codes of the same length with the base
             values determined at step 2. Codes that are never used
             (which have a bit length of zero) must not be assigned a
             value.

                for (n = 0;  n <= max_code; n++) {
                    len = tree[n].Len;
                    if (len != 0) {
                        tree[n].Code = next_code[len];
                        next_code[len]++;
                    }



Deutsch                      Informational                      [Page 8]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


                }

         Example:

         Consider the alphabet ABCDEFGH, with bit lengths (3, 3, 3, 3,
         3, 2, 4, 4).  After step 1, we have:

            N      bl_count[N]
            -      -----------
            2      1
            3      5
            4      2

         Step 2 computes the following next_code values:

            N      next_code[N]
            -      ------------
            1      0
            2      0
            3      2
            4      14

         Step 3 produces the following code values:

            Symbol Length   Code
            ------ ------   ----
            A       3        010
            B       3        011
            C       3        100
            D       3        101
            E       3        110
            F       2         00
            G       4       1110
            H       4       1111

      3.2.3. Details of block format

         Each block of compressed data begins with 3 header bits
         containing the following data:

            first bit       BFINAL
            next 2 bits     BTYPE

         Note that the header bits do not necessarily begin on a byte
         boundary, since a block does not necessarily occupy an integral
         number of bytes.





Deutsch                      Informational                      [Page 9]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


         BFINAL is set if and only if this is the last block of the data
         set.

         BTYPE specifies how the data are compressed, as follows:

            00 - no compression
            01 - compressed with fixed Huffman codes
            10 - compressed with dynamic Huffman codes
            11 - reserved (error)

         The only difference between the two compressed cases is how the
         Huffman codes for the literal/length and distance alphabets are
         defined.

         In all cases, the decoding algorithm for the actual data is as
         follows:

            do
               read block header from input stream.
               if stored with no compression
                  skip any remaining bits in current partially
                     processed byte
                  read LEN and NLEN (see next section)
                  copy LEN bytes of data to output
               otherwise
                  if compressed with dynamic Huffman codes
                     read representation of code trees (see
                        subsection below)
                  loop (until end of block code recognized)
                     decode literal/length value from input stream
                     if value < 256
                        copy value (literal byte) to output stream
                     otherwise
                        if value = end of block (256)
                           break from loop
                        otherwise (value = 257..285)
                           decode distance from input stream

                           move backwards distance bytes in the output
                           stream, and copy length bytes from this
                           position to the output stream.
                  end loop
            while not last block

         Note that a duplicated string reference may refer to a string
         in a previous block; i.e., the backward distance may cross one
         or more block boundaries.  However a distance cannot refer past
         the beginning of the output stream.  (An application using a



Deutsch                      Informational                     [Page 10]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


         preset dictionary might discard part of the output stream; a
         distance can refer to that part of the output stream anyway)
         Note also that the referenced string may overlap the current
         position; for example, if the last 2 bytes decoded have values
         X and Y, a string reference with <length = 5, distance = 2>
         adds X,Y,X,Y,X to the output stream.

         We now specify each compression method in turn.

      3.2.4. Non-compressed blocks (BTYPE=00)

         Any bits of input up to the next byte boundary are ignored.
         The rest of the block consists of the following information:

              0   1   2   3   4...
            +---+---+---+---+================================+
            |  LEN  | NLEN  |... LEN bytes of literal data...|
            +---+---+---+---+================================+

         LEN is the number of data bytes in the block.  NLEN is the
         one's complement of LEN.

      3.2.5. Compressed blocks (length and distance codes)

         As noted above, encoded data blocks in the "deflate" format
         consist of sequences of symbols drawn from three conceptually
         distinct alphabets: either literal bytes, from the alphabet of
         byte values (0..255), or <length, backward distance> pairs,
         where the length is drawn from (3..258) and the distance is
         drawn from (1..32,768).  In fact, the literal and length
         alphabets are merged into a single alphabet (0..285), where
         values 0..255 represent literal bytes, the value 256 indicates
         end-of-block, and values 257..285 represent length codes
         (possibly in conjunction with extra bits following the symbol
         code) as follows:
















Deutsch                      Informational                     [Page 11]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


                 Extra               Extra               Extra
            Code Bits Length(s) Code Bits Lengths   Code Bits Length(s)
            ---- ---- ------     ---- ---- -------   ---- ---- -------
             257   0     3       267   1   15,16     277   4   67-82
             258   0     4       268   1   17,18     278   4   83-98
             259   0     5       269   2   19-22     279   4   99-114
             260   0     6       270   2   23-26     280   4  115-130
             261   0     7       271   2   27-30     281   5  131-162
             262   0     8       272   2   31-34     282   5  163-194
             263   0     9       273   3   35-42     283   5  195-226
             264   0    10       274   3   43-50     284   5  227-257
             265   1  11,12      275   3   51-58     285   0    258
             266   1  13,14      276   3   59-66

         The extra bits should be interpreted as a machine integer
         stored with the most-significant bit first, e.g., bits 1110
         represent the value 14.

                  Extra           Extra               Extra
             Code Bits Dist  Code Bits   Dist     Code Bits Distance
             ---- ---- ----  ---- ----  ------    ---- ---- --------
               0   0    1     10   4     33-48    20    9   1025-1536
               1   0    2     11   4     49-64    21    9   1537-2048
               2   0    3     12   5     65-96    22   10   2049-3072
               3   0    4     13   5     97-128   23   10   3073-4096
               4   1   5,6    14   6    129-192   24   11   4097-6144
               5   1   7,8    15   6    193-256   25   11   6145-8192
               6   2   9-12   16   7    257-384   26   12  8193-12288
               7   2  13-16   17   7    385-512   27   12 12289-16384
               8   3  17-24   18   8    513-768   28   13 16385-24576
               9   3  25-32   19   8   769-1024   29   13 24577-32768

      3.2.6. Compression with fixed Huffman codes (BTYPE=01)

         The Huffman codes for the two alphabets are fixed, and are not
         represented explicitly in the data.  The Huffman code lengths
         for the literal/length alphabet are:

                   Lit Value    Bits        Codes
                   ---------    ----        -----
                     0 - 143     8          00110000 through
                                            10111111
                   144 - 255     9          110010000 through
                                            111111111
                   256 - 279     7          0000000 through
                                            0010111
                   280 - 287     8          11000000 through
                                            11000111



Deutsch                      Informational                     [Page 12]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


         The code lengths are sufficient to generate the actual codes,
         as described above; we show the codes in the table for added
         clarity.  Literal/length values 286-287 will never actually
         occur in the compressed data, but participate in the code
         construction.

         Distance codes 0-31 are represented by (fixed-length) 5-bit
         codes, with possible additional bits as shown in the table
         shown in Paragraph 3.2.5, above.  Note that distance codes 30-
         31 will never actually occur in the compressed data.

      3.2.7. Compression with dynamic Huffman codes (BTYPE=10)

         The Huffman codes for the two alphabets appear in the block
         immediately after the header bits and before the actual
         compressed data, first the literal/length code and then the
         distance code.  Each code is defined by a sequence of code
         lengths, as discussed in Paragraph 3.2.2, above.  For even
         greater compactness, the code length sequences themselves are
         compressed using a Huffman code.  The alphabet for code lengths
         is as follows:

               0 - 15: Represent code lengths of 0 - 15
                   16: Copy the previous code length 3 - 6 times.
                       The next 2 bits indicate repeat length
                             (0 = 3, ... , 3 = 6)
                          Example:  Codes 8, 16 (+2 bits 11),
                                    16 (+2 bits 10) will expand to
                                    12 code lengths of 8 (1 + 6 + 5)
                   17: Repeat a code length of 0 for 3 - 10 times.
                       (3 bits of length)
                   18: Repeat a code length of 0 for 11 - 138 times
                       (7 bits of length)

         A code length of 0 indicates that the corresponding symbol in
         the literal/length or distance alphabet will not occur in the
         block, and should not participate in the Huffman code
         construction algorithm given earlier.  If only one distance
         code is used, it is encoded using one bit, not zero bits; in
         this case there is a single code length of one, with one unused
         code.  One distance code of zero bits means that there are no
         distance codes used at all (the data is all literals).

         We can now define the format of the block:

               5 Bits: HLIT, # of Literal/Length codes - 257 (257 - 286)
               5 Bits: HDIST, # of Distance codes - 1        (1 - 32)
               4 Bits: HCLEN, # of Code Length codes - 4     (4 - 19)



Deutsch                      Informational                     [Page 13]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


               (HCLEN + 4) x 3 bits: code lengths for the code length
                  alphabet given just above, in the order: 16, 17, 18,
                  0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15

                  These code lengths are interpreted as 3-bit integers
                  (0-7); as above, a code length of 0 means the
                  corresponding symbol (literal/length or distance code
                  length) is not used.

               HLIT + 257 code lengths for the literal/length alphabet,
                  encoded using the code length Huffman code

               HDIST + 1 code lengths for the distance alphabet,
                  encoded using the code length Huffman code

               The actual compressed data of the block,
                  encoded using the literal/length and distance Huffman
                  codes

               The literal/length symbol 256 (end of data),
                  encoded using the literal/length Huffman code

         The code length repeat codes can cross from HLIT + 257 to the
         HDIST + 1 code lengths.  In other words, all code lengths form
         a single sequence of HLIT + HDIST + 258 values.

   3.3. Compliance

      A compressor may limit further the ranges of values specified in
      the previous section and still be compliant; for example, it may
      limit the range of backward pointers to some value smaller than
      32K.  Similarly, a compressor may limit the size of blocks so that
      a compressible block fits in memory.

      A compliant decompressor must accept the full range of possible
      values defined in the previous section, and must accept blocks of
      arbitrary size.

4. Compression algorithm details

   While it is the intent of this document to define the "deflate"
   compressed data format without reference to any particular
   compression algorithm, the format is related to the compressed
   formats produced by LZ77 (Lempel-Ziv 1977, see reference [2] below);
   since many variations of LZ77 are patented, it is strongly
   recommended that the implementor of a compressor follow the general
   algorithm presented here, which is known not to be patented per se.
   The material in this section is not part of the definition of the



Deutsch                      Informational                     [Page 14]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


   specification per se, and a compressor need not follow it in order to
   be compliant.

   The compressor terminates a block when it determines that starting a
   new block with fresh trees would be useful, or when the block size
   fills up the compressor's block buffer.

   The compressor uses a chained hash table to find duplicated strings,
   using a hash function that operates on 3-byte sequences.  At any
   given point during compression, let XYZ be the next 3 input bytes to
   be examined (not necessarily all different, of course).  First, the
   compressor examines the hash chain for XYZ.  If the chain is empty,
   the compressor simply writes out X as a literal byte and advances one
   byte in the input.  If the hash chain is not empty, indicating that
   the sequence XYZ (or, if we are unlucky, some other 3 bytes with the
   same hash function value) has occurred recently, the compressor
   compares all strings on the XYZ hash chain with the actual input data
   sequence starting at the current point, and selects the longest
   match.

   The compressor searches the hash chains starting with the most recent
   strings, to favor small distances and thus take advantage of the
   Huffman encoding.  The hash chains are singly linked. There are no
   deletions from the hash chains; the algorithm simply discards matches
   that are too old.  To avoid a worst-case situation, very long hash
   chains are arbitrarily truncated at a certain length, determined by a
   run-time parameter.

   To improve overall compression, the compressor optionally defers the
   selection of matches ("lazy matching"): after a match of length N has
   been found, the compressor searches for a longer match starting at
   the next input byte.  If it finds a longer match, it truncates the
   previous match to a length of one (thus producing a single literal
   byte) and then emits the longer match.  Otherwise, it emits the
   original match, and, as described above, advances N bytes before
   continuing.

   Run-time parameters also control this "lazy match" procedure.  If
   compression ratio is most important, the compressor attempts a
   complete second search regardless of the length of the first match.
   In the normal case, if the current match is "long enough", the
   compressor reduces the search for a longer match, thus speeding up
   the process.  If speed is most important, the compressor inserts new
   strings in the hash table only when no match was found, or when the
   match is not "too long".  This degrades the compression ratio but
   saves time since there are both fewer insertions and fewer searches.





Deutsch                      Informational                     [Page 15]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


5. References

   [1] Huffman, D. A., "A Method for the Construction of Minimum
       Redundancy Codes", Proceedings of the Institute of Radio
       Engineers, September 1952, Volume 40, Number 9, pp. 1098-1101.

   [2] Ziv J., Lempel A., "A Universal Algorithm for Sequential Data
       Compression", IEEE Transactions on Information Theory, Vol. 23,
       No. 3, pp. 337-343.

   [3] Gailly, J.-L., and Adler, M., ZLIB documentation and sources,
       available in ftp://ftp.uu.net/pub/archiving/zip/doc/

   [4] Gailly, J.-L., and Adler, M., GZIP documentation and sources,
       available as gzip-*.tar in ftp://prep.ai.mit.edu/pub/gnu/

   [5] Schwartz, E. S., and Kallick, B. "Generating a canonical prefix
       encoding." Comm. ACM, 7,3 (Mar. 1964), pp. 166-169.

   [6] Hirschberg and Lelewer, "Efficient decoding of prefix codes,"
       Comm. ACM, 33,4, April 1990, pp. 449-459.

6. Security Considerations

   Any data compression method involves the reduction of redundancy in
   the data.  Consequently, any corruption of the data is likely to have
   severe effects and be difficult to correct.  Uncompressed text, on
   the other hand, will probably still be readable despite the presence
   of some corrupted bytes.

   It is recommended that systems using this data format provide some
   means of validating the integrity of the compressed data.  See
   reference [3], for example.

7. Source code

   Source code for a C language implementation of a "deflate" compliant
   compressor and decompressor is available within the zlib package at
   ftp://ftp.uu.net/pub/archiving/zip/zlib/.

8. Acknowledgements

   Trademarks cited in this document are the property of their
   respective owners.

   Phil Katz designed the deflate format.  Jean-Loup Gailly and Mark
   Adler wrote the related software described in this specification.
   Glenn Randers-Pehrson converted this document to RFC and HTML format.



Deutsch                      Informational                     [Page 16]
 
RFC 1951      DEFLATE Compressed Data Format Specification      May 1996


9. Author's Address

   L. Peter Deutsch
   Aladdin Enterprises
   203 Santa Margarita Ave.
   Menlo Park, CA 94025

   Phone: (415) 322-0103 (AM only)
   FAX:   (415) 322-1734
   EMail: <ghost@aladdin.com>

   Questions about the technical content of this specification can be
   sent by email to:

   Jean-Loup Gailly <gzip@prep.ai.mit.edu> and
   Mark Adler <madler@alumni.caltech.edu>

   Editorial comments on this specification can be sent by email to:

   L. Peter Deutsch <ghost@aladdin.com> and
   Glenn Randers-Pehrson <randeg@alumni.rpi.edu>






























Deutsch                      Informational                     [Page 17]
 
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






















































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































Deleted compat/zlib/doc/rfc1952.txt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675






Network Working Group                                         P. Deutsch
Request for Comments: 1952                           Aladdin Enterprises
Category: Informational                                         May 1996


               GZIP file format specification version 4.3

Status of This Memo

   This memo provides information for the Internet community.  This memo
   does not specify an Internet standard of any kind.  Distribution of
   this memo is unlimited.

IESG Note:

   The IESG takes no position on the validity of any Intellectual
   Property Rights statements contained in this document.

Notices

   Copyright (c) 1996 L. Peter Deutsch

   Permission is granted to copy and distribute this document for any
   purpose and without charge, including translations into other
   languages and incorporation into compilations, provided that the
   copyright notice and this notice are preserved, and that any
   substantive changes or deletions from the original are clearly
   marked.

   A pointer to the latest version of this and related documentation in
   HTML format can be found at the URL
   <ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html>.

Abstract

   This specification defines a lossless compressed data format that is
   compatible with the widely used GZIP utility.  The format includes a
   cyclic redundancy check value for detecting data corruption.  The
   format presently uses the DEFLATE method of compression but can be
   easily extended to use other compression methods.  The format can be
   implemented readily in a manner not covered by patents.










Deutsch                      Informational                      [Page 1]
 
RFC 1952             GZIP File Format Specification             May 1996


Table of Contents

   1. Introduction ................................................... 2
      1.1. Purpose ................................................... 2
      1.2. Intended audience ......................................... 3
      1.3. Scope ..................................................... 3
      1.4. Compliance ................................................ 3
      1.5. Definitions of terms and conventions used ................. 3
      1.6. Changes from previous versions ............................ 3
   2. Detailed specification ......................................... 4
      2.1. Overall conventions ....................................... 4
      2.2. File format ............................................... 5
      2.3. Member format ............................................. 5
          2.3.1. Member header and trailer ........................... 6
              2.3.1.1. Extra field ................................... 8
              2.3.1.2. Compliance .................................... 9
      3. References .................................................. 9
      4. Security Considerations .................................... 10
      5. Acknowledgements ........................................... 10
      6. Author's Address ........................................... 10
      7. Appendix: Jean-Loup Gailly's gzip utility .................. 11
      8. Appendix: Sample CRC Code .................................. 11

1. Introduction

   1.1. Purpose

      The purpose of this specification is to define a lossless
      compressed data format that:

          * Is independent of CPU type, operating system, file system,
            and character set, and hence can be used for interchange;
          * Can compress or decompress a data stream (as opposed to a
            randomly accessible file) to produce another data stream,
            using only an a priori bounded amount of intermediate
            storage, and hence can be used in data communications or
            similar structures such as Unix filters;
          * Compresses data with efficiency comparable to the best
            currently available general-purpose compression methods,
            and in particular considerably better than the "compress"
            program;
          * Can be implemented readily in a manner not covered by
            patents, and hence can be practiced freely;
          * Is compatible with the file format produced by the current
            widely used gzip utility, in that conforming decompressors
            will be able to read data produced by the existing gzip
            compressor.




Deutsch                      Informational                      [Page 2]
 
RFC 1952             GZIP File Format Specification             May 1996


      The data format defined by this specification does not attempt to:

          * Provide random access to compressed data;
          * Compress specialized data (e.g., raster graphics) as well as
            the best currently available specialized algorithms.

   1.2. Intended audience

      This specification is intended for use by implementors of software
      to compress data into gzip format and/or decompress data from gzip
      format.

      The text of the specification assumes a basic background in
      programming at the level of bits and other primitive data
      representations.

   1.3. Scope

      The specification specifies a compression method and a file format
      (the latter assuming only that a file can store a sequence of
      arbitrary bytes).  It does not specify any particular interface to
      a file system or anything about character sets or encodings
      (except for file names and comments, which are optional).

   1.4. Compliance

      Unless otherwise indicated below, a compliant decompressor must be
      able to accept and decompress any file that conforms to all the
      specifications presented here; a compliant compressor must produce
      files that conform to all the specifications presented here.  The
      material in the appendices is not part of the specification per se
      and is not relevant to compliance.

   1.5. Definitions of terms and conventions used

      byte: 8 bits stored or transmitted as a unit (same as an octet).
      (For this specification, a byte is exactly 8 bits, even on
      machines which store a character on a number of bits different
      from 8.)  See below for the numbering of bits within a byte.

   1.6. Changes from previous versions

      There have been no technical changes to the gzip format since
      version 4.1 of this specification.  In version 4.2, some
      terminology was changed, and the sample CRC code was rewritten for
      clarity and to eliminate the requirement for the caller to do pre-
      and post-conditioning.  Version 4.3 is a conversion of the
      specification to RFC style.



Deutsch                      Informational                      [Page 3]
 
RFC 1952             GZIP File Format Specification             May 1996


2. Detailed specification

   2.1. Overall conventions

      In the diagrams below, a box like this:

         +---+
         |   | <-- the vertical bars might be missing
         +---+

      represents one byte; a box like this:

         +==============+
         |              |
         +==============+

      represents a variable number of bytes.

      Bytes stored within a computer do not have a "bit order", since
      they are always treated as a unit.  However, a byte considered as
      an integer between 0 and 255 does have a most- and least-
      significant bit, and since we write numbers with the most-
      significant digit on the left, we also write bytes with the most-
      significant bit on the left.  In the diagrams below, we number the
      bits of a byte so that bit 0 is the least-significant bit, i.e.,
      the bits are numbered:

         +--------+
         |76543210|
         +--------+

      This document does not address the issue of the order in which
      bits of a byte are transmitted on a bit-sequential medium, since
      the data format described here is byte- rather than bit-oriented.

      Within a computer, a number may occupy multiple bytes.  All
      multi-byte numbers in the format described here are stored with
      the least-significant byte first (at the lower memory address).
      For example, the decimal number 520 is stored as:

             0        1
         +--------+--------+
         |00001000|00000010|
         +--------+--------+
          ^        ^
          |        |
          |        + more significant byte = 2 x 256
          + less significant byte = 8



Deutsch                      Informational                      [Page 4]
 
RFC 1952             GZIP File Format Specification             May 1996


   2.2. File format

      A gzip file consists of a series of "members" (compressed data
      sets).  The format of each member is specified in the following
      section.  The members simply appear one after another in the file,
      with no additional information before, between, or after them.

   2.3. Member format

      Each member has the following structure:

         +---+---+---+---+---+---+---+---+---+---+
         |ID1|ID2|CM |FLG|     MTIME     |XFL|OS | (more-->)
         +---+---+---+---+---+---+---+---+---+---+

      (if FLG.FEXTRA set)

         +---+---+=================================+
         | XLEN  |...XLEN bytes of "extra field"...| (more-->)
         +---+---+=================================+

      (if FLG.FNAME set)

         +=========================================+
         |...original file name, zero-terminated...| (more-->)
         +=========================================+

      (if FLG.FCOMMENT set)

         +===================================+
         |...file comment, zero-terminated...| (more-->)
         +===================================+

      (if FLG.FHCRC set)

         +---+---+
         | CRC16 |
         +---+---+

         +=======================+
         |...compressed blocks...| (more-->)
         +=======================+

           0   1   2   3   4   5   6   7
         +---+---+---+---+---+---+---+---+
         |     CRC32     |     ISIZE     |
         +---+---+---+---+---+---+---+---+




Deutsch                      Informational                      [Page 5]
 
RFC 1952             GZIP File Format Specification             May 1996


      2.3.1. Member header and trailer

         ID1 (IDentification 1)
         ID2 (IDentification 2)
            These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139
            (0x8b, \213), to identify the file as being in gzip format.

         CM (Compression Method)
            This identifies the compression method used in the file.  CM
            = 0-7 are reserved.  CM = 8 denotes the "deflate"
            compression method, which is the one customarily used by
            gzip and which is documented elsewhere.

         FLG (FLaGs)
            This flag byte is divided into individual bits as follows:

               bit 0   FTEXT
               bit 1   FHCRC
               bit 2   FEXTRA
               bit 3   FNAME
               bit 4   FCOMMENT
               bit 5   reserved
               bit 6   reserved
               bit 7   reserved

            If FTEXT is set, the file is probably ASCII text.  This is
            an optional indication, which the compressor may set by
            checking a small amount of the input data to see whether any
            non-ASCII characters are present.  In case of doubt, FTEXT
            is cleared, indicating binary data. For systems which have
            different file formats for ascii text and binary data, the
            decompressor can use FTEXT to choose the appropriate format.
            We deliberately do not specify the algorithm used to set
            this bit, since a compressor always has the option of
            leaving it cleared and a decompressor always has the option
            of ignoring it and letting some other program handle issues
            of data conversion.

            If FHCRC is set, a CRC16 for the gzip header is present,
            immediately before the compressed data. The CRC16 consists
            of the two least significant bytes of the CRC32 for all
            bytes of the gzip header up to and not including the CRC16.
            [The FHCRC bit was never set by versions of gzip up to
            1.2.4, even though it was documented with a different
            meaning in gzip 1.2.4.]

            If FEXTRA is set, optional extra fields are present, as
            described in a following section.



Deutsch                      Informational                      [Page 6]
 
RFC 1952             GZIP File Format Specification             May 1996


            If FNAME is set, an original file name is present,
            terminated by a zero byte.  The name must consist of ISO
            8859-1 (LATIN-1) characters; on operating systems using
            EBCDIC or any other character set for file names, the name
            must be translated to the ISO LATIN-1 character set.  This
            is the original name of the file being compressed, with any
            directory components removed, and, if the file being
            compressed is on a file system with case insensitive names,
            forced to lower case. There is no original file name if the
            data was compressed from a source other than a named file;
            for example, if the source was stdin on a Unix system, there
            is no file name.

            If FCOMMENT is set, a zero-terminated file comment is
            present.  This comment is not interpreted; it is only
            intended for human consumption.  The comment must consist of
            ISO 8859-1 (LATIN-1) characters.  Line breaks should be
            denoted by a single line feed character (10 decimal).

            Reserved FLG bits must be zero.

         MTIME (Modification TIME)
            This gives the most recent modification time of the original
            file being compressed.  The time is in Unix format, i.e.,
            seconds since 00:00:00 GMT, Jan.  1, 1970.  (Note that this
            may cause problems for MS-DOS and other systems that use
            local rather than Universal time.)  If the compressed data
            did not come from a file, MTIME is set to the time at which
            compression started.  MTIME = 0 means no time stamp is
            available.

         XFL (eXtra FLags)
            These flags are available for use by specific compression
            methods.  The "deflate" method (CM = 8) sets these flags as
            follows:

               XFL = 2 - compressor used maximum compression,
                         slowest algorithm
               XFL = 4 - compressor used fastest algorithm

         OS (Operating System)
            This identifies the type of file system on which compression
            took place.  This may be useful in determining end-of-line
            convention for text files.  The currently defined values are
            as follows:






Deutsch                      Informational                      [Page 7]
 
RFC 1952             GZIP File Format Specification             May 1996


                 0 - FAT filesystem (MS-DOS, OS/2, NT/Win32)
                 1 - Amiga
                 2 - VMS (or OpenVMS)
                 3 - Unix
                 4 - VM/CMS
                 5 - Atari TOS
                 6 - HPFS filesystem (OS/2, NT)
                 7 - Macintosh
                 8 - Z-System
                 9 - CP/M
                10 - TOPS-20
                11 - NTFS filesystem (NT)
                12 - QDOS
                13 - Acorn RISCOS
               255 - unknown

         XLEN (eXtra LENgth)
            If FLG.FEXTRA is set, this gives the length of the optional
            extra field.  See below for details.

         CRC32 (CRC-32)
            This contains a Cyclic Redundancy Check value of the
            uncompressed data computed according to CRC-32 algorithm
            used in the ISO 3309 standard and in section 8.1.1.6.2 of
            ITU-T recommendation V.42.  (See http://www.iso.ch for
            ordering ISO documents. See gopher://info.itu.ch for an
            online version of ITU-T V.42.)

         ISIZE (Input SIZE)
            This contains the size of the original (uncompressed) input
            data modulo 2^32.

      2.3.1.1. Extra field

         If the FLG.FEXTRA bit is set, an "extra field" is present in
         the header, with total length XLEN bytes.  It consists of a
         series of subfields, each of the form:

            +---+---+---+---+==================================+
            |SI1|SI2|  LEN  |... LEN bytes of subfield data ...|
            +---+---+---+---+==================================+

         SI1 and SI2 provide a subfield ID, typically two ASCII letters
         with some mnemonic value.  Jean-Loup Gailly
         <gzip@prep.ai.mit.edu> is maintaining a registry of subfield
         IDs; please send him any subfield ID you wish to use.  Subfield
         IDs with SI2 = 0 are reserved for future use.  The following
         IDs are currently defined:



Deutsch                      Informational                      [Page 8]
 
RFC 1952             GZIP File Format Specification             May 1996


            SI1         SI2         Data
            ----------  ----------  ----
            0x41 ('A')  0x70 ('P')  Apollo file type information

         LEN gives the length of the subfield data, excluding the 4
         initial bytes.

      2.3.1.2. Compliance

         A compliant compressor must produce files with correct ID1,
         ID2, CM, CRC32, and ISIZE, but may set all the other fields in
         the fixed-length part of the header to default values (255 for
         OS, 0 for all others).  The compressor must set all reserved
         bits to zero.

         A compliant decompressor must check ID1, ID2, and CM, and
         provide an error indication if any of these have incorrect
         values.  It must examine FEXTRA/XLEN, FNAME, FCOMMENT and FHCRC
         at least so it can skip over the optional fields if they are
         present.  It need not examine any other part of the header or
         trailer; in particular, a decompressor may ignore FTEXT and OS
         and always produce binary output, and still be compliant.  A
         compliant decompressor must give an error indication if any
         reserved bit is non-zero, since such a bit could indicate the
         presence of a new field that would cause subsequent data to be
         interpreted incorrectly.

3. References

   [1] "Information Processing - 8-bit single-byte coded graphic
       character sets - Part 1: Latin alphabet No.1" (ISO 8859-1:1987).
       The ISO 8859-1 (Latin-1) character set is a superset of 7-bit
       ASCII. Files defining this character set are available as
       iso_8859-1.* in ftp://ftp.uu.net/graphics/png/documents/

   [2] ISO 3309

   [3] ITU-T recommendation V.42

   [4] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification",
       available in ftp://ftp.uu.net/pub/archiving/zip/doc/

   [5] Gailly, J.-L., GZIP documentation, available as gzip-*.tar in
       ftp://prep.ai.mit.edu/pub/gnu/

   [6] Sarwate, D.V., "Computation of Cyclic Redundancy Checks via Table
       Look-Up", Communications of the ACM, 31(8), pp.1008-1013.




Deutsch                      Informational                      [Page 9]
 
RFC 1952             GZIP File Format Specification             May 1996


   [7] Schwaderer, W.D., "CRC Calculation", April 85 PC Tech Journal,
       pp.118-133.

   [8] ftp://ftp.adelaide.edu.au/pub/rocksoft/papers/crc_v3.txt,
       describing the CRC concept.

4. Security Considerations

   Any data compression method involves the reduction of redundancy in
   the data.  Consequently, any corruption of the data is likely to have
   severe effects and be difficult to correct.  Uncompressed text, on
   the other hand, will probably still be readable despite the presence
   of some corrupted bytes.

   It is recommended that systems using this data format provide some
   means of validating the integrity of the compressed data, such as by
   setting and checking the CRC-32 check value.

5. Acknowledgements

   Trademarks cited in this document are the property of their
   respective owners.

   Jean-Loup Gailly designed the gzip format and wrote, with Mark Adler,
   the related software described in this specification.  Glenn
   Randers-Pehrson converted this document to RFC and HTML format.

6. Author's Address

   L. Peter Deutsch
   Aladdin Enterprises
   203 Santa Margarita Ave.
   Menlo Park, CA 94025

   Phone: (415) 322-0103 (AM only)
   FAX:   (415) 322-1734
   EMail: <ghost@aladdin.com>

   Questions about the technical content of this specification can be
   sent by email to:

   Jean-Loup Gailly <gzip@prep.ai.mit.edu> and
   Mark Adler <madler@alumni.caltech.edu>

   Editorial comments on this specification can be sent by email to:

   L. Peter Deutsch <ghost@aladdin.com> and
   Glenn Randers-Pehrson <randeg@alumni.rpi.edu>



Deutsch                      Informational                     [Page 10]
 
RFC 1952             GZIP File Format Specification             May 1996


7. Appendix: Jean-Loup Gailly's gzip utility

   The most widely used implementation of gzip compression, and the
   original documentation on which this specification is based, were
   created by Jean-Loup Gailly <gzip@prep.ai.mit.edu>.  Since this
   implementation is a de facto standard, we mention some more of its
   features here.  Again, the material in this section is not part of
   the specification per se, and implementations need not follow it to
   be compliant.

   When compressing or decompressing a file, gzip preserves the
   protection, ownership, and modification time attributes on the local
   file system, since there is no provision for representing protection
   attributes in the gzip file format itself.  Since the file format
   includes a modification time, the gzip decompressor provides a
   command line switch that assigns the modification time from the file,
   rather than the local modification time of the compressed input, to
   the decompressed output.

8. Appendix: Sample CRC Code

   The following sample code represents a practical implementation of
   the CRC (Cyclic Redundancy Check). (See also ISO 3309 and ITU-T V.42
   for a formal specification.)

   The sample code is in the ANSI C programming language. Non C users
   may find it easier to read with these hints:

      &      Bitwise AND operator.
      ^      Bitwise exclusive-OR operator.
      >>     Bitwise right shift operator. When applied to an
             unsigned quantity, as here, right shift inserts zero
             bit(s) at the left.
      !      Logical NOT operator.
      ++     "n++" increments the variable n.
      0xNNN  0x introduces a hexadecimal (base 16) constant.
             Suffix L indicates a long value (at least 32 bits).

      /* Table of CRCs of all 8-bit messages. */
      unsigned long crc_table[256];

      /* Flag: has the table been computed? Initially false. */
      int crc_table_computed = 0;

      /* Make the table for a fast CRC. */
      void make_crc_table(void)
      {
        unsigned long c;



Deutsch                      Informational                     [Page 11]
 
RFC 1952             GZIP File Format Specification             May 1996


        int n, k;
        for (n = 0; n < 256; n++) {
          c = (unsigned long) n;
          for (k = 0; k < 8; k++) {
            if (c & 1) {
              c = 0xedb88320L ^ (c >> 1);
            } else {
              c = c >> 1;
            }
          }
          crc_table[n] = c;
        }
        crc_table_computed = 1;
      }

      /*
         Update a running crc with the bytes buf[0..len-1] and return
       the updated crc. The crc should be initialized to zero. Pre- and
       post-conditioning (one's complement) is performed within this
       function so it shouldn't be done by the caller. Usage example:

         unsigned long crc = 0L;

         while (read_buffer(buffer, length) != EOF) {
           crc = update_crc(crc, buffer, length);
         }
         if (crc != original_crc) error();
      */
      unsigned long update_crc(unsigned long crc,
                      unsigned char *buf, int len)
      {
        unsigned long c = crc ^ 0xffffffffL;
        int n;

        if (!crc_table_computed)
          make_crc_table();
        for (n = 0; n < len; n++) {
          c = crc_table[(c ^ buf[n]) & 0xff] ^ (c >> 8);
        }
        return c ^ 0xffffffffL;
      }

      /* Return the CRC of the bytes buf[0..len-1]. */
      unsigned long crc(unsigned char *buf, int len)
      {
        return update_crc(0L, buf, len);
      }




Deutsch                      Informational                     [Page 12]
 
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































Deleted compat/zlib/doc/txtvsbin.txt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
A Fast Method for Identifying Plain Text Files
==============================================


Introduction
------------

Given a file coming from an unknown source, it is sometimes desirable
to find out whether the format of that file is plain text.  Although
this may appear like a simple task, a fully accurate detection of the
file type requires heavy-duty semantic analysis on the file contents.
It is, however, possible to obtain satisfactory results by employing
various heuristics.

Previous versions of PKZip and other zip-compatible compression tools
were using a crude detection scheme: if more than 80% (4/5) of the bytes
found in a certain buffer are within the range [7..127], the file is
labeled as plain text, otherwise it is labeled as binary.  A prominent
limitation of this scheme is the restriction to Latin-based alphabets.
Other alphabets, like Greek, Cyrillic or Asian, make extensive use of
the bytes within the range [128..255], and texts using these alphabets
are most often misidentified by this scheme; in other words, the rate
of false negatives is sometimes too high, which means that the recall
is low.  Another weakness of this scheme is a reduced precision, due to
the false positives that may occur when binary files containing large
amounts of textual characters are misidentified as plain text.

In this article we propose a new, simple detection scheme that features
a much increased precision and a near-100% recall.  This scheme is
designed to work on ASCII, Unicode and other ASCII-derived alphabets,
and it handles single-byte encodings (ISO-8859, MacRoman, KOI8, etc.)
and variable-sized encodings (ISO-2022, UTF-8, etc.).  Wider encodings
(UCS-2/UTF-16 and UCS-4/UTF-32) are not handled, however.


The Algorithm
-------------

The algorithm works by dividing the set of bytecodes [0..255] into three
categories:
- The white list of textual bytecodes:
  9 (TAB), 10 (LF), 13 (CR), 32 (SPACE) to 255.
- The gray list of tolerated bytecodes:
  7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB), 27 (ESC).
- The black list of undesired, non-textual bytecodes:
  0 (NUL) to 6, 14 to 31.

If a file contains at least one byte that belongs to the white list and
no byte that belongs to the black list, then the file is categorized as
plain text; otherwise, it is categorized as binary.  (The boundary case,
when the file is empty, automatically falls into the latter category.)


Rationale
---------

The idea behind this algorithm relies on two observations.

The first observation is that, although the full range of 7-bit codes
[0..127] is properly specified by the ASCII standard, most control
characters in the range [0..31] are not used in practice.  The only
widely-used, almost universally-portable control codes are 9 (TAB),
10 (LF) and 13 (CR).  There are a few more control codes that are
recognized on a reduced range of platforms and text viewers/editors:
7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB) and 27 (ESC); but these
codes are rarely (if ever) used alone, without being accompanied by
some printable text.  Even the newer, portable text formats such as
XML avoid using control characters outside the list mentioned here.

The second observation is that most of the binary files tend to contain
control characters, especially 0 (NUL).  Even though the older text
detection schemes observe the presence of non-ASCII codes from the range
[128..255], the precision rarely has to suffer if this upper range is
labeled as textual, because the files that are genuinely binary tend to
contain both control characters and codes from the upper range.  On the
other hand, the upper range needs to be labeled as textual, because it
is used by virtually all ASCII extensions.  In particular, this range is
used for encoding non-Latin scripts.

Since there is no counting involved, other than simply observing the
presence or the absence of some byte values, the algorithm produces
consistent results, regardless what alphabet encoding is being used.
(If counting were involved, it could be possible to obtain different
results on a text encoded, say, using ISO-8859-16 versus UTF-8.)

There is an extra category of plain text files that are "polluted" with
one or more black-listed codes, either by mistake or by peculiar design
considerations.  In such cases, a scheme that tolerates a small fraction
of black-listed codes would provide an increased recall (i.e. more true
positives).  This, however, incurs a reduced precision overall, since
false positives are more likely to appear in binary files that contain
large chunks of textual data.  Furthermore, "polluted" plain text should
be regarded as binary by general-purpose text detection schemes, because
general-purpose text processing algorithms might not be applicable.
Under this premise, it is safe to say that our detection method provides
a near-100% recall.

Experiments have been run on many files coming from various platforms
and applications.  We tried plain text files, system logs, source code,
formatted office documents, compiled object code, etc.  The results
confirm the optimistic assumptions about the capabilities of this
algorithm.


--
Cosmin Truta
Last updated: 2006-May-28
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






















































































































































































































Changes to src/allrepo.c.

416
417
418
419
420
421
422

423
424
425
426
427
428
429
    zSyscmd = mprintf("%s %s %s%s",
                      zFossil, zCmd, zQFilename, blob_str(&extra));
    if( showLabel ){
      int len = (int)strlen(zFilename);
      int nStar = 80 - (len + 15);
      if( nStar<2 ) nStar = 1;
      fossil_print("%.13c %s %.*c\n", '*', zFilename, nStar, '*');

    }
    if( !quiet || dryRunFlag ){
      fossil_print("%s\n", zSyscmd);
      fflush(stdout);
    }
    rc = dryRunFlag ? 0 : fossil_system(zSyscmd);
    free(zSyscmd);







>







416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
    zSyscmd = mprintf("%s %s %s%s",
                      zFossil, zCmd, zQFilename, blob_str(&extra));
    if( showLabel ){
      int len = (int)strlen(zFilename);
      int nStar = 80 - (len + 15);
      if( nStar<2 ) nStar = 1;
      fossil_print("%.13c %s %.*c\n", '*', zFilename, nStar, '*');
      fflush(stdout);
    }
    if( !quiet || dryRunFlag ){
      fossil_print("%s\n", zSyscmd);
      fflush(stdout);
    }
    rc = dryRunFlag ? 0 : fossil_system(zSyscmd);
    free(zSyscmd);

Changes to src/attach.c.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
/*
** Copyright (c) 2010 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)

** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
/*
** Copyright (c) 2010 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/

Changes to src/blob.c.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
...
645
646
647
648
649
650
651
652





653
654
655
656
657
658
659
660
661
662
663
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)

** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
................................................................................
  if( pTo ){
    blob_append(pTo, &pFrom->aData[pFrom->iCursor], i - pFrom->iCursor);
  }
  pFrom->iCursor = i;
}

/*
** Return true if the blob contains a valid UUID_SIZE-digit base16 identifier.





*/
int blob_is_uuid(Blob *pBlob){
  return blob_size(pBlob)==UUID_SIZE
         && validate16(blob_buffer(pBlob), UUID_SIZE);
}

/*
** Return true if the blob contains a valid filename
*/
int blob_is_filename(Blob *pBlob){
  return file_is_simple_pathname(blob_str(pBlob), 1);






|







 







|
>
>
>
>
>

|
<
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
...
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659

660
661
662
663
664
665
666
667
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
................................................................................
  if( pTo ){
    blob_append(pTo, &pFrom->aData[pFrom->iCursor], i - pFrom->iCursor);
  }
  pFrom->iCursor = i;
}

/*
** Return true if the blob contains a valid base16 identifier artifact hash.
**
** The value returned is actually one of HNAME_SHA1 OR HNAME_K256 if the
** hash is valid.  Both of these are non-zero and therefore "true".
** If the hash is not valid, then HNAME_ERROR is returned, which is zero or
** false.
*/
int blob_is_hname(Blob *pBlob){

  return hname_validate(blob_buffer(pBlob), blob_size(pBlob));
}

/*
** Return true if the blob contains a valid filename
*/
int blob_is_filename(Blob *pBlob){
  return file_is_simple_pathname(blob_str(pBlob), 1);

Changes to src/browse.c.

131
132
133
134
135
136
137

138
139
140
141
142
143
144
...
297
298
299
300
301
302
303
























































304
305
306
307
308
309
310
...
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
...
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
...
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
  int rid = 0;
  char *zUuid = 0;
  Blob dirname;
  Manifest *pM = 0;
  const char *zSubdirLink;
  int linkTrunk = 1;
  int linkTip = 1;

  HQuery sURI;

  if( strcmp(PD("type","flat"),"tree")==0 ){ page_tree(); return; }
  login_check_credentials();
  if( !g.perm.Read ){ login_needed(g.anon.Read); return; }
  while( nD>1 && zD[nD-2]=='/' ){ zD[(--nD)-1] = 0; }
  style_header("File List");
................................................................................
      }
      @ <li class="%z(fileext_class(zFN))">%z(zLink)%h(zFN)</a></li>
    }
  }
  db_finalize(&q);
  manifest_destroy(pM);
  @ </ul></td></tr></table>
























































  style_footer();
}

/*
** Objects used by the "tree" webpage.
*/
typedef struct FileTreeNode FileTreeNode;
................................................................................
  FileTreeNode *pNext;      /* Next entry in an ordered list of them all */
  FileTreeNode *pParent;    /* Directory containing this entry */
  FileTreeNode *pSibling;   /* Next element in the same subdirectory */
  FileTreeNode *pChild;     /* List of child nodes */
  FileTreeNode *pLastChild; /* Last child on the pChild list */
  char *zName;              /* Name of this entry.  The "tail" */
  char *zFullName;          /* Full pathname of this entry */
  char *zUuid;              /* SHA1 hash of this file.  May be NULL. */
  double mtime;             /* Modification time for this entry */
  unsigned nFullName;       /* Length of zFullName */
  unsigned iLevel;          /* Levels of parent directories */
};

/*
** A complete file hierarchy
................................................................................
** When constructing a list of FileTreeNodes, all entries that have
** a common directory prefix must be added consecutively in order for
** the tree to be constructed properly.
*/
static void tree_add_node(
  FileTree *pTree,         /* Tree into which nodes are added */
  const char *zPath,       /* The full pathname of file to add */
  const char *zUuid,       /* UUID of the file.  Might be NULL. */
  double mtime             /* Modification time for this entry */
){
  int i;
  FileTreeNode *pParent;   /* Parent (directory) of the next node to insert */

  /* Make pParent point to the most recent ancestor of zPath, or
  ** NULL if there are no prior entires that are a container for zPath.
................................................................................
  i = pParent ? pParent->nFullName+1 : 0;
  while( zPath[i] ){
    FileTreeNode *pNew;
    int iStart = i;
    int nByte;
    while( zPath[i] && zPath[i]!='/' ){ i++; }
    nByte = sizeof(*pNew) + i + 1;
    if( zUuid!=0 && zPath[i]==0 ) nByte += UUID_SIZE+1;
    pNew = fossil_malloc( nByte );
    memset(pNew, 0, sizeof(*pNew));
    pNew->zFullName = (char*)&pNew[1];
    memcpy(pNew->zFullName, zPath, i);
    pNew->zFullName[i] = 0;
    pNew->nFullName = i;
    if( zUuid!=0 && zPath[i]==0 ){
      pNew->zUuid = pNew->zFullName + i + 1;
      memcpy(pNew->zUuid, zUuid, UUID_SIZE+1);
    }
    pNew->zName = pNew->zFullName + iStart;
    if( pTree->pLast ){
      pTree->pLast->pNext = pNew;
    }else{
      pTree->pFirst = pNew;
    }







>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|







 







|







 







|








|







131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
...
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
...
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
...
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
...
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
  int rid = 0;
  char *zUuid = 0;
  Blob dirname;
  Manifest *pM = 0;
  const char *zSubdirLink;
  int linkTrunk = 1;
  int linkTip = 1;
  char *zReadme;
  HQuery sURI;

  if( strcmp(PD("type","flat"),"tree")==0 ){ page_tree(); return; }
  login_check_credentials();
  if( !g.perm.Read ){ login_needed(g.anon.Read); return; }
  while( nD>1 && zD[nD-2]=='/' ){ zD[(--nD)-1] = 0; }
  style_header("File List");
................................................................................
      }
      @ <li class="%z(fileext_class(zFN))">%z(zLink)%h(zFN)</a></li>
    }
  }
  db_finalize(&q);
  manifest_destroy(pM);
  @ </ul></td></tr></table>

  /* If the directory contains a readme file, then display its content below
  ** the list of files
  */
  db_prepare(&q,
    "SELECT x, u FROM localfiles"
    " WHERE x COLLATE nocase IN"
    " ('readme','readme.txt','readme.md','readme.wiki','readme.markdown',"
    " 'readme.html') ORDER BY x LIMIT 1;"
  );
  if( db_step(&q)==SQLITE_ROW ){
    const char *zName = db_column_text(&q,0);
    const char *zUuid = db_column_text(&q,1);
    if( zUuid ){
      rid = fast_uuid_to_rid(zUuid);
    }else{
      if( zD ){
        rid = db_int(0,
           "SELECT fid FROM filename, mlink, event"
           " WHERE name='%q/%q'"
           "   AND mlink.fnid=filename.fnid"
           "   AND event.objid=mlink.mid"
           " ORDER BY event.mtime DESC LIMIT 1",
           zD, zName
        );
      }else{
        rid = db_int(0,
           "SELECT fid FROM filename, mlink, event"
           " WHERE name='%q'"
           "   AND mlink.fnid=filename.fnid"
           "   AND event.objid=mlink.mid"
           " ORDER BY event.mtime DESC LIMIT 1",
           zName
        );
      }
    }
    if( rid ){
      @ <hr>
      if( sqlite3_strlike("readme.html", zName, 0)==0 ){
        if( zUuid==0 ){
          zUuid = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", rid);
        }
        @ <iframe src="%R/raw/%s(zUuid)"
        @ width="100%%" frameborder="0" marginwidth="0" marginheight="0"
        @ sandbox="allow-same-origin"
        @ onload="this.height=this.contentDocument.documentElement.scrollHeight;">
        @ </iframe>
      }else{
        Blob content;
        const char *zMime = mimetype_from_name(zName);
        content_get(rid, &content);
        wiki_render_by_mimetype(&content, zMime);
      }
    }
  }
  db_finalize(&q);
  style_footer();
}

/*
** Objects used by the "tree" webpage.
*/
typedef struct FileTreeNode FileTreeNode;
................................................................................
  FileTreeNode *pNext;      /* Next entry in an ordered list of them all */
  FileTreeNode *pParent;    /* Directory containing this entry */
  FileTreeNode *pSibling;   /* Next element in the same subdirectory */
  FileTreeNode *pChild;     /* List of child nodes */
  FileTreeNode *pLastChild; /* Last child on the pChild list */
  char *zName;              /* Name of this entry.  The "tail" */
  char *zFullName;          /* Full pathname of this entry */
  char *zUuid;              /* Artifact hash of this file.  May be NULL. */
  double mtime;             /* Modification time for this entry */
  unsigned nFullName;       /* Length of zFullName */
  unsigned iLevel;          /* Levels of parent directories */
};

/*
** A complete file hierarchy
................................................................................
** When constructing a list of FileTreeNodes, all entries that have
** a common directory prefix must be added consecutively in order for
** the tree to be constructed properly.
*/
static void tree_add_node(
  FileTree *pTree,         /* Tree into which nodes are added */
  const char *zPath,       /* The full pathname of file to add */
  const char *zUuid,       /* Hash of the file.  Might be NULL. */
  double mtime             /* Modification time for this entry */
){
  int i;
  FileTreeNode *pParent;   /* Parent (directory) of the next node to insert */

  /* Make pParent point to the most recent ancestor of zPath, or
  ** NULL if there are no prior entires that are a container for zPath.
................................................................................
  i = pParent ? pParent->nFullName+1 : 0;
  while( zPath[i] ){
    FileTreeNode *pNew;
    int iStart = i;
    int nByte;
    while( zPath[i] && zPath[i]!='/' ){ i++; }
    nByte = sizeof(*pNew) + i + 1;
    if( zUuid!=0 && zPath[i]==0 ) nByte += HNAME_MAX+1;
    pNew = fossil_malloc( nByte );
    memset(pNew, 0, sizeof(*pNew));
    pNew->zFullName = (char*)&pNew[1];
    memcpy(pNew->zFullName, zPath, i);
    pNew->zFullName[i] = 0;
    pNew->nFullName = i;
    if( zUuid!=0 && zPath[i]==0 ){
      pNew->zUuid = pNew->zFullName + i + 1;
      memcpy(pNew->zUuid, zUuid, strlen(zUuid)+1);
    }
    pNew->zName = pNew->zFullName + iStart;
    if( pTree->pLast ){
      pTree->pLast->pNext = pNew;
    }else{
      pTree->pFirst = pNew;
    }

Changes to src/bundle.c.

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
...
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
...
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
...
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
...
595
596
597
598
599
600
601
602
603

604
605
606
607
608
609
610
static const char zBundleInit[] =
@ CREATE TABLE IF NOT EXISTS "%w".bconfig(
@   bcname TEXT,
@   bcvalue ANY
@ );
@ CREATE TABLE IF NOT EXISTS "%w".bblob(
@   blobid INTEGER PRIMARY KEY,      -- Blob ID
@   uuid TEXT NOT NULL,              -- SHA1 hash of expanded blob
@   sz INT NOT NULL,                 -- Size of blob after expansion
@   delta ANY,                       -- Delta compression basis, or NULL
@   notes TEXT,                      -- Description of content
@   data BLOB                        -- compressed content
@ );
;

................................................................................
     "SELECT uuid, data, bblob.delta, bix.blobid"
     "  FROM bix, bblob"
     " WHERE bix.delta=%d"
     "   AND bix.blobid=bblob.blobid;",
     iSrc
  );
  while( db_step(&q)==SQLITE_ROW ){
    Blob h1, h2, c1, c2;
    int rid;
    blob_zero(&h1);
    db_column_blob(&q, 0, &h1);
    blob_zero(&c1);
    db_column_blob(&q, 1, &c1);
    blob_uncompress(&c1, &c1);
    blob_zero(&c2);
    if( db_column_type(&q,2)==SQLITE_TEXT && db_column_bytes(&q,2)==40 ){
      Blob basis;
      rid = db_int(0,"SELECT rid FROM blob WHERE uuid=%Q",
                   db_column_text(&q,2));
      content_get(rid, &basis);
      blob_delta_apply(&basis, &c1, &c2);
      blob_reset(&basis);
      blob_reset(&c1);
    }else if( pBasis ){
      blob_delta_apply(pBasis, &c1, &c2);
      blob_reset(&c1);
    }else{
      c2 = c1;
    }
    sha1sum_blob(&c2, &h2);
    if( blob_compare(&h1, &h2)!=0 ){
      fossil_fatal("SHA1 hash mismatch - wanted %s, got %s",
                   blob_str(&h1), blob_str(&h2));
    }
    blob_reset(&h2);
    rid = content_put_ex(&c2, blob_str(&h1), 0, 0, isPriv);
    if( rid==0 ){
      fossil_fatal("%s", g.zErrMsg);
    }else{
      if( !isPriv ) content_make_public(rid);
      content_get(rid, &c1);
      manifest_crosslink(rid, &c1, MC_NO_ERRORS);
................................................................................
** Extract an item from content from the bundle
*/
static void bundle_extract_item(
  int blobid,          /* ID of the item to extract */
  Blob *pOut           /* Write the content into this blob */
){
  Stmt q;
  Blob x, basis, h1, h2;
  static Bag busy;

  db_prepare(&q, "SELECT uuid, delta, data FROM bblob"
                 " WHERE blobid=%d", blobid);
  if( db_step(&q)!=SQLITE_ROW ){
    db_finalize(&q);
    fossil_fatal("no such item: %d", blobid);
................................................................................
    blob_reset(&basis);
    blob_reset(&x);
  }else{
    *pOut = x;
  }
  blob_zero(&h1);
  db_column_blob(&q, 0, &h1);
  sha1sum_blob(pOut, &h2);
  if( blob_compare(&h1, &h2)!=0 ){
    fossil_fatal("SHA1 hash mismatch - wanted %s, got %s",
                 blob_str(&h1), blob_str(&h2));
  }
  blob_reset(&h1);
  blob_reset(&h2);
  bag_remove(&busy, blobid);
  db_finalize(&q);
}

/* fossil bundle cat BUNDLE UUID...
**
** Write elements of a bundle on standard output
................................................................................
  /* If the bundle contains deltas with a basis that is external to the
  ** bundle and those external basis files are missing from the local
  ** repo, then the delta encodings cannot be decoded and the bundle cannot
  ** be extracted. */
  zMissingDeltas = db_text(0,
      "SELECT group_concat(substr(delta,1,10),' ')"
      "  FROM bblob"
      " WHERE typeof(delta)='text' AND length(delta)=40"
      "   AND NOT EXISTS(SELECT 1 FROM blob WHERE uuid=bblob.delta)");

  if( zMissingDeltas && zMissingDeltas[0] ){
    fossil_fatal("delta basis artifacts not found in repository: %s",
                 zMissingDeltas);
  }

  db_begin_transaction();
  db_multi_exec(







|







 







|







|













|
|
<
<

<







 







|







 







|
|
<
<


<







 







|
|
>







33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
...
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467


468

469
470
471
472
473
474
475
...
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
...
520
521
522
523
524
525
526
527
528


529
530

531
532
533
534
535
536
537
...
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
static const char zBundleInit[] =
@ CREATE TABLE IF NOT EXISTS "%w".bconfig(
@   bcname TEXT,
@   bcvalue ANY
@ );
@ CREATE TABLE IF NOT EXISTS "%w".bblob(
@   blobid INTEGER PRIMARY KEY,      -- Blob ID
@   uuid TEXT NOT NULL,              -- hash of expanded blob
@   sz INT NOT NULL,                 -- Size of blob after expansion
@   delta ANY,                       -- Delta compression basis, or NULL
@   notes TEXT,                      -- Description of content
@   data BLOB                        -- compressed content
@ );
;

................................................................................
     "SELECT uuid, data, bblob.delta, bix.blobid"
     "  FROM bix, bblob"
     " WHERE bix.delta=%d"
     "   AND bix.blobid=bblob.blobid;",
     iSrc
  );
  while( db_step(&q)==SQLITE_ROW ){
    Blob h1, c1, c2;
    int rid;
    blob_zero(&h1);
    db_column_blob(&q, 0, &h1);
    blob_zero(&c1);
    db_column_blob(&q, 1, &c1);
    blob_uncompress(&c1, &c1);
    blob_zero(&c2);
    if( db_column_type(&q,2)==SQLITE_TEXT && db_column_bytes(&q,2)>=HNAME_MIN ){
      Blob basis;
      rid = db_int(0,"SELECT rid FROM blob WHERE uuid=%Q",
                   db_column_text(&q,2));
      content_get(rid, &basis);
      blob_delta_apply(&basis, &c1, &c2);
      blob_reset(&basis);
      blob_reset(&c1);
    }else if( pBasis ){
      blob_delta_apply(pBasis, &c1, &c2);
      blob_reset(&c1);
    }else{
      c2 = c1;
    }
    if( hname_verify_hash(&c2, blob_buffer(&h1), blob_size(&h1))==0 ){
      fossil_fatal("artifact hash error on %b", &h1);


    }

    rid = content_put_ex(&c2, blob_str(&h1), 0, 0, isPriv);
    if( rid==0 ){
      fossil_fatal("%s", g.zErrMsg);
    }else{
      if( !isPriv ) content_make_public(rid);
      content_get(rid, &c1);
      manifest_crosslink(rid, &c1, MC_NO_ERRORS);
................................................................................
** Extract an item from content from the bundle
*/
static void bundle_extract_item(
  int blobid,          /* ID of the item to extract */
  Blob *pOut           /* Write the content into this blob */
){
  Stmt q;
  Blob x, basis, h1;
  static Bag busy;

  db_prepare(&q, "SELECT uuid, delta, data FROM bblob"
                 " WHERE blobid=%d", blobid);
  if( db_step(&q)!=SQLITE_ROW ){
    db_finalize(&q);
    fossil_fatal("no such item: %d", blobid);
................................................................................
    blob_reset(&basis);
    blob_reset(&x);
  }else{
    *pOut = x;
  }
  blob_zero(&h1);
  db_column_blob(&q, 0, &h1);
  if( hname_verify_hash(pOut, blob_buffer(&h1), blob_size(&h1))==0 ){
    fossil_fatal("incorrect hash for artifact %b", &h1);


  }
  blob_reset(&h1);

  bag_remove(&busy, blobid);
  db_finalize(&q);
}

/* fossil bundle cat BUNDLE UUID...
**
** Write elements of a bundle on standard output
................................................................................
  /* If the bundle contains deltas with a basis that is external to the
  ** bundle and those external basis files are missing from the local
  ** repo, then the delta encodings cannot be decoded and the bundle cannot
  ** be extracted. */
  zMissingDeltas = db_text(0,
      "SELECT group_concat(substr(delta,1,10),' ')"
      "  FROM bblob"
      " WHERE typeof(delta)='text' AND length(delta)>=%d"
      "   AND NOT EXISTS(SELECT 1 FROM blob WHERE uuid=bblob.delta)",
      HNAME_MIN);
  if( zMissingDeltas && zMissingDeltas[0] ){
    fossil_fatal("delta basis artifacts not found in repository: %s",
                 zMissingDeltas);
  }

  db_begin_transaction();
  db_multi_exec(

Changes to src/cgi.c.

1442
1443
1444
1445
1446
1447
1448


1449
1450
1451
1452
1453
1454
1455
      cgi_setenv("HTTP_IF_NONE_MATCH", zVal);
    }else if( fossil_strcmp(zFieldName,"if-modified-since:")==0 ){
      cgi_setenv("HTTP_IF_MODIFIED_SINCE", zVal);
    }else if( fossil_strcmp(zFieldName,"referer:")==0 ){
      cgi_setenv("HTTP_REFERER", zVal);
    }else if( fossil_strcmp(zFieldName,"user-agent:")==0 ){
      cgi_setenv("HTTP_USER_AGENT", zVal);


    }else if( fossil_strcmp(zFieldName,"x-forwarded-for:")==0 ){
      const char *zIpAddr = cgi_accept_forwarded_for(zVal);
      if( zIpAddr!=0 ){
        g.zIpAddr = mprintf("%s", zIpAddr);
        cgi_replace_parameter("REMOTE_ADDR", g.zIpAddr);
      }
    }







>
>







1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
      cgi_setenv("HTTP_IF_NONE_MATCH", zVal);
    }else if( fossil_strcmp(zFieldName,"if-modified-since:")==0 ){
      cgi_setenv("HTTP_IF_MODIFIED_SINCE", zVal);
    }else if( fossil_strcmp(zFieldName,"referer:")==0 ){
      cgi_setenv("HTTP_REFERER", zVal);
    }else if( fossil_strcmp(zFieldName,"user-agent:")==0 ){
      cgi_setenv("HTTP_USER_AGENT", zVal);
    }else if( fossil_strcmp(zFieldName,"authorization:")==0 ){
      cgi_setenv("HTTP_AUTHORIZATION", zVal);
    }else if( fossil_strcmp(zFieldName,"x-forwarded-for:")==0 ){
      const char *zIpAddr = cgi_accept_forwarded_for(zVal);
      if( zIpAddr!=0 ){
        g.zIpAddr = mprintf("%s", zIpAddr);
        cgi_replace_parameter("REMOTE_ADDR", g.zIpAddr);
      }
    }

Changes to src/checkin.c.

375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
...
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
...
462
463
464
465
466
467
468
469

470
471
472
473
474
475
476
...
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
...
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
...
900
901
902
903
904
905
906
907
908

909
910
911
912
913
914
915
916
...
997
998
999
1000
1001
1002
1003
1004

1005
1006
1007
1008
1009
1010
1011
....
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
....
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
....
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
....
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
....
2068
2069
2070
2071
2072
2073
2074
2075

2076
2077
2078
2079
2080
2081
2082
....
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
** can be overridden by using one or more filter options (listed below),
** in which case only files with the specified change type(s) are shown.
** As a special case, the --no-merge option does not inhibit this default.
** This default shows exactly the set of changes that would be checked
** in by the commit command.
**
** If no filter options are used, or if the --merge option is used, the
** SHA1 hash of each merge contributor check-in version is displayed at
** the end of the report.  The --no-merge option is useful to display the
** default set of changed files without the merge contributors.
**
** If change type classification is enabled, each output line starts with
** a code describing the file's change type, e.g. EDITED or RENAMED.  It
** is enabled by default unless exactly one change type is selected.  For
** the purposes of determining the default, --changed counts as selecting
................................................................................
**
** The "fossil changes --extra" command is equivalent to "fossil extras".
**
** General options:
**    --abs-paths       Display absolute pathnames.
**    --rel-paths       Display pathnames relative to the current working
**                      directory.
**    --sha1sum         Verify file status using SHA1 hashing rather than
**                      relying on file mtimes.
**    --case-sensitive <BOOL>  Override case-sensitive setting.
**    --dotfiles        Include unmanaged files beginning with a dot.
**    --ignore <CSG>    Ignore unmanaged files matching CSG glob patterns.
**    --no-dir-symlinks Disables support for directory symlinks.
**
** Options specific to the changes command:
................................................................................
    {"classify", C_CLASSIFY},
  }, noFlagDefs[] = {
    {"no-merge", C_MERGE   }, {"no-classify", C_CLASSIFY },
  };

  Blob report = BLOB_INITIALIZER;
  enum {CHANGES, STATUS} command = *g.argv[1]=='s' ? STATUS : CHANGES;
  int useSha1sum = find_option("sha1sum", 0, 0)!=0;

  int showHdr = command==CHANGES && find_option("header", 0, 0);
  int verboseFlag = command==CHANGES && find_option("verbose", "v", 0);
  const char *zIgnoreFlag = find_option("ignore", 0, 1);
  unsigned scanFlags = 0;
  unsigned flags = 0;
  int vid, i;

................................................................................
    scanFlags = SCAN_ALL;
  }

  /* We should be done with options. */
  verify_all_options();

  /* Check for changed files. */
  vfile_check_signature(vid, useSha1sum ? CKSIG_SHA1 : 0);

  /* Search for unmanaged files if requested. */
  if( flags & C_EXTRA ){
    Glob *pIgnore = glob_create(zIgnoreFlag);
    locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
    glob_free(pIgnore);
  }
................................................................................
  /* We should be done with options.. */
  verify_all_options();

  if( zIgnoreFlag==0 ){
    zIgnoreFlag = db_get("ignore-glob", 0);
  }
  pIgnore = glob_create(zIgnoreFlag);
  /* Always consider symlinks. */
  g.allowSymlinks = db_allow_symlinks_by_default();
  locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
  glob_free(pIgnore);

  blob_zero(&report);
  status_report(&report, flags);
  if( blob_size(&report) ){
    if( showHdr ){
................................................................................
** ever be deleted.  Files and subdirectories whose names begin with "."
** are automatically ignored unless the --dotfiles option is used.
**
** The default values for --clean, --ignore, and --keep are determined by
** the (versionable) clean-glob, ignore-glob, and keep-glob settings.
**
** The --verily option ignores the keep-glob and ignore-glob settings and
** turns on --force, --emptydirs, --dotfiles, and --disable-undo.  Use the
** --verily option when you really want to clean up everything.  Extreme

** care should be exercised when using the --verily option.
**
** Options:
**    --allckouts      Check for empty directories within any checkouts
**                     that may be nested within the current one.  This
**                     option should be used with great care because the
**                     empty-dirs setting (and other applicable settings)
**                     belonging to the other repositories, if any, will
................................................................................
  db_must_be_within_tree();
  if( find_option("verily","x",0)!=0 ){
    verilyFlag = allFileFlag = allDirFlag = 1;
    emptyDirsFlag = 1;
    disableUndo = 1;
    scanFlags |= SCAN_ALL;
    zCleanFlag = 0;
    g.fNoDirSymlinks = 1;

  }
  if( zIgnoreFlag==0 && !verilyFlag ){
    zIgnoreFlag = db_get("ignore-glob", 0);
  }
  if( zKeepFlag==0 && !verilyFlag ){
    zKeepFlag = db_get("keep-glob", 0);
  }
................................................................................
  }
  if( db_get_boolean("dotfiles", 0) ) scanFlags |= SCAN_ALL;
  verify_all_options();
  pIgnore = glob_create(zIgnoreFlag);
  pKeep = glob_create(zKeepFlag);
  pClean = glob_create(zCleanFlag);
  nRoot = (int)strlen(g.zLocalRoot);
  /* Always consider symlinks. */
  g.allowSymlinks = db_allow_symlinks_by_default();
  if( !dirsOnlyFlag ){
    Stmt q;
    Blob repo;
    if( !dryRunFlag && !disableUndo ) undo_begin();
    locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
    db_prepare(&q,
        "SELECT %Q || pathname FROM sfile"
................................................................................
** blank commit message.  The default value is "N", do not commit.
**
** The --private option creates a private check-in that is never synced.
** Children of private check-ins are automatically private.
**
** The --tag option applies the symbolic tag name to the check-in.
**
** The --sha1sum option detects edited files by computing each file's
** SHA1 hash rather than just checking for changes to its size or mtime.
**
** Options:
**    --allow-conflict           allow unresolved merge conflicts
**    --allow-empty              allow a commit with no changes
**    --allow-fork               allow the commit to fork
**    --allow-older              allow a commit older than its ancestor
**    --baseline                 use a baseline manifest in the commit process
................................................................................
**    -n|--dry-run               If given, display instead of run actions
**    --no-prompt                This option disables prompting the user for
**                               input and assumes an answer of 'No' for every
**                               question.
**    --no-warnings              omit all warnings about file contents
**    --nosign                   do not attempt to sign this commit with gpg
**    --private                  do not sync changes and their descendants
**    --sha1sum                  verify file status using SHA1 hashing rather
**                               than relying on file mtimes
**    --tag TAG-NAME             assign given tag TAG-NAME to the check-in
**    --date-override DATETIME   DATE to use instead of 'now'
**    --user-override USER       USER to use instead of the current default
**
** DATETIME may be "now" or "YYYY-MM-DDTHH:MM:SS.SSS". If in
** year-month-day form, it may be truncated, the "T" may be replaced by
................................................................................
  int vid;               /* blob-id of parent version */
  int nrid;              /* blob-id of a modified file */
  int nvid;              /* Blob-id of the new check-in */
  Blob comment;          /* Check-in comment */
  const char *zComment;  /* Check-in comment */
  Stmt q;                /* Various queries */
  char *zUuid;           /* UUID of the new check-in */
  int useSha1sum = 0;    /* True to verify file status using SHA1 hashing */
  int noSign = 0;        /* True to omit signing the manifest using GPG */
  int isAMerge = 0;      /* True if checking in a merge */
  int noWarningFlag = 0; /* True if skipping all warnings */
  int noPrompt = 0;      /* True if skipping all prompts */
  int forceFlag = 0;     /* Undocumented: Disables all checks */
  int forceDelta = 0;    /* Force a delta-manifest */
  int forceBaseline = 0; /* Force a baseline-manifest */
................................................................................
  int nConflict = 0;     /* Number of unresolved merge conflicts */
  int abortCommit = 0;
  Blob ans;
  char cReply;

  memset(&sCiInfo, 0, sizeof(sCiInfo));
  url_proxy_options();
  useSha1sum = find_option("sha1sum", 0, 0)!=0;

  noSign = find_option("nosign",0,0)!=0;
  forceDelta = find_option("delta",0,0)!=0;
  forceBaseline = find_option("baseline",0,0)!=0;
  if( forceDelta && forceBaseline ){
    fossil_fatal("cannot use --delta and --baseline together");
  }
  dryRunFlag = find_option("dry-run","n",0)!=0;
................................................................................
  /*
  ** Check that the user exists.
  */
  if( !db_exists("SELECT 1 FROM user WHERE login=%Q", g.zLogin) ){
    fossil_fatal("no such user: %s", g.zLogin);
  }

  hasChanges = unsaved_changes(useSha1sum ? CKSIG_SHA1 : 0);
  db_begin_transaction();
  db_record_repository_filename(0);
  if( hasChanges==0 && !isAMerge && !allowEmpty && !forceFlag ){
    fossil_fatal("nothing has changed; use --allow-empty to override");
  }

  /* If none of the files that were named on the command line have







|







 







|







 







|
>







 







|







 







<
<







 







|
|
>
|







 







|
>







 







<
<







 







|
|







 







|







 







|







 







|
>







 







|







375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
...
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
...
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
...
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
...
854
855
856
857
858
859
860


861
862
863
864
865
866
867
...
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
...
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
....
1015
1016
1017
1018
1019
1020
1021


1022
1023
1024
1025
1026
1027
1028
....
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
....
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
....
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
....
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
....
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
** can be overridden by using one or more filter options (listed below),
** in which case only files with the specified change type(s) are shown.
** As a special case, the --no-merge option does not inhibit this default.
** This default shows exactly the set of changes that would be checked
** in by the commit command.
**
** If no filter options are used, or if the --merge option is used, the
** artifact hash of each merge contributor check-in version is displayed at
** the end of the report.  The --no-merge option is useful to display the
** default set of changed files without the merge contributors.
**
** If change type classification is enabled, each output line starts with
** a code describing the file's change type, e.g. EDITED or RENAMED.  It
** is enabled by default unless exactly one change type is selected.  For
** the purposes of determining the default, --changed counts as selecting
................................................................................
**
** The "fossil changes --extra" command is equivalent to "fossil extras".
**
** General options:
**    --abs-paths       Display absolute pathnames.
**    --rel-paths       Display pathnames relative to the current working
**                      directory.
**    --hash            Verify file status using hashing rather than
**                      relying on file mtimes.
**    --case-sensitive <BOOL>  Override case-sensitive setting.
**    --dotfiles        Include unmanaged files beginning with a dot.
**    --ignore <CSG>    Ignore unmanaged files matching CSG glob patterns.
**    --no-dir-symlinks Disables support for directory symlinks.
**
** Options specific to the changes command:
................................................................................
    {"classify", C_CLASSIFY},
  }, noFlagDefs[] = {
    {"no-merge", C_MERGE   }, {"no-classify", C_CLASSIFY },
  };

  Blob report = BLOB_INITIALIZER;
  enum {CHANGES, STATUS} command = *g.argv[1]=='s' ? STATUS : CHANGES;
  /* --sha1sum is an undocumented alias for --hash for backwards compatiblity */
  int useHash = find_option("hash",0,0)!=0 || find_option("sha1sum",0,0)!=0;
  int showHdr = command==CHANGES && find_option("header", 0, 0);
  int verboseFlag = command==CHANGES && find_option("verbose", "v", 0);
  const char *zIgnoreFlag = find_option("ignore", 0, 1);
  unsigned scanFlags = 0;
  unsigned flags = 0;
  int vid, i;

................................................................................
    scanFlags = SCAN_ALL;
  }

  /* We should be done with options. */
  verify_all_options();

  /* Check for changed files. */
  vfile_check_signature(vid, useHash ? CKSIG_HASH : 0);

  /* Search for unmanaged files if requested. */
  if( flags & C_EXTRA ){
    Glob *pIgnore = glob_create(zIgnoreFlag);
    locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
    glob_free(pIgnore);
  }
................................................................................
  /* We should be done with options.. */
  verify_all_options();

  if( zIgnoreFlag==0 ){
    zIgnoreFlag = db_get("ignore-glob", 0);
  }
  pIgnore = glob_create(zIgnoreFlag);


  locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
  glob_free(pIgnore);

  blob_zero(&report);
  status_report(&report, flags);
  if( blob_size(&report) ){
    if( showHdr ){
................................................................................
** ever be deleted.  Files and subdirectories whose names begin with "."
** are automatically ignored unless the --dotfiles option is used.
**
** The default values for --clean, --ignore, and --keep are determined by
** the (versionable) clean-glob, ignore-glob, and keep-glob settings.
**
** The --verily option ignores the keep-glob and ignore-glob settings and
** turns on the options --force, --emptydirs, --dotfiles, --disable-undo,
** and --no-dir-symlinks.  Use the --verily option when you really want
** to clean up everything.  Extreme care should be exercised when using
** the --verily option.
**
** Options:
**    --allckouts      Check for empty directories within any checkouts
**                     that may be nested within the current one.  This
**                     option should be used with great care because the
**                     empty-dirs setting (and other applicable settings)
**                     belonging to the other repositories, if any, will
................................................................................
  db_must_be_within_tree();
  if( find_option("verily","x",0)!=0 ){
    verilyFlag = allFileFlag = allDirFlag = 1;
    emptyDirsFlag = 1;
    disableUndo = 1;
    scanFlags |= SCAN_ALL;
    zCleanFlag = 0;
    g.fNoDirSymlinks = 1; /* Forbid symlink directory traversal. */
    g.allowSymlinks = 1;  /* Treat symlink files as content. */
  }
  if( zIgnoreFlag==0 && !verilyFlag ){
    zIgnoreFlag = db_get("ignore-glob", 0);
  }
  if( zKeepFlag==0 && !verilyFlag ){
    zKeepFlag = db_get("keep-glob", 0);
  }
................................................................................
  }
  if( db_get_boolean("dotfiles", 0) ) scanFlags |= SCAN_ALL;
  verify_all_options();
  pIgnore = glob_create(zIgnoreFlag);
  pKeep = glob_create(zKeepFlag);
  pClean = glob_create(zCleanFlag);
  nRoot = (int)strlen(g.zLocalRoot);


  if( !dirsOnlyFlag ){
    Stmt q;
    Blob repo;
    if( !dryRunFlag && !disableUndo ) undo_begin();
    locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
    db_prepare(&q,
        "SELECT %Q || pathname FROM sfile"
................................................................................
** blank commit message.  The default value is "N", do not commit.
**
** The --private option creates a private check-in that is never synced.
** Children of private check-ins are automatically private.
**
** The --tag option applies the symbolic tag name to the check-in.
**
** The --hash option detects edited files by computing each file's
** artifact hash rather than just checking for changes to its size or mtime.
**
** Options:
**    --allow-conflict           allow unresolved merge conflicts
**    --allow-empty              allow a commit with no changes
**    --allow-fork               allow the commit to fork
**    --allow-older              allow a commit older than its ancestor
**    --baseline                 use a baseline manifest in the commit process
................................................................................
**    -n|--dry-run               If given, display instead of run actions
**    --no-prompt                This option disables prompting the user for
**                               input and assumes an answer of 'No' for every
**                               question.
**    --no-warnings              omit all warnings about file contents
**    --nosign                   do not attempt to sign this commit with gpg
**    --private                  do not sync changes and their descendants
**    --hash                     verify file status using hashing rather
**                               than relying on file mtimes
**    --tag TAG-NAME             assign given tag TAG-NAME to the check-in
**    --date-override DATETIME   DATE to use instead of 'now'
**    --user-override USER       USER to use instead of the current default
**
** DATETIME may be "now" or "YYYY-MM-DDTHH:MM:SS.SSS". If in
** year-month-day form, it may be truncated, the "T" may be replaced by
................................................................................
  int vid;               /* blob-id of parent version */
  int nrid;              /* blob-id of a modified file */
  int nvid;              /* Blob-id of the new check-in */
  Blob comment;          /* Check-in comment */
  const char *zComment;  /* Check-in comment */
  Stmt q;                /* Various queries */
  char *zUuid;           /* UUID of the new check-in */
  int useHash = 0;       /* True to verify file status using hashing */
  int noSign = 0;        /* True to omit signing the manifest using GPG */
  int isAMerge = 0;      /* True if checking in a merge */
  int noWarningFlag = 0; /* True if skipping all warnings */
  int noPrompt = 0;      /* True if skipping all prompts */
  int forceFlag = 0;     /* Undocumented: Disables all checks */
  int forceDelta = 0;    /* Force a delta-manifest */
  int forceBaseline = 0; /* Force a baseline-manifest */
................................................................................
  int nConflict = 0;     /* Number of unresolved merge conflicts */
  int abortCommit = 0;
  Blob ans;
  char cReply;

  memset(&sCiInfo, 0, sizeof(sCiInfo));
  url_proxy_options();
  /* --sha1sum is an undocumented alias for --hash for backwards compatiblity */
  useHash = find_option("hash",0,0)!=0 || find_option("sha1sum",0,0)!=0;
  noSign = find_option("nosign",0,0)!=0;
  forceDelta = find_option("delta",0,0)!=0;
  forceBaseline = find_option("baseline",0,0)!=0;
  if( forceDelta && forceBaseline ){
    fossil_fatal("cannot use --delta and --baseline together");
  }
  dryRunFlag = find_option("dry-run","n",0)!=0;
................................................................................
  /*
  ** Check that the user exists.
  */
  if( !db_exists("SELECT 1 FROM user WHERE login=%Q", g.zLogin) ){
    fossil_fatal("no such user: %s", g.zLogin);
  }

  hasChanges = unsaved_changes(useHash ? CKSIG_HASH : 0);
  db_begin_transaction();
  db_record_repository_filename(0);
  if( hasChanges==0 && !isAMerge && !allowEmpty && !forceFlag ){
    fossil_fatal("nothing has changed; use --allow-empty to override");
  }

  /* If none of the files that were named on the command line have

Changes to src/checkout.c.

132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163

164

165
166
167
168
169
170
171
** If the manifest setting is set, but is not a boolean value, then treat
** each character as a flag to enable writing "manifest", "manifest.uuid" or
** "manifest.tags".
*/
void manifest_to_disk(int vid){
  char *zManFile;
  Blob manifest;
  Blob hash;
  Blob taglist;
  int flg;

  flg = db_get_manifest_setting();

  if( flg & (MFESTFLG_RAW|MFESTFLG_UUID) ){
    blob_zero(&manifest);
    content_get(vid, &manifest);
    blob_zero(&hash);
    sha1sum_blob(&manifest, &hash);
    sterilize_manifest(&manifest);
  }
  if( flg & MFESTFLG_RAW ){
    zManFile = mprintf("%smanifest", g.zLocalRoot);
    blob_write_to_file(&manifest, zManFile);
    free(zManFile);
  }else{
    if( !db_exists("SELECT 1 FROM vfile WHERE pathname='manifest'") ){
      zManFile = mprintf("%smanifest", g.zLocalRoot);
      file_delete(zManFile);
      free(zManFile);
    }
  }
  if( flg & MFESTFLG_UUID ){

    zManFile = mprintf("%smanifest.uuid", g.zLocalRoot);

    blob_append(&hash, "\n", 1);
    blob_write_to_file(&hash, zManFile);
    free(zManFile);
    blob_reset(&hash);
  }else{
    if( !db_exists("SELECT 1 FROM vfile WHERE pathname='manifest.uuid'") ){
      zManFile = mprintf("%smanifest.uuid", g.zLocalRoot);







<





|


<
<

<
<











>

>







132
133
134
135
136
137
138

139
140
141
142
143
144
145
146


147


148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
** If the manifest setting is set, but is not a boolean value, then treat
** each character as a flag to enable writing "manifest", "manifest.uuid" or
** "manifest.tags".
*/
void manifest_to_disk(int vid){
  char *zManFile;
  Blob manifest;

  Blob taglist;
  int flg;

  flg = db_get_manifest_setting();

  if( flg & MFESTFLG_RAW ){
    blob_zero(&manifest);
    content_get(vid, &manifest);


    sterilize_manifest(&manifest);


    zManFile = mprintf("%smanifest", g.zLocalRoot);
    blob_write_to_file(&manifest, zManFile);
    free(zManFile);
  }else{
    if( !db_exists("SELECT 1 FROM vfile WHERE pathname='manifest'") ){
      zManFile = mprintf("%smanifest", g.zLocalRoot);
      file_delete(zManFile);
      free(zManFile);
    }
  }
  if( flg & MFESTFLG_UUID ){
    Blob hash;
    zManFile = mprintf("%smanifest.uuid", g.zLocalRoot);
    blob_set_dynamic(&hash, rid_to_uuid(vid));
    blob_append(&hash, "\n", 1);
    blob_write_to_file(&hash, zManFile);
    free(zManFile);
    blob_reset(&hash);
  }else{
    if( !db_exists("SELECT 1 FROM vfile WHERE pathname='manifest.uuid'") ){
      zManFile = mprintf("%smanifest.uuid", g.zLocalRoot);

Changes to src/clone.c.

173
174
175
176
177
178
179

180
181
182
183
184
185
186
    db_begin_transaction();
    db_record_repository_filename(g.argv[3]);
    db_initial_setup(0, 0, zDefaultUser);
    user_select();
    db_set("content-schema", CONTENT_SCHEMA, 0);
    db_set("aux-schema", AUX_SCHEMA_MAX, 0);
    db_set("rebuilt", get_version(), 0);

    remember_or_get_http_auth(zHttpAuth, urlFlags & URL_REMEMBER, g.argv[2]);
    url_remember();
    if( g.zSSLIdentity!=0 ){
      /* If the --ssl-identity option was specified, store it as a setting */
      Blob fn;
      blob_zero(&fn);
      file_canonical_name(g.zSSLIdentity, &fn, 0);







>







173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
    db_begin_transaction();
    db_record_repository_filename(g.argv[3]);
    db_initial_setup(0, 0, zDefaultUser);
    user_select();
    db_set("content-schema", CONTENT_SCHEMA, 0);
    db_set("aux-schema", AUX_SCHEMA_MAX, 0);
    db_set("rebuilt", get_version(), 0);
    db_unset("hash-policy", 0);
    remember_or_get_http_auth(zHttpAuth, urlFlags & URL_REMEMBER, g.argv[2]);
    url_remember();
    if( g.zSSLIdentity!=0 ){
      /* If the --ssl-identity option was specified, store it as a setting */
      Blob fn;
      blob_zero(&fn);
      file_canonical_name(g.zSSLIdentity, &fn, 0);

Changes to src/config.h.

36
37
38
39
40
41
42





43
44
45
46
47
48
49
...
166
167
168
169
170
171
172

173
174
175
176
177
178
179
#if defined(_WIN32) && !defined(_WIN64) && !defined(_MSC_VER) && !defined(_USE_32BIT_TIME_T)
#  define _USE_32BIT_TIME_T
#endif

#ifdef HAVE_AUTOCONFIG_H
#include "autoconfig.h"
#endif






#ifndef _RC_COMPILE_

/*
** System header files used by all modules
*/
#include <unistd.h>
................................................................................
#  else
#    define COMPILER_NAME "unknown"
#  endif
#endif

#if !defined(_RC_COMPILE_) && !defined(SQLITE_AMALGAMATION)


#include "sqlite3.h"

/*
** On Solaris, getpass() will only return up to 8 characters. getpassphrase() returns up to 257.
*/
#if HAVE_GETPASSPHRASE
  #define getpass getpassphrase







>
>
>
>
>







 







>







36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
...
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
#if defined(_WIN32) && !defined(_WIN64) && !defined(_MSC_VER) && !defined(_USE_32BIT_TIME_T)
#  define _USE_32BIT_TIME_T
#endif

#ifdef HAVE_AUTOCONFIG_H
#include "autoconfig.h"
#endif

/* Enable the hardened SHA1 implemenation by default */
#ifndef FOSSIL_HARDENED_SHA1
# define FOSSIL_HARDENED_SHA1 1
#endif

#ifndef _RC_COMPILE_

/*
** System header files used by all modules
*/
#include <unistd.h>
................................................................................
#  else
#    define COMPILER_NAME "unknown"
#  endif
#endif

#if !defined(_RC_COMPILE_) && !defined(SQLITE_AMALGAMATION)

#include <stdint.h>
#include "sqlite3.h"

/*
** On Solaris, getpass() will only return up to 8 characters. getpassphrase() returns up to 257.
*/
#if HAVE_GETPASSPHRASE
  #define getpass getpassphrase

Changes to src/configure.c.

127
128
129
130
131
132
133

134
135
136
137
138
139
140
  { "crnl-glob",              CONFIGSET_PROJ },
  { "encoding-glob",          CONFIGSET_PROJ },
  { "empty-dirs",             CONFIGSET_PROJ },
  { "allow-symlinks",         CONFIGSET_PROJ },
  { "dotfiles",               CONFIGSET_PROJ },
  { "parent-project-code",    CONFIGSET_PROJ },
  { "parent-project-name",    CONFIGSET_PROJ },


#ifdef FOSSIL_ENABLE_LEGACY_MV_RM
  { "mv-rm-files",            CONFIGSET_PROJ },
#endif

  { "ticket-table",           CONFIGSET_TKT  },
  { "ticket-common",          CONFIGSET_TKT  },







>







127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
  { "crnl-glob",              CONFIGSET_PROJ },
  { "encoding-glob",          CONFIGSET_PROJ },
  { "empty-dirs",             CONFIGSET_PROJ },
  { "allow-symlinks",         CONFIGSET_PROJ },
  { "dotfiles",               CONFIGSET_PROJ },
  { "parent-project-code",    CONFIGSET_PROJ },
  { "parent-project-name",    CONFIGSET_PROJ },
  { "hash-policy",            CONFIGSET_PROJ },

#ifdef FOSSIL_ENABLE_LEGACY_MV_RM
  { "mv-rm-files",            CONFIGSET_PROJ },
#endif

  { "ticket-table",           CONFIGSET_TKT  },
  { "ticket-common",          CONFIGSET_TKT  },

Changes to src/content.c.

310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
...
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
...
511
512
513
514
515
516
517

518
519
520










521
522




523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
...
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
...
903
904
905
906
907
908
909

910
911
912
913
914
915
916
...
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
...
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
....
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
}

/*
** COMMAND: artifact*
**
** Usage: %fossil artifact ARTIFACT-ID ?OUTPUT-FILENAME? ?OPTIONS?
**
** Extract an artifact by its SHA1 hash and write the results on
** standard output, or if the optional 4th argument is given, in
** the named output file.
**
** Options:
**    -R|--repository FILE       Extract artifacts from repository FILE
**
** See also: finfo
................................................................................
**
** The original content of pBlob is not disturbed.  The caller continues
** to be responsible for pBlob.  This routine does *not* take over
** responsibility for freeing pBlob.
*/
int content_put_ex(
  Blob *pBlob,              /* Content to add to the repository */
  const char *zUuid,        /* SHA1 hash of reconstructed pBlob */
  int srcId,                /* pBlob is a delta from this entry */
  int nBlob,                /* pBlob is compressed. Original size is this */
  int isPrivate             /* The content should be marked private */
){
  int size;
  int rid;
  Stmt s1;
................................................................................
  Blob hash;
  int markAsUnclustered = 0;
  int isDephantomize = 0;

  assert( g.repositoryOpen );
  assert( pBlob!=0 );
  assert( srcId==0 || zUuid!=0 );

  if( zUuid==0 ){
    assert( nBlob==0 );
    sha1sum_blob(pBlob, &hash);










  }else{
    blob_init(&hash, zUuid, -1);




  }
  if( nBlob ){
    size = nBlob;
  }else{
    size = blob_size(pBlob);
    if( srcId ){
      size = delta_output_size(blob_buffer(pBlob), size);
    }
  }
  db_begin_transaction();

  /* Check to see if the entry already exists and if it does whether
  ** or not the entry is a phantom
  */
  db_prepare(&s1, "SELECT rid, size FROM blob WHERE uuid=%B", &hash);
  if( db_step(&s1)==SQLITE_ROW ){
    rid = db_column_int(&s1, 0);
................................................................................
**
**    --parse            Parse all manifests, wikis, tickets, events, and
**                       so forth, reporting any errors found.
*/
void test_integrity(void){
  Stmt q;
  Blob content;
  Blob cksum;
  int n1 = 0;
  int n2 = 0;
  int nErr = 0;
  int total;
  int nCA = 0;
  int anCA[10];
  int bParse = find_option("parse",0,0)!=0;
................................................................................
  db_finalize(&q);

  db_prepare(&q, "SELECT rid, uuid, size FROM blob ORDER BY rid");
  total = db_int(0, "SELECT max(rid) FROM blob");
  while( db_step(&q)==SQLITE_ROW ){
    int rid = db_column_int(&q, 0);
    const char *zUuid = db_column_text(&q, 1);

    int size = db_column_int(&q, 2);
    n1++;
    fossil_print("  %d/%d\r", n1, total);
    fflush(stdout);
    if( size<0 ){
      fossil_print("skip phantom %d %s\n", rid, zUuid);
      continue;  /* Ignore phantoms */
................................................................................
    }
    content_get(rid, &content);
    if( blob_size(&content)!=size ){
      fossil_print("size mismatch on artifact %d: wanted %d but got %d\n",
                     rid, size, blob_size(&content));
      nErr++;
    }
    sha1sum_blob(&content, &cksum);
    if( fossil_strcmp(blob_str(&cksum), zUuid)!=0 ){
      fossil_print("wrong hash on artifact %d: wanted %s but got %s\n",
                   rid, zUuid, blob_str(&cksum));
      nErr++;
    }
    if( bParse && looks_like_control_artifact(&content) ){
      Blob err;
      int i, n;
      char *z;
      Manifest *p;
................................................................................
      n = blob_size(&content);
      for(i=0; i<n && z[i] && z[i]!='\n' && i<sizeof(zFirstLine)-1; i++){}
      memcpy(zFirstLine, z, i);
      zFirstLine[i] = 0;
      p = manifest_parse(&content, 0, &err);
      if( p==0 ){
        fossil_print("manifest_parse failed for %s:\n%s\n",
               blob_str(&cksum), blob_str(&err));
        if( strncmp(blob_str(&err), "line 1:", 7)==0 ){
          fossil_print("\"%s\"\n", zFirstLine);
        }
      }else{
        anCA[p->type]++;
        manifest_destroy(p);
        nCA++;
      }
      blob_reset(&err);
    }else{
      blob_reset(&content);
    }
    blob_reset(&cksum);
    n2++;
  }
  db_finalize(&q);
  fossil_print("%d non-phantom blobs (out of %d total) checked:  %d errors\n",
               n2, n1, nErr);
  if( bParse ){
    static const char *const azType[] = { 0, "manifest", "cluster",
................................................................................
** WARNING: This command destroys data and can cause you to lose work.
** Make sure you have a backup copy before using this command!
**
** WARNING: You must run "fossil rebuild" after this command to rebuild
** the metadata.
**
** Note that the arguments are the integer raw RID values from the BLOB table,
** not SHA1 hashs or labels.
*/
void test_content_erase(void){
  int i;
  Blob x;
  char c;
  Stmt q;
  prompt_user("This command erases information from the repository and\n"







|







 







|







 







>


<
>
>
>
>
>
>
>
>
>
>


>
>
>
>









<







 







<







 







>







 







|
<
|
<







 







|












<







 







|







310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
...
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
...
511
512
513
514
515
516
517
518
519
520

521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545

546
547
548
549
550
551
552
...
879
880
881
882
883
884
885

886
887
888
889
890
891
892
...
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
...
930
931
932
933
934
935
936
937

938

939
940
941
942
943
944
945
...
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969

970
971
972
973
974
975
976
....
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
}

/*
** COMMAND: artifact*
**
** Usage: %fossil artifact ARTIFACT-ID ?OUTPUT-FILENAME? ?OPTIONS?
**
** Extract an artifact by its artifact hash and write the results on
** standard output, or if the optional 4th argument is given, in
** the named output file.
**
** Options:
**    -R|--repository FILE       Extract artifacts from repository FILE
**
** See also: finfo
................................................................................
**
** The original content of pBlob is not disturbed.  The caller continues
** to be responsible for pBlob.  This routine does *not* take over
** responsibility for freeing pBlob.
*/
int content_put_ex(
  Blob *pBlob,              /* Content to add to the repository */
  const char *zUuid,        /* artifact hash of reconstructed pBlob */
  int srcId,                /* pBlob is a delta from this entry */
  int nBlob,                /* pBlob is compressed. Original size is this */
  int isPrivate             /* The content should be marked private */
){
  int size;
  int rid;
  Stmt s1;
................................................................................
  Blob hash;
  int markAsUnclustered = 0;
  int isDephantomize = 0;

  assert( g.repositoryOpen );
  assert( pBlob!=0 );
  assert( srcId==0 || zUuid!=0 );
  db_begin_transaction();
  if( zUuid==0 ){
    assert( nBlob==0 );

    /* First check the auxiliary hash to see if there is already an artifact
    ** that uses the auxiliary hash name */
    hname_hash(pBlob, 1, &hash);
    rid = fast_uuid_to_rid(blob_str(&hash));
    if( rid==0 ){
      /* No existing artifact with the auxiliary hash name.  Therefore, use
      ** the primary hash name. */
      blob_reset(&hash);
      hname_hash(pBlob, 0, &hash);
    }
  }else{
    blob_init(&hash, zUuid, -1);
  }
  if( g.eHashPolicy==HPOLICY_AUTO && blob_size(&hash)>HNAME_LEN_SHA1 ){
    g.eHashPolicy = HPOLICY_SHA3;
    db_set_int("hash-policy", HPOLICY_SHA3, 0);
  }
  if( nBlob ){
    size = nBlob;
  }else{
    size = blob_size(pBlob);
    if( srcId ){
      size = delta_output_size(blob_buffer(pBlob), size);
    }
  }


  /* Check to see if the entry already exists and if it does whether
  ** or not the entry is a phantom
  */
  db_prepare(&s1, "SELECT rid, size FROM blob WHERE uuid=%B", &hash);
  if( db_step(&s1)==SQLITE_ROW ){
    rid = db_column_int(&s1, 0);
................................................................................
**
**    --parse            Parse all manifests, wikis, tickets, events, and
**                       so forth, reporting any errors found.
*/
void test_integrity(void){
  Stmt q;
  Blob content;

  int n1 = 0;
  int n2 = 0;
  int nErr = 0;
  int total;
  int nCA = 0;
  int anCA[10];
  int bParse = find_option("parse",0,0)!=0;
................................................................................
  db_finalize(&q);

  db_prepare(&q, "SELECT rid, uuid, size FROM blob ORDER BY rid");
  total = db_int(0, "SELECT max(rid) FROM blob");
  while( db_step(&q)==SQLITE_ROW ){
    int rid = db_column_int(&q, 0);
    const char *zUuid = db_column_text(&q, 1);
    int nUuid = db_column_bytes(&q, 1);
    int size = db_column_int(&q, 2);
    n1++;
    fossil_print("  %d/%d\r", n1, total);
    fflush(stdout);
    if( size<0 ){
      fossil_print("skip phantom %d %s\n", rid, zUuid);
      continue;  /* Ignore phantoms */
................................................................................
    }
    content_get(rid, &content);
    if( blob_size(&content)!=size ){
      fossil_print("size mismatch on artifact %d: wanted %d but got %d\n",
                     rid, size, blob_size(&content));
      nErr++;
    }
    if( !hname_verify_hash(&content, zUuid, nUuid) ){

      fossil_print("wrong hash on artifact %d\n",rid);

      nErr++;
    }
    if( bParse && looks_like_control_artifact(&content) ){
      Blob err;
      int i, n;
      char *z;
      Manifest *p;
................................................................................
      n = blob_size(&content);
      for(i=0; i<n && z[i] && z[i]!='\n' && i<sizeof(zFirstLine)-1; i++){}
      memcpy(zFirstLine, z, i);
      zFirstLine[i] = 0;
      p = manifest_parse(&content, 0, &err);
      if( p==0 ){
        fossil_print("manifest_parse failed for %s:\n%s\n",
               zUuid, blob_str(&err));
        if( strncmp(blob_str(&err), "line 1:", 7)==0 ){
          fossil_print("\"%s\"\n", zFirstLine);
        }
      }else{
        anCA[p->type]++;
        manifest_destroy(p);
        nCA++;
      }
      blob_reset(&err);
    }else{
      blob_reset(&content);
    }

    n2++;
  }
  db_finalize(&q);
  fossil_print("%d non-phantom blobs (out of %d total) checked:  %d errors\n",
               n2, n1, nErr);
  if( bParse ){
    static const char *const azType[] = { 0, "manifest", "cluster",
................................................................................
** WARNING: This command destroys data and can cause you to lose work.
** Make sure you have a backup copy before using this command!
**
** WARNING: You must run "fossil rebuild" after this command to rebuild
** the metadata.
**
** Note that the arguments are the integer raw RID values from the BLOB table,
** not artifact hashs or labels.
*/
void test_content_erase(void){
  int i;
  Blob x;
  char c;
  Stmt q;
  prompt_user("This command erases information from the repository and\n"

Changes to src/db.c.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
....
1434
1435
1436
1437
1438
1439
1440
1441




1442

1443
1444
1445
1446
1447
1448
1449
....
1478
1479
1480
1481
1482
1483
1484
1485

1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500


1501
1502
1503
1504
1505
1506
1507
1508
1509

1510
1511
1512
1513
1514
1515
1516
....
1670
1671
1672
1673
1674
1675
1676

1677
1678
1679
1680
1681
1682
1683
....
1841
1842
1843
1844
1845
1846
1847

1848
1849
1850
1851
1852
1853
1854
....
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922

1923
1924
1925
1926
1927
1928
1929
....
1930
1931
1932
1933
1934
1935
1936


1937
1938
1939
1940

1941
1942
1943
1944
1945
1946
1947
....
1950
1951
1952
1953
1954
1955
1956




1957
1958
1959
1960
1961
1962
1963
....
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
....
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)

** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
................................................................................
  return 1;
#endif
}

/*
** Returns non-zero if support for symlinks is currently enabled.
*/
int db_allow_symlinks(void){




  return g.allowSymlinks;

}

/*
** Open the repository database given by zDbName.  If zDbName==NULL then
** get the name from the already open local database.
*/
void db_open_repository(const char *zDbName){
................................................................................
  g.zRepositoryName = mprintf("%s", zDbName);
  db_open_or_attach(g.zRepositoryName, "repository");
  g.repositoryOpen = 1;
  /* Cache "allow-symlinks" option, because we'll need it on every stat call */
  g.allowSymlinks = db_get_boolean("allow-symlinks",
                                   db_allow_symlinks_by_default());
  g.zAuxSchema = db_get("aux-schema","");


  /* Verify that the PLINK table has a new column added by the
  ** 2014-11-28 schema change.  Create it if necessary.  This code
  ** can be removed in the future, once all users have upgraded to the
  ** 2014-11-28 or later schema.
  */
  if( !db_table_has_column("repository","plink","baseid") ){
    db_multi_exec(
      "ALTER TABLE repository.plink ADD COLUMN baseid;"
    );
  }

  /* Verify that the MLINK table has the newer columns added by the
  ** 2015-01-24 schema change.  Create them if necessary.  This code
  ** can be removed in the future, once all users have upgraded to the
  ** 2015-01-24 or later schema.


  */
  if( !db_table_has_column("repository","mlink","isaux") ){
    db_begin_transaction();
    db_multi_exec(
      "ALTER TABLE repository.mlink ADD COLUMN pmid INTEGER DEFAULT 0;"
      "ALTER TABLE repository.mlink ADD COLUMN isaux BOOLEAN DEFAULT 0;"
    );
    db_end_transaction(0);
  }

}

/*
** Flags for the db_find_and_open_repository() function.
*/
#if INTERFACE
#define OPEN_OK_NOT_FOUND    0x001      /* Do not error out if not found */
................................................................................
    fprintf(stderr, "-- prepared statements    %10d\n", db.nPrepare);
  }
  while( db.pAllStmt ){
    db_finalize(db.pAllStmt);
  }
  db_end_transaction(1);
  pStmt = 0;

  db_close_config();

  /* If the localdb has a lot of unused free space,
  ** then VACUUM it as we shut down.
  */
  if( db_database_slot("localdb")>=0 ){
    int nFree = db_int(0, "PRAGMA localdb.freelist_count");
................................................................................
      " SELECT name,value,mtime FROM settingSrc.config"
      "  WHERE (name IN %s OR name IN %s)"
      "    AND name NOT GLOB 'project-*'"
      "    AND name NOT GLOB 'short-project-*';",
      configure_inop_rhs(CONFIGSET_ALL),
      db_setting_inop_rhs()
    );

    db_multi_exec(
      "REPLACE INTO reportfmt SELECT * FROM settingSrc.reportfmt;"
    );

    /*
    ** Copy the user permissions, contact information, last modified
    ** time, and photo for all the "system" users from the supplied
................................................................................
** repository is used, almost all of the settings accessible from the setup
** page, either directly or indirectly, will be copied.  Normal users and
** their associated permissions will not be copied; however, the system
** default users "anonymous", "nobody", "reader", "developer", and their
** associated permissions will be copied.
**
** Options:
**    --template      FILE      copy settings from repository file
**    --admin-user|-A USERNAME  select given USERNAME as admin user
**    --date-override DATETIME  use DATETIME as time of the initial check-in

**
** DATETIME may be "now" or "YYYY-MM-DDTHH:MM:SS.SSS". If in
** year-month-day form, it may be truncated, the "T" may be replaced by
** a space, and it may also name a timezone offset from UTC as "-HH:MM"
** (westward) or "+HH:MM" (eastward). Either no timezone suffix or "Z"
** means UTC.
**
................................................................................
** See also: clone
*/
void create_repository_cmd(void){
  char *zPassword;
  const char *zTemplate;      /* Repository from which to copy settings */
  const char *zDate;          /* Date of the initial check-in */
  const char *zDefaultUser;   /* Optional name of the default user */



  zTemplate = find_option("template",0,1);
  zDate = find_option("date-override",0,1);
  zDefaultUser = find_option("admin-user","A",1);

  /* We should be done with options.. */
  verify_all_options();

  if( g.argc!=3 ){
    usage("REPOSITORY-NAME");
  }

................................................................................
  }

  db_create_repository(g.argv[2]);
  db_open_repository(g.argv[2]);
  db_open_config(0, 0);
  if( zTemplate ) db_attach(zTemplate, "settingSrc");
  db_begin_transaction();




  if( zDate==0 ) zDate = "now";
  db_initial_setup(zTemplate, zDate, zDefaultUser);
  db_end_transaction(0);
  if( zTemplate ) db_detach("settingSrc");
  fossil_print("project-id: %s\n", db_get("project-code", 0));
  fossil_print("server-id:  %s\n", db_get("server-code", 0));
  zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
................................................................................
    assert( rc==0 || rc==1 );
    if( sqlite3_value_type(argv[2-rc])==SQLITE_NULL ) rc = 1-rc;
    sqlite3_result_value(context, argv[2-rc]);
  }
}

/*
** Convert the input string into an SHA1.  Make a notation in the
** CONCEALED table so that the hash can be undo using the db_reveal()
** function at some later time.
**
** The value returned is stored in static space and will be overwritten
** on subsequent calls.
**
** If zContent is already a well-formed SHA1 hash, then return a copy
** of that hash, not a hash of the hash.
**
** The CONCEALED table is meant to obscure email addresses.  Every valid
** email address will contain a "@" character and "@" is not valid within
** an SHA1 hash so there is no chance that a valid email address will go
** unconcealed.
*/
char *db_conceal(const char *zContent, int n){
  static char zHash[42];
  Blob out;
  if( n==40 && validate16(zContent, n) ){
    memcpy(zHash, zContent, n);
    zHash[n] = 0;
  }else{
    sha1sum_step_text(zContent, n);
    sha1sum_finish(&out);
    sqlite3_snprintf(sizeof(zHash), zHash, "%s", blob_str(&out));
    blob_reset(&out);
................................................................................
  if( zVersionedSetting!=0 && zNonVersionedSetting!=0
   && zNonVersionedSetting[0]!='\0' && !noWarn
  ){
    /* There's a versioned setting, and a non-versioned setting. Tell
    ** the user about the conflict */
    fossil_warning(
        "setting %s has both versioned and non-versioned values: using "
        "versioned value from file .fossil-settings/%s (to silence this "
        "warning, either create an empty file named "
        ".fossil-settings/%s.no-warn in the check-out root, "
        "or delete the non-versioned setting "
        "with \"fossil unset %s\")", zName, zName, zName, zName
    );
  }
  /* Prefer the versioned setting */
  return ( zVersionedSetting!=0 ) ? zVersionedSetting : zNonVersionedSetting;
}








|







 







|
>
>
>
>
|
>







 







<
>
|
|
|
<
<
<
<
<
<


<
<
<
<
>
>

<
<
<
<
<
<
<
<
>







 







>







 







>







 







|
|
|
>







 







>
>




>







 







>
>
>
>







 







|






|




|



|

|







 







|
|
|
|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
....
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
....
1483
1484
1485
1486
1487
1488
1489

1490
1491
1492
1493






1494
1495




1496
1497
1498








1499
1500
1501
1502
1503
1504
1505
1506
....
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
....
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
....
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
....
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
....
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
....
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
....
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
................................................................................
  return 1;
#endif
}

/*
** Returns non-zero if support for symlinks is currently enabled.
*/
int db_allow_symlinks(int traversal){
  if( traversal ){
    if( g.allowSymlinks ) return 1;
    return g.fNoDirSymlinks;
  }else{
    return g.allowSymlinks;
  }
}

/*
** Open the repository database given by zDbName.  If zDbName==NULL then
** get the name from the already open local database.
*/
void db_open_repository(const char *zDbName){
................................................................................
  g.zRepositoryName = mprintf("%s", zDbName);
  db_open_or_attach(g.zRepositoryName, "repository");
  g.repositoryOpen = 1;
  /* Cache "allow-symlinks" option, because we'll need it on every stat call */
  g.allowSymlinks = db_get_boolean("allow-symlinks",
                                   db_allow_symlinks_by_default());
  g.zAuxSchema = db_get("aux-schema","");

  g.eHashPolicy = db_get_int("hash-policy",-1);
  if( g.eHashPolicy<0 ){
    g.eHashPolicy = hname_default_policy();
    db_set_int("hash-policy", g.eHashPolicy, 0);






  }





  /* Make a change to the CHECK constraint on the BLOB table for
  ** version 2.0 and later.
  */








  rebuild_schema_update_2_0();   /* Do the Fossil-2.0 schema updates */
}

/*
** Flags for the db_find_and_open_repository() function.
*/
#if INTERFACE
#define OPEN_OK_NOT_FOUND    0x001      /* Do not error out if not found */
................................................................................
    fprintf(stderr, "-- prepared statements    %10d\n", db.nPrepare);
  }
  while( db.pAllStmt ){
    db_finalize(db.pAllStmt);
  }
  db_end_transaction(1);
  pStmt = 0;
  sqlite3_exec(g.db, "PRAGMA optimize", 0, 0, 0);
  db_close_config();

  /* If the localdb has a lot of unused free space,
  ** then VACUUM it as we shut down.
  */
  if( db_database_slot("localdb")>=0 ){
    int nFree = db_int(0, "PRAGMA localdb.freelist_count");
................................................................................
      " SELECT name,value,mtime FROM settingSrc.config"
      "  WHERE (name IN %s OR name IN %s)"
      "    AND name NOT GLOB 'project-*'"
      "    AND name NOT GLOB 'short-project-*';",
      configure_inop_rhs(CONFIGSET_ALL),
      db_setting_inop_rhs()
    );
    g.eHashPolicy = db_get_int("hash-policy", g.eHashPolicy);
    db_multi_exec(
      "REPLACE INTO reportfmt SELECT * FROM settingSrc.reportfmt;"
    );

    /*
    ** Copy the user permissions, contact information, last modified
    ** time, and photo for all the "system" users from the supplied
................................................................................
** repository is used, almost all of the settings accessible from the setup
** page, either directly or indirectly, will be copied.  Normal users and
** their associated permissions will not be copied; however, the system
** default users "anonymous", "nobody", "reader", "developer", and their
** associated permissions will be copied.
**
** Options:
**    --template      FILE         Copy settings from repository file
**    --admin-user|-A USERNAME     Select given USERNAME as admin user
**    --date-override DATETIME     Use DATETIME as time of the initial check-in
**    --sha1                       Use a initial hash policy of "sha1"
**
** DATETIME may be "now" or "YYYY-MM-DDTHH:MM:SS.SSS". If in
** year-month-day form, it may be truncated, the "T" may be replaced by
** a space, and it may also name a timezone offset from UTC as "-HH:MM"
** (westward) or "+HH:MM" (eastward). Either no timezone suffix or "Z"
** means UTC.
**
................................................................................
** See also: clone
*/
void create_repository_cmd(void){
  char *zPassword;
  const char *zTemplate;      /* Repository from which to copy settings */
  const char *zDate;          /* Date of the initial check-in */
  const char *zDefaultUser;   /* Optional name of the default user */
  int bUseSha1 = 0;           /* True to set the hash-policy to sha1 */


  zTemplate = find_option("template",0,1);
  zDate = find_option("date-override",0,1);
  zDefaultUser = find_option("admin-user","A",1);
  bUseSha1 = find_option("sha1",0,0)!=0;
  /* We should be done with options.. */
  verify_all_options();

  if( g.argc!=3 ){
    usage("REPOSITORY-NAME");
  }

................................................................................
  }

  db_create_repository(g.argv[2]);
  db_open_repository(g.argv[2]);
  db_open_config(0, 0);
  if( zTemplate ) db_attach(zTemplate, "settingSrc");
  db_begin_transaction();
  if( bUseSha1 ){
    g.eHashPolicy = HPOLICY_SHA1;
    db_set_int("hash-policy", HPOLICY_SHA1, 0);
  }
  if( zDate==0 ) zDate = "now";
  db_initial_setup(zTemplate, zDate, zDefaultUser);
  db_end_transaction(0);
  if( zTemplate ) db_detach("settingSrc");
  fossil_print("project-id: %s\n", db_get("project-code", 0));
  fossil_print("server-id:  %s\n", db_get("server-code", 0));
  zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
................................................................................
    assert( rc==0 || rc==1 );
    if( sqlite3_value_type(argv[2-rc])==SQLITE_NULL ) rc = 1-rc;
    sqlite3_result_value(context, argv[2-rc]);
  }
}

/*
** Convert the input string into a artifact hash.  Make a notation in the
** CONCEALED table so that the hash can be undo using the db_reveal()
** function at some later time.
**
** The value returned is stored in static space and will be overwritten
** on subsequent calls.
**
** If zContent is already a well-formed artifact hash, then return a copy
** of that hash, not a hash of the hash.
**
** The CONCEALED table is meant to obscure email addresses.  Every valid
** email address will contain a "@" character and "@" is not valid within
** a SHA1 hash so there is no chance that a valid email address will go
** unconcealed.
*/
char *db_conceal(const char *zContent, int n){
  static char zHash[HNAME_MAX+1];
  Blob out;
  if( hname_validate(zContent, n) ){
    memcpy(zHash, zContent, n);
    zHash[n] = 0;
  }else{
    sha1sum_step_text(zContent, n);
    sha1sum_finish(&out);
    sqlite3_snprintf(sizeof(zHash), zHash, "%s", blob_str(&out));
    blob_reset(&out);
................................................................................
  if( zVersionedSetting!=0 && zNonVersionedSetting!=0
   && zNonVersionedSetting[0]!='\0' && !noWarn
  ){
    /* There's a versioned setting, and a non-versioned setting. Tell
    ** the user about the conflict */
    fossil_warning(
        "setting %s has both versioned and non-versioned values: using "
        "versioned value from file \"%/.fossil-settings/%s\" (to silence "
        "this warning, either create an empty file named "
        "\"%/.fossil-settings/%s.no-warn\" in the check-out root, or delete "
        "the non-versioned setting with \"fossil unset %s\")", zName,
        g.zLocalRoot, zName, g.zLocalRoot, zName, zName
    );
  }
  /* Prefer the versioned setting */
  return ( zVersionedSetting!=0 ) ? zVersionedSetting : zNonVersionedSetting;
}


Changes to src/diff.c.

2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
** WEBPAGE: praise
**
** URL: /annotate?checkin=ID&filename=FILENAME
** URL: /blame?checkin=ID&filename=FILENAME
** URL: /praise?checkin=ID&filename=FILENAME
**
** Show the most recent change to each line of a text file.  /annotate shows
** the date of the changes and the check-in SHA1 hash (with a link to the
** check-in).  /blame and /praise also show the user who made the check-in.
**
** Query parameters:
**
**    checkin=ID          The manifest ID at which to start the annotation
**    filename=FILENAME   The filename.
**    filevers            Show file versions rather than check-in versions







|







2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
** WEBPAGE: praise
**
** URL: /annotate?checkin=ID&filename=FILENAME
** URL: /blame?checkin=ID&filename=FILENAME
** URL: /praise?checkin=ID&filename=FILENAME
**
** Show the most recent change to each line of a text file.  /annotate shows
** the date of the changes and the check-in hash (with a link to the
** check-in).  /blame and /praise also show the user who made the check-in.
**
** Query parameters:
**
**    checkin=ID          The manifest ID at which to start the annotation
**    filename=FILENAME   The filename.
**    filevers            Show file versions rather than check-in versions

Changes to src/diffcmd.c.

149
150
151
152
153
154
155



156
157
158
159
160
161
162
...
165
166
167
168
169
170
171
172

173
174
175
176
177
178
179
...
192
193
194
195
196
197
198



199

200
201
202
203
204
205
206
...
250
251
252
253
254
255
256



257


258
259

260
261
262
263
264
265
266
...
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
...
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
...
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
}

/*
** Show the difference between two files, one in memory and one on disk.
**
** The difference is the set of edits needed to transform pFile1 into
** zFile2.  The content of pFile1 is in memory.  zFile2 exists on disk.



**
** Use the internal diff logic if zDiffCmd is NULL.  Otherwise call the
** command zDiffCmd to do the diffing.
**
** When using an external diff program, zBinGlob contains the GLOB patterns
** for file names to treat as binary.  If fIncludeBinary is zero, these files
** will be skipped in addition to files that may contain binary content.
................................................................................
  Blob *pFile1,             /* In memory content to compare from */
  int isBin1,               /* Does the 'from' content appear to be binary */
  const char *zFile2,       /* On disk content to compare to */
  const char *zName,        /* Display name of the file */
  const char *zDiffCmd,     /* Command for comparison */
  const char *zBinGlob,     /* Treat file names matching this as binary */
  int fIncludeBinary,       /* Include binary files for external diff */
  u64 diffFlags             /* Flags to control the diff */

){
  if( zDiffCmd==0 ){
    Blob out;                 /* Diff output text */
    Blob file2;               /* Content of zFile2 */
    const char *zName2;       /* Name of zFile2 for display */

    /* Read content of zFile2 into memory */
................................................................................
    /* Compute and output the differences */
    if( diffFlags & DIFF_BRIEF ){
      if( blob_compare(pFile1, &file2) ){
        fossil_print("CHANGED  %s\n", zName);
      }
    }else{
      blob_zero(&out);



      text_diff(pFile1, &file2, &out, 0, diffFlags);

      if( blob_size(&out) ){
        diff_print_filenames(zName, zName2, diffFlags);
        fossil_print("%s\n", blob_str(&out));
      }
      blob_reset(&out);
    }

................................................................................
      blob_appendf(&nameFile1, "%s~%d", zFile2, cnt++);
    }while( file_access(blob_str(&nameFile1),F_OK)==0 );
    blob_write_to_file(pFile1, blob_str(&nameFile1));

    /* Construct the external diff command */
    blob_zero(&cmd);
    blob_appendf(&cmd, "%s ", zDiffCmd);



    shell_escape(&cmd, blob_str(&nameFile1));


    blob_append(&cmd, " ", 1);
    shell_escape(&cmd, zFile2);


    /* Run the external diff command */
    fossil_system(blob_str(&cmd));

    /* Delete the temporary file and clean up memory used */
    file_delete(blob_str(&nameFile1));
    blob_reset(&nameFile1);
................................................................................
        content_get(srcid, &content);
      }else{
        blob_zero(&content);
      }
      isBin = fIncludeBinary ? 0 : looks_like_binary(&content);
      diff_print_index(zPathname, diffFlags);
      diff_file(&content, isBin, zFullName, zPathname, zDiffCmd,
                zBinGlob, fIncludeBinary, diffFlags);
      blob_reset(&content);
    }
    blob_reset(&fname);
  }
  db_finalize(&q);
  db_end_transaction(1);  /* ROLLBACK */
}
................................................................................
  while( db_step(&q)==SQLITE_ROW ){
    char *zFullName;
    const char *zFile = (const char*)db_column_text(&q, 0);
    if( !file_dir_match(pFileDir, zFile) ) continue;
    zFullName = mprintf("%s%s", g.zLocalRoot, zFile);
    db_column_blob(&q, 1, &content);
    diff_file(&content, 0, zFullName, zFile,
              zDiffCmd, zBinGlob, fIncludeBinary, diffFlags);
    fossil_free(zFullName);
    blob_reset(&content);
  }
  db_finalize(&q);
}

/*
................................................................................

/*
** Returns non-zero if files that may be binary should be used with external
** diff programs.
*/
int diff_include_binary_files(void){
  const char* zArgIncludeBinary = find_option("diff-binary", 0, 1);
  
  /* Command line argument have priority on settings */
  if( zArgIncludeBinary ){
    return is_truth(zArgIncludeBinary);
  }else{
    return db_get_boolean("diff-binary", 1);
  }
}







>
>
>







 







|
>







 







>
>
>
|
>







 







>
>
>
|
>
>
|
|
>







 







|







 







|







 







|







149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
...
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
...
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
...
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
...
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
...
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
...
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
}

/*
** Show the difference between two files, one in memory and one on disk.
**
** The difference is the set of edits needed to transform pFile1 into
** zFile2.  The content of pFile1 is in memory.  zFile2 exists on disk.
**
** If fSwapDiff is 1, show the set of edits to transform zFile2 into pFile1
** instead of the opposite.
**
** Use the internal diff logic if zDiffCmd is NULL.  Otherwise call the
** command zDiffCmd to do the diffing.
**
** When using an external diff program, zBinGlob contains the GLOB patterns
** for file names to treat as binary.  If fIncludeBinary is zero, these files
** will be skipped in addition to files that may contain binary content.
................................................................................
  Blob *pFile1,             /* In memory content to compare from */
  int isBin1,               /* Does the 'from' content appear to be binary */
  const char *zFile2,       /* On disk content to compare to */
  const char *zName,        /* Display name of the file */
  const char *zDiffCmd,     /* Command for comparison */
  const char *zBinGlob,     /* Treat file names matching this as binary */
  int fIncludeBinary,       /* Include binary files for external diff */
  u64 diffFlags,            /* Flags to control the diff */
  int fSwapDiff             /* Diff from Zfile2 to Pfile1 */
){
  if( zDiffCmd==0 ){
    Blob out;                 /* Diff output text */
    Blob file2;               /* Content of zFile2 */
    const char *zName2;       /* Name of zFile2 for display */

    /* Read content of zFile2 into memory */
................................................................................
    /* Compute and output the differences */
    if( diffFlags & DIFF_BRIEF ){
      if( blob_compare(pFile1, &file2) ){
        fossil_print("CHANGED  %s\n", zName);
      }
    }else{
      blob_zero(&out);
      if( fSwapDiff ){
        text_diff(&file2, pFile1, &out, 0, diffFlags);
      }else{
        text_diff(pFile1, &file2, &out, 0, diffFlags);
      }
      if( blob_size(&out) ){
        diff_print_filenames(zName, zName2, diffFlags);
        fossil_print("%s\n", blob_str(&out));
      }
      blob_reset(&out);
    }

................................................................................
      blob_appendf(&nameFile1, "%s~%d", zFile2, cnt++);
    }while( file_access(blob_str(&nameFile1),F_OK)==0 );
    blob_write_to_file(pFile1, blob_str(&nameFile1));

    /* Construct the external diff command */
    blob_zero(&cmd);
    blob_appendf(&cmd, "%s ", zDiffCmd);
    if( fSwapDiff ){
      shell_escape(&cmd, zFile2);
      blob_append(&cmd, " ", 1);
      shell_escape(&cmd, blob_str(&nameFile1));
    }else{
      shell_escape(&cmd, blob_str(&nameFile1));
      blob_append(&cmd, " ", 1);
      shell_escape(&cmd, zFile2);
    }

    /* Run the external diff command */
    fossil_system(blob_str(&cmd));

    /* Delete the temporary file and clean up memory used */
    file_delete(blob_str(&nameFile1));
    blob_reset(&nameFile1);
................................................................................
        content_get(srcid, &content);
      }else{
        blob_zero(&content);
      }
      isBin = fIncludeBinary ? 0 : looks_like_binary(&content);
      diff_print_index(zPathname, diffFlags);
      diff_file(&content, isBin, zFullName, zPathname, zDiffCmd,
                zBinGlob, fIncludeBinary, diffFlags, 0);
      blob_reset(&content);
    }
    blob_reset(&fname);
  }
  db_finalize(&q);
  db_end_transaction(1);  /* ROLLBACK */
}
................................................................................
  while( db_step(&q)==SQLITE_ROW ){
    char *zFullName;
    const char *zFile = (const char*)db_column_text(&q, 0);
    if( !file_dir_match(pFileDir, zFile) ) continue;
    zFullName = mprintf("%s%s", g.zLocalRoot, zFile);
    db_column_blob(&q, 1, &content);
    diff_file(&content, 0, zFullName, zFile,
              zDiffCmd, zBinGlob, fIncludeBinary, diffFlags, 0);
    fossil_free(zFullName);
    blob_reset(&content);
  }
  db_finalize(&q);
}

/*
................................................................................

/*
** Returns non-zero if files that may be binary should be used with external
** diff programs.
*/
int diff_include_binary_files(void){
  const char* zArgIncludeBinary = find_option("diff-binary", 0, 1);

  /* Command line argument have priority on settings */
  if( zArgIncludeBinary ){
    return is_truth(zArgIncludeBinary);
  }else{
    return db_get_boolean("diff-binary", 1);
  }
}

Changes to src/doc.c.

525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
...
706
707
708
709
710
711
712







713
714
715


716
717
718
719
720
721
722
...
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739

/*
** WEBPAGE: uv
** WEBPAGE: doc
** URL: /uv/FILE
** URL: /doc/CHECKIN/FILE
**
** CHECKIN can be either tag or SHA1 hash or timestamp identifying a
** particular check, or the name of a branch (meaning the most recent
** check-in on that branch) or one of various magic words:
**
**     "tip"      means the most recent check-in
**
**     "ckout"    means the current check-out, if the server is run from
**                within a check-out, otherwise it is the same as "tip"
................................................................................
    convert_href_and_output(&filebody);
    style_footer();
#ifdef FOSSIL_ENABLE_TH1_DOCS
  }else if( Th_AreDocsEnabled() &&
            fossil_strcmp(zMime, "application/x-th1")==0 ){
    int raw = P("raw")!=0;
    if( !raw ){







      style_header("%h", zName);
    }
    Th_Render(blob_str(&filebody));


    if( !raw ){
      style_footer();
    }
#endif
  }else{
    cgi_set_content_type(zMime);
    cgi_set_content(&filebody);
................................................................................
  db_end_transaction(0);
  return;

  /* Jump here when unable to locate the document */
doc_not_found:
  db_end_transaction(0);
  if( isUV && P("name")==0 ){
    uvstat_page();
    return;
  }
  cgi_set_status(404, "Not Found");
  style_header("Not Found");
  @ <p>Document %h(zOrigName) not found
  if( fossil_strcmp(zCheckin,"ckout")!=0 ){
    @ in %z(href("%R/tree?ci=%T",zCheckin))%h(zCheckin)</a>







|







 







>
>
>
>
>
>
>
|
<
|
>
>







 







|







525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
...
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720

721
722
723
724
725
726
727
728
729
730
...
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747

/*
** WEBPAGE: uv
** WEBPAGE: doc
** URL: /uv/FILE
** URL: /doc/CHECKIN/FILE
**
** CHECKIN can be either tag or hash prefix or timestamp identifying a
** particular check, or the name of a branch (meaning the most recent
** check-in on that branch) or one of various magic words:
**
**     "tip"      means the most recent check-in
**
**     "ckout"    means the current check-out, if the server is run from
**                within a check-out, otherwise it is the same as "tip"
................................................................................
    convert_href_and_output(&filebody);
    style_footer();
#ifdef FOSSIL_ENABLE_TH1_DOCS
  }else if( Th_AreDocsEnabled() &&
            fossil_strcmp(zMime, "application/x-th1")==0 ){
    int raw = P("raw")!=0;
    if( !raw ){
      Blob tail;
      blob_zero(&tail);
      if( wiki_find_title(&filebody, &title, &tail) ){
        style_header("%s", blob_str(&title));
        Th_Render(blob_str(&tail));
        blob_reset(&tail);
      }else{
        style_header("%h", zName);

        Th_Render(blob_str(&filebody));
      }
    }
    if( !raw ){
      style_footer();
    }
#endif
  }else{
    cgi_set_content_type(zMime);
    cgi_set_content(&filebody);
................................................................................
  db_end_transaction(0);
  return;

  /* Jump here when unable to locate the document */
doc_not_found:
  db_end_transaction(0);
  if( isUV && P("name")==0 ){
    uvlist_page();
    return;
  }
  cgi_set_status(404, "Not Found");
  style_header("Not Found");
  @ <p>Document %h(zOrigName) not found
  if( fossil_strcmp(zCheckin,"ckout")!=0 ){
    @ in %z(href("%R/tree?ci=%T",zCheckin))%h(zCheckin)</a>

Changes to src/encode.c.

334
335
336
337
338
339
340


























































































341
342
343
344
345
346
347
      }
    }
    z[j++] = c;
  }
  if( z[j] ) z[j] = 0;
}




























































































/*
** The characters used for HTTP base64 encoding.
*/
static unsigned char zBase[] =
  "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
      }
    }
    z[j++] = c;
  }
  if( z[j] ) z[j] = 0;
}


/*
** The *pz variable points to a UTF8 string.  Read the next character
** off of that string and return its codepoint value.  Advance *pz to the
** next character
*/
u32 fossil_utf8_read(
  const unsigned char **pz    /* Pointer to string from which to read char */
){
  unsigned int c;

  /*
  ** This lookup table is used to help decode the first byte of
  ** a multi-byte UTF8 character.
  */
  static const unsigned char utf8Trans1[] = {
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
    0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
    0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
    0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
    0x00, 0x01, 0x02, 0x03, 0x00, 0x01, 0x00, 0x00,
  };

  c = *((*pz)++);
  if( c>=0xc0 ){
    c = utf8Trans1[c-0xc0];
    while( (*(*pz) & 0xc0)==0x80 ){
      c = (c<<6) + (0x3f & *((*pz)++));
    }
    if( c<0x80
        || (c&0xFFFFF800)==0xD800
        || (c&0xFFFFFFFE)==0xFFFE ){  c = 0xFFFD; }
  }
  return c;
}

/*
** Encode a UTF8 string for JSON.  All special characters are escaped.
*/
void blob_append_json_string(Blob *pBlob, const char *zStr){
  const unsigned char *z;
  char *zOut;
  u32 c;
  int n, i, j;
  z = (const unsigned char*)zStr;
  n = 0;
  while( (c = fossil_utf8_read(&z))!=0 ){
    if( c=='\\' || c=='"' ){
      n += 2;
    }else if( c<' ' || c>=0x7f ){
      if( c=='\n' || c=='\r' ){
        n += 2;
      }else{
        n += 6;
      }
    }else{
      n++;
    }
  }
  i = blob_size(pBlob);
  blob_resize(pBlob, i+n);
  zOut = blob_buffer(pBlob);
  z = (const unsigned char*)zStr;
  while( (c = fossil_utf8_read(&z))!=0 ){
    if( c=='\\' ){
      zOut[i++] = '\\';
      zOut[i++] = c;
    }else if( c<' ' || c>=0x7f ){
      zOut[i++] = '\\';
      if( c=='\n' ){
        zOut[i++] = 'n';
      }else if( c=='\r' ){
        zOut[i++] = 'r';
      }else{
        zOut[i++] = 'u';
        for(j=3; j>=0; j--){
          zOut[i+j] = "0123456789abcdef"[c&0xf];
          c >>= 4;
        }
        i += 4;
      }
    }else{
      zOut[i++] = c;
    }
  }
  zOut[i] = 0;
}

/*
** The characters used for HTTP base64 encoding.
*/
static unsigned char zBase[] =
  "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

Changes to src/event.c.

382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
  }
  zTag = mprintf("event-%s", zId);
  rid = db_int(0,
    "SELECT rid FROM tagxref"
    " WHERE tagid=(SELECT tagid FROM tag WHERE tagname GLOB '%q*')"
    " ORDER BY mtime DESC", zTag
  );
  if( rid && strlen(zId)<40 ){
    zId = db_text(0,
      "SELECT substr(tagname,7) FROM tag WHERE tagname GLOB '%q*'",
      zTag
    );
  }
  free(zTag);








|







382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
  }
  zTag = mprintf("event-%s", zId);
  rid = db_int(0,
    "SELECT rid FROM tagxref"
    " WHERE tagid=(SELECT tagid FROM tag WHERE tagname GLOB '%q*')"
    " ORDER BY mtime DESC", zTag
  );
  if( rid && strlen(zId)<HNAME_MIN ){
    zId = db_text(0,
      "SELECT substr(tagname,7) FROM tag WHERE tagname GLOB '%q*'",
      zTag
    );
  }
  free(zTag);

Changes to src/export.c.

259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
  }

  cur_tok = strtok(NULL, " \t");
  if( !cur_tok ){
    /* This mark was generated by an older version of Fossil and doesn't
    ** include the mark name and uuid.  create_mark() will name the new mark
    ** exactly as it was when exported to git, so that we should have a
    ** valid mapping from git sha1<->mark name<->fossil sha1. */
    unsigned int mid;
    if( type_=='c' ){
      mid = COMMITMARK(mark->rid);
    }
    else{
      mid = BLOBMARK(mark->rid);
    }







|







259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
  }

  cur_tok = strtok(NULL, " \t");
  if( !cur_tok ){
    /* This mark was generated by an older version of Fossil and doesn't
    ** include the mark name and uuid.  create_mark() will name the new mark
    ** exactly as it was when exported to git, so that we should have a
    ** valid mapping from git hash<->mark name<->fossil hash. */
    unsigned int mid;
    if( type_=='c' ){
      mid = COMMITMARK(mark->rid);
    }
    else{
      mid = BLOBMARK(mark->rid);
    }

Changes to src/file.c.

83
84
85
86
87
88
89
90





91
92
93
94
95
96
97
98
99
100
101
102
103
104








105
106
107
108
109
110
111
...
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
...
155
156
157
158
159
160
161
















162
163
164
165
166
167
168
...
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
...
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
...
299
300
301
302
303
304
305

306
307
308
309

310
311
312
313
314

315
316
317
318
319
320
321
...
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
...
930
931
932
933
934
935
936
937
938

























































































939
940
941
942
943
944






945
946
947
948
949

950


951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
....
1073
1074
1075
1076
1077
1078
1079




1080
1081
1082
1083
1084
1085
1086
static struct fossilStat fileStat;

/*
** Fill stat buf with information received from stat() or lstat().
** lstat() is called on Unix if isWd is TRUE and allow-symlinks setting is on.
**
*/
static int fossil_stat(const char *zFilename, struct fossilStat *buf, int isWd){





  int rc;
  void *zMbcs = fossil_utf8_to_path(zFilename, 0);
#if !defined(_WIN32)
  if( isWd && db_allow_symlinks() ){
    rc = lstat(zMbcs, buf);
  }else{
    rc = stat(zMbcs, buf);
  }
#else
  rc = win32_stat(zMbcs, buf, isWd);
#endif
  fossil_path_free(zMbcs);
  return rc;
}









/*
** Fill in the fileStat variable for the file named zFilename.
** If zFilename==0, then use the previous value of fileStat if
** there is a previous value.
**
** If isWd is TRUE, do lstat() instead of stat() if allow-symlinks is on.
................................................................................
** Return the number of errors.  No error messages are generated.
*/
static int getStat(const char *zFilename, int isWd){
  int rc = 0;
  if( zFilename==0 ){
    if( fileStatValid==0 ) rc = 1;
  }else{
    if( fossil_stat(zFilename, &fileStat, isWd)!=0 ){
      fileStatValid = 0;
      rc = 1;
    }else{
      fileStatValid = 1;
      rc = 0;
    }
  }
................................................................................

/*
** Same as file_mtime(), but takes into account symlinks.
*/
i64 file_wd_mtime(const char *zFilename){
  return getStat(zFilename, 1) ? -1 : fileStat.st_mtime;
}

















/*
** Return TRUE if the named file is an ordinary file or symlink
** and symlinks are allowed.
** Return false for directories, devices, fifos, etc.
*/
int file_wd_isfile_or_link(const char *zFilename){
................................................................................
** Create symlink to file on Unix, or plain-text file with
** symlink target if "allow-symlinks" is off or we're on Windows.
**
** Arguments: target file (symlink will point to it), link file
**/
void symlink_create(const char *zTargetFile, const char *zLinkFile){
#if !defined(_WIN32)
  if( db_allow_symlinks() ){
    int i, nName;
    char *zName, zBuf[1000];

    nName = strlen(zLinkFile);
    if( nName>=sizeof(zBuf) ){
      zName = mprintf("%s", zLinkFile);
    }else{
................................................................................
**   - PERM_REG for all other cases (regular file, directory, fifo, etc).
*/
int file_wd_perm(const char *zFilename){
#if !defined(_WIN32)
  if( !getStat(zFilename, 1) ){
     if( S_ISREG(fileStat.st_mode) && ((S_IXUSR)&fileStat.st_mode)!=0 )
      return PERM_EXE;
    else if( db_allow_symlinks() && S_ISLNK(fileStat.st_mode) )
      return PERM_LNK;
  }
#endif
  return PERM_REG;
}

/*
................................................................................
** zFilename is a directory -OR- a symlink that points to a directory.
** Return 0 if zFilename does not exist.  Return 2 if zFilename exists
** but is something other than a directory.
*/
int file_wd_isdir(const char *zFilename){
  int rc;
  char *zFN;


  zFN = mprintf("%s", zFilename);
  file_simplify_name(zFN, -1, 0);
  rc = getStat(zFN, 1);

  if( rc ){
    rc = 0; /* It does not exist at all. */
  }else if( S_ISDIR(fileStat.st_mode) ){
    rc = 1; /* It exists and is a real directory. */
  }else if( !g.fNoDirSymlinks && S_ISLNK(fileStat.st_mode) ){

    Blob content;
    blob_read_link(&content, zFN); /* It exists and is a link. */
    rc = file_wd_isdir(blob_str(&content)); /* Points to directory? */
    blob_reset(&content);
  }else{
    rc = 2; /* It exists and is something else. */
  }
................................................................................
** Set or clear the execute bit on a file.  Return true if a change
** occurred and false if this routine is a no-op.
*/
int file_wd_setexe(const char *zFilename, int onoff){
  int rc = 0;
#if !defined(_WIN32)
  struct stat buf;
  if( fossil_stat(zFilename, &buf, 1)!=0 || S_ISLNK(buf.st_mode) ) return 0;
  if( onoff ){
    int targetMode = (buf.st_mode & 0444)>>2;
    if( (buf.st_mode & 0100)==0 ){
      chmod(zFilename, buf.st_mode | targetMode);
      rc = 1;
    }
  }else{
................................................................................
      zOut[0] = fossil_toupper(zOut[0]);
    }
  }
#endif
  blob_resize(pOut, file_simplify_name(blob_buffer(pOut),
                                       blob_size(pOut), slash));
}

/*

























































































** COMMAND: test-canonical-name
**
** Usage: %fossil test-canonical-name FILENAME...
**
** Test the operation of the canonical name generator.
** Also test Fossil's ability to measure attributes of a file.






*/
void cmd_test_canonical_name(void){
  int i;
  Blob x;
  int slashFlag = find_option("slash",0,0)!=0;

  blob_zero(&x);


  for(i=2; i<g.argc; i++){
    char zBuf[100];
    const char *zName = g.argv[i];
    file_canonical_name(zName, &x, slashFlag);
    fossil_print("[%s] -> [%s]\n", zName, blob_buffer(&x));
    blob_reset(&x);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", file_wd_size(zName));
    fossil_print("  file_size   = %s\n", zBuf);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", file_wd_mtime(zName));
    fossil_print("  file_mtime  = %s\n", zBuf);
    fossil_print("  file_isfile = %d\n", file_wd_isfile(zName));
    fossil_print("  file_isfile_or_link = %d\n",file_wd_isfile_or_link(zName));
    fossil_print("  file_islink = %d\n", file_wd_islink(zName));
    fossil_print("  file_isexe  = %d\n", file_wd_isexe(zName));
    fossil_print("  file_isdir  = %d\n", file_wd_isdir(zName));
  }
}

/*
** Return TRUE if the given filename is canonical.
**
** Canonical names are full pathnames using "/" not "\" and which
................................................................................
  }
}

/*
** COMMAND: test-relative-name
**
** Test the operation of the relative name generator.




*/
void cmd_test_relative_name(void){
  int i;
  Blob x;
  int slashFlag = find_option("slash",0,0)!=0;
  blob_zero(&x);
  for(i=2; i<g.argc; i++){







|
>
>
>
>
>



|










>
>
>
>
>
>
>
>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|







 







|







 







>



|
>


|

<
>







 







|







 









>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>






>
>
>
>
>
>



<

>
|
>
>

<
<
|
<
<
<
<
<
<
<
<
<
<
<







 







>
>
>
>







83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
...
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
...
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
...
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
...
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
...
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344

345
346
347
348
349
350
351
352
...
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
...
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073

1074
1075
1076
1077
1078
1079


1080











1081
1082
1083
1084
1085
1086
1087
....
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
static struct fossilStat fileStat;

/*
** Fill stat buf with information received from stat() or lstat().
** lstat() is called on Unix if isWd is TRUE and allow-symlinks setting is on.
**
*/
static int fossil_stat(
  const char *zFilename,  /* name of file or directory to inspect. */
  struct fossilStat *buf, /* pointer to buffer where info should go. */
  int isWd,               /* non-zero to consider look at symlink itself. */
  int forceWd             /* non-zero to force look at symlink itself. */
){
  int rc;
  void *zMbcs = fossil_utf8_to_path(zFilename, 0);
#if !defined(_WIN32)
  if( isWd && (forceWd || db_allow_symlinks(0)) ){
    rc = lstat(zMbcs, buf);
  }else{
    rc = stat(zMbcs, buf);
  }
#else
  rc = win32_stat(zMbcs, buf, isWd);
#endif
  fossil_path_free(zMbcs);
  return rc;
}

/*
** Clears the fileStat variable and its associated validity flag.
*/
static void resetStat(){
  fileStatValid = 0;
  memset(&fileStat, 0, sizeof(struct fossilStat));
}

/*
** Fill in the fileStat variable for the file named zFilename.
** If zFilename==0, then use the previous value of fileStat if
** there is a previous value.
**
** If isWd is TRUE, do lstat() instead of stat() if allow-symlinks is on.
................................................................................
** Return the number of errors.  No error messages are generated.
*/
static int getStat(const char *zFilename, int isWd){
  int rc = 0;
  if( zFilename==0 ){
    if( fileStatValid==0 ) rc = 1;
  }else{
    if( fossil_stat(zFilename, &fileStat, isWd, 0)!=0 ){
      fileStatValid = 0;
      rc = 1;
    }else{
      fileStatValid = 1;
      rc = 0;
    }
  }
................................................................................

/*
** Same as file_mtime(), but takes into account symlinks.
*/
i64 file_wd_mtime(const char *zFilename){
  return getStat(zFilename, 1) ? -1 : fileStat.st_mtime;
}

/*
** Return the mode bits for a file.  Return -1 if the file does not
** exist.  If zFilename is NULL return the size of the most recently
** stat-ed file.
*/
int file_mode(const char *zFilename){
  return getStat(zFilename, 0) ? -1 : fileStat.st_mode;
}

/*
** Same as file_mode(), but takes into account symlinks.
*/
int file_wd_mode(const char *zFilename){
  return getStat(zFilename, 1) ? -1 : fileStat.st_mode;
}

/*
** Return TRUE if the named file is an ordinary file or symlink
** and symlinks are allowed.
** Return false for directories, devices, fifos, etc.
*/
int file_wd_isfile_or_link(const char *zFilename){
................................................................................
** Create symlink to file on Unix, or plain-text file with
** symlink target if "allow-symlinks" is off or we're on Windows.
**
** Arguments: target file (symlink will point to it), link file
**/
void symlink_create(const char *zTargetFile, const char *zLinkFile){
#if !defined(_WIN32)
  if( db_allow_symlinks(0) ){
    int i, nName;
    char *zName, zBuf[1000];

    nName = strlen(zLinkFile);
    if( nName>=sizeof(zBuf) ){
      zName = mprintf("%s", zLinkFile);
    }else{
................................................................................
**   - PERM_REG for all other cases (regular file, directory, fifo, etc).
*/
int file_wd_perm(const char *zFilename){
#if !defined(_WIN32)
  if( !getStat(zFilename, 1) ){
     if( S_ISREG(fileStat.st_mode) && ((S_IXUSR)&fileStat.st_mode)!=0 )
      return PERM_EXE;
    else if( db_allow_symlinks(0) && S_ISLNK(fileStat.st_mode) )
      return PERM_LNK;
  }
#endif
  return PERM_REG;
}

/*
................................................................................
** zFilename is a directory -OR- a symlink that points to a directory.
** Return 0 if zFilename does not exist.  Return 2 if zFilename exists
** but is something other than a directory.
*/
int file_wd_isdir(const char *zFilename){
  int rc;
  char *zFN;
  struct fossilStat dirFileStat;

  zFN = mprintf("%s", zFilename);
  file_simplify_name(zFN, -1, 0);
  memset(&dirFileStat, 0, sizeof(struct fossilStat));
  rc = fossil_stat(zFN, &dirFileStat, 1, 1);
  if( rc ){
    rc = 0; /* It does not exist at all. */
  }else if( S_ISDIR(dirFileStat.st_mode) ){
    rc = 1; /* It exists and is a real directory. */

  }else if( !db_allow_symlinks(1) && S_ISLNK(dirFileStat.st_mode) ){
    Blob content;
    blob_read_link(&content, zFN); /* It exists and is a link. */
    rc = file_wd_isdir(blob_str(&content)); /* Points to directory? */
    blob_reset(&content);
  }else{
    rc = 2; /* It exists and is something else. */
  }
................................................................................
** Set or clear the execute bit on a file.  Return true if a change
** occurred and false if this routine is a no-op.
*/
int file_wd_setexe(const char *zFilename, int onoff){
  int rc = 0;
#if !defined(_WIN32)
  struct stat buf;
  if( fossil_stat(zFilename, &buf, 1, 0)!=0 || S_ISLNK(buf.st_mode) ) return 0;
  if( onoff ){
    int targetMode = (buf.st_mode & 0444)>>2;
    if( (buf.st_mode & 0100)==0 ){
      chmod(zFilename, buf.st_mode | targetMode);
      rc = 1;
    }
  }else{
................................................................................
      zOut[0] = fossil_toupper(zOut[0]);
    }
  }
#endif
  blob_resize(pOut, file_simplify_name(blob_buffer(pOut),
                                       blob_size(pOut), slash));
}

/*
** Emits the effective or raw stat() information for the specified
** file or directory, optionally preserving the trailing slash and
** resetting the cached stat() information.
*/
static void emitFileStat(
  const char *zPath,
  int raw,
  int slash,
  int reset
){
  char zBuf[100];
  Blob x;
  memset(zBuf, 0, sizeof(zBuf));
  blob_zero(&x);
  file_canonical_name(zPath, &x, slash);
  fossil_print("%s[%s] -> [%s]\n", raw ? "RAW " : "", zPath, blob_buffer(&x));
  blob_reset(&x);
  if( raw ){
    int rc;
    struct fossilStat testFileStat;
    memset(&testFileStat, 0, sizeof(struct fossilStat));
    rc = fossil_stat(zPath, &testFileStat, 0, 0);
    fossil_print("  stat_rc      = %d\n", rc);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", testFileStat.st_size);
    fossil_print("  stat_size    = %s\n", zBuf);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", testFileStat.st_mtime);
    fossil_print("  stat_mtime   = %s\n", zBuf);
    fossil_print("  stat_mode    = %d\n", testFileStat.st_mode);
    memset(&testFileStat, 0, sizeof(struct fossilStat));
    rc = fossil_stat(zPath, &testFileStat, 1, 1);
    fossil_print("  l_stat_rc    = %d\n", rc);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", testFileStat.st_size);
    fossil_print("  l_stat_size  = %s\n", zBuf);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", testFileStat.st_mtime);
    fossil_print("  l_stat_mtime = %s\n", zBuf);
    fossil_print("  l_stat_mode  = %d\n", testFileStat.st_mode);
  }else{
    if( reset ) resetStat();
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", file_wd_size(zPath));
    fossil_print("  file_size           = %s\n", zBuf);
    sqlite3_snprintf(sizeof(zBuf), zBuf, "%lld", file_wd_mtime(zPath));
    fossil_print("  file_mtime          = %s\n", zBuf);
    fossil_print("  file_mode           = %d\n", file_wd_mode(zPath));
    fossil_print("  file_isfile         = %d\n", file_wd_isfile(zPath));
    fossil_print("  file_isfile_or_link = %d\n", file_wd_isfile_or_link(zPath));
    fossil_print("  file_islink         = %d\n", file_wd_islink(zPath));
    fossil_print("  file_isexe          = %d\n", file_wd_isexe(zPath));
    fossil_print("  file_isdir          = %d\n", file_wd_isdir(zPath));
    if( reset ) resetStat();
  }
}

/*
** COMMAND: test-file-environment
**
** Usage: %fossil test-file-environment FILENAME...
**
** Display the effective file handling subsystem "settings" and then
** display file system information about the files specified, if any.
**
** Options:
**
**     --open-config        Open the configuration database first.
**     --slash              Trailing slashes, if any, are retained.
**     --reset              Reset cached stat() info for each file.
*/
void cmd_test_file_environment(void){
  int i;
  int slashFlag = find_option("slash",0,0)!=0;
  int resetFlag = find_option("reset",0,0)!=0;
  if( find_option("open-config", 0, 0)!=0 ){
    Th_OpenConfig(1);
  }
  fossil_print("Th_IsLocalOpen() = %d\n", Th_IsLocalOpen());
  fossil_print("Th_IsRepositoryOpen() = %d\n", Th_IsRepositoryOpen());
  fossil_print("Th_IsConfigOpen() = %d\n", Th_IsConfigOpen());
  fossil_print("filenames_are_case_sensitive() = %d\n",
               filenames_are_case_sensitive());
  fossil_print("db_allow_symlinks_by_default() = %d\n",
               db_allow_symlinks_by_default());
  fossil_print("db_allow_symlinks(0) = %d\n", db_allow_symlinks(0));
  fossil_print("db_allow_symlinks(1) = %d\n", db_allow_symlinks(1));
  for(i=2; i<g.argc; i++){
    emitFileStat(g.argv[i], 1, slashFlag, resetFlag);
    emitFileStat(g.argv[i], 0, slashFlag, resetFlag);
  }
}

/*
** COMMAND: test-canonical-name
**
** Usage: %fossil test-canonical-name FILENAME...
**
** Test the operation of the canonical name generator.
** Also test Fossil's ability to measure attributes of a file.
**
** Options:
**
**     --open-config        Open the configuration database first.
**     --slash              Trailing slashes, if any, are retained.
**     --reset              Reset cached stat() info for each file.
*/
void cmd_test_canonical_name(void){
  int i;

  int slashFlag = find_option("slash",0,0)!=0;
  int resetFlag = find_option("reset",0,0)!=0;
  if( find_option("open-config", 0, 0)!=0 ){
    Th_OpenConfig(1);
  }
  for(i=2; i<g.argc; i++){


    emitFileStat(g.argv[i], 0, slashFlag, resetFlag);











  }
}

/*
** Return TRUE if the given filename is canonical.
**
** Canonical names are full pathnames using "/" not "\" and which
................................................................................
  }
}

/*
** COMMAND: test-relative-name
**
** Test the operation of the relative name generator.
**
** Options:
**
**     --slash              Trailing slashes, if any, are retained.
*/
void cmd_test_relative_name(void){
  int i;
  Blob x;
  int slashFlag = find_option("slash",0,0)!=0;
  blob_zero(&x);
  for(i=2; i<g.argc; i++){

Changes to src/foci.c.

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
..
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
**    SELECT * FROM files_of_checkin('trunk');
**
** The "schema" for the temp.foci table is:
**
**     CREATE TABLE files_of_checkin(
**       checkinID    INTEGER,    -- RID for the check-in manifest
**       filename     TEXT,       -- Name of a file
**       uuid         TEXT,       -- SHA1 hash of the file
**       previousName TEXT,       -- Name of the file in previous check-in
**       perm         TEXT,       -- Permissions on the file
**       symname      TEXT HIDDEN -- Symbolic name of the check-in.
**     );
**
** The hidden symname column is (optionally) used as a query parameter to
** identify the particular check-in to parse.  The checkinID parameter
................................................................................
/*
** The schema for the virtual table:
*/
static const char zFociSchema[] =
@ CREATE TABLE files_of_checkin(
@  checkinID    INTEGER,    -- RID for the check-in manifest
@  filename     TEXT,       -- Name of a file
@  uuid         TEXT,       -- SHA1 hash of the file
@  previousName TEXT,       -- Name of the file in previous check-in
@  perm         TEXT,       -- Permissions on the file
@  symname      TEXT HIDDEN -- Symbolic name of the check-in
@ );
;

#define FOCI_CHECKINID   0







|







 







|







26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
..
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
**    SELECT * FROM files_of_checkin('trunk');
**
** The "schema" for the temp.foci table is:
**
**     CREATE TABLE files_of_checkin(
**       checkinID    INTEGER,    -- RID for the check-in manifest
**       filename     TEXT,       -- Name of a file
**       uuid         TEXT,       -- hash of the file
**       previousName TEXT,       -- Name of the file in previous check-in
**       perm         TEXT,       -- Permissions on the file
**       symname      TEXT HIDDEN -- Symbolic name of the check-in.
**     );
**
** The hidden symname column is (optionally) used as a query parameter to
** identify the particular check-in to parse.  The checkinID parameter
................................................................................
/*
** The schema for the virtual table:
*/
static const char zFociSchema[] =
@ CREATE TABLE files_of_checkin(
@  checkinID    INTEGER,    -- RID for the check-in manifest
@  filename     TEXT,       -- Name of a file
@  uuid         TEXT,       -- hash of the file
@  previousName TEXT,       -- Name of the file in previous check-in
@  perm         TEXT,       -- Permissions on the file
@  symname      TEXT HIDDEN -- Symbolic name of the check-in
@ );
;

#define FOCI_CHECKINID   0

Changes to src/fusefs.c.

292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
** Usage: %fossil fusefs [--debug] DIRECTORY
**
** This command uses the Fuse Filesystem (FuseFS) to mount a directory
** at DIRECTORY that contains the content of all check-ins in the
** repository.  The names of files are DIRECTORY/checkins/VERSION/PATH
** where DIRECTORY is the root of the mount, VERSION is any valid
** check-in name (examples: "trunk" or "tip" or a tag or any unique
** prefix of a SHA1 hash, etc) and PATH is the pathname of the file in
** the check-in.  If DIRECTORY does not exist, then an attempt is made
** to create it.
**
** The DIRECTORY/checkins directory is not searchable so one cannot
** do "ls DIRECTORY/checkins" to get a listing of all possible check-in
** names.  There are countless variations on check-in names and it is
** impractical to list them all.  But all other directories are searchable







|







292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
** Usage: %fossil fusefs [--debug] DIRECTORY
**
** This command uses the Fuse Filesystem (FuseFS) to mount a directory
** at DIRECTORY that contains the content of all check-ins in the
** repository.  The names of files are DIRECTORY/checkins/VERSION/PATH
** where DIRECTORY is the root of the mount, VERSION is any valid
** check-in name (examples: "trunk" or "tip" or a tag or any unique
** prefix of an artifact hash, etc) and PATH is the pathname of the file in
** the check-in.  If DIRECTORY does not exist, then an attempt is made
** to create it.
**
** The DIRECTORY/checkins directory is not searchable so one cannot
** do "ls DIRECTORY/checkins" to get a listing of all possible check-in
** names.  There are countless variations on check-in names and it is
** impractical to list them all.  But all other directories are searchable

Changes to src/graph.c.

177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
int graph_add_row(
  GraphContext *p,     /* The context to which the row is added */
  int rid,             /* RID for the check-in */
  int nParent,         /* Number of parents */
  int *aParent,        /* Array of parents */
  const char *zBranch, /* Branch for this check-in */
  const char *zBgClr,  /* Background color. NULL or "" for white. */
  const char *zUuid,   /* SHA1 hash of the object being graphed */
  int isLeaf           /* True if this row is a leaf */
){
  GraphRow *pRow;
  int nByte;

  if( p->nErr ) return 0;
  nByte = sizeof(GraphRow);







|







177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
int graph_add_row(
  GraphContext *p,     /* The context to which the row is added */
  int rid,             /* RID for the check-in */
  int nParent,         /* Number of parents */
  int *aParent,        /* Array of parents */
  const char *zBranch, /* Branch for this check-in */
  const char *zBgClr,  /* Background color. NULL or "" for white. */
  const char *zUuid,   /* hash name of the object being graphed */
  int isLeaf           /* True if this row is a leaf */
){
  GraphRow *pRow;
  int nByte;

  if( p->nErr ) return 0;
  nByte = sizeof(GraphRow);

Added src/hname.c.

























































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
/*
** Copyright (c) 2017 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
**
*******************************************************************************
**
** This file contains generic code for dealing with hashes used for
** naming artifacts.  Specific hash algorithms are implemented separately
** (for example in sha1.c and sha3.c).  This file contains the generic
** interface logic.
**
** "hname" is intended to be an abbreviation of "hash name".
*/
#include "config.h"
#include "hname.h"


#if INTERFACE
/*
** Code numbers for the allowed hash algorithms.
*/
#define HNAME_ERROR  0      /* Not a valid hash */
#define HNAME_SHA1   1      /* SHA1 */
#define HNAME_K256   2      /* SHA3-256 */

/*
** Minimum and maximum lengths for a hash value when hex encoded.
*/
#define HNAME_MIN  40     /* Length for SHA1 */
#define HNAME_MAX  64     /* Length for SHA3-256 */

/*
** Hash lengths for the various algorithms
*/
#define HNAME_LEN_SHA1   40
#define HNAME_LEN_K256   64

/*
** The number of distinct hash algorithms:
*/
#define HNAME_COUNT 2     /* Just SHA1 and SHA3-256. Let's keep it that way! */

/*
** Hash naming policies
*/
#define HPOLICY_SHA1           0      /* Use SHA1 hashes */
#define HPOLICY_AUTO           1      /* SHA1 but auto-promote to SHA3 */
#define HPOLICY_SHA3           2      /* Use SHA3 hashes */
#define HPOLICY_SHA3_ONLY      3      /* Use SHA3 hashes exclusively */
#define HPOLICY_SHUN_SHA1      4      /* Shun all SHA1 objects */

#endif /* INTERFACE */

/*
** Return a human-readable name for the hash algorithm given a hash with
** a length of nHash hexadecimal digits.
*/
const char *hname_alg(int nHash){
  if( nHash==HNAME_LEN_SHA1 ) return "SHA1";
  if( nHash==HNAME_LEN_K256 ) return "SHA3-256";
  return "?";
}

/*
** Return the integer hash algorithm code number (ex: HNAME_K256) for
** the hash string provided.  Or return HNAME_ERROR (0) if the input string
** is not a valid artifact hash string.
*/
int hname_validate(const char *zHash, int nHash){
  int id;
  switch( nHash ){
    case HNAME_LEN_SHA1:   id = HNAME_SHA1;  break;
    case HNAME_LEN_K256:   id = HNAME_K256;  break;
    default:               return HNAME_ERROR;
  }
  if( !validate16(zHash, nHash) ) return HNAME_ERROR;
  return id;
}

/*
** Verify that zHash is a valid hash for the content in pContent.
** Return true if the hash is correct.  Return false if the content
** does not match the hash.
**
** Actually, the returned value is one of the hash algorithm constants
** corresponding to the hash that matched if the hash is correct.
** (Examples: HNAME_SHA1 or HNAME_K256).  And the return is HNAME_ERROR
** if the hash does not match.
*/
int hname_verify_hash(Blob *pContent, const char *zHash, int nHash){
  int id = HNAME_ERROR;
  switch( nHash ){
    case HNAME_LEN_SHA1: {
      Blob hash;
      sha1sum_blob(pContent, &hash);
      if( memcmp(blob_buffer(&hash),zHash,HNAME_LEN_SHA1)==0 ) id = HNAME_SHA1;
      blob_reset(&hash);
      break;
    }
    case HNAME_LEN_K256: {
      sha3sum_init(256);
      sha3sum_step_blob(pContent);
      if( memcmp(sha3sum_finish(0),zHash,64)==0 ) id = HNAME_K256;
      break;
    }
  }
  return id;
}

/*
** Verify that zHash is a valid hash for the content of a file on
** disk named zFile.
**
** Return true if the hash is correct.  Return false if the content
** does not match the hash.
**
** Actually, the returned value is one of the hash algorithm constants
** corresponding to the hash that matched if the hash is correct.
** (Examples: HNAME_SHA1 or HNAME_K256).  And the return is HNAME_ERROR
** if the hash does not match.
*/
int hname_verify_file_hash(const char *zFile, const char *zHash, int nHash){
  int id = HNAME_ERROR;
  switch( nHash ){
    case HNAME_LEN_SHA1: {
      Blob hash;
      sha1sum_file(zFile, &hash);
      if( memcmp(blob_buffer(&hash),zHash,HNAME_LEN_SHA1)==0 ) id = HNAME_SHA1;
      blob_reset(&hash);
      break;
    }
    case HNAME_LEN_K256: {
      Blob hash;
      sha3sum_file(zFile, 256, &hash);
      if( memcmp(blob_buffer(&hash),zHash,64)==0 ) id = HNAME_LEN_K256;
      blob_reset(&hash);
      break;
    }
  }
  return id;
}

/*
** Compute a hash on blob pContent.  Write the hash into blob pHashOut.
** This routine assumes that pHashOut is uninitialized.
**
** The preferred hash is used for iHType==0 and the alternative hash is
** used if iHType==1.  (The interface is designed to accommodate more than
** just two hashes, but HNAME_COUNT is currently fixed at 2.)
**
** Depending on the hash policy, the alternative hash may be disallowed.
** If the alterative hash is disallowed, the routine returns 0.  This
** routine returns 1 if iHType>0 and the alternative hash is allowed,
** and it always returns 1 when iHType==0.
**
** Alternative hash is disallowed for all hash policies except auto,
** sha1 and sha3.
*/
int hname_hash(const Blob *pContent, unsigned int iHType, Blob *pHashOut){
  assert( iHType==0 || iHType==1 );
  if( iHType==1 ){
    switch( g.eHashPolicy ){
      case HPOLICY_AUTO:
      case HPOLICY_SHA1:
        sha3sum_blob(pContent, 256, pHashOut);
        return 1;
      case HPOLICY_SHA3:
        sha1sum_blob(pContent, pHashOut);
        return 1;
    }
  }
  if( iHType==0 ){
    switch( g.eHashPolicy ){
      case HPOLICY_SHA1:
      case HPOLICY_AUTO:
        sha1sum_blob(pContent, pHashOut);
        return 1;
      case HPOLICY_SHA3:
      case HPOLICY_SHA3_ONLY:
      case HPOLICY_SHUN_SHA1:
        sha3sum_blob(pContent, 256, pHashOut);
        return 1;
    }
  }
  blob_init(pHashOut, 0, 0);
  return 0;
}

/*
** Return the default hash policy for repositories that do not currently
** have an assigned hash policy.
**
** Make the default HPOLICY_AUTO if there are SHA1 artficates but no SHA3
** artifacts in the repository.  Make the default HPOLICY_SHA3 if there
** are one or more SHA3 artifacts or if the repository is initially empty.
*/
int hname_default_policy(void){
  if( db_exists("SELECT 1 FROM blob WHERE length(uuid)>40")
   || !db_exists("SELECT 1 FROM blob WHERE length(uuid)==40")
  ){
    return HPOLICY_SHA3;
  }else{
    return HPOLICY_AUTO;
  }
}

/*
** Names of the hash policies.
*/
static const char *azPolicy[] = {
  "sha1", "auto", "sha3", "sha3-only", "shun-sha1"
};

/* Return the name of the current hash policy.
*/
const char *hpolicy_name(void){
  return azPolicy[g.eHashPolicy];
}


/*
** COMMAND: hash-policy*
**
** Usage: fossil hash-policy ?NEW-POLICY?
**
** Query or set the hash policy for the current repository.  Available hash
** policies are as follows:
**
**   sha1              New artifact names are created using SHA1
**
**   auto              New artifact names are created using SHA1, but
**                     automatically change the policy to "sha3" when
**                     any SHA3 artifact enters the repository.
**
**   sha3              New artifact names are created using SHA3, but
**                     older artifacts with SHA1 names may be reused.
**
**   sha3-only         Use only SHA3 artifact names.  Do not reuse legacy
**                     SHA1 names.
**
**   shun-sha1         Shun any SHA1 artifacts received by sync operations
**                     other than clones.  Older legacy SHA1 artifacts are
**                     allowed during a clone.
**
** The default hash policy for existing repositories is "auto", which will
** immediately promote to "sha3" if the repository contains one or more
** artifacts with SHA3 names.  The default hash policy for new repositories
** is "shun-sha1".
*/
void hash_policy_command(void){
  int i;
  db_find_and_open_repository(0, 0);
  if( g.argc!=2 && g.argc!=3 ) usage("?NEW-POLICY?");
  if( g.argc==2 ){
    fossil_print("%s\n", azPolicy[g.eHashPolicy]);
    return;
  }
  for(i=HPOLICY_SHA1; i<=HPOLICY_SHUN_SHA1; i++){
    if( fossil_strcmp(g.argv[2],azPolicy[i])==0 ){
      if( i==HPOLICY_AUTO
       && db_exists("SELECT 1 FROM blob WHERE length(uuid)>40")
      ){
        i = HPOLICY_SHA3;
      }
      g.eHashPolicy = i;
      db_set_int("hash-policy", i, 0);
      fossil_print("%s\n", azPolicy[i]);
      return;
    }
  }
  fossil_fatal("unknown hash policy \"%s\" - should be one of: sha1 auto"
               " sha3 sha3-only shun-sha1", g.argv[2]);
}

Changes to src/http_ssl.c.

250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
...
291
292
293
294
295
296
297
298
299

300
301
302
303
304
305
306
    X509_STORE_add_cert(SSL_CTX_get_cert_store(sslCtx), cert);
    X509_free(cert);
    hasSavedCertificate = 1;
  }

  if( pUrlData->useProxy ){
    int rc;
    BIO *sBio;
    char *connStr;
    connStr = mprintf("%s:%d", g.url.name, pUrlData->port);
    sBio = BIO_new_connect(connStr);
    free(connStr);
    if( BIO_do_connect(sBio)<=0 ){
      ssl_set_errmsg("SSL: cannot connect to proxy %s:%d (%s)",
            pUrlData->name, pUrlData->port, ERR_reason_error_string(ERR_get_error()));
      ssl_close();
      return 1;
    }
................................................................................
                  "continuing without it.\n");
  }
#endif

  SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);

  if( !pUrlData->useProxy ){
    BIO_set_conn_hostname(iBio, pUrlData->name);
    BIO_ctrl(iBio,BIO_C_SET_CONNECT,3,(char *)&pUrlData->port);

    if( BIO_do_connect(iBio)<=0 ){
      ssl_set_errmsg("SSL: cannot connect to host %s:%d (%s)",
          pUrlData->name, pUrlData->port, ERR_reason_error_string(ERR_get_error()));
      ssl_close();
      return 1;
    }
  }







<
<
|
|







 







|
|
>







250
251
252
253
254
255
256


257
258
259
260
261
262
263
264
265
...
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
    X509_STORE_add_cert(SSL_CTX_get_cert_store(sslCtx), cert);
    X509_free(cert);
    hasSavedCertificate = 1;
  }

  if( pUrlData->useProxy ){
    int rc;


    char *connStr = mprintf("%s:%d", g.url.name, pUrlData->port);
    BIO *sBio = BIO_new_connect(connStr);
    free(connStr);
    if( BIO_do_connect(sBio)<=0 ){
      ssl_set_errmsg("SSL: cannot connect to proxy %s:%d (%s)",
            pUrlData->name, pUrlData->port, ERR_reason_error_string(ERR_get_error()));
      ssl_close();
      return 1;
    }
................................................................................
                  "continuing without it.\n");
  }
#endif

  SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);

  if( !pUrlData->useProxy ){
    char *connStr = mprintf("%s:%d", pUrlData->name, pUrlData->port);
    BIO_set_conn_hostname(iBio, connStr);
    free(connStr);
    if( BIO_do_connect(iBio)<=0 ){
      ssl_set_errmsg("SSL: cannot connect to host %s:%d (%s)",
          pUrlData->name, pUrlData->port, ERR_reason_error_string(ERR_get_error()));
      ssl_close();
      return 1;
    }
  }

Changes to src/import.c.

147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
....
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
**
** If saveUuid is true, then pContent is a commit record.  Record its
** UUID in gg.zPrevCheckin.
*/
static int fast_insert_content(
  Blob *pContent,          /* Content to insert */
  const char *zMark,       /* Label using this mark, if not NULL */
  int saveUuid,            /* Save SHA1 hash in gg.zPrevCheckin */
  int doParse              /* Invoke manifest_crosslink() */
){
  Blob hash;
  Blob cmpr;
  int rid;

  sha1sum_blob(pContent, &hash);
  rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);
  if( rid==0 ){
    static Stmt ins;
    db_static_prepare(&ins,
        "INSERT INTO blob(uuid, size, content) VALUES(:uuid, :size, :content)"
    );
    db_bind_text(&ins, ":uuid", blob_str(&hash));
................................................................................
    /* Get --svn related options here, so verify_all_options() fails when
     * svn-only options are specified with --git
     */
    const char *zIgnTree;
    unsigned nIgnTree = 0;
    while( (zIgnTree = find_option("ignore-tree", 0, 1)) ){
      if ( *zIgnTree ){
        gsvn.azIgnTree = fossil_realloc(gsvn.azIgnTree,
            sizeof(*gsvn.azIgnTree) * (nIgnTree + 2));
        gsvn.azIgnTree[nIgnTree++] = zIgnTree;
        gsvn.azIgnTree[nIgnTree] = 0;
      }
    }
    zBase = find_option("base", 0, 1);
    flatFlag = find_option("flat", 0, 0)!=0;







|






|







 







|







147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
....
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
**
** If saveUuid is true, then pContent is a commit record.  Record its
** UUID in gg.zPrevCheckin.
*/
static int fast_insert_content(
  Blob *pContent,          /* Content to insert */
  const char *zMark,       /* Label using this mark, if not NULL */
  int saveUuid,            /* Save artifact hash in gg.zPrevCheckin */
  int doParse              /* Invoke manifest_crosslink() */
){
  Blob hash;
  Blob cmpr;
  int rid;

  hname_hash(pContent, 0, &hash);
  rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);
  if( rid==0 ){
    static Stmt ins;
    db_static_prepare(&ins,
        "INSERT INTO blob(uuid, size, content) VALUES(:uuid, :size, :content)"
    );
    db_bind_text(&ins, ":uuid", blob_str(&hash));
................................................................................
    /* Get --svn related options here, so verify_all_options() fails when
     * svn-only options are specified with --git
     */
    const char *zIgnTree;
    unsigned nIgnTree = 0;
    while( (zIgnTree = find_option("ignore-tree", 0, 1)) ){
      if ( *zIgnTree ){
        gsvn.azIgnTree = fossil_realloc((void *)gsvn.azIgnTree,
            sizeof(*gsvn.azIgnTree) * (nIgnTree + 2));
        gsvn.azIgnTree[nIgnTree++] = zIgnTree;
        gsvn.azIgnTree[nIgnTree] = 0;
      }
    }
    zBase = find_option("base", 0, 1);
    flatFlag = find_option("flat", 0, 0)!=0;

Changes to src/info.c.

66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
..
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
...
570
571
572
573
574
575
576

577
578
579
580
581
582
583
...
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
....
1383
1384
1385
1386
1387
1388
1389

1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
....
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
....
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856


1857
1858
1859
1860
1861
1862
1863
....
1873
1874
1875
1876
1877
1878
1879


1880





1881
1882
1883
1884
1885
1886
1887
1888












1889
1890
1891
1892
1893
1894
1895
  zUuid = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", rid);
  if( zUuid ){
    zDate = db_text(0,
      "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
      rid
    );
         /* 01234567890123 */
    fossil_print("%-13s %s %s\n", zUuidName, zUuid, zDate ? zDate : "");
    free(zDate);
    if( showComment ){
      zComment = db_text(0,
        "SELECT coalesce(ecomment,comment) || "
        "       ' (user: ' || coalesce(euser,user,'?') || ')' "
        "  FROM event WHERE objid=%d",
        rid
................................................................................
    while( db_step(&q)==SQLITE_ROW ){
      const char *zUuid = db_column_text(&q, 0);
      const char *zType = db_column_int(&q, 2) ? "parent:" : "merged-from:";
      zDate = db_text("",
        "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
        db_column_int(&q, 1)
      );
      fossil_print("%-13s %s %s\n", zType, zUuid, zDate);
      free(zDate);
    }
    db_finalize(&q);
    db_prepare(&q, "SELECT uuid, cid, isprim FROM plink JOIN blob ON cid=rid "
                   " WHERE pid=%d"
                   " ORDER BY isprim DESC, mtime DESC /*sort*/", rid);
    while( db_step(&q)==SQLITE_ROW ){
      const char *zUuid = db_column_text(&q, 0);
      const char *zType = db_column_int(&q, 2) ? "child:" : "merged-into:";
      zDate = db_text("",
        "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
        db_column_int(&q, 1)
      );
      fossil_print("%-13s %s %s\n", zType, zUuid, zDate);
      free(zDate);
    }
    db_finalize(&q);
  }
  zTags = info_tags_of_checkin(rid, 0);
  if( zTags && zTags[0] ){
    fossil_print("tags:         %s\n", zTags);
................................................................................
     " WHERE blob.rid=%d"
     "   AND event.objid=%d",
     rid, rid
  );
  sideBySide = !is_false(PD("sbs","1"));
  if( db_step(&q1)==SQLITE_ROW ){
    const char *zUuid = db_column_text(&q1, 0);

    char *zEUser, *zEComment;
    const char *zUser;
    const char *zComment;
    const char *zDate;
    const char *zOrigDate;

    style_header("Check-in [%S]", zUuid);
................................................................................
                   TAG_COMMENT, rid);
    zUser = db_column_text(&q1, 2);
    zComment = db_column_text(&q1, 3);
    zDate = db_column_text(&q1,1);
    zOrigDate = db_column_text(&q1, 4);
    @ <div class="section">Overview</div>
    @ <table class="label-value">
    @ <tr><th>SHA1&nbsp;Hash:</th><td>%s(zUuid)
    if( g.perm.Setup ){
      @ (Record ID: %d(rid))
    }
    @ </td></tr>
    @ <tr><th>Date:</th><td>
    hyperlink_to_date(zDate, "</td></tr>");
    if( zOrigDate && fossil_strcmp(zDate, zOrigDate)!=0 ){
................................................................................
    "  FROM attachment"
    " WHERE src=(SELECT uuid FROM blob WHERE rid=%d)"
    " ORDER BY mtime DESC /*sort*/",
    rid
  );
  while( db_step(&q)==SQLITE_ROW ){
    const char *zTarget = db_column_text(&q, 0);

    const char *zFilename = db_column_text(&q, 1);
    const char *zDate = db_column_text(&q, 2);
    const char *zUser = db_column_text(&q, 3);
    /* const char *zSrc = db_column_text(&q, 4); */
    if( cnt>0 ){
      @ Also attachment "%h(zFilename)" to
    }else{
      @ Attachment "%h(zFilename)" to
    }
    objType |= OBJTYPE_ATTACHMENT;
    if( strlen(zTarget)==UUID_SIZE && validate16(zTarget,UUID_SIZE) ){
      if ( db_exists("SELECT 1 FROM tag WHERE tagname='tkt-%q'",
            zTarget)
      ){
        if( g.perm.Hyperlink && g.anon.RdTkt ){
          @ ticket [%z(href("%R/tktview?name=%!S",zTarget))%S(zTarget)</a>]
        }else{
          @ ticket [%S(zTarget)]
................................................................................
/*
** WEBPAGE: artifact
** WEBPAGE: file
** WEBPAGE: whatis
**
** Typical usage:
**
**    /artifact/SHA1HASH
**    /whatis/SHA1HASH
**    /file/NAME
**
** Additional query parameters:
**
**   ln              - show line numbers
**   ln=N            - highlight line number N
**   ln=M-N          - highlight lines M through N inclusive
................................................................................
**   download        - redirect to the download (artifact page only)
**   name=SHA1HASH   - Provide the SHA1HASH as a query parameter
**   filename=NAME   - Show information for content file NAME
**   fn=NAME         - "fn" is shorthand for "filename"
**   ci=VERSION      - The specific check-in to use for "filename=".
**
** The /artifact page show the complete content of a file
** identified by SHA1HASH as preformatted text.  The
** /whatis page shows only a description of the file.  The /file
** page shows the most recent version of the named file.


*/
void artifact_page(void){
  int rid = 0;
  Blob content;
  const char *zMime;
  Blob downloadName;
  int renderAsWiki = 0;
................................................................................
  HQuery url;

  url_initialize(&url, g.zPath);
  rid = artifact_from_ci_and_filename(&url);
  if( rid==0 ){
    url_add_parameter(&url, "name", zName);
    if( isFile ){


      if( zName==0 ) zName = "";





      rid = db_int(0,
         "SELECT fid FROM filename, mlink, event"
         " WHERE name=%Q"
         "   AND mlink.fnid=filename.fnid"
         "   AND event.objid=mlink.mid"
         " ORDER BY event.mtime DESC LIMIT 1",
         zName
      );












      if( rid==0 ){
        style_header("No such file");
        @ File '%h(zName)' does not exist in this repository.
        style_footer();
        return;
      }
    }else{







|







 







|













|







 







>







 







|







 







>










|







 







|
|







 







|

|
>
>







 







>
>
|
>
>
>
>
>








>
>
>
>
>
>
>
>
>
>
>
>







66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
..
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
...
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
...
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
....
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
....
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
....
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
....
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
  zUuid = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", rid);
  if( zUuid ){
    zDate = db_text(0,
      "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
      rid
    );
         /* 01234567890123 */
    fossil_print("%-13s %.40s %s\n", zUuidName, zUuid, zDate ? zDate : "");
    free(zDate);
    if( showComment ){
      zComment = db_text(0,
        "SELECT coalesce(ecomment,comment) || "
        "       ' (user: ' || coalesce(euser,user,'?') || ')' "
        "  FROM event WHERE objid=%d",
        rid
................................................................................
    while( db_step(&q)==SQLITE_ROW ){
      const char *zUuid = db_column_text(&q, 0);
      const char *zType = db_column_int(&q, 2) ? "parent:" : "merged-from:";
      zDate = db_text("",
        "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
        db_column_int(&q, 1)
      );
      fossil_print("%-13s %.40s %s\n", zType, zUuid, zDate);
      free(zDate);
    }
    db_finalize(&q);
    db_prepare(&q, "SELECT uuid, cid, isprim FROM plink JOIN blob ON cid=rid "
                   " WHERE pid=%d"
                   " ORDER BY isprim DESC, mtime DESC /*sort*/", rid);
    while( db_step(&q)==SQLITE_ROW ){
      const char *zUuid = db_column_text(&q, 0);
      const char *zType = db_column_int(&q, 2) ? "child:" : "merged-into:";
      zDate = db_text("",
        "SELECT datetime(mtime) || ' UTC' FROM event WHERE objid=%d",
        db_column_int(&q, 1)
      );
      fossil_print("%-13s %.40s %s\n", zType, zUuid, zDate);
      free(zDate);
    }
    db_finalize(&q);
  }
  zTags = info_tags_of_checkin(rid, 0);
  if( zTags && zTags[0] ){
    fossil_print("tags:         %s\n", zTags);
................................................................................
     " WHERE blob.rid=%d"
     "   AND event.objid=%d",
     rid, rid
  );
  sideBySide = !is_false(PD("sbs","1"));
  if( db_step(&q1)==SQLITE_ROW ){
    const char *zUuid = db_column_text(&q1, 0);
    int nUuid = db_column_bytes(&q1, 0);
    char *zEUser, *zEComment;
    const char *zUser;
    const char *zComment;
    const char *zDate;
    const char *zOrigDate;

    style_header("Check-in [%S]", zUuid);
................................................................................
                   TAG_COMMENT, rid);
    zUser = db_column_text(&q1, 2);
    zComment = db_column_text(&q1, 3);
    zDate = db_column_text(&q1,1);
    zOrigDate = db_column_text(&q1, 4);
    @ <div class="section">Overview</div>
    @ <table class="label-value">
    @ <tr><th>%s(hname_alg(nUuid)):</th><td>%s(zUuid)
    if( g.perm.Setup ){
      @ (Record ID: %d(rid))
    }
    @ </td></tr>
    @ <tr><th>Date:</th><td>
    hyperlink_to_date(zDate, "</td></tr>");
    if( zOrigDate && fossil_strcmp(zDate, zOrigDate)!=0 ){
................................................................................
    "  FROM attachment"
    " WHERE src=(SELECT uuid FROM blob WHERE rid=%d)"
    " ORDER BY mtime DESC /*sort*/",
    rid
  );
  while( db_step(&q)==SQLITE_ROW ){
    const char *zTarget = db_column_text(&q, 0);
    int nTarget = db_column_bytes(&q, 0);
    const char *zFilename = db_column_text(&q, 1);
    const char *zDate = db_column_text(&q, 2);
    const char *zUser = db_column_text(&q, 3);
    /* const char *zSrc = db_column_text(&q, 4); */
    if( cnt>0 ){
      @ Also attachment "%h(zFilename)" to
    }else{
      @ Attachment "%h(zFilename)" to
    }
    objType |= OBJTYPE_ATTACHMENT;
    if( nTarget==UUID_SIZE && validate16(zTarget,UUID_SIZE) ){
      if ( db_exists("SELECT 1 FROM tag WHERE tagname='tkt-%q'",
            zTarget)
      ){
        if( g.perm.Hyperlink && g.anon.RdTkt ){
          @ ticket [%z(href("%R/tktview?name=%!S",zTarget))%S(zTarget)</a>]
        }else{
          @ ticket [%S(zTarget)]
................................................................................
/*
** WEBPAGE: artifact
** WEBPAGE: file
** WEBPAGE: whatis
**
** Typical usage:
**
**    /artifact/HASH
**    /whatis/HASH
**    /file/NAME
**
** Additional query parameters:
**
**   ln              - show line numbers
**   ln=N            - highlight line number N
**   ln=M-N          - highlight lines M through N inclusive
................................................................................
**   download        - redirect to the download (artifact page only)
**   name=SHA1HASH   - Provide the SHA1HASH as a query parameter
**   filename=NAME   - Show information for content file NAME
**   fn=NAME         - "fn" is shorthand for "filename"
**   ci=VERSION      - The specific check-in to use for "filename=".
**
** The /artifact page show the complete content of a file
** identified by HASH as preformatted text.  The
** /whatis page shows only a description of the file.  The /file
** page shows the most recent version of the file or directory
** called NAME, or a list of the top-level directory if NAME is
** omitted.
*/
void artifact_page(void){
  int rid = 0;
  Blob content;
  const char *zMime;
  Blob downloadName;
  int renderAsWiki = 0;
................................................................................
  HQuery url;

  url_initialize(&url, g.zPath);
  rid = artifact_from_ci_and_filename(&url);
  if( rid==0 ){
    url_add_parameter(&url, "name", zName);
    if( isFile ){
      /* Do a top-level directory listing in /file mode if no argument
      ** specified */
      if( zName==0 || zName[0]==0 ){
        if( P("ci")==0 ) cgi_set_query_parameter("ci","tip");
        page_tree();
        return;
      }
      /* Look for a single file with the given name */
      rid = db_int(0,
         "SELECT fid FROM filename, mlink, event"
         " WHERE name=%Q"
         "   AND mlink.fnid=filename.fnid"
         "   AND event.objid=mlink.mid"
         " ORDER BY event.mtime DESC LIMIT 1",
         zName
      );
      /* If no file called NAME exists, instead look for a directory
      ** with that name, and do a directory listing */
      if( rid==0 && db_exists(
         "SELECT 1 FROM filename"
         " WHERE name GLOB '%q/*' AND substr(name,1,length(%Q)+1)=='%q/';",
         zName, zName, zName
      ) ){
        if( P("ci")==0 ) cgi_set_query_parameter("ci","tip");
        page_tree();
        return;
      }
      /* If no file or directory called NAME: issue an error */
      if( rid==0 ){
        style_header("No such file");
        @ File '%h(zName)' does not exist in this repository.
        style_footer();
        return;
      }
    }else{

Changes to src/json.c.

2247
2248
2249
2250
2251
2252
2253





2254
2255
2256
2257
2258
2259
2260
** COMMAND: json
**
** Usage: %fossil json SUBCOMMAND ?OPTIONS?
**
** In CLI mode, the -R REPO common option is supported. Due to limitations
** in the argument dispatching code, any -FLAGS must come after the final
** sub- (or subsub-) command.





**
** The commands include:
**
**   anonymousPassword
**   artifact
**   branch
**   cap







>
>
>
>
>







2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
** COMMAND: json
**
** Usage: %fossil json SUBCOMMAND ?OPTIONS?
**
** In CLI mode, the -R REPO common option is supported. Due to limitations
** in the argument dispatching code, any -FLAGS must come after the final
** sub- (or subsub-) command.
**
** The -json-input FILE option can be used to read JSON data and process
** it like the HTTP interface would. For example:
**
**   %fossil json -json-input my.json
**
** The commands include:
**
**   anonymousPassword
**   artifact
**   branch
**   cap

Changes to src/login.c.

46
47
48
49
50
51
52

53
54
55
56
57
58
59
...
844
845
846
847
848
849
850




















































851
852
853
854
855
856
857
...
964
965
966
967
968
969
970








971
972
973
974
975
976
977
#if defined(_WIN32)
#  include <windows.h>           /* for Sleep */
#  if defined(__MINGW32__) || defined(_MSC_VER)
#    define sleep Sleep            /* windows does not have sleep, but Sleep */
#  endif
#endif
#include <time.h>


/*
** Return the login-group name.  Or return 0 if this repository is
** not a member of a login-group.
*/
const char *login_group_name(void){
  static const char *zGroup = 0;
................................................................................
int login_wants_https_redirect(void){
  if( g.sslNotAvailable ) return 0;
  if( db_get_boolean("redirect-to-https",0)==0 ) return 0;
  if( P("HTTPS")!=0 ) return 0;
  return 1;
}





















































/*
** This routine examines the login cookie to see if it exists and
** is valid.  If the login cookie checks out, it then sets global
** variables appropriately.
**
**    g.userUid      Database USER.UID value.  Might be -1 for "nobody"
**    g.zLogin       Database USER.LOGIN value.  NULL for user "nobody"
................................................................................
  if( uid==0 ){
    const char *zRemoteUser = P("REMOTE_USER");
    if( zRemoteUser && db_get_boolean("remote_user_ok",0) ){
      uid = db_int(0, "SELECT uid FROM user WHERE login=%Q"
                      " AND length(cap)>0 AND length(pw)>0", zRemoteUser);
    }
  }









  /* If no user found yet, try to log in as "nobody" */
  if( uid==0 ){
    uid = db_int(0, "SELECT uid FROM user WHERE login='nobody'");
    if( uid==0 ){
      /* If there is no user "nobody", then make one up - with no privileges */
      uid = -1;







>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>







46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
...
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
....
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
#if defined(_WIN32)
#  include <windows.h>           /* for Sleep */
#  if defined(__MINGW32__) || defined(_MSC_VER)
#    define sleep Sleep            /* windows does not have sleep, but Sleep */
#  endif
#endif
#include <time.h>


/*
** Return the login-group name.  Or return 0 if this repository is
** not a member of a login-group.
*/
const char *login_group_name(void){
  static const char *zGroup = 0;
................................................................................
int login_wants_https_redirect(void){
  if( g.sslNotAvailable ) return 0;
  if( db_get_boolean("redirect-to-https",0)==0 ) return 0;
  if( P("HTTPS")!=0 ) return 0;
  return 1;
}


/*
** Attempt to use Basic Authentication to establish the user.  Return the
** (non-zero) uid if successful.  Return 0 if it does not work.
*/
static int logic_basic_authentication(const char *zIpAddr){
  const char *zAuth = PD("HTTP_AUTHORIZATION", 0);
  int i;
  int uid = 0;
  int nDecode = 0;
  char *zDecode = 0;
  const char *zUsername = 0;
  const char *zPasswd = 0;
  

  if( zAuth==0 ) return 0;                    /* Fail: No Authentication: header */
  while( fossil_isspace(zAuth[0]) ) zAuth++;  /* Skip leading whitespace */
  if( strncmp(zAuth, "Basic ", 6)!=0 ) return 0;  /* Fail: Not Basic Authentication */

  /* Parse out the username and password, separated by a ":" */
  zAuth += 6;
  while( fossil_isspace(zAuth[0]) ) zAuth++;
  zDecode = decode64(zAuth, &nDecode);

  for(i=0; zDecode[i] && zDecode[i]!=':'; i++){}
  if( zDecode[i] ){
    zDecode[i] = 0;
    zUsername = zDecode;
    zPasswd = &zDecode[i+1];

    /* Attempting to log in as the user provided by HTTP
    ** basic auth
    */
    uid = login_search_uid(zUsername, zPasswd);
    if( uid>0 ){
      record_login_attempt(zUsername, zIpAddr, 1);
    }else{
      record_login_attempt(zUsername, zIpAddr, 0);

      /* The user attempted to login specifically with HTTP basic
      ** auth, but provided invalid credentials. Inform them of
      ** the failed login attempt via 401.
      */
      cgi_set_status(401, "Unauthorized");
      cgi_reply();
      fossil_exit(0);
    }
  }
  fossil_free(zDecode);
  return uid;
}

/*
** This routine examines the login cookie to see if it exists and
** is valid.  If the login cookie checks out, it then sets global
** variables appropriately.
**
**    g.userUid      Database USER.UID value.  Might be -1 for "nobody"
**    g.zLogin       Database USER.LOGIN value.  NULL for user "nobody"
................................................................................
  if( uid==0 ){
    const char *zRemoteUser = P("REMOTE_USER");
    if( zRemoteUser && db_get_boolean("remote_user_ok",0) ){
      uid = db_int(0, "SELECT uid FROM user WHERE login=%Q"
                      " AND length(cap)>0 AND length(pw)>0", zRemoteUser);
    }
  }

  /* If the request didn't provide a login cookie or the login cookie didn't
  ** match a known valid user, check the HTTP "Authorization" header and
  ** see if those credentials are valid for a known user.
  */
  if( uid==0 && db_get_boolean("http_authentication_ok",0) ){
    uid = logic_basic_authentication(zIpAddr);
  }

  /* If no user found yet, try to log in as "nobody" */
  if( uid==0 ){
    uid = db_int(0, "SELECT uid FROM user WHERE login='nobody'");
    if( uid==0 ){
      /* If there is no user "nobody", then make one up - with no privileges */
      uid = -1;

Changes to src/main.c.

48
49
50
51
52
53
54
55





56
57
58
59
60
61
62
...
133
134
135
136
137
138
139

140
141
142
143
144
145
146
...
928
929
930
931
932
933
934



935
936
937
938
939
940
941
....
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
....
2012
2013
2014
2015
2016
2017
2018


2019
2020
2021
2022
2023
2024
2025
#endif
#ifdef FOSSIL_ENABLE_JSON
#  include "cson_amalgamation.h" /* JSON API. */
#  include "json_detail.h"
#endif

/*
** Size of a UUID in characters





*/
#define UUID_SIZE 40

/*
** Maximum number of auxiliary parameters on reports
*/
#define MX_AUX  5
................................................................................
  char *zRepositoryOption; /* Most recent cached repository option value */
  char *zRepositoryName;  /* Name of the repository database file */
  char *zLocalDbName;     /* Name of the local database file */
  char *zOpenRevision;    /* Check-in version to use during database open */
  int localOpen;          /* True if the local database is open */
  char *zLocalRoot;       /* The directory holding the  local database */
  int minPrefix;          /* Number of digits needed for a distinct UUID */

  int fNoDirSymlinks;     /* True if --no-dir-symlinks flag is present */
  int fSqlTrace;          /* True if --sqltrace flag is present */
  int fSqlStats;          /* True if --sqltrace or --sqlstats are present */
  int fSqlPrint;          /* True if -sqlprint flag is present */
  int fQuiet;             /* True if -quiet flag is present */
  int fJail;              /* True if running with a chroot jail */
  int fHttpTrace;         /* Trace outbound HTTP requests */
................................................................................
               __DATE__, __TIME__, COMPILER_NAME, sizeof(void*)*8);
  blob_appendf(pOut, "Schema version %s\n", AUX_SCHEMA_MAX);
#if defined(FOSSIL_ENABLE_MINIZ)
  blob_appendf(pOut, "miniz %s, loaded %s\n", MZ_VERSION, mz_version());
#else
  blob_appendf(pOut, "zlib %s, loaded %s\n", ZLIB_VERSION, zlibVersion());
#endif



#if defined(FOSSIL_ENABLE_SSL)
  blob_appendf(pOut, "SSL (%s)\n", SSLeay_version(SSLEAY_VERSION));
#endif
#if defined(FOSSIL_HAVE_FUSEFS)
  blob_appendf(pOut, "libfuse %s, loaded %s\n", fusefs_inc_version(),
               fusefs_lib_version());
#endif
................................................................................
** Open the repository to be served if it is known.  If g.argv[arg] is
** a directory full of repositories, then set g.zRepositoryName to
** the name of that directory and the specific repository will be
** opened later by process_one_web_page() based on the content of
** the PATH_INFO variable.
**
** If the fCreate flag is set, then create the repository if it
** does not already exist.
*/
static void find_server_repository(int arg, int fCreate){
  if( g.argc<=arg ){
    db_must_be_within_tree();
  }else{
    const char *zRepo = g.argv[arg];
    int isDir = file_isdir(zRepo);
................................................................................
      file_simplify_name(g.zRepositoryName, -1, 0);
    }else{
      if( isDir==0 && fCreate ){
        const char *zPassword;
        db_create_repository(zRepo);
        db_open_repository(zRepo);
        db_begin_transaction();


        db_initial_setup(0, "now", g.zLogin);
        db_end_transaction(0);
        fossil_print("project-id: %s\n", db_get("project-code", 0));
        fossil_print("server-id:  %s\n", db_get("server-code", 0));
        zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
        fossil_print("admin-user: %s (initial password is \"%s\")\n",
                     g.zLogin, zPassword);







|
>
>
>
>
>







 







>







 







>
>
>







 







|







 







>
>







48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
...
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
...
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
....
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
....
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
#endif
#ifdef FOSSIL_ENABLE_JSON
#  include "cson_amalgamation.h" /* JSON API. */
#  include "json_detail.h"
#endif

/*
** Size of a UUID in characters.   A UUID is a randomly generated
** lower-case hexadecimal number used to identify tickets.
**
** In Fossil 1.x, UUID also referred to a SHA1 artifact hash.  But that
** usage is now obsolete.  The term UUID should now mean only a very large
** random number used as a unique identifier for tickets or other objects.
*/
#define UUID_SIZE 40

/*
** Maximum number of auxiliary parameters on reports
*/
#define MX_AUX  5
................................................................................
  char *zRepositoryOption; /* Most recent cached repository option value */
  char *zRepositoryName;  /* Name of the repository database file */
  char *zLocalDbName;     /* Name of the local database file */
  char *zOpenRevision;    /* Check-in version to use during database open */
  int localOpen;          /* True if the local database is open */
  char *zLocalRoot;       /* The directory holding the  local database */
  int minPrefix;          /* Number of digits needed for a distinct UUID */
  int eHashPolicy;        /* Current hash policy.  One of HPOLICY_* */
  int fNoDirSymlinks;     /* True if --no-dir-symlinks flag is present */
  int fSqlTrace;          /* True if --sqltrace flag is present */
  int fSqlStats;          /* True if --sqltrace or --sqlstats are present */
  int fSqlPrint;          /* True if -sqlprint flag is present */
  int fQuiet;             /* True if -quiet flag is present */
  int fJail;              /* True if running with a chroot jail */
  int fHttpTrace;         /* Trace outbound HTTP requests */
................................................................................
               __DATE__, __TIME__, COMPILER_NAME, sizeof(void*)*8);
  blob_appendf(pOut, "Schema version %s\n", AUX_SCHEMA_MAX);
#if defined(FOSSIL_ENABLE_MINIZ)
  blob_appendf(pOut, "miniz %s, loaded %s\n", MZ_VERSION, mz_version());
#else
  blob_appendf(pOut, "zlib %s, loaded %s\n", ZLIB_VERSION, zlibVersion());
#endif
#if FOSSIL_HARDENED_SHA1
  blob_appendf(pOut, "hardened-SHA1 by Marc Stevens and Dan Shumow\n");
#endif
#if defined(FOSSIL_ENABLE_SSL)
  blob_appendf(pOut, "SSL (%s)\n", SSLeay_version(SSLEAY_VERSION));
#endif
#if defined(FOSSIL_HAVE_FUSEFS)
  blob_appendf(pOut, "libfuse %s, loaded %s\n", fusefs_inc_version(),
               fusefs_lib_version());
#endif
................................................................................
** Open the repository to be served if it is known.  If g.argv[arg] is
** a directory full of repositories, then set g.zRepositoryName to
** the name of that directory and the specific repository will be
** opened later by process_one_web_page() based on the content of
** the PATH_INFO variable.
**
** If the fCreate flag is set, then create the repository if it
** does not already exist. Always use "auto" hash-policy in this case.
*/
static void find_server_repository(int arg, int fCreate){
  if( g.argc<=arg ){
    db_must_be_within_tree();
  }else{
    const char *zRepo = g.argv[arg];
    int isDir = file_isdir(zRepo);
................................................................................
      file_simplify_name(g.zRepositoryName, -1, 0);
    }else{
      if( isDir==0 && fCreate ){
        const char *zPassword;
        db_create_repository(zRepo);
        db_open_repository(zRepo);
        db_begin_transaction();
        g.eHashPolicy = HPOLICY_AUTO;
        db_set_int("hash-policy", HPOLICY_AUTO, 0);
        db_initial_setup(0, "now", g.zLogin);
        db_end_transaction(0);
        fossil_print("project-id: %s\n", db_get("project-code", 0));
        fossil_print("server-id:  %s\n", db_get("server-code", 0));
        zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
        fossil_print("admin-user: %s (initial password is \"%s\")\n",
                     g.zLogin, zPassword);

Changes to src/main.mk.

50
51
52
53
54
55
56

57
58
59
60
61
62
63
...
100
101
102
103
104
105
106


107
108
109
110
111
112
113
...
225
226
227
228
229
230
231

232
233
234
235
236
237
238
...
275
276
277
278
279
280
281


282
283
284
285
286
287
288
...
349
350
351
352
353
354
355

356
357
358
359
360
361
362
...
399
400
401
402
403
404
405


406
407
408
409
410
411
412
...
634
635
636
637
638
639
640

641
642
643
644
645
646
647
...
684
685
686
687
688
689
690


691
692
693
694
695
696
697
....
1035
1036
1037
1038
1039
1040
1041








1042
1043
1044
1045
1046
1047
1048
....
1435
1436
1437
1438
1439
1440
1441
















1442
1443
1444
1445
1446
1447
1448
  $(SRCDIR)/finfo.c \
  $(SRCDIR)/foci.c \
  $(SRCDIR)/fshell.c \
  $(SRCDIR)/fusefs.c \
  $(SRCDIR)/glob.c \
  $(SRCDIR)/graph.c \
  $(SRCDIR)/gzip.c \

  $(SRCDIR)/http.c \
  $(SRCDIR)/http_socket.c \
  $(SRCDIR)/http_ssl.c \
  $(SRCDIR)/http_transport.c \
  $(SRCDIR)/import.c \
  $(SRCDIR)/info.c \
  $(SRCDIR)/json.c \
................................................................................
  $(SRCDIR)/regexp.c \
  $(SRCDIR)/report.c \
  $(SRCDIR)/rss.c \
  $(SRCDIR)/schema.c \
  $(SRCDIR)/search.c \
  $(SRCDIR)/setup.c \
  $(SRCDIR)/sha1.c \


  $(SRCDIR)/shun.c \
  $(SRCDIR)/sitemap.c \
  $(SRCDIR)/skins.c \
  $(SRCDIR)/sqlcmd.c \
  $(SRCDIR)/stash.c \
  $(SRCDIR)/stat.c \
  $(SRCDIR)/statrep.c \
................................................................................
  $(OBJDIR)/finfo_.c \
  $(OBJDIR)/foci_.c \
  $(OBJDIR)/fshell_.c \
  $(OBJDIR)/fusefs_.c \
  $(OBJDIR)/glob_.c \
  $(OBJDIR)/graph_.c \
  $(OBJDIR)/gzip_.c \

  $(OBJDIR)/http_.c \
  $(OBJDIR)/http_socket_.c \
  $(OBJDIR)/http_ssl_.c \
  $(OBJDIR)/http_transport_.c \
  $(OBJDIR)/import_.c \
  $(OBJDIR)/info_.c \
  $(OBJDIR)/json_.c \
................................................................................
  $(OBJDIR)/regexp_.c \
  $(OBJDIR)/report_.c \
  $(OBJDIR)/rss_.c \
  $(OBJDIR)/schema_.c \
  $(OBJDIR)/search_.c \
  $(OBJDIR)/setup_.c \
  $(OBJDIR)/sha1_.c \


  $(OBJDIR)/shun_.c \
  $(OBJDIR)/sitemap_.c \
  $(OBJDIR)/skins_.c \
  $(OBJDIR)/sqlcmd_.c \
  $(OBJDIR)/stash_.c \
  $(OBJDIR)/stat_.c \
  $(OBJDIR)/statrep_.c \
................................................................................
 $(OBJDIR)/finfo.o \
 $(OBJDIR)/foci.o \
 $(OBJDIR)/fshell.o \
 $(OBJDIR)/fusefs.o \
 $(OBJDIR)/glob.o \
 $(OBJDIR)/graph.o \
 $(OBJDIR)/gzip.o \

 $(OBJDIR)/http.o \
 $(OBJDIR)/http_socket.o \
 $(OBJDIR)/http_ssl.o \
 $(OBJDIR)/http_transport.o \
 $(OBJDIR)/import.o \
 $(OBJDIR)/info.o \
 $(OBJDIR)/json.o \
................................................................................
 $(OBJDIR)/regexp.o \
 $(OBJDIR)/report.o \
 $(OBJDIR)/rss.o \
 $(OBJDIR)/schema.o \
 $(OBJDIR)/search.o \
 $(OBJDIR)/setup.o \
 $(OBJDIR)/sha1.o \


 $(OBJDIR)/shun.o \
 $(OBJDIR)/sitemap.o \
 $(OBJDIR)/skins.o \
 $(OBJDIR)/sqlcmd.o \
 $(OBJDIR)/stash.o \
 $(OBJDIR)/stat.o \
 $(OBJDIR)/statrep.o \
................................................................................
	$(OBJDIR)/finfo_.c:$(OBJDIR)/finfo.h \
	$(OBJDIR)/foci_.c:$(OBJDIR)/foci.h \
	$(OBJDIR)/fshell_.c:$(OBJDIR)/fshell.h \
	$(OBJDIR)/fusefs_.c:$(OBJDIR)/fusefs.h \
	$(OBJDIR)/glob_.c:$(OBJDIR)/glob.h \
	$(OBJDIR)/graph_.c:$(OBJDIR)/graph.h \
	$(OBJDIR)/gzip_.c:$(OBJDIR)/gzip.h \

	$(OBJDIR)/http_.c:$(OBJDIR)/http.h \
	$(OBJDIR)/http_socket_.c:$(OBJDIR)/http_socket.h \
	$(OBJDIR)/http_ssl_.c:$(OBJDIR)/http_ssl.h \
	$(OBJDIR)/http_transport_.c:$(OBJDIR)/http_transport.h \
	$(OBJDIR)/import_.c:$(OBJDIR)/import.h \
	$(OBJDIR)/info_.c:$(OBJDIR)/info.h \
	$(OBJDIR)/json_.c:$(OBJDIR)/json.h \
................................................................................
	$(OBJDIR)/regexp_.c:$(OBJDIR)/regexp.h \
	$(OBJDIR)/report_.c:$(OBJDIR)/report.h \
	$(OBJDIR)/rss_.c:$(OBJDIR)/rss.h \
	$(OBJDIR)/schema_.c:$(OBJDIR)/schema.h \
	$(OBJDIR)/search_.c:$(OBJDIR)/search.h \
	$(OBJDIR)/setup_.c:$(OBJDIR)/setup.h \
	$(OBJDIR)/sha1_.c:$(OBJDIR)/sha1.h \


	$(OBJDIR)/shun_.c:$(OBJDIR)/shun.h \
	$(OBJDIR)/sitemap_.c:$(OBJDIR)/sitemap.h \
	$(OBJDIR)/skins_.c:$(OBJDIR)/skins.h \
	$(OBJDIR)/sqlcmd_.c:$(OBJDIR)/sqlcmd.h \
	$(OBJDIR)/stash_.c:$(OBJDIR)/stash.h \
	$(OBJDIR)/stat_.c:$(OBJDIR)/stat.h \
	$(OBJDIR)/statrep_.c:$(OBJDIR)/statrep.h \
................................................................................
$(OBJDIR)/gzip_.c:	$(SRCDIR)/gzip.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/gzip.c >$@

$(OBJDIR)/gzip.o:	$(OBJDIR)/gzip_.c $(OBJDIR)/gzip.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/gzip.o -c $(OBJDIR)/gzip_.c

$(OBJDIR)/gzip.h:	$(OBJDIR)/headers









$(OBJDIR)/http_.c:	$(SRCDIR)/http.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/http.c >$@

$(OBJDIR)/http.o:	$(OBJDIR)/http_.c $(OBJDIR)/http.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/http.o -c $(OBJDIR)/http_.c

................................................................................
$(OBJDIR)/sha1_.c:	$(SRCDIR)/sha1.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/sha1.c >$@

$(OBJDIR)/sha1.o:	$(OBJDIR)/sha1_.c $(OBJDIR)/sha1.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/sha1.o -c $(OBJDIR)/sha1_.c

$(OBJDIR)/sha1.h:	$(OBJDIR)/headers

















$(OBJDIR)/shun_.c:	$(SRCDIR)/shun.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/shun.c >$@

$(OBJDIR)/shun.o:	$(OBJDIR)/shun_.c $(OBJDIR)/shun.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/shun.o -c $(OBJDIR)/shun_.c








>







 







>
>







 







>







 







>
>







 







>







 







>
>







 







>







 







>
>







 







>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
...
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
...
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
...
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
...
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
...
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
...
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
....
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
....
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
  $(SRCDIR)/finfo.c \
  $(SRCDIR)/foci.c \
  $(SRCDIR)/fshell.c \
  $(SRCDIR)/fusefs.c \
  $(SRCDIR)/glob.c \
  $(SRCDIR)/graph.c \
  $(SRCDIR)/gzip.c \
  $(SRCDIR)/hname.c \
  $(SRCDIR)/http.c \
  $(SRCDIR)/http_socket.c \
  $(SRCDIR)/http_ssl.c \
  $(SRCDIR)/http_transport.c \
  $(SRCDIR)/import.c \
  $(SRCDIR)/info.c \
  $(SRCDIR)/json.c \
................................................................................
  $(SRCDIR)/regexp.c \
  $(SRCDIR)/report.c \
  $(SRCDIR)/rss.c \
  $(SRCDIR)/schema.c \
  $(SRCDIR)/search.c \
  $(SRCDIR)/setup.c \
  $(SRCDIR)/sha1.c \
  $(SRCDIR)/sha1hard.c \
  $(SRCDIR)/sha3.c \
  $(SRCDIR)/shun.c \
  $(SRCDIR)/sitemap.c \
  $(SRCDIR)/skins.c \
  $(SRCDIR)/sqlcmd.c \
  $(SRCDIR)/stash.c \
  $(SRCDIR)/stat.c \
  $(SRCDIR)/statrep.c \
................................................................................
  $(OBJDIR)/finfo_.c \
  $(OBJDIR)/foci_.c \
  $(OBJDIR)/fshell_.c \
  $(OBJDIR)/fusefs_.c \
  $(OBJDIR)/glob_.c \
  $(OBJDIR)/graph_.c \
  $(OBJDIR)/gzip_.c \
  $(OBJDIR)/hname_.c \
  $(OBJDIR)/http_.c \
  $(OBJDIR)/http_socket_.c \
  $(OBJDIR)/http_ssl_.c \
  $(OBJDIR)/http_transport_.c \
  $(OBJDIR)/import_.c \
  $(OBJDIR)/info_.c \
  $(OBJDIR)/json_.c \
................................................................................
  $(OBJDIR)/regexp_.c \
  $(OBJDIR)/report_.c \
  $(OBJDIR)/rss_.c \
  $(OBJDIR)/schema_.c \
  $(OBJDIR)/search_.c \
  $(OBJDIR)/setup_.c \
  $(OBJDIR)/sha1_.c \
  $(OBJDIR)/sha1hard_.c \
  $(OBJDIR)/sha3_.c \
  $(OBJDIR)/shun_.c \
  $(OBJDIR)/sitemap_.c \
  $(OBJDIR)/skins_.c \
  $(OBJDIR)/sqlcmd_.c \
  $(OBJDIR)/stash_.c \
  $(OBJDIR)/stat_.c \
  $(OBJDIR)/statrep_.c \
................................................................................
 $(OBJDIR)/finfo.o \
 $(OBJDIR)/foci.o \
 $(OBJDIR)/fshell.o \
 $(OBJDIR)/fusefs.o \
 $(OBJDIR)/glob.o \
 $(OBJDIR)/graph.o \
 $(OBJDIR)/gzip.o \
 $(OBJDIR)/hname.o \
 $(OBJDIR)/http.o \
 $(OBJDIR)/http_socket.o \
 $(OBJDIR)/http_ssl.o \
 $(OBJDIR)/http_transport.o \
 $(OBJDIR)/import.o \
 $(OBJDIR)/info.o \
 $(OBJDIR)/json.o \
................................................................................
 $(OBJDIR)/regexp.o \
 $(OBJDIR)/report.o \
 $(OBJDIR)/rss.o \
 $(OBJDIR)/schema.o \
 $(OBJDIR)/search.o \
 $(OBJDIR)/setup.o \
 $(OBJDIR)/sha1.o \
 $(OBJDIR)/sha1hard.o \
 $(OBJDIR)/sha3.o \
 $(OBJDIR)/shun.o \
 $(OBJDIR)/sitemap.o \
 $(OBJDIR)/skins.o \
 $(OBJDIR)/sqlcmd.o \
 $(OBJDIR)/stash.o \
 $(OBJDIR)/stat.o \
 $(OBJDIR)/statrep.o \
................................................................................
	$(OBJDIR)/finfo_.c:$(OBJDIR)/finfo.h \
	$(OBJDIR)/foci_.c:$(OBJDIR)/foci.h \
	$(OBJDIR)/fshell_.c:$(OBJDIR)/fshell.h \
	$(OBJDIR)/fusefs_.c:$(OBJDIR)/fusefs.h \
	$(OBJDIR)/glob_.c:$(OBJDIR)/glob.h \
	$(OBJDIR)/graph_.c:$(OBJDIR)/graph.h \
	$(OBJDIR)/gzip_.c:$(OBJDIR)/gzip.h \
	$(OBJDIR)/hname_.c:$(OBJDIR)/hname.h \
	$(OBJDIR)/http_.c:$(OBJDIR)/http.h \
	$(OBJDIR)/http_socket_.c:$(OBJDIR)/http_socket.h \
	$(OBJDIR)/http_ssl_.c:$(OBJDIR)/http_ssl.h \
	$(OBJDIR)/http_transport_.c:$(OBJDIR)/http_transport.h \
	$(OBJDIR)/import_.c:$(OBJDIR)/import.h \
	$(OBJDIR)/info_.c:$(OBJDIR)/info.h \
	$(OBJDIR)/json_.c:$(OBJDIR)/json.h \
................................................................................
	$(OBJDIR)/regexp_.c:$(OBJDIR)/regexp.h \
	$(OBJDIR)/report_.c:$(OBJDIR)/report.h \
	$(OBJDIR)/rss_.c:$(OBJDIR)/rss.h \
	$(OBJDIR)/schema_.c:$(OBJDIR)/schema.h \
	$(OBJDIR)/search_.c:$(OBJDIR)/search.h \
	$(OBJDIR)/setup_.c:$(OBJDIR)/setup.h \
	$(OBJDIR)/sha1_.c:$(OBJDIR)/sha1.h \
	$(OBJDIR)/sha1hard_.c:$(OBJDIR)/sha1hard.h \
	$(OBJDIR)/sha3_.c:$(OBJDIR)/sha3.h \
	$(OBJDIR)/shun_.c:$(OBJDIR)/shun.h \
	$(OBJDIR)/sitemap_.c:$(OBJDIR)/sitemap.h \
	$(OBJDIR)/skins_.c:$(OBJDIR)/skins.h \
	$(OBJDIR)/sqlcmd_.c:$(OBJDIR)/sqlcmd.h \
	$(OBJDIR)/stash_.c:$(OBJDIR)/stash.h \
	$(OBJDIR)/stat_.c:$(OBJDIR)/stat.h \
	$(OBJDIR)/statrep_.c:$(OBJDIR)/statrep.h \
................................................................................
$(OBJDIR)/gzip_.c:	$(SRCDIR)/gzip.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/gzip.c >$@

$(OBJDIR)/gzip.o:	$(OBJDIR)/gzip_.c $(OBJDIR)/gzip.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/gzip.o -c $(OBJDIR)/gzip_.c

$(OBJDIR)/gzip.h:	$(OBJDIR)/headers

$(OBJDIR)/hname_.c:	$(SRCDIR)/hname.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/hname.c >$@

$(OBJDIR)/hname.o:	$(OBJDIR)/hname_.c $(OBJDIR)/hname.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/hname.o -c $(OBJDIR)/hname_.c

$(OBJDIR)/hname.h:	$(OBJDIR)/headers

$(OBJDIR)/http_.c:	$(SRCDIR)/http.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/http.c >$@

$(OBJDIR)/http.o:	$(OBJDIR)/http_.c $(OBJDIR)/http.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/http.o -c $(OBJDIR)/http_.c

................................................................................
$(OBJDIR)/sha1_.c:	$(SRCDIR)/sha1.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/sha1.c >$@

$(OBJDIR)/sha1.o:	$(OBJDIR)/sha1_.c $(OBJDIR)/sha1.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/sha1.o -c $(OBJDIR)/sha1_.c

$(OBJDIR)/sha1.h:	$(OBJDIR)/headers

$(OBJDIR)/sha1hard_.c:	$(SRCDIR)/sha1hard.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/sha1hard.c >$@

$(OBJDIR)/sha1hard.o:	$(OBJDIR)/sha1hard_.c $(OBJDIR)/sha1hard.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/sha1hard.o -c $(OBJDIR)/sha1hard_.c

$(OBJDIR)/sha1hard.h:	$(OBJDIR)/headers

$(OBJDIR)/sha3_.c:	$(SRCDIR)/sha3.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/sha3.c >$@

$(OBJDIR)/sha3.o:	$(OBJDIR)/sha3_.c $(OBJDIR)/sha3.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/sha3.o -c $(OBJDIR)/sha3_.c

$(OBJDIR)/sha3.h:	$(OBJDIR)/headers

$(OBJDIR)/shun_.c:	$(SRCDIR)/shun.c $(OBJDIR)/translate
	$(OBJDIR)/translate $(SRCDIR)/shun.c >$@

$(OBJDIR)/shun.o:	$(OBJDIR)/shun_.c $(OBJDIR)/shun.h $(SRCDIR)/config.h
	$(XTCC) -o $(OBJDIR)/shun.o -c $(OBJDIR)/shun_.c

Changes to src/makeheaders.c.

1015
1016
1017
1018
1019
1020
1021
1022



1023
1024
1025
1026
1027
1028
1029
....
2179
2180
2181
2182
2183
2184
2185


2186
2187
2188
2189
2190
2191
2192
  while( 1 ){
    nErr += GetToken(pIn,pToken);
    /* printf("%04d: Type=%d nIf=%d [%.*s]\n",
       pToken->nLine,pToken->eType,nIf,pToken->nText,
       pToken->eType!=TT_Space ? pToken->zText : "<space>"); */
    pToken->pComment = blockComment;
    switch( pToken->eType ){
      case TT_Comment:



      case TT_Space:
        break;

      case TT_BlockComment:
        if( doc_flag ){
          blockComment = SafeMalloc( sizeof(Token) );
          *blockComment = *pToken;
................................................................................
    if( *zArg==0 || *zArg=='\n' ){ return 0; }
    nArg = pToken->nText + (int)(pToken->zText - zArg);
    if( nArg==9 && strncmp(zArg,"INTERFACE",9)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Interface);
    }else if( nArg==16 && strncmp(zArg,"EXPORT_INTERFACE",16)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Export);
    }else if( nArg==15 && strncmp(zArg,"LOCAL_INTERFACE",15)==0 ){


      PushIfMacro(0,0,0,pToken->nLine,PS_Local);
    }else{
      PushIfMacro(0,zArg,nArg,pToken->nLine,0);
    }
  }else if( nCmd==5 && strncmp(zCmd,"ifdef",5)==0 ){
    /*
    ** Push an #ifdef.







|
>
>
>







 







>
>







1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
....
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
  while( 1 ){
    nErr += GetToken(pIn,pToken);
    /* printf("%04d: Type=%d nIf=%d [%.*s]\n",
       pToken->nLine,pToken->eType,nIf,pToken->nText,
       pToken->eType!=TT_Space ? pToken->zText : "<space>"); */
    pToken->pComment = blockComment;
    switch( pToken->eType ){
      case TT_Comment:          /*0123456789 12345678 */
       if( strncmp(pToken->zText, "/*MAKEHEADERS-STOP", 18)==0 ) return nErr;
       break;

      case TT_Space:
        break;

      case TT_BlockComment:
        if( doc_flag ){
          blockComment = SafeMalloc( sizeof(Token) );
          *blockComment = *pToken;
................................................................................
    if( *zArg==0 || *zArg=='\n' ){ return 0; }
    nArg = pToken->nText + (int)(pToken->zText - zArg);
    if( nArg==9 && strncmp(zArg,"INTERFACE",9)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Interface);
    }else if( nArg==16 && strncmp(zArg,"EXPORT_INTERFACE",16)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Export);
    }else if( nArg==15 && strncmp(zArg,"LOCAL_INTERFACE",15)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Local);
    }else if( nArg==15 && strncmp(zArg,"MAKEHEADERS_STOPLOCAL_INTERFACE",15)==0 ){
      PushIfMacro(0,0,0,pToken->nLine,PS_Local);
    }else{
      PushIfMacro(0,zArg,nArg,pToken->nLine,0);
    }
  }else if( nCmd==5 && strncmp(zCmd,"ifdef",5)==0 ){
    /*
    ** Push an #ifdef.

Changes to src/makemake.tcl.

56
57
58
59
60
61
62

63
64
65
66
67
68
69
...
105
106
107
108
109
110
111


112
113
114
115
116
117
118
  finfo
  foci
  fshell
  fusefs
  glob
  graph
  gzip

  http
  http_socket
  http_transport
  import
  info
  json
  json_artifact
................................................................................
  regexp
  report
  rss
  schema
  search
  setup
  sha1


  shun
  sitemap
  skins
  sqlcmd
  stash
  stat
  statrep







>







 







>
>







56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
...
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
  finfo
  foci
  fshell
  fusefs
  glob
  graph
  gzip
  hname
  http
  http_socket
  http_transport
  import
  info
  json
  json_artifact
................................................................................
  regexp
  report
  rss
  schema
  search
  setup
  sha1
  sha1hard
  sha3
  shun
  sitemap
  skins
  sqlcmd
  stash
  stat
  statrep

Changes to src/manifest.c.

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
..
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
...
325
326
327
328
329
330
331
332
333
334

335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
...
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
...
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
...
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
...
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
...
543
544
545
546
547
548
549
550
551

552
553
554
555
556
557
558
...
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650

651
652
653
654
655
656
657
...
681
682
683
684
685
686
687
688
689

690
691
692
693
694
695
696
...
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
...
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
...
949
950
951
952
953
954
955
956
957
958
959
960
961
962



963
964
965
966
967
968
969
....
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
....
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
....
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627

1628
1629
1630
1631
1632
1633
1634
1635




1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651

1652
1653
1654
1655
1656
1657
1658
....
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
#define MC_NO_ERRORS      2  /*  do not issue errors for a bad parse */

/*
** A single F-card within a manifest
*/
struct ManifestFile {
  char *zName;           /* Name of a file */
  char *zUuid;           /* UUID of the file */
  char *zPerm;           /* File permissions */
  char *zPrior;          /* Prior name if the name was changed */
};


/*
** A parsed manifest or cluster.
................................................................................
  double rDate;         /* Date and time from D card.  0.0 if no D card. */
  char *zUser;          /* Name of the user from the U card. */
  char *zRepoCksum;     /* MD5 checksum of the baseline content.  R card. */
  char *zWiki;          /* Text of the wiki page.  W card. */
  char *zWikiTitle;     /* Name of the wiki page. L card. */
  char *zMimetype;      /* Mime type of wiki or comment text.  N card.  */
  double rEventDate;    /* Date of an event.  E card. */
  char *zEventId;       /* UUID for an event.  E card. */
  char *zTicketUuid;    /* UUID for a ticket. K card. */
  char *zAttachName;    /* Filename of an attachment. A card. */
  char *zAttachSrc;     /* UUID of document being attached. A card. */
  char *zAttachTarget;  /* Ticket or wiki that attachment applies to.  A card */
  int nFile;            /* Number of F cards */
  int nFileAlloc;       /* Slots allocated in aFile[] */
  int iFile;            /* Index of current file in iterator */
  ManifestFile *aFile;  /* One entry for each F-card */
  int nParent;          /* Number of parents. */
  int nParentAlloc;     /* Slots allocated in azParent[] */
  char **azParent;      /* UUIDs of parents.  One for each P card argument */
  int nCherrypick;      /* Number of entries in aCherrypick[] */
  struct {
    char *zCPTarget;    /* UUID of cherry-picked version w/ +|- prefix */
    char *zCPBase;      /* UUID of cherry-pick baseline. NULL for singletons */
  } *aCherrypick;
  int nCChild;          /* Number of cluster children */
  int nCChildAlloc;     /* Number of closts allocated in azCChild[] */
  char **azCChild;      /* UUIDs of referenced objects in a cluster. M cards */
  int nTag;             /* Number of T Cards */
  int nTagAlloc;        /* Slots allocated in aTag[] */
  struct TagType {
    char *zName;           /* Name of the tag */
    char *zUuid;           /* UUID that the tag is applied to */
    char *zValue;          /* Value if the tag is really a property */
  } *aTag;              /* One for each T card */
  int nField;           /* Number of J cards */
  int nFieldAlloc;      /* Slots allocated in aField[] */
  struct {
    char *zName;           /* Key or field name */
    char *zValue;          /* Value of the field */
................................................................................
/*
** Parse a blob into a Manifest object.  The Manifest object
** takes over the input blob and will free it when the
** Manifest object is freed.  Zeros are inserted into the blob
** as string terminators so that blob should not be used again.
**
** Return a pointer to an allocated Manifest object if the content
** really is a control file of some kind.  This object needs to be
** freed by a subsequent call to manifest_destroy().  Return NULL
** if there are syntax errors.

**
** This routine is strict about the format of a control file.
** The format must match exactly or else it is rejected.  This
** rule minimizes the risk that a content file will be mistaken
** for a control file simply because they look the same.
**
** The pContent is reset.  If a pointer is returned, then pContent will
** be reset when the Manifest object is cleared.  If NULL is
** returned then the Manifest object is cleared automatically
** and pContent is reset before the return.
**
** The entire file can be PGP clear-signed.  The signature is ignored.
** The file consists of zero or more cards, one card per line.
** (Except: the content of the W card can extend of multiple lines.)
** Each card is divided into tokens by a single space character.
** The first token is a single upper-case letter which is the card type.
** The card type determines the other parameters to the card.
** Cards must occur in lexicographical order.
*/
Manifest *manifest_parse(Blob *pContent, int rid, Blob *pErr){
................................................................................
  char cPrevType = 0;
  char cType;
  char *z;
  int n;
  char *zUuid;
  int sz = 0;
  int isRepeat, hasSelfRefTag = 0;
  Blob bUuid = BLOB_INITIALIZER;
  static Bag seen;
  const char *zErr = 0;

  if( rid==0 ){
    isRepeat = 1;
  }else if( bag_find(&seen, rid) ){
    isRepeat = 1;
  }else{
    isRepeat = 0;
    bag_insert(&seen, rid);
  }

  /* Every control artifact ends with a '\n' character.  Exit early
  ** if that is not the case for this artifact.
  */
  if( !isRepeat ) g.parseCnt[0]++;
  z = blob_materialize(pContent);
  n = blob_size(pContent);
  if( n<=0 || z[n-1]!='\n' ){
    blob_reset(pContent);
................................................................................
  */
  if( verify_z_card(z, n)==2 ){
    blob_reset(pContent);
    blob_appendf(pErr, "incorrect Z-card cksum");
    return 0;
  }

  /* Store the UUID (before modifying the blob) only for error
  ** reporting purposes.
  */
  sha1sum_blob(pContent, &bUuid);

  /* Allocate a Manifest object to hold the parsed control artifact.
  */
  p = fossil_malloc( sizeof(*p) );
  memset(p, 0, sizeof(*p));
  memcpy(&p->content, pContent, sizeof(p->content));
  p->rid = rid;
  blob_zero(pContent);
................................................................................
        if( zName==0 || zTarget==0 ) goto manifest_syntax_error;
        if( p->zAttachName!=0 ) goto manifest_syntax_error;
        defossilize(zName);
        if( !file_is_simple_pathname(zName, 0) ){
          SYNTAX("invalid filename on A-card");
        }
        defossilize(zTarget);
        if( (nTarget!=UUID_SIZE || !validate16(zTarget, UUID_SIZE))
           && !wiki_name_is_wellformed((const unsigned char *)zTarget) ){
          SYNTAX("invalid target on A-card");
        }
        if( zSrc && (nSrc!=UUID_SIZE || !validate16(zSrc, UUID_SIZE)) ){
          SYNTAX("invalid source on A-card");
        }
        p->zAttachName = (char*)file_tail(zName);
        p->zAttachSrc = zSrc;
        p->zAttachTarget = zTarget;
        break;
      }

      /*
      **    B <uuid>
      **
      ** A B-line gives the UUID for the baseline of a delta-manifest.
      */
      case 'B': {
        if( p->zBaseline ) SYNTAX("more than one B-card");
        p->zBaseline = next_token(&x, &sz);
        if( p->zBaseline==0 ) SYNTAX("missing UUID on B-card");
        if( sz!=UUID_SIZE || !validate16(p->zBaseline, UUID_SIZE) ){
          SYNTAX("invalid UUID on B-card");
        }
        break;
      }


      /*
      **     C <comment>
................................................................................
      ** is when the specific event is said to occur.
      */
      case 'E': {
        if( p->rEventDate>0.0 ) SYNTAX("more than one E-card");
        p->rEventDate = db_double(0.0,"SELECT julianday(%Q)", next_token(&x,0));
        if( p->rEventDate<=0.0 ) SYNTAX("malformed date on E-card");
        p->zEventId = next_token(&x, &sz);
        if( sz!=UUID_SIZE || !validate16(p->zEventId, UUID_SIZE) ){
          SYNTAX("malformed UUID on E-card");
        }
        break;
      }

      /*
      **     F <filename> ?<uuid>? ?<permissions>? ?<old-name>?
      **
................................................................................
        if( zName==0 ) SYNTAX("missing filename on F-card");
        defossilize(zName);
        if( !file_is_simple_pathname(zName, 0) ){
          SYNTAX("F-card filename is not a simple path");
        }
        zUuid = next_token(&x, &sz);
        if( p->zBaseline==0 || zUuid!=0 ){
          if( sz!=UUID_SIZE ) SYNTAX("F-card UUID is the wrong size");
          if( !validate16(zUuid, UUID_SIZE) ) SYNTAX("F-card UUID invalid");

        }
        zPerm = next_token(&x,0);
        zPriorName = next_token(&x,0);
        if( zPriorName ){
          defossilize(zPriorName);
          if( !file_is_simple_pathname(zPriorName, 0) ){
            SYNTAX("F-card old filename is not a simple path");
................................................................................
        if( !wiki_name_is_wellformed((const unsigned char *)p->zWikiTitle) ){
          SYNTAX("L-card has malformed wiki name");
        }
        break;
      }

      /*
      **    M <uuid>
      **
      ** An M-line identifies another artifact by its UUID.  M-lines
      ** occur in clusters only.
      */
      case 'M': {
        zUuid = next_token(&x, &sz);
        if( zUuid==0 ) SYNTAX("missing UUID on M-card");
        if( sz!=UUID_SIZE ) SYNTAX("wrong size for UUID on M-card");
        if( !validate16(zUuid, UUID_SIZE) ) SYNTAX("UUID invalid on M-card");

        if( p->nCChild>=p->nCChildAlloc ){
          p->nCChildAlloc = p->nCChildAlloc*2 + 10;
          p->azCChild = fossil_realloc(p->azCChild
                                 , p->nCChildAlloc*sizeof(p->azCChild[0]) );
        }
        i = p->nCChild++;
        p->azCChild[i] = zUuid;
................................................................................
      ** this artifact.  The first parent is the primary parent.  All
      ** others are parents by merge. Note that the initial empty
      ** check-in historically has an empty P-card, so empty P-cards
      ** must be accepted.
      */
      case 'P': {
        while( (zUuid = next_token(&x, &sz))!=0 ){
          if( sz!=UUID_SIZE ) SYNTAX("wrong size UUID on P-card");
          if( !validate16(zUuid, UUID_SIZE) )SYNTAX("invalid UUID on P-card");

          if( p->nParent>=p->nParentAlloc ){
            p->nParentAlloc = p->nParentAlloc*2 + 5;
            p->azParent = fossil_realloc(p->azParent,
                               p->nParentAlloc*sizeof(char*));
          }
          i = p->nParent++;
          p->azParent[i] = zUuid;
................................................................................
      /*
      **     Q (+|-)<uuid> ?<uuid>?
      **
      ** Specify one or a range of check-ins that are cherrypicked into
      ** this check-in ("+") or backed out of this check-in ("-").
      */
      case 'Q': {
        if( (zUuid=next_token(&x, &sz))==0 ) SYNTAX("missing UUID on Q-card");
        if( sz!=UUID_SIZE+1 ) SYNTAX("wrong size UUID on Q-card");
        if( zUuid[0]!='+' && zUuid[0]!='-' ){
          SYNTAX("Q-card does not begin with '+' or '-'");
        }
        if( !validate16(&zUuid[1], UUID_SIZE) ){
          SYNTAX("invalid UUID on Q-card");
        }
        n = p->nCherrypick;
        p->nCherrypick++;
        p->aCherrypick = fossil_realloc(p->aCherrypick,
                                 p->nCherrypick*sizeof(p->aCherrypick[0]));
        p->aCherrypick[n].zCPTarget = zUuid;
        p->aCherrypick[n].zCPBase = zUuid = next_token(&x, &sz);
        if( zUuid ){
          if( sz!=UUID_SIZE ) SYNTAX("wrong size second UUID in Q-card");
          if( !validate16(zUuid, UUID_SIZE) ){
            SYNTAX("invalid second UUID on Q-card");
          }
        }
        break;
      }

      /*
      **     R <md5sum>
      **
................................................................................
      ** Tags are not allowed in clusters.  Multiple T lines are allowed.
      */
      case 'T': {
        char *zName, *zValue;
        zName = next_token(&x, 0);
        if( zName==0 ) SYNTAX("missing name on T-card");
        zUuid = next_token(&x, &sz);
        if( zUuid==0 ) SYNTAX("missing UUID on T-card");
        zValue = next_token(&x, 0);
        if( zValue ) defossilize(zValue);
        if( sz==UUID_SIZE && validate16(zUuid, UUID_SIZE) ){
          /* A valid uuid */
          if( p->zEventId ) SYNTAX("non-self-referential T-card in event");
        }else if( sz==1 && zUuid[0]=='*' ){
          zUuid = 0;
          hasSelfRefTag = 1;
          if( p->zEventId && zName[0]!='+' ){
            SYNTAX("propagating T-card in event");
          }
        }else{
          SYNTAX("malformed UUID on T-card");
        }
        defossilize(zName);
        if( zName[0]!='-' && zName[0]!='+' && zName[0]!='*' ){
          SYNTAX("T-card name does not begin with '-', '+', or '*'");
        }
        if( validate16(&zName[1], strlen(&zName[1])) ){
          /* Do not allow tags whose names look like UUIDs */
          SYNTAX("T-card name looks like a UUID");
        }
        if( p->nTag>=p->nTagAlloc ){
          p->nTagAlloc = p->nTagAlloc*2 + 10;
          p->aTag = fossil_realloc(p->aTag, p->nTagAlloc*sizeof(p->aTag[0]) );
        }
        i = p->nTag++;
        p->aTag[i].zName = zName;
................................................................................
    if( p->rDate<=0.0 ) SYNTAX("missing date on control");
    if( p->zMimetype ) SYNTAX("N-card in control");
    if( !seenZ ) SYNTAX("missing Z-card on control");
    p->type = CFTYPE_CONTROL;
  }
  md5sum_init();
  if( !isRepeat ) g.parseCnt[p->type]++;
  blob_reset(&bUuid);
  return p;

manifest_syntax_error:
  if(bUuid.nUsed){
    blob_appendf(pErr, "manifest [%.40s] ", blob_str(&bUuid));
    blob_reset(&bUuid);



  }
  if( zErr ){
    blob_appendf(pErr, "line %d: %s", lineNo, zErr);
  }else{
    blob_appendf(pErr, "unknown error on line %d", lineNo);
  }
  md5sum_init();
................................................................................
** An mlink entry is always created if isPrimary is true.  But if
** isPrimary is false (meaning that pmid is a merge parent of mid)
** then the mlink entry is only created if there is already an mlink
** from primary parent for the same file.
*/
static void add_one_mlink(
  int pmid,                 /* The parent manifest */
  const char *zFromUuid,    /* UUID for content in parent */
  int mid,                  /* The record ID of the manifest */
  const char *zToUuid,      /* UUID for content in child */
  const char *zFilename,    /* Filename */
  const char *zPrior,       /* Previous filename. NULL if unchanged */
  int isPublic,             /* True if mid is not a private manifest */
  int isPrimary,            /* pmid is the primary parent of mid */
  int mperm                 /* 1: exec, 2: symlink */
){
  int fnid, pfnid, pid, fid;
................................................................................
      add_mlink(pmid, 0, mid, pChild, 0);
    }
  }
}

/*
** For a check-in with RID "rid" that has nParent parent check-ins given
** by the UUIDs in azParent[], create all appropriate plink and mlink table
** entries.
**
** The primary parent is the first UUID on the azParent[] list.
**
** Return the RID of the primary parent.
*/
static int manifest_add_checkin_linkages(
  int rid,                   /* The RID of the check-in */
  Manifest *p,               /* Manifest for this check-in */
  int nParent,               /* Number of parents for this check-in */
  char **azParent            /* UUIDs for each parent */
){
  int i;
  int parentid = 0;
  char zBaseId[30];    /* Baseline manifest RID for deltas.  "NULL" otherwise */
  Stmt q;

  if( p->zBaseline ){
................................................................................
    }
  }
  return parentid;
}

/*
** There exists a "parent" tag against checkin rid that has value zValue.
** If value is well-formed (meaning that it is a list of UUIDs), then use
** zValue to reparent check-in rid.
*/
void manifest_reparent_checkin(int rid, const char *zValue){
  int nParent;
  char *zCopy = 0;
  char **azParent = 0;
  Manifest *p = 0;
  int i;
  int n = (int)strlen(zValue);
  nParent = (n+1)/(UUID_SIZE+1);
  if( nParent*(UUID_SIZE+1) - 1 !=n ) return;

  if( nParent<1 ) return;
  zCopy = fossil_strdup(zValue);
  azParent = fossil_malloc( sizeof(azParent[0])*nParent );
  for(i=0; i<nParent; i++){
    azParent[i] = &zCopy[i*(UUID_SIZE+1)];
    if( i<nParent-1 && azParent[i][UUID_SIZE]!=' ' ) break;
    azParent[i][UUID_SIZE] = 0;
    if( !validate16(azParent[i],UUID_SIZE) ) break;




  }
  if( i==nParent
   && !db_exists("SELECT 1 FROM plink WHERE cid=%d AND pid=%d",
                 rid, uuid_to_rid(azParent[0],0))
  ){
    p = manifest_get(rid, CFTYPE_MANIFEST, 0);
  }
  if( p!=0 ){
    db_multi_exec(
       "DELETE FROM plink WHERE cid=%d;"
       "DELETE FROM mlink WHERE mid=%d;",
       rid, rid
    );
    manifest_add_checkin_linkages(rid,p,nParent,azParent);
  }
  manifest_destroy(p);

  fossil_free(azParent);
  fossil_free(zCopy);
}

/*
** Setup to do multiple manifest_crosslink() calls.
**
................................................................................
    const char *zValue;
    const char *zTagUuid;
    int branchMove = 0;
    blob_zero(&comment);
    if( p->zComment ){
      blob_appendf(&comment, " %s.", p->zComment);
    }
    /* Next loop expects tags to be sorted on UUID, so sort it. */
    qsort(p->aTag, p->nTag, sizeof(p->aTag[0]), tag_compare);
    for(i=0; i<p->nTag; i++){
      zTagUuid = p->aTag[i].zUuid;
      if( !zTagUuid ) continue;
      if( i==0 || fossil_strcmp(zTagUuid, p->aTag[i-1].zUuid)!=0 ){
        blob_appendf(&comment,
           " Edit [%!S|%S]:",







|







 







|


|







|


|
|



|




|







 







|
|
|
>

|

|
|






|
|







 







<












|







 







<
<
<
<
<







 







|



|











|




|
|
|







 







|
|







 







|
|
>







 







|

|




|
|
|
>







 







|
|
>







 







|
<



|
|







|
<
<
|
<







 







|


|
|








|






|
|







 







<



|
|
|
>
>
>







 







|

|







 







|


|







|







 







|



|



|

|
<
>
|

|
|
|
|
|
|
>
>
>
>

<
|













>







 







|







52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
..
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
...
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
...
360
361
362
363
364
365
366

367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
...
404
405
406
407
408
409
410





411
412
413
414
415
416
417
...
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
...
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
...
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
...
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
...
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
...
699
700
701
702
703
704
705
706

707
708
709
710
711
712
713
714
715
716
717
718
719


720

721
722
723
724
725
726
727
...
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
...
943
944
945
946
947
948
949

950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
....
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
....
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
....
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622

1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636

1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
....
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
#define MC_NO_ERRORS      2  /*  do not issue errors for a bad parse */

/*
** A single F-card within a manifest
*/
struct ManifestFile {
  char *zName;           /* Name of a file */
  char *zUuid;           /* Artifact hash for the file */
  char *zPerm;           /* File permissions */
  char *zPrior;          /* Prior name if the name was changed */
};


/*
** A parsed manifest or cluster.
................................................................................
  double rDate;         /* Date and time from D card.  0.0 if no D card. */
  char *zUser;          /* Name of the user from the U card. */
  char *zRepoCksum;     /* MD5 checksum of the baseline content.  R card. */
  char *zWiki;          /* Text of the wiki page.  W card. */
  char *zWikiTitle;     /* Name of the wiki page. L card. */
  char *zMimetype;      /* Mime type of wiki or comment text.  N card.  */
  double rEventDate;    /* Date of an event.  E card. */
  char *zEventId;       /* Artifact hash for an event.  E card. */
  char *zTicketUuid;    /* UUID for a ticket. K card. */
  char *zAttachName;    /* Filename of an attachment. A card. */
  char *zAttachSrc;     /* Artifact hash for document being attached. A card. */
  char *zAttachTarget;  /* Ticket or wiki that attachment applies to.  A card */
  int nFile;            /* Number of F cards */
  int nFileAlloc;       /* Slots allocated in aFile[] */
  int iFile;            /* Index of current file in iterator */
  ManifestFile *aFile;  /* One entry for each F-card */
  int nParent;          /* Number of parents. */
  int nParentAlloc;     /* Slots allocated in azParent[] */
  char **azParent;      /* Hashes of parents.  One for each P card argument */
  int nCherrypick;      /* Number of entries in aCherrypick[] */
  struct {
    char *zCPTarget;    /* Hash for cherry-picked version w/ +|- prefix */
    char *zCPBase;      /* Hash for cherry-pick baseline. NULL for singletons */
  } *aCherrypick;
  int nCChild;          /* Number of cluster children */
  int nCChildAlloc;     /* Number of closts allocated in azCChild[] */
  char **azCChild;      /* Hashes of referenced objects in a cluster. M cards */
  int nTag;             /* Number of T Cards */
  int nTagAlloc;        /* Slots allocated in aTag[] */
  struct TagType {
    char *zName;           /* Name of the tag */
    char *zUuid;           /* Hash of artifact that the tag is applied to */
    char *zValue;          /* Value if the tag is really a property */
  } *aTag;              /* One for each T card */
  int nField;           /* Number of J cards */
  int nFieldAlloc;      /* Slots allocated in aField[] */
  struct {
    char *zName;           /* Key or field name */
    char *zValue;          /* Value of the field */
................................................................................
/*
** Parse a blob into a Manifest object.  The Manifest object
** takes over the input blob and will free it when the
** Manifest object is freed.  Zeros are inserted into the blob
** as string terminators so that blob should not be used again.
**
** Return a pointer to an allocated Manifest object if the content
** really is a structural artifact of some kind.  The returned Manifest
** object needs to be freed by a subsequent call to manifest_destroy().
** Return NULL if there are syntax errors or if the input blob does
** not describe a valid structural artifact.
**
** This routine is strict about the format of a structural artifacts.
** The format must match exactly or else it is rejected.  This
** rule minimizes the risk that a content artifact will be mistaken
** for a structural artifact simply because they look the same.
**
** The pContent is reset.  If a pointer is returned, then pContent will
** be reset when the Manifest object is cleared.  If NULL is
** returned then the Manifest object is cleared automatically
** and pContent is reset before the return.
**
** The entire input blob can be PGP clear-signed.  The signature is ignored.
** The artifact consists of zero or more cards, one card per line.
** (Except: the content of the W card can extend of multiple lines.)
** Each card is divided into tokens by a single space character.
** The first token is a single upper-case letter which is the card type.
** The card type determines the other parameters to the card.
** Cards must occur in lexicographical order.
*/
Manifest *manifest_parse(Blob *pContent, int rid, Blob *pErr){
................................................................................
  char cPrevType = 0;
  char cType;
  char *z;
  int n;
  char *zUuid;
  int sz = 0;
  int isRepeat, hasSelfRefTag = 0;

  static Bag seen;
  const char *zErr = 0;

  if( rid==0 ){
    isRepeat = 1;
  }else if( bag_find(&seen, rid) ){
    isRepeat = 1;
  }else{
    isRepeat = 0;
    bag_insert(&seen, rid);
  }

  /* Every structural artifact ends with a '\n' character.  Exit early
  ** if that is not the case for this artifact.
  */
  if( !isRepeat ) g.parseCnt[0]++;
  z = blob_materialize(pContent);
  n = blob_size(pContent);
  if( n<=0 || z[n-1]!='\n' ){
    blob_reset(pContent);
................................................................................
  */
  if( verify_z_card(z, n)==2 ){
    blob_reset(pContent);
    blob_appendf(pErr, "incorrect Z-card cksum");
    return 0;
  }






  /* Allocate a Manifest object to hold the parsed control artifact.
  */
  p = fossil_malloc( sizeof(*p) );
  memset(p, 0, sizeof(*p));
  memcpy(&p->content, pContent, sizeof(p->content));
  p->rid = rid;
  blob_zero(pContent);
................................................................................
        if( zName==0 || zTarget==0 ) goto manifest_syntax_error;
        if( p->zAttachName!=0 ) goto manifest_syntax_error;
        defossilize(zName);
        if( !file_is_simple_pathname(zName, 0) ){
          SYNTAX("invalid filename on A-card");
        }
        defossilize(zTarget);
        if( !hname_validate(zTarget,nTarget)
           && !wiki_name_is_wellformed((const unsigned char *)zTarget) ){
          SYNTAX("invalid target on A-card");
        }
        if( zSrc && !hname_validate(zSrc,nSrc) ){
          SYNTAX("invalid source on A-card");
        }
        p->zAttachName = (char*)file_tail(zName);
        p->zAttachSrc = zSrc;
        p->zAttachTarget = zTarget;
        break;
      }

      /*
      **    B <uuid>
      **
      ** A B-line gives the artifact hash for the baseline of a delta-manifest.
      */
      case 'B': {
        if( p->zBaseline ) SYNTAX("more than one B-card");
        p->zBaseline = next_token(&x, &sz);
        if( p->zBaseline==0 ) SYNTAX("missing hash on B-card");
        if( !hname_validate(p->zBaseline,sz) ){
          SYNTAX("invalid hash on B-card");
        }
        break;
      }


      /*
      **     C <comment>
................................................................................
      ** is when the specific event is said to occur.
      */
      case 'E': {
        if( p->rEventDate>0.0 ) SYNTAX("more than one E-card");
        p->rEventDate = db_double(0.0,"SELECT julianday(%Q)", next_token(&x,0));
        if( p->rEventDate<=0.0 ) SYNTAX("malformed date on E-card");
        p->zEventId = next_token(&x, &sz);
        if( !hname_validate(p->zEventId, sz) ){
          SYNTAX("malformed hash on E-card");
        }
        break;
      }

      /*
      **     F <filename> ?<uuid>? ?<permissions>? ?<old-name>?
      **
................................................................................
        if( zName==0 ) SYNTAX("missing filename on F-card");
        defossilize(zName);
        if( !file_is_simple_pathname(zName, 0) ){
          SYNTAX("F-card filename is not a simple path");
        }
        zUuid = next_token(&x, &sz);
        if( p->zBaseline==0 || zUuid!=0 ){
          if( !hname_validate(zUuid,sz) ){
            SYNTAX("F-card hash invalid");
          }
        }
        zPerm = next_token(&x,0);
        zPriorName = next_token(&x,0);
        if( zPriorName ){
          defossilize(zPriorName);
          if( !file_is_simple_pathname(zPriorName, 0) ){
            SYNTAX("F-card old filename is not a simple path");
................................................................................
        if( !wiki_name_is_wellformed((const unsigned char *)p->zWikiTitle) ){
          SYNTAX("L-card has malformed wiki name");
        }
        break;
      }

      /*
      **    M <hash>
      **
      ** An M-line identifies another artifact by its hash.  M-lines
      ** occur in clusters only.
      */
      case 'M': {
        zUuid = next_token(&x, &sz);
        if( zUuid==0 ) SYNTAX("missing hash on M-card");
        if( !hname_validate(zUuid,sz) ){
          SYNTAX("Invalid hash on M-card");
        }
        if( p->nCChild>=p->nCChildAlloc ){
          p->nCChildAlloc = p->nCChildAlloc*2 + 10;
          p->azCChild = fossil_realloc(p->azCChild
                                 , p->nCChildAlloc*sizeof(p->azCChild[0]) );
        }
        i = p->nCChild++;
        p->azCChild[i] = zUuid;
................................................................................
      ** this artifact.  The first parent is the primary parent.  All
      ** others are parents by merge. Note that the initial empty
      ** check-in historically has an empty P-card, so empty P-cards
      ** must be accepted.
      */
      case 'P': {
        while( (zUuid = next_token(&x, &sz))!=0 ){
          if( !hname_validate(zUuid, sz) ){
             SYNTAX("invalid hash on P-card");
          }
          if( p->nParent>=p->nParentAlloc ){
            p->nParentAlloc = p->nParentAlloc*2 + 5;
            p->azParent = fossil_realloc(p->azParent,
                               p->nParentAlloc*sizeof(char*));
          }
          i = p->nParent++;
          p->azParent[i] = zUuid;
................................................................................
      /*
      **     Q (+|-)<uuid> ?<uuid>?
      **
      ** Specify one or a range of check-ins that are cherrypicked into
      ** this check-in ("+") or backed out of this check-in ("-").
      */
      case 'Q': {
        if( (zUuid=next_token(&x, &sz))==0 ) SYNTAX("missing hash on Q-card");

        if( zUuid[0]!='+' && zUuid[0]!='-' ){
          SYNTAX("Q-card does not begin with '+' or '-'");
        }
        if( !hname_validate(&zUuid[1], sz-1) ){
          SYNTAX("invalid hash on Q-card");
        }
        n = p->nCherrypick;
        p->nCherrypick++;
        p->aCherrypick = fossil_realloc(p->aCherrypick,
                                 p->nCherrypick*sizeof(p->aCherrypick[0]));
        p->aCherrypick[n].zCPTarget = zUuid;
        p->aCherrypick[n].zCPBase = zUuid = next_token(&x, &sz);
        if( zUuid && !hname_validate(zUuid,sz) ){


          SYNTAX("invalid second hash on Q-card");

        }
        break;
      }

      /*
      **     R <md5sum>
      **
................................................................................
      ** Tags are not allowed in clusters.  Multiple T lines are allowed.
      */
      case 'T': {
        char *zName, *zValue;
        zName = next_token(&x, 0);
        if( zName==0 ) SYNTAX("missing name on T-card");
        zUuid = next_token(&x, &sz);
        if( zUuid==0 ) SYNTAX("missing artifact hash on T-card");
        zValue = next_token(&x, 0);
        if( zValue ) defossilize(zValue);
        if( hname_validate(zUuid, sz) ){
          /* A valid artifact hash */
          if( p->zEventId ) SYNTAX("non-self-referential T-card in event");
        }else if( sz==1 && zUuid[0]=='*' ){
          zUuid = 0;
          hasSelfRefTag = 1;
          if( p->zEventId && zName[0]!='+' ){
            SYNTAX("propagating T-card in event");
          }
        }else{
          SYNTAX("malformed artifact hash on T-card");
        }
        defossilize(zName);
        if( zName[0]!='-' && zName[0]!='+' && zName[0]!='*' ){
          SYNTAX("T-card name does not begin with '-', '+', or '*'");
        }
        if( validate16(&zName[1], strlen(&zName[1])) ){
          /* Do not allow tags whose names look like a hash */
          SYNTAX("T-card name looks like a hexadecimal hash");
        }
        if( p->nTag>=p->nTagAlloc ){
          p->nTagAlloc = p->nTagAlloc*2 + 10;
          p->aTag = fossil_realloc(p->aTag, p->nTagAlloc*sizeof(p->aTag[0]) );
        }
        i = p->nTag++;
        p->aTag[i].zName = zName;
................................................................................
    if( p->rDate<=0.0 ) SYNTAX("missing date on control");
    if( p->zMimetype ) SYNTAX("N-card in control");
    if( !seenZ ) SYNTAX("missing Z-card on control");
    p->type = CFTYPE_CONTROL;
  }
  md5sum_init();
  if( !isRepeat ) g.parseCnt[p->type]++;

  return p;

manifest_syntax_error:
  {
    char *zUuid = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", rid);
    if( zUuid ){
      blob_appendf(pErr, "manifest [%s] ", zUuid);
      fossil_free(zUuid);
    }
  }
  if( zErr ){
    blob_appendf(pErr, "line %d: %s", lineNo, zErr);
  }else{
    blob_appendf(pErr, "unknown error on line %d", lineNo);
  }
  md5sum_init();
................................................................................
** An mlink entry is always created if isPrimary is true.  But if
** isPrimary is false (meaning that pmid is a merge parent of mid)
** then the mlink entry is only created if there is already an mlink
** from primary parent for the same file.
*/
static void add_one_mlink(
  int pmid,                 /* The parent manifest */
  const char *zFromUuid,    /* Artifact hash for content in parent */
  int mid,                  /* The record ID of the manifest */
  const char *zToUuid,      /* artifact hash for content in child */
  const char *zFilename,    /* Filename */
  const char *zPrior,       /* Previous filename. NULL if unchanged */
  int isPublic,             /* True if mid is not a private manifest */
  int isPrimary,            /* pmid is the primary parent of mid */
  int mperm                 /* 1: exec, 2: symlink */
){
  int fnid, pfnid, pid, fid;
................................................................................
      add_mlink(pmid, 0, mid, pChild, 0);
    }
  }
}

/*
** For a check-in with RID "rid" that has nParent parent check-ins given
** by the hashes in azParent[], create all appropriate plink and mlink table
** entries.
**
** The primary parent is the first hash on the azParent[] list.
**
** Return the RID of the primary parent.
*/
static int manifest_add_checkin_linkages(
  int rid,                   /* The RID of the check-in */
  Manifest *p,               /* Manifest for this check-in */
  int nParent,               /* Number of parents for this check-in */
  char **azParent            /* hashes for each parent */
){
  int i;
  int parentid = 0;
  char zBaseId[30];    /* Baseline manifest RID for deltas.  "NULL" otherwise */
  Stmt q;

  if( p->zBaseline ){
................................................................................
    }
  }
  return parentid;
}

/*
** There exists a "parent" tag against checkin rid that has value zValue.
** If value is well-formed (meaning that it is a list of hashes), then use
** zValue to reparent check-in rid.
*/
void manifest_reparent_checkin(int rid, const char *zValue){
  int nParent = 0;
  char *zCopy = 0;
  char **azParent = 0;
  Manifest *p = 0;
  int i, j;
  int n = (int)strlen(zValue);
  int mxParent = (n+1)/(HNAME_MIN+1);


  if( mxParent<1 ) return;
  zCopy = fossil_strdup(zValue);
  azParent = fossil_malloc( sizeof(azParent[0])*mxParent );
  for(nParent=0, i=0; zCopy[i]; i++){
    char *z = &zCopy[i];
    azParent[nParent++] = z;
    if( nParent>mxParent ) goto reparent_abort;
    for(j=HNAME_MIN; z[j]>' '; j++){}
    if( !hname_validate(z, j) ) goto reparent_abort;
    if( z[j]==0 ) break;
    z[j] = 0;
    i += j;
  }

  if( !db_exists("SELECT 1 FROM plink WHERE cid=%d AND pid=%d",
                 rid, uuid_to_rid(azParent[0],0))
  ){
    p = manifest_get(rid, CFTYPE_MANIFEST, 0);
  }
  if( p!=0 ){
    db_multi_exec(
       "DELETE FROM plink WHERE cid=%d;"
       "DELETE FROM mlink WHERE mid=%d;",
       rid, rid
    );
    manifest_add_checkin_linkages(rid,p,nParent,azParent);
  }
  manifest_destroy(p);
reparent_abort:
  fossil_free(azParent);
  fossil_free(zCopy);
}

/*
** Setup to do multiple manifest_crosslink() calls.
**
................................................................................
    const char *zValue;
    const char *zTagUuid;
    int branchMove = 0;
    blob_zero(&comment);
    if( p->zComment ){
      blob_appendf(&comment, " %s.", p->zComment);
    }
    /* Next loop expects tags to be sorted on hash, so sort it. */
    qsort(p->aTag, p->nTag, sizeof(p->aTag[0]), tag_compare);
    for(i=0; i<p->nTag; i++){
      zTagUuid = p->aTag[i].zUuid;
      if( !zTagUuid ) continue;
      if( i==0 || fossil_strcmp(zTagUuid, p->aTag[i-1].zUuid)!=0 ){
        blob_appendf(&comment,
           " Edit [%!S|%S]:",

Changes to src/markdown.c.

457
458
459
460
461
462
463
464
465
466
467
468
469
470


471
472
473
474
475
476
477
/* find_emph_char -- looks for the next emph char, skipping other constructs */
static size_t find_emph_char(char *data, size_t size, char c){
  size_t i = 1;

  while( i<size ){
    while( i<size && data[i]!=c && data[i]!='`' && data[i]!='[' ){ i++; }
    if( i>=size ) return 0;
    if( data[i]==c ) return i;

    /* not counting escaped chars */
    if( i && data[i-1]=='\\' ){
      i++;
      continue;
    }



    /* skipping a code span */
    if( data[i]=='`' ){
      size_t span_nb = 0, bt;
      size_t tmp_i = 0;

      /* counting the number of opening backticks */







<






>
>







457
458
459
460
461
462
463

464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
/* find_emph_char -- looks for the next emph char, skipping other constructs */
static size_t find_emph_char(char *data, size_t size, char c){
  size_t i = 1;

  while( i<size ){
    while( i<size && data[i]!=c && data[i]!='`' && data[i]!='[' ){ i++; }
    if( i>=size ) return 0;


    /* not counting escaped chars */
    if( i && data[i-1]=='\\' ){
      i++;
      continue;
    }

    if( data[i]==c ) return i;

    /* skipping a code span */
    if( data[i]=='`' ){
      size_t span_nb = 0, bt;
      size_t tmp_i = 0;

      /* counting the number of opening backticks */

Changes to src/md5.c.

20
21
22
23
24
25
26










27
28
29
30
31
32
33
..
40
41
42
43
44
45
46

47


48
49
50
51
52
53
54
...
264
265
266
267
268
269
270

271
272
273
274
275
276
277
 */
#include "config.h"
#include <string.h>
#include <stdio.h>
#include <sqlite3.h>
#include "md5.h"











/*
 * If compiled on a machine that doesn't have a 32-bit integer,
 * you just set "uint32" to the appropriate datatype for an
 * unsigned 32-bit integer.  For example:
 *
 *       cc -Duint32='unsigned long' md5.c
 *
................................................................................
  int isInit;
  uint32 buf[4];
  uint32 bits[2];
  unsigned char in[64];
};
typedef struct Context MD5Context;


#if defined(__i386__) || defined(__x86_64__) || defined(_WIN32)


# define byteReverse(A,B)
#else
/*
 * Convert an array of integers to little-endian.
 * Note: this code is a no-op on little-endian machines.
 */
static void byteReverse (unsigned char *buf, unsigned longs){
................................................................................
        memcpy(&ctx->in[14*sizeof(uint32)], ctx->bits, 2*sizeof(uint32));

        MD5Transform(ctx->buf, (uint32 *)ctx->in);
        byteReverse((unsigned char *)ctx->buf, 4);
        memcpy(digest, ctx->buf, 16);
        memset(ctx, 0, sizeof(*ctx));    /* In case it's sensitive */
}


/*
** Convert a digest into base-16.  digest should be declared as
** "unsigned char digest[16]" in the calling function.  The MD5
** digest is stored in the first 16 bytes.  zBuf should
** be "char zBuf[33]".
*/







>
>
>
>
>
>
>
>
>
>







 







>
|
>
>







 







>







20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
..
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
...
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
 */
#include "config.h"
#include <string.h>
#include <stdio.h>
#include <sqlite3.h>
#include "md5.h"

#ifdef FOSSIL_ENABLE_SSL

# include <openssl/md5.h>
# define MD5Context MD5_CTX
# define MD5Init MD5_Init
# define MD5Update MD5_Update
# define MD5Final MD5_Final

#else

/*
 * If compiled on a machine that doesn't have a 32-bit integer,
 * you just set "uint32" to the appropriate datatype for an
 * unsigned 32-bit integer.  For example:
 *
 *       cc -Duint32='unsigned long' md5.c
 *
................................................................................
  int isInit;
  uint32 buf[4];
  uint32 bits[2];
  unsigned char in[64];
};
typedef struct Context MD5Context;

#if defined(i386)     || defined(__i386__)   || defined(_M_IX86) ||    \
    defined(__x86_64) || defined(__x86_64__) || defined(_M_X64)  ||    \
    defined(_M_AMD64) || defined(_M_ARM)     || defined(__x86)   ||    \
    defined(__arm__)  || defined(_WIN32)
# define byteReverse(A,B)
#else
/*
 * Convert an array of integers to little-endian.
 * Note: this code is a no-op on little-endian machines.
 */
static void byteReverse (unsigned char *buf, unsigned longs){
................................................................................
        memcpy(&ctx->in[14*sizeof(uint32)], ctx->bits, 2*sizeof(uint32));

        MD5Transform(ctx->buf, (uint32 *)ctx->in);
        byteReverse((unsigned char *)ctx->buf, 4);
        memcpy(digest, ctx->buf, 16);
        memset(ctx, 0, sizeof(*ctx));    /* In case it's sensitive */
}
#endif

/*
** Convert a digest into base-16.  digest should be declared as
** "unsigned char digest[16]" in the calling function.  The MD5
** digest is stored in the first 16 bytes.  zBuf should
** be "char zBuf[33]".
*/

Changes to src/mkversion.c.

10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
..
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char *argv[]){
    FILE *m,*u,*v;
    char *z;
    int i, x, d;

    char b[1000];
    char vx[1000];
    memset(b,0,sizeof(b));
    memset(vx,0,sizeof(vx));
    u = fopen(argv[1],"r");
    if( fgets(b, sizeof(b)-1,u)==0 ){
      fprintf(stderr, "malformed manifest.uuid file: %s\n", argv[1]);
................................................................................
      exit(1);
    }
    fclose(v);
    for(z=b; z[0] && z[0]!='\r' && z[0]!='\n'; z++){}
    *z = 0;
    printf("#define RELEASE_VERSION \"%s\"\n", b);
    x=0;
    i=0;
    z=b;

    while(1){
      if( z[0]>='0' && z[0]<='9' ){
        x = x*10 + z[0] - '0';
      }else{
        sprintf(&vx[i],"%02d",x);
        i += 2;
        x = 0;
        if( z[0]==0 ) break;
      }
      z++;
    }
    for(z=vx; z[0]=='0'; z++){}
    printf("#define RELEASE_VERSION_NUMBER %s\n", z);
    memset(vx,0,sizeof(vx));
    strcpy(vx,b);
    d = 0;
    for(z=vx; z[0]; z++){
      if( z[0]=='-' ){
        z[0] = 0;
        break;







|
>







 







|

>




|
<






|







10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
..
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

59
60
61
62
63
64
65
66
67
68
69
70
71
72
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char *argv[]){
    FILE *m,*u,*v;
    char *z;
    int i, j, x, d;
    int vn[3];
    char b[1000];
    char vx[1000];
    memset(b,0,sizeof(b));
    memset(vx,0,sizeof(vx));
    u = fopen(argv[1],"r");
    if( fgets(b, sizeof(b)-1,u)==0 ){
      fprintf(stderr, "malformed manifest.uuid file: %s\n", argv[1]);
................................................................................
      exit(1);
    }
    fclose(v);
    for(z=b; z[0] && z[0]!='\r' && z[0]!='\n'; z++){}
    *z = 0;
    printf("#define RELEASE_VERSION \"%s\"\n", b);
    x=0;
    i=j=0;
    z=b;
    vn[0] = vn[1] = vn[2] = 0;
    while(1){
      if( z[0]>='0' && z[0]<='9' ){
        x = x*10 + z[0] - '0';
      }else{
        if( j<3 ) vn[j++] = x;

        x = 0;
        if( z[0]==0 ) break;
      }
      z++;
    }
    for(z=vx; z[0]=='0'; z++){}
    printf("#define RELEASE_VERSION_NUMBER %d%02d%02d\n", vn[0], vn[1], vn[2]);
    memset(vx,0,sizeof(vx));
    strcpy(vx,b);
    d = 0;
    for(z=vx; z[0]; z++){
      if( z[0]=='-' ){
        z[0] = 0;
        break;

Changes to src/name.c.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
..
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
...
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
...
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
...
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
...
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
...
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
...
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
...
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
...
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
....
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
....
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
....
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)

** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
**
*******************************************************************************
**
** This file contains code used to convert user-supplied object names into
** canonical UUIDs.
**
** A user-supplied object name is any unique prefix of a valid UUID but
** not necessarily in canonical form.
*/
#include "config.h"
#include "name.h"
#include <assert.h>

/*
** Return TRUE if the string begins with something that looks roughly
................................................................................
  db_finalize(&q);
  return rid;
}

/*
** Convert a symbolic name into a RID.  Acceptable forms:
**
**   *  SHA1 hash
**   *  SHA1 hash prefix of at least 4 characters
**   *  Symbolic Name
**   *  "tag:" + symbolic name
**   *  Date or date-time
**   *  "date:" + Date or date-time
**   *  symbolic-name ":" date-time
**   *  "tip"
**
................................................................................
** The zType parameter specifies the type of artifact: ci, t, w, e, g.
** If zType is NULL or "" or "*" then any type of artifact will serve.
** If zType is "br" then find the first check-in of the named branch
** rather than the last.
** zType is "ci" in most use cases since we are usually searching for
** a check-in.
**
** Note that the input zTag for types "t" and "e" is the SHA1 hash of
** the ticket-change or event-change artifact, not the randomly generated
** hexadecimal identifier assigned to tickets and events.  Those identifiers
** live in a separate namespace.
*/
int symbolic_name_to_rid(const char *zTag, const char *zType){
  int vid;
  int rid = 0;
................................................................................
      "   AND event.mtime<=julianday(%Q)"
      "   AND event.type GLOB '%q'",
      zTagBase, zDate, zType
    );
    return rid;
  }

  /* SHA1 hash or prefix */
  if( nTag>=4 && nTag<=UUID_SIZE && validate16(zTag, nTag) ){
    Stmt q;
    char zUuid[UUID_SIZE+1];
    memcpy(zUuid, zTag, nTag+1);
    canonical16(zUuid, nTag);
    rid = 0;
    if( zType[0]=='*' ){
      db_prepare(&q, "SELECT rid FROM blob WHERE uuid GLOB '%q*'", zUuid);
    }else{
      db_prepare(&q,
................................................................................
** collisions of a given UUID based on its length on UUIDs no shorter
** than 4 characters in length.
*/
int name_collisions(const char *zName){
  int c = 0;         /* count of collisions for zName */
  int nLen;          /* length of zName */
  nLen = strlen(zName);
  if( nLen>=4 && nLen<=UUID_SIZE && validate16(zName, nLen) ){
    c = db_int(0,
      "SELECT"
      " (SELECT count(*) FROM ticket"
      "   WHERE tkt_uuid GLOB '%q*') +"
      " (SELECT count(*) FROM tag"
      "   WHERE tagname GLOB 'event-%q*') +"
      " (SELECT count(*) FROM blob"
................................................................................
  }
}

/*
** Convert a name to a rid.  If the name can be any of the various forms
** accepted:
**
**   * SHA1 hash or prefix thereof
**   * symbolic name
**   * date
**   * label:date
**   * prev, previous
**   * next
**   * tip
**
................................................................................
}
int name_to_rid(const char *zName){
  return name_to_typed_rid(zName, "*");
}

/*
** WEBPAGE: ambiguous
** URL: /ambiguous?name=UUID&src=WEBPAGE
**
** The UUID given by the name parameter is ambiguous.  Display a page
** that shows all possible choices and let the user select between them.
*/
void ambiguous_page(void){
  Stmt q;
  const char *zName = P("name");
  const char *zSrc = P("src");
  char *z;
................................................................................
}

/*
** COMMAND: whatis*
**
** Usage: %fossil whatis NAME
**
** Resolve the symbol NAME into its canonical 40-character SHA1-hash
** artifact name and provide a description of what role that artifact
** plays.
**
** Options:
**
**    --type TYPE          Only find artifacts of TYPE (one of: 'ci', 't',
**                         'w', 'g', or 'e').
................................................................................


/*
** COMMAND: test-ambiguous
**
** Usage: %fossil test-ambiguous [--minsize N]
**
** Show a list of ambiguous SHA1-hash abbreviations of N characters or
** more where N defaults to 4.  Change N to a different value using
** the "--minsize N" command-line option.
*/
void test_ambiguous_cmd(void){
  Stmt q, ins;
  int i;
  int minSize = 4;
................................................................................

/*
** Schema for the description table
*/
static const char zDescTab[] =
@ CREATE TEMP TABLE IF NOT EXISTS description(
@   rid INTEGER PRIMARY KEY,       -- RID of the object
@   uuid TEXT,                     -- SHA1 hash of the object
@   ctime DATETIME,                -- Time of creation
@   isPrivate BOOLEAN DEFAULT 0,   -- True for unpublished artifacts
@   type TEXT,                     -- file, checkin, wiki, ticket, etc.
@   summary TEXT,                  -- Summary comment for the object
@   detail TEXT                    -- File name, check-in comment, etc
@ );
;
................................................................................
    "       datetime(description.ctime)"
    "  FROM description, blob LEFT JOIN delta ON delta.rid=blob.rid"
    " WHERE description.rid=blob.rid"
    " ORDER BY length(content) DESC"
  );
  @ <table cellpadding="2" cellspacing="0" border="1" id="bigblobtab">
  @ <thead><tr><th align="right">Size<th align="right">RID
  @ <th align="right">Delta From<th>SHA1<th>Description<th>Date</tr></thead>
  @ <tbody>
  while( db_step(&q)==SQLITE_ROW ){
    int rid = db_column_int(&q,0);
    const char *zUuid = db_column_text(&q, 1);
    const char *zDesc = db_column_text(&q, 2);
    int sz = db_column_int(&q,3);
    const char *zSrcId = db_column_text(&q,4);
................................................................................
  describe_artifacts_to_stdout("IN (SELECT rid FROM blob WHERE size<0)", 0);
}

/* Maximum number of collision examples to remember */
#define MAX_COLLIDE 25

/*
** Generate a report on the number of collisions in SHA1 hashes
** generated by the SQL given in the argument.
*/
static void collision_report(const char *zSql){
  int i, j, kk;
  int nHash = 0;
  Stmt q;
  char zPrev[UUID_SIZE+1];
  struct {
    int cnt;
    char *azHit[MAX_COLLIDE];
    char z[UUID_SIZE+1];
  } aCollide[UUID_SIZE+1];
  memset(aCollide, 0, sizeof(aCollide));
  memset(zPrev, 0, sizeof(zPrev));
  db_prepare(&q,"%s",zSql/*safe-for-%s*/);
  while( db_step(&q)==SQLITE_ROW ){
    const char *zUuid = db_column_text(&q,0);
    int n = db_column_bytes(&q,0);
    int i;
    nHash++;
    for(i=0; zPrev[i] && zPrev[i]==zUuid[i]; i++){}
    if( i>0 && i<=UUID_SIZE ){
      if( i>=4 && aCollide[i].cnt<MAX_COLLIDE ){
        aCollide[i].azHit[aCollide[i].cnt] = mprintf("%.*s", i, zPrev);
      }
      aCollide[i].cnt++;
      if( aCollide[i].z[0]==0 ) memcpy(aCollide[i].z, zPrev, n+1);
    }
    memcpy(zPrev, zUuid, n+1);
  }
  db_finalize(&q);
  @ <table border=1><thead>
  @ <tr><th>Length<th>Instances<th>First Instance</tr>
  @ </thead><tbody>
  for(i=1; i<=UUID_SIZE; i++){
    if( aCollide[i].cnt==0 ) continue;
    @ <tr><td>%d(i)<td>%d(aCollide[i].cnt)<td>%h(aCollide[i].z)</tr>
  }
  @ </tbody></table>
  @ <p>Total number of hashes: %d(nHash)</p>
  kk = 0;
  for(i=UUID_SIZE; i>=4; i--){
    if( aCollide[i].cnt==0 ) continue;
    if( aCollide[i].cnt>200 ) break;
    kk += aCollide[i].cnt;
    if( aCollide[i].cnt<25 ){
      @ <p>Collisions of length %d(i):
    }else{
      @ <p>First 25 collisions of length %d(i):
................................................................................
** WEBPAGE: hash-collisions
**
** Show the number of hash collisions for hash prefixes of various lengths.
*/
void hash_collisions_webpage(void){
  login_check_credentials();
  if( !g.perm.Read ){ login_needed(g.anon.Read); return; }
  style_header("SHA1 Prefix Collisions");
  style_submenu_element("Activity Reports", "reports");
  style_submenu_element("Stats", "stat");
  @ <h1>Hash Prefix Collisions on Check-ins</h1>
  collision_report("SELECT (SELECT uuid FROM blob WHERE rid=objid)"
                   "  FROM event WHERE event.type='ci'"
                   " ORDER BY 1");
  @ <h1>Hash Prefix Collisions on All Artifacts</h1>
  collision_report("SELECT uuid FROM blob ORDER BY 1");
  style_footer();
}






|










|
<
<
<
<







 







|
|







 







|







 







|
|

|







 







|







 







|







 







|

|







 







|







 







|







 







|







 







|







 







|






|



|
|









|












|






|







 







|










1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18




19
20
21
22
23
24
25
..
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
..
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
...
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
...
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
...
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
...
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
...
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
...
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
...
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
....
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
....
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
....
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
**
*******************************************************************************
**
** This file contains code used to resolved user-supplied object names.




*/
#include "config.h"
#include "name.h"
#include <assert.h>

/*
** Return TRUE if the string begins with something that looks roughly
................................................................................
  db_finalize(&q);
  return rid;
}

/*
** Convert a symbolic name into a RID.  Acceptable forms:
**
**   *  artifact hash
**   *  4-character or larger prefix of a artifact
**   *  Symbolic Name
**   *  "tag:" + symbolic name
**   *  Date or date-time
**   *  "date:" + Date or date-time
**   *  symbolic-name ":" date-time
**   *  "tip"
**
................................................................................
** The zType parameter specifies the type of artifact: ci, t, w, e, g.
** If zType is NULL or "" or "*" then any type of artifact will serve.
** If zType is "br" then find the first check-in of the named branch
** rather than the last.
** zType is "ci" in most use cases since we are usually searching for
** a check-in.
**
** Note that the input zTag for types "t" and "e" is the artifact hash of
** the ticket-change or event-change artifact, not the randomly generated
** hexadecimal identifier assigned to tickets and events.  Those identifiers
** live in a separate namespace.
*/
int symbolic_name_to_rid(const char *zTag, const char *zType){
  int vid;
  int rid = 0;
................................................................................
      "   AND event.mtime<=julianday(%Q)"
      "   AND event.type GLOB '%q'",
      zTagBase, zDate, zType
    );
    return rid;
  }

  /* artifact hash or prefix */
  if( nTag>=4 && nTag<=HNAME_MAX && validate16(zTag, nTag) ){
    Stmt q;
    char zUuid[HNAME_MAX+1];
    memcpy(zUuid, zTag, nTag+1);
    canonical16(zUuid, nTag);
    rid = 0;
    if( zType[0]=='*' ){
      db_prepare(&q, "SELECT rid FROM blob WHERE uuid GLOB '%q*'", zUuid);
    }else{
      db_prepare(&q,
................................................................................
** collisions of a given UUID based on its length on UUIDs no shorter
** than 4 characters in length.
*/
int name_collisions(const char *zName){
  int c = 0;         /* count of collisions for zName */
  int nLen;          /* length of zName */
  nLen = strlen(zName);
  if( nLen>=4 && nLen<=HNAME_MAX && validate16(zName, nLen) ){
    c = db_int(0,
      "SELECT"
      " (SELECT count(*) FROM ticket"
      "   WHERE tkt_uuid GLOB '%q*') +"
      " (SELECT count(*) FROM tag"
      "   WHERE tagname GLOB 'event-%q*') +"
      " (SELECT count(*) FROM blob"
................................................................................
  }
}

/*
** Convert a name to a rid.  If the name can be any of the various forms
** accepted:
**
**   * artifact hash or prefix thereof
**   * symbolic name
**   * date
**   * label:date
**   * prev, previous
**   * next
**   * tip
**
................................................................................
}
int name_to_rid(const char *zName){
  return name_to_typed_rid(zName, "*");
}

/*
** WEBPAGE: ambiguous
** URL: /ambiguous?name=NAME&src=WEBPAGE
**
** The NAME given by the name parameter is ambiguous.  Display a page
** that shows all possible choices and let the user select between them.
*/
void ambiguous_page(void){
  Stmt q;
  const char *zName = P("name");
  const char *zSrc = P("src");
  char *z;
................................................................................
}

/*
** COMMAND: whatis*
**
** Usage: %fossil whatis NAME
**
** Resolve the symbol NAME into its canonical artifact hash
** artifact name and provide a description of what role that artifact
** plays.
**
** Options:
**
**    --type TYPE          Only find artifacts of TYPE (one of: 'ci', 't',
**                         'w', 'g', or 'e').
................................................................................


/*
** COMMAND: test-ambiguous
**
** Usage: %fossil test-ambiguous [--minsize N]
**
** Show a list of ambiguous artifact hash abbreviations of N characters or
** more where N defaults to 4.  Change N to a different value using
** the "--minsize N" command-line option.
*/
void test_ambiguous_cmd(void){
  Stmt q, ins;
  int i;
  int minSize = 4;
................................................................................

/*
** Schema for the description table
*/
static const char zDescTab[] =
@ CREATE TEMP TABLE IF NOT EXISTS description(
@   rid INTEGER PRIMARY KEY,       -- RID of the object
@   uuid TEXT,                     -- hash of the object
@   ctime DATETIME,                -- Time of creation
@   isPrivate BOOLEAN DEFAULT 0,   -- True for unpublished artifacts
@   type TEXT,                     -- file, checkin, wiki, ticket, etc.
@   summary TEXT,                  -- Summary comment for the object
@   detail TEXT                    -- File name, check-in comment, etc
@ );
;
................................................................................
    "       datetime(description.ctime)"
    "  FROM description, blob LEFT JOIN delta ON delta.rid=blob.rid"
    " WHERE description.rid=blob.rid"
    " ORDER BY length(content) DESC"
  );
  @ <table cellpadding="2" cellspacing="0" border="1" id="bigblobtab">
  @ <thead><tr><th align="right">Size<th align="right">RID
  @ <th align="right">Delta From<th>Hash<th>Description<th>Date</tr></thead>
  @ <tbody>
  while( db_step(&q)==SQLITE_ROW ){
    int rid = db_column_int(&q,0);
    const char *zUuid = db_column_text(&q, 1);
    const char *zDesc = db_column_text(&q, 2);
    int sz = db_column_int(&q,3);
    const char *zSrcId = db_column_text(&q,4);
................................................................................
  describe_artifacts_to_stdout("IN (SELECT rid FROM blob WHERE size<0)", 0);
}

/* Maximum number of collision examples to remember */
#define MAX_COLLIDE 25

/*
** Generate a report on the number of collisions in artifact hashes
** generated by the SQL given in the argument.
*/
static void collision_report(const char *zSql){
  int i, j, kk;
  int nHash = 0;
  Stmt q;
  char zPrev[HNAME_MAX+1];
  struct {
    int cnt;
    char *azHit[MAX_COLLIDE];
    char z[HNAME_MAX+1];
  } aCollide[HNAME_MAX+1];
  memset(aCollide, 0, sizeof(aCollide));
  memset(zPrev, 0, sizeof(zPrev));
  db_prepare(&q,"%s",zSql/*safe-for-%s*/);
  while( db_step(&q)==SQLITE_ROW ){
    const char *zUuid = db_column_text(&q,0);
    int n = db_column_bytes(&q,0);
    int i;
    nHash++;
    for(i=0; zPrev[i] && zPrev[i]==zUuid[i]; i++){}
    if( i>0 && i<=HNAME_MAX ){
      if( i>=4 && aCollide[i].cnt<MAX_COLLIDE ){
        aCollide[i].azHit[aCollide[i].cnt] = mprintf("%.*s", i, zPrev);
      }
      aCollide[i].cnt++;
      if( aCollide[i].z[0]==0 ) memcpy(aCollide[i].z, zPrev, n+1);
    }
    memcpy(zPrev, zUuid, n+1);
  }
  db_finalize(&q);
  @ <table border=1><thead>
  @ <tr><th>Length<th>Instances<th>First Instance</tr>
  @ </thead><tbody>
  for(i=1; i<=HNAME_MAX; i++){
    if( aCollide[i].cnt==0 ) continue;
    @ <tr><td>%d(i)<td>%d(aCollide[i].cnt)<td>%h(aCollide[i].z)</tr>
  }
  @ </tbody></table>
  @ <p>Total number of hashes: %d(nHash)</p>
  kk = 0;
  for(i=HNAME_MAX; i>=4; i--){
    if( aCollide[i].cnt==0 ) continue;
    if( aCollide[i].cnt>200 ) break;
    kk += aCollide[i].cnt;
    if( aCollide[i].cnt<25 ){
      @ <p>Collisions of length %d(i):
    }else{
      @ <p>First 25 collisions of length %d(i):
................................................................................
** WEBPAGE: hash-collisions
**
** Show the number of hash collisions for hash prefixes of various lengths.
*/
void hash_collisions_webpage(void){
  login_check_credentials();
  if( !g.perm.Read ){ login_needed(g.anon.Read); return; }
  style_header("Hash Prefix Collisions");
  style_submenu_element("Activity Reports", "reports");
  style_submenu_element("Stats", "stat");
  @ <h1>Hash Prefix Collisions on Check-ins</h1>
  collision_report("SELECT (SELECT uuid FROM blob WHERE rid=objid)"
                   "  FROM event WHERE event.type='ci'"
                   " ORDER BY 1");
  @ <h1>Hash Prefix Collisions on All Artifacts</h1>
  collision_report("SELECT uuid FROM blob ORDER BY 1");
  style_footer();
}

Changes to src/printf.c.

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
..
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#include "printf.h"
#if defined(_WIN32)
#   include <io.h>
#   include <fcntl.h>
#endif
#include <time.h>

/* Two custom conversions are used to show a prefix of SHA1 hashes:
**
**      %!S       Prefix of a length appropriate for URLs
**      %S        Prefix of a length appropriate for human display
**
** The following macros help determine those lengths.  FOSSIL_HASH_DIGITS
** is the default number of digits to display to humans.  This value can
** be overridden using the hash-digits setting.  FOSSIL_HASH_DIGITS_URL
................................................................................
# define FOSSIL_HASH_DIGITS 10       /* For %S (human display) */
#endif
#ifndef FOSSIL_HASH_DIGITS_URL
# define FOSSIL_HASH_DIGITS_URL 16   /* For %!S (embedded in URLs) */
#endif

/*
** Return the number of SHA1 hash digits to display.  The number is for
** human output if the bForUrl is false and is destined for a URL if
** bForUrl is false.
*/
static int hashDigits(int bForUrl){
  static int nDigitHuman = 0;
  static int nDigitUrl = 0;
  if( nDigitHuman==0 ){







|







 







|







22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
..
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#include "printf.h"
#if defined(_WIN32)
#   include <io.h>
#   include <fcntl.h>
#endif
#include <time.h>

/* Two custom conversions are used to show a prefix of artifact hashes:
**
**      %!S       Prefix of a length appropriate for URLs
**      %S        Prefix of a length appropriate for human display
**
** The following macros help determine those lengths.  FOSSIL_HASH_DIGITS
** is the default number of digits to display to humans.  This value can
** be overridden using the hash-digits setting.  FOSSIL_HASH_DIGITS_URL
................................................................................
# define FOSSIL_HASH_DIGITS 10       /* For %S (human display) */
#endif
#ifndef FOSSIL_HASH_DIGITS_URL
# define FOSSIL_HASH_DIGITS_URL 16   /* For %!S (embedded in URLs) */
#endif

/*
** Return the number of artifact hash digits to display.  The number is for
** human output if the bForUrl is false and is destined for a URL if
** bForUrl is false.
*/
static int hashDigits(int bForUrl){
  static int nDigitHuman = 0;
  static int nDigitUrl = 0;
  if( nDigitHuman==0 ){

Changes to src/purge.c.

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
...
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
...
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
...
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
@   ctime DATETIME,            -- When purge occurred.  Seconds since 1970.
@   pnotes TEXT                -- Human-readable notes about the purge event
@ );
@ CREATE TABLE IF NOT EXISTS "%w".purgeitem(
@   piid INTEGER PRIMARY KEY,  -- ID for the purge item
@   peid INTEGER REFERENCES purgeevent ON DELETE CASCADE, -- Purge event
@   orid INTEGER,              -- Original RID before purged
@   uuid TEXT NOT NULL,        -- SHA1 hash of the purged artifact
@   srcid INTEGER,             -- Basis purgeitem for delta compression
@   isPrivate BOOLEAN,         -- True if artifact was originally private
@   sz INT NOT NULL,           -- Uncompressed size of the purged artifact
@   desc TEXT,                 -- Brief description of this artifact
@   data BLOB                  -- Compressed artifact content
@ );
;
................................................................................
*/
static int purge_extract_item(
  int piid,            /* ID of the item to extract */
  Blob *pOut           /* Write the content into this blob */
){
  Stmt q;
  int srcid;
  Blob h1, h2, x;
  static Bag busy;

  db_prepare(&q, "SELECT uuid, srcid, data FROM purgeitem"
                 " WHERE piid=%d", piid);
  if( db_step(&q)!=SQLITE_ROW ){
    db_finalize(&q);
    fossil_fatal("missing purge-item %d", piid);
................................................................................
    blob_reset(pOut);
    *pOut = out;
    blob_reset(&baseline);
  }
  bag_remove(&busy, piid);
  blob_zero(&h1);
  db_column_blob(&q, 0, &h1);
  sha1sum_blob(pOut, &h2);
  if( blob_compare(&h1, &h2)!=0 ){
    fossil_fatal("SHA1 hash mismatch - wanted %s, got %s",
                 blob_str(&h1), blob_str(&h2));
  }
  blob_reset(&h1);
  blob_reset(&h2);
  db_finalize(&q);
  return 0;
}

/*
** There is a TEMP table ix(piid,srcid) containing a set of purgeitems
** that need to be transferred to the BLOB table.  This routine does
................................................................................
     "SELECT uuid, data, isPrivate, ix.piid"
     "  FROM ix, purgeitem"
     " WHERE ix.srcid=%d"
     "   AND ix.piid=purgeitem.piid;",
     iSrc
  );
  while( db_step(&q)==SQLITE_ROW ){
    Blob h1, h2, c1, c2;
    int isPriv, rid;
    blob_zero(&h1);
    db_column_blob(&q, 0, &h1);
    blob_zero(&c1);
    db_column_blob(&q, 1, &c1);
    blob_uncompress(&c1, &c1);
    blob_zero(&c2);
    if( pBasis ){
      blob_delta_apply(pBasis, &c1, &c2);
      blob_reset(&c1);
    }else{
      c2 = c1;
    }
    sha1sum_blob(&c2, &h2);
    if( blob_compare(&h1, &h2)!=0 ){
      fossil_fatal("SHA1 hash mismatch - wanted %s, got %s",
                   blob_str(&h1), blob_str(&h2));
    }
    blob_reset(&h2);
    isPriv = db_column_int(&q, 2);
    rid = content_put_ex(&c2, blob_str(&h1), 0, 0, isPriv);
    if( rid==0 ){
      fossil_fatal("%s", g.zErrMsg);
    }else{
      if( !isPriv ) content_make_public(rid);
      content_get(rid, &c1);







|







 







|







 







|
|
<
<


<







 







|













|
|
<
<

<







41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
...
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
...
374
375
376
377
378
379
380
381
382


383
384

385
386
387
388
389
390
391
...
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428


429

430
431
432
433
434
435
436
@   ctime DATETIME,            -- When purge occurred.  Seconds since 1970.
@   pnotes TEXT                -- Human-readable notes about the purge event
@ );
@ CREATE TABLE IF NOT EXISTS "%w".purgeitem(
@   piid INTEGER PRIMARY KEY,  -- ID for the purge item
@   peid INTEGER REFERENCES purgeevent ON DELETE CASCADE, -- Purge event
@   orid INTEGER,              -- Original RID before purged
@   uuid TEXT NOT NULL,        -- hash of the purged artifact
@   srcid INTEGER,             -- Basis purgeitem for delta compression
@   isPrivate BOOLEAN,         -- True if artifact was originally private
@   sz INT NOT NULL,           -- Uncompressed size of the purged artifact
@   desc TEXT,                 -- Brief description of this artifact
@   data BLOB                  -- Compressed artifact content
@ );
;
................................................................................
*/
static int purge_extract_item(
  int piid,            /* ID of the item to extract */
  Blob *pOut           /* Write the content into this blob */
){
  Stmt q;
  int srcid;
  Blob h1, x;
  static Bag busy;

  db_prepare(&q, "SELECT uuid, srcid, data FROM purgeitem"
                 " WHERE piid=%d", piid);
  if( db_step(&q)!=SQLITE_ROW ){
    db_finalize(&q);
    fossil_fatal("missing purge-item %d", piid);
................................................................................
    blob_reset(pOut);
    *pOut = out;
    blob_reset(&baseline);
  }
  bag_remove(&busy, piid);
  blob_zero(&h1);
  db_column_blob(&q, 0, &h1);
  if( hname_verify_hash(pOut, blob_buffer(&h1), blob_size(&h1))==0 ){
    fossil_fatal("incorrect artifact hash on %b", &h1);


  }
  blob_reset(&h1);

  db_finalize(&q);
  return 0;
}

/*
** There is a TEMP table ix(piid,srcid) containing a set of purgeitems
** that need to be transferred to the BLOB table.  This routine does
................................................................................
     "SELECT uuid, data, isPrivate, ix.piid"
     "  FROM ix, purgeitem"
     " WHERE ix.srcid=%d"
     "   AND ix.piid=purgeitem.piid;",
     iSrc
  );
  while( db_step(&q)==SQLITE_ROW ){
    Blob h1, c1, c2;
    int isPriv, rid;
    blob_zero(&h1);
    db_column_blob(&q, 0, &h1);
    blob_zero(&c1);
    db_column_blob(&q, 1, &c1);
    blob_uncompress(&c1, &c1);
    blob_zero(&c2);
    if( pBasis ){
      blob_delta_apply(pBasis, &c1, &c2);
      blob_reset(&c1);
    }else{
      c2 = c1;
    }
    if( hname_verify_hash(&c2, blob_buffer(&h1), blob_size(&h1))==0 ){
      fossil_fatal("incorrect hash on %b", &h1);


    }

    isPriv = db_column_int(&q, 2);
    rid = content_put_ex(&c2, blob_str(&h1), 0, 0, isPriv);
    if( rid==0 ){
      fossil_fatal("%s", g.zErrMsg);
    }else{
      if( !isPriv ) content_make_public(rid);
      content_get(rid, &c1);

Changes to src/rebuild.c.

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
...
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125


126
127
128
129
130
131
132
133
134
135













136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157































158
159
160
161
162
163
164
...
328
329
330
331
332
333
334
335
336
337
338

339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361



362
363
364
365
366
367
368
...
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
*/
#include "config.h"
#include "rebuild.h"
#include <assert.h>
#include <errno.h>

/*
** Make changes to the stable part of the schema (the part that is not
** simply deleted and reconstructed on a rebuild) to bring the schema
** up to the latest.
*/
static const char zSchemaUpdates1[] =
@ -- Index on the delta table
@ --
@ CREATE INDEX IF NOT EXISTS delta_i1 ON delta(srcid);
@
@ -- Artifacts that should not be processed are identified in the
@ -- "shun" table.  Artifacts that are control-file forgeries or
@ -- spam or artifacts whose contents violate administrative policy
@ -- can be shunned in order to prevent them from contaminating
@ -- the repository.
@ --
@ -- Shunned artifacts do not exist in the blob table.  Hence they
@ -- have not artifact ID (rid) and we thus must store their full
@ -- UUID.
@ --
@ CREATE TABLE IF NOT EXISTS shun(
@   uuid UNIQUE,          -- UUID of artifact to be shunned. Canonical form
@   mtime INTEGER,        -- When added.  Seconds since 1970
@   scom TEXT             -- Optional text explaining why the shun occurred
@ );
@
@ -- Artifacts that should not be pushed are stored in the "private"
@ -- table.
@ --
@ CREATE TABLE IF NOT EXISTS private(rid INTEGER PRIMARY KEY);
@
@ -- Some ticket content (such as the originators email address or contact
@ -- information) needs to be obscured to protect privacy.  This is achieved
@ -- by storing an SHA1 hash of the content.  For display, the hash is
@ -- mapped back into the original text using this table.
@ --
@ -- This table contains sensitive information and should not be shared
@ -- with unauthorized users.
@ --
@ CREATE TABLE IF NOT EXISTS concealed(
@   hash TEXT PRIMARY KEY,    -- The SHA1 hash of content
@   mtime INTEGER,            -- Time created.  Seconds since 1970
@   content TEXT              -- Content intended to be concealed
@ );
;
static const char zSchemaUpdates2[] =
@ -- An entry in this table describes a database query that generates a
@ -- table of tickets.
@ --
@ CREATE TABLE IF NOT EXISTS reportfmt(
@    rn INTEGER PRIMARY KEY,  -- Report number
@    owner TEXT,              -- Owner of this report format (not used)
@    title TEXT UNIQUE,       -- Title of this report
@    mtime INTEGER,           -- Time last modified.  Seconds since 1970
@    cols TEXT,               -- A color-key specification
@    sqlcode TEXT             -- An SQL SELECT statement for this report
@ );
;

static void rebuild_update_schema(void){
  int rc;
  db_multi_exec("%s", zSchemaUpdates1 /*safe-for-%s*/);
  db_multi_exec("%s", zSchemaUpdates2 /*safe-for-%s*/);

  rc = db_exists("SELECT 1 FROM sqlite_master"
                 " WHERE name='user' AND sql GLOB '* mtime *'");
  if( rc==0 ){
    db_multi_exec(
      "CREATE TEMP TABLE temp_user AS SELECT * FROM user;"
      "DROP TABLE user;"
      "CREATE TABLE user(\n"
      "  uid INTEGER PRIMARY KEY,\n"
      "  login TEXT UNIQUE,\n"
      "  pw TEXT,\n"
................................................................................
      "INSERT OR IGNORE INTO user"
        " SELECT uid, login, pw, cap, cookie,"
               " ipaddr, cexpire, info, now(), photo FROM temp_user;"
      "DROP TABLE temp_user;"
    );
  }

  rc = db_exists("SELECT 1 FROM sqlite_master"
                 " WHERE name='config' AND sql GLOB '* mtime *'");
  if( rc==0 ){
    db_multi_exec(
      "ALTER TABLE config ADD COLUMN mtime INTEGER;"
      "UPDATE config SET mtime=now();"
    );
  }

  rc = db_exists("SELECT 1 FROM sqlite_master"
                 " WHERE name='shun' AND sql GLOB '* mtime *'");
  if( rc==0 ){


    db_multi_exec(
      "ALTER TABLE shun ADD COLUMN mtime INTEGER;"
      "ALTER TABLE shun ADD COLUMN scom TEXT;"
      "UPDATE shun SET mtime=now();"
    );
  }

  rc = db_exists("SELECT 1 FROM sqlite_master"
                 " WHERE name='reportfmt' AND sql GLOB '* mtime *'");
  if( rc==0 ){













    db_multi_exec(
      "CREATE TEMP TABLE old_fmt AS SELECT * FROM reportfmt;"
      "DROP TABLE reportfmt;"
    );
    db_multi_exec("%s", zSchemaUpdates2/*safe-for-%s*/);
    db_multi_exec(
      "INSERT OR IGNORE INTO reportfmt(rn,owner,title,cols,sqlcode,mtime)"
        " SELECT rn, owner, title, cols, sqlcode, now() FROM old_fmt;"
      "INSERT OR IGNORE INTO reportfmt(rn,owner,title,cols,sqlcode,mtime)"
        " SELECT rn, owner, title || ' (' || rn || ')', cols, sqlcode, now()"
        "   FROM old_fmt;"
    );
  }

  rc = db_exists("SELECT 1 FROM sqlite_master"
                 " WHERE name='concealed' AND sql GLOB '* mtime *'");
  if( rc==0 ){
    db_multi_exec(
      "ALTER TABLE concealed ADD COLUMN mtime INTEGER;"
      "UPDATE concealed SET mtime=now();"
    );
  }































}

/*
** Variables used to store state information about an on-going "rebuild"
** or "deconstruct".
*/
static int totalSize;       /* Total number of artifacts to process */
................................................................................
**
** If the randomize parameter is true, then the BLOBs are deliberately
** extracted in a random order.  This feature is used to test the
** ability of fossil to accept records in any order and still
** construct a sane repository.
*/
int rebuild_db(int randomize, int doOut, int doClustering){
  Stmt s;
  int errCnt = 0;
  char *zTable;
  int incrSize;


  bag_init(&bagDone);
  ttyOutput = doOut;
  processCnt = 0;
  if (ttyOutput && !g.fQuiet) {
    percent_complete(0);
  }
  rebuild_update_schema();
  for(;;){
    zTable = db_text(0,
       "SELECT name FROM sqlite_master /*scan*/"
       " WHERE type='table'"
       " AND name NOT IN ('admin_log', 'blob','delta','rcvfrom','user',"
                         "'config','shun','private','reportfmt',"
                         "'concealed','accesslog','modreq',"
                         "'purgeevent','purgeitem','unversioned')"
       " AND name NOT GLOB 'sqlite_*'"
       " AND name NOT GLOB 'fx_*'"
    );
    if( zTable==0 ) break;
    db_multi_exec("DROP TABLE %Q", zTable);
    free(zTable);
  }



  db_multi_exec("%s", zRepositorySchema2/*safe-for-%s*/);
  ticket_create_table(0);
  shun_artifacts();

  db_multi_exec(
     "INSERT INTO unclustered"
     " SELECT rid FROM blob EXCEPT SELECT rid FROM private"
................................................................................
**
** Usage %fossil deconstruct ?OPTIONS? DESTINATION
**
**
** This command exports all artifacts of a given repository and
** writes all artifacts to the file system. The DESTINATION directory
** will be populated with subdirectories AA and files AA/BBBBBBBBB.., where
** AABBBBBBBBB.. is the 40 character artifact ID, AA the first 2 characters.
** If -L|--prefixlength is given, the length (default 2) of the directory
** prefix can be set to 0,1,..,9 characters.
**
** Options:
**   -R|--repository REPOSITORY  deconstruct given REPOSITORY
**   -L|--prefixlength N         set the length of the names of the DESTINATION
**                               subdirectories to N







|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







|
|
|






|
|
<
>
>







|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>




|









|
|
|





>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|

<

>








|
|
|
|
|
|
|
|
|
|
|
|
|
<

>
>
>







 







|







19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56



































57
58
59
60
61
62
63
..
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
...
338
339
340
341
342
343
344
345
346

347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369

370
371
372
373
374
375
376
377
378
379
380
....
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
*/
#include "config.h"
#include "rebuild.h"
#include <assert.h>
#include <errno.h>

/*
** Update the schema as necessary
*/
static void rebuild_update_schema(void){
  /* Verify that the PLINK table has a new column added by the
  ** 2014-11-28 schema change.  Create it if necessary.  This code
  ** can be removed in the future, once all users have upgraded to the
  ** 2014-11-28 or later schema.
  */
  if( !db_table_has_column("repository","plink","baseid") ){
    db_multi_exec(
      "ALTER TABLE repository.plink ADD COLUMN baseid;"
    );
  }

  /* Verify that the MLINK table has the newer columns added by the
  ** 2015-01-24 schema change.  Create them if necessary.  This code
  ** can be removed in the future, once all users have upgraded to the
  ** 2015-01-24 or later schema.
  */
  if( !db_table_has_column("repository","mlink","isaux") ){
    db_begin_transaction();
    db_multi_exec(
      "ALTER TABLE repository.mlink ADD COLUMN pmid INTEGER DEFAULT 0;"
      "ALTER TABLE repository.mlink ADD COLUMN isaux BOOLEAN DEFAULT 0;"
    );
    db_end_transaction(0);
  }

  /* Add the user.mtime column if it is missing. (2011-04-27)
  */
  if( !db_table_has_column("repository", "user", "mtime") ){



































    db_multi_exec(
      "CREATE TEMP TABLE temp_user AS SELECT * FROM user;"
      "DROP TABLE user;"
      "CREATE TABLE user(\n"
      "  uid INTEGER PRIMARY KEY,\n"
      "  login TEXT UNIQUE,\n"
      "  pw TEXT,\n"
................................................................................
      "INSERT OR IGNORE INTO user"
        " SELECT uid, login, pw, cap, cookie,"
               " ipaddr, cexpire, info, now(), photo FROM temp_user;"
      "DROP TABLE temp_user;"
    );
  }

  /* Add the config.mtime column if it is missing.  (2011-04-27)
  */
  if( !db_table_has_column("repository", "config", "mtime") ){
    db_multi_exec(
      "ALTER TABLE config ADD COLUMN mtime INTEGER;"
      "UPDATE config SET mtime=now();"
    );
  }

  /* Add the shun.mtime and shun.scom columns if they are missing.
  ** (2011-04-27)

  */
  if( !db_table_has_column("repository", "shun", "mtime") ){
    db_multi_exec(
      "ALTER TABLE shun ADD COLUMN mtime INTEGER;"
      "ALTER TABLE shun ADD COLUMN scom TEXT;"
      "UPDATE shun SET mtime=now();"
    );
  }

  /* Add the reportfmt.mtime column if it is missing. (2011-04-27)
  */
  if( !db_table_has_column("repository", "reportfmt", "mtime") ){
    static const char zCreateReportFmtTable[] =
    @ -- An entry in this table describes a database query that generates a
    @ -- table of tickets.
    @ --
    @ CREATE TABLE IF NOT EXISTS reportfmt(
    @    rn INTEGER PRIMARY KEY,  -- Report number
    @    owner TEXT,              -- Owner of this report format (not used)
    @    title TEXT UNIQUE,       -- Title of this report
    @    mtime INTEGER,           -- Time last modified.  Seconds since 1970
    @    cols TEXT,               -- A color-key specification
    @    sqlcode TEXT             -- An SQL SELECT statement for this report
    @ );
    ;
    db_multi_exec(
      "CREATE TEMP TABLE old_fmt AS SELECT * FROM reportfmt;"
      "DROP TABLE reportfmt;"
    );
    db_multi_exec("%s", zCreateReportFmtTable/*safe-for-%s*/);
    db_multi_exec(
      "INSERT OR IGNORE INTO reportfmt(rn,owner,title,cols,sqlcode,mtime)"
        " SELECT rn, owner, title, cols, sqlcode, now() FROM old_fmt;"
      "INSERT OR IGNORE INTO reportfmt(rn,owner,title,cols,sqlcode,mtime)"
        " SELECT rn, owner, title || ' (' || rn || ')', cols, sqlcode, now()"
        "   FROM old_fmt;"
    );
  }

  /* Add the concealed.mtime column if it is missing. (2011-04-27)
  */
  if( !db_table_has_column("repository", "concealed", "mtime") ){
    db_multi_exec(
      "ALTER TABLE concealed ADD COLUMN mtime INTEGER;"
      "UPDATE concealed SET mtime=now();"
    );
  }

  /* Do the fossil-2.0 updates to the schema.  (2017-02-28)
  */
  rebuild_schema_update_2_0();
}

/*
** Update the repository schema for Fossil version 2.0.  (2017-02-28)
**   (1) Change the CHECK constraint on BLOB.UUID so that the length
**       is greater than or equal to 40, not exactly equal to 40.
*/
void rebuild_schema_update_2_0(void){
  char *z = db_text(0, "SELECT sql FROM repository.sqlite_master WHERE name='blob'");
  if( z ){
    /* Search for:  length(uuid)==40
    **              0123456789 12345   */
    int i;
    for(i=10; z[i]; i++){
      if( z[i]=='=' && strncmp(&z[i-6],"(uuid)==40",10)==0 ){
        z[i] = '>';
        db_multi_exec(
           "PRAGMA writable_schema=ON;"
           "UPDATE repository.sqlite_master SET sql=%Q WHERE name LIKE 'blob';"
           "PRAGMA writable_schema=OFF;",
           z
        );
        break;
      }
    }
    fossil_free(z);
  }
}

/*
** Variables used to store state information about an on-going "rebuild"
** or "deconstruct".
*/
static int totalSize;       /* Total number of artifacts to process */
................................................................................
**
** If the randomize parameter is true, then the BLOBs are deliberately
** extracted in a random order.  This feature is used to test the
** ability of fossil to accept records in any order and still
** construct a sane repository.
*/
int rebuild_db(int randomize, int doOut, int doClustering){
  Stmt s, q;
  int errCnt = 0;

  int incrSize;
  Blob sql;

  bag_init(&bagDone);
  ttyOutput = doOut;
  processCnt = 0;
  if (ttyOutput && !g.fQuiet) {
    percent_complete(0);
  }
  rebuild_update_schema();
  blob_init(&sql, 0, 0);
  db_prepare(&q,
     "SELECT name FROM sqlite_master /*scan*/"
     " WHERE type='table'"
     " AND name NOT IN ('admin_log', 'blob','delta','rcvfrom','user','alias',"
                       "'config','shun','private','reportfmt',"
                       "'concealed','accesslog','modreq',"
                       "'purgeevent','purgeitem','unversioned')"
     " AND name NOT GLOB 'sqlite_*'"
     " AND name NOT GLOB 'fx_*'"
  );
  while( db_step(&q)==SQLITE_ROW ){
    blob_appendf(&sql, "DROP TABLE \"%w\";\n", db_column_text(&q,0));

  }
  db_finalize(&q);
  db_multi_exec("%s", blob_str(&sql)/*safe-for-%s*/);
  blob_reset(&sql);
  db_multi_exec("%s", zRepositorySchema2/*safe-for-%s*/);
  ticket_create_table(0);
  shun_artifacts();

  db_multi_exec(
     "INSERT INTO unclustered"
     " SELECT rid FROM blob EXCEPT SELECT rid FROM private"
................................................................................
**
** Usage %fossil deconstruct ?OPTIONS? DESTINATION
**
**
** This command exports all artifacts of a given repository and
** writes all artifacts to the file system. The DESTINATION directory
** will be populated with subdirectories AA and files AA/BBBBBBBBB.., where
** AABBBBBBBBB.. is the 40+ character artifact ID, AA the first 2 characters.
** If -L|--prefixlength is given, the length (default 2) of the directory
** prefix can be set to 0,1,..,9 characters.
**
** Options:
**   -R|--repository REPOSITORY  deconstruct given REPOSITORY
**   -L|--prefixlength N         set the length of the names of the DESTINATION
**                               subdirectories to N

Changes to src/schema.c.

77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
@ -- The blob and delta tables collectively hold the "global state" of
@ -- a Fossil repository.
@ --
@ CREATE TABLE blob(
@   rid INTEGER PRIMARY KEY,        -- Record ID
@   rcvid INTEGER,                  -- Origin of this record
@   size INTEGER,                   -- Size of content. -1 for a phantom.
@   uuid TEXT UNIQUE NOT NULL,      -- SHA1 hash of the content
@   content BLOB,                   -- Compressed content of this record
@   CHECK( length(uuid)==40 AND rid>0 )
@ );
@ CREATE TABLE delta(
@   rid INTEGER PRIMARY KEY,                 -- BLOB that is delta-compressed
@   srcid INTEGER NOT NULL REFERENCES blob   -- Baseline for delta-compression
@ );
@ CREATE INDEX delta_i1 ON delta(srcid);
@







|

|







77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
@ -- The blob and delta tables collectively hold the "global state" of
@ -- a Fossil repository.
@ --
@ CREATE TABLE blob(
@   rid INTEGER PRIMARY KEY,        -- Record ID
@   rcvid INTEGER,                  -- Origin of this record
@   size INTEGER,                   -- Size of content. -1 for a phantom.
@   uuid TEXT UNIQUE NOT NULL,      -- hash of the content
@   content BLOB,                   -- Compressed content of this record
@   CHECK( length(uuid)>=40 AND rid>0 )
@ );
@ CREATE TABLE delta(
@   rid INTEGER PRIMARY KEY,                 -- BLOB that is delta-compressed
@   srcid INTEGER NOT NULL REFERENCES blob   -- Baseline for delta-compression
@ );
@ CREATE INDEX delta_i1 ON delta(srcid);
@

Changes to src/setup.c.

1106
1107
1108
1109
1110
1111
1112








1113
1114
1115
1116
1117
1118
1119
  @ <hr />
  onoff_attribute("Allow REMOTE_USER authentication",
     "remote_user_ok", "remote_user_ok", 0, 0);
  @ <p>When enabled, if the REMOTE_USER environment variable is set to the
  @ login name of a valid user and no other login credentials are available,
  @ then the REMOTE_USER is accepted as an authenticated user.
  @ </p>








  @
  @ <hr />
  entry_attribute("IP address terms used in login cookie", 3,
                  "ip-prefix-terms", "ipt", "2", 0);
  @ <p>The number of octets of of the IP address used in the login cookie.
  @ Set to zero to omit the IP address from the login cookie.  A value of
  @ 2 is recommended.







>
>
>
>
>
>
>
>







1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
  @ <hr />
  onoff_attribute("Allow REMOTE_USER authentication",
     "remote_user_ok", "remote_user_ok", 0, 0);
  @ <p>When enabled, if the REMOTE_USER environment variable is set to the
  @ login name of a valid user and no other login credentials are available,
  @ then the REMOTE_USER is accepted as an authenticated user.
  @ </p>
  @
  @ <hr />
  onoff_attribute("Allow HTTP_AUTHENTICATION authentication",
     "http_authentication_ok", "http_authentication_ok", 0, 0);
  @ <p>When enabled, allow the use of the HTTP_AUTHENTICATION environment
  @ variable or the "Authentication:" HTTP header to find the username and
  @ password. This is another way of supporting Basic Authenitication.
  @ </p>
  @
  @ <hr />
  entry_attribute("IP address terms used in login cookie", 3,
                  "ip-prefix-terms", "ipt", "2", 0);
  @ <p>The number of octets of of the IP address used in the login cookie.
  @ Set to zero to omit the IP address from the login cookie.  A value of
  @ 2 is recommended.

Changes to src/sha1.c.

1
















2
3
4
5

6
7













































8
9
10
11
12
13
14
15





16
17
18
19
20
21
22
23
24
..
51
52
53
54
55
56
57



58
59
60
61
62
63
64
...
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
...
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
...
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
...
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
...
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
...
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
...
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
/*
















** This implementation of SHA1.
*/
#include "config.h"
#include <sys/types.h>

#include "sha1.h"














































#ifdef FOSSIL_ENABLE_SSL

# include <openssl/sha.h>
# define SHA1Context SHA_CTX
# define SHA1Init SHA1_Init
# define SHA1Update SHA1_Update
# define SHA1Final(a,b) SHA1_Final(b,a)






#else

/*
** The SHA1 implementation below is adapted from:
**
**  $NetBSD: sha1.c,v 1.6 2009/11/06 20:31:18 joerg Exp $
**  $OpenBSD: sha1.c,v 1.9 1997/07/23 21:12:32 kstailey Exp $
**
** SHA-1 in C
................................................................................

#else
/* Generic C equivalent */
#define SHA_ROT(x,l,r) ((x) << (l) | (x) >> (r))
#define rol(x,k) SHA_ROT(x,k,32-(k))
#define ror(x,k) SHA_ROT(x,32-(k),k)
#endif





#define blk0le(i) (block[i] = (ror(block[i],8)&0xFF00FF00) \
    |(rol(block[i],8)&0x00FF00FF))
#define blk0be(i) block[i]
#define blk(i) (block[i&15] = rol(block[(i+13)&15]^block[(i+8)&15] \
    ^block[(i+2)&15]^block[i&15],1))
................................................................................
    (void)memcpy(&context->buffer[j], &data[i], len - i);
}


/*
 * Add padding and return the message digest.
 */
static void SHA1Final(SHA1Context *context, unsigned char digest[20]){
    unsigned int i;
    unsigned char finalcount[8];

    for (i = 0; i < 8; i++) {
        finalcount[i] = (unsigned char)((context->count[(i >= 4 ? 0 : 1)]
         >> ((3-(i & 3)) * 8) ) & 255); /* Endian independent */
    }
................................................................................

    if (digest) {
        for (i = 0; i < 20; i++)
            digest[i] = (unsigned char)
                ((context->state[i>>2] >> ((3-(i & 3)) * 8) ) & 255);
    }
}
#endif


/*
** Convert a digest into base-16.  digest should be declared as
** "unsigned char digest[20]" in the calling function.  The SHA1
** digest is stored in the first 20 bytes.  zBuf should
** be "char zBuf[41]".
*/
................................................................................
** of computation.  The return pointer points to a static buffer that
** is overwritten by subsequent calls to this function.
*/
char *sha1sum_finish(Blob *pOut){
  unsigned char zResult[20];
  static char zOut[41];
  sha1sum_step_text(0,0);
  SHA1Final(&incrCtx, zResult);
  incrInit = 0;
  DigestToBase16(zResult, zOut);
  if( pOut ){
    blob_zero(pOut);
    blob_append(pOut, zOut, 40);
  }
  return zOut;
................................................................................
    n = fread(zBuf, 1, sizeof(zBuf), in);
    if( n<=0 ) break;
    SHA1Update(&ctx, (unsigned char*)zBuf, (unsigned)n);
  }
  fclose(in);
  blob_zero(pCksum);
  blob_resize(pCksum, 40);
  SHA1Final(&ctx, zResult);
  DigestToBase16(zResult, blob_buffer(pCksum));
  return 0;
}

/*
** Compute the SHA1 checksum of a blob in memory.  Store the resulting
** checksum in the blob pCksum.  pCksum is assumed to be either
................................................................................
  SHA1Update(&ctx, (unsigned char*)blob_buffer(pIn), blob_size(pIn));
  if( pIn==pCksum ){
    blob_reset(pCksum);
  }else{
    blob_zero(pCksum);
  }
  blob_resize(pCksum, 40);
  SHA1Final(&ctx, zResult);
  DigestToBase16(zResult, blob_buffer(pCksum));
  return 0;
}

/*
** Compute the SHA1 checksum of a zero-terminated string.  The
** result is held in memory obtained from mprintf().
................................................................................
char *sha1sum(const char *zIn){
  SHA1Context ctx;
  unsigned char zResult[20];
  char zDigest[41];

  SHA1Init(&ctx);
  SHA1Update(&ctx, (unsigned const char*)zIn, strlen(zIn));
  SHA1Final(&ctx, zResult);
  DigestToBase16(zResult, zDigest);
  return mprintf("%s", zDigest);
}

/*
** Convert a cleartext password for a specific user into a SHA1 hash.
**
................................................................................
    zProjCode = zProjectId;
  }
  SHA1Update(&ctx, (unsigned char*)zProjCode, strlen(zProjCode));
  SHA1Update(&ctx, (unsigned char*)"/", 1);
  SHA1Update(&ctx, (unsigned char*)zLogin, strlen(zLogin));
  SHA1Update(&ctx, (unsigned char*)"/", 1);
  SHA1Update(&ctx, (unsigned const char*)zPw, strlen(zPw));
  SHA1Final(&ctx, zResult);
  DigestToBase16(zResult, zDigest);
  return mprintf("%s", zDigest);
}

/*
** Implement the shared_secret() SQL function.  shared_secret() takes two or
** three arguments; the third argument is optional.

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>




>


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|





|

>
>
>
>
>

<







 







>
>
>







 







|







 







|
<







 







|







 







|







 







|







 







|







 







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

84
85
86
87
88
89
90
...
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
...
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
...
274
275
276
277
278
279
280
281

282
283
284
285
286
287
288
...
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
...
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
...
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
...
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
...
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
/*
** Copyright (c) 2006 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
**
*******************************************************************************
**
** This implementation of SHA1.
*/
#include "config.h"
#include <sys/types.h>
#include <stdint.h>
#include "sha1.h"


/*
** SHA1 Implementation #1 is the hardened SHA1 implementation by
** Marc Stevens.  Code obtained from GitHub
**
**     https://github.com/cr-marcstevens/sha1collisiondetection
**
** Downloaded on 2017-03-01 then repackaged to work with Fossil
** and makeheaders.
*/
#if FOSSIL_HARDENED_SHA1

#if INTERFACE
typedef void(*collision_block_callback)(uint64_t, const uint32_t*, const uint32_t*, const uint32_t*, const uint32_t*);
struct SHA1_CTX {
  uint64_t total;
  uint32_t ihv[5];
  unsigned char buffer[64];
  int bigendian;
  int found_collision;
  int safe_hash;
  int detect_coll;
  int ubc_check;
  int reduced_round_coll;
  collision_block_callback callback;

  uint32_t ihv1[5];
  uint32_t ihv2[5];
  uint32_t m1[80];
  uint32_t m2[80];
  uint32_t states[80][5];
};
#endif
void SHA1DCInit(SHA1_CTX*);
void SHA1DCUpdate(SHA1_CTX*, const unsigned char*, unsigned);
int SHA1DCFinal(unsigned char[20], SHA1_CTX*);

#define SHA1Context SHA1_CTX
#define SHA1Init SHA1DCInit
#define SHA1Update SHA1DCUpdate
#define SHA1Final SHA1DCFinal

/*
** SHA1 Implementation #2: use the SHA1 algorithm built into SSL
*/
#elif  defined(FOSSIL_ENABLE_SSL)

# include <openssl/sha.h>
# define SHA1Context SHA_CTX
# define SHA1Init SHA1_Init
# define SHA1Update SHA1_Update
# define SHA1Final SHA1_Final

/*
** SHA1 Implementation #3:  If none of the previous two SHA1
** algorithms work, there is this built-in.  This built-in was the
** original implementation used by Fossil.
*/
#else

/*
** The SHA1 implementation below is adapted from:
**
**  $NetBSD: sha1.c,v 1.6 2009/11/06 20:31:18 joerg Exp $
**  $OpenBSD: sha1.c,v 1.9 1997/07/23 21:12:32 kstailey Exp $
**
** SHA-1 in C
................................................................................

#else
/* Generic C equivalent */
#define SHA_ROT(x,l,r) ((x) << (l) | (x) >> (r))
#define rol(x,k) SHA_ROT(x,k,32-(k))
#define ror(x,k) SHA_ROT(x,32-(k),k)
#endif





#define blk0le(i) (block[i] = (ror(block[i],8)&0xFF00FF00) \
    |(rol(block[i],8)&0x00FF00FF))
#define blk0be(i) block[i]
#define blk(i) (block[i&15] = rol(block[(i+13)&15]^block[(i+8)&15] \
    ^block[(i+2)&15]^block[i&15],1))
................................................................................
    (void)memcpy(&context->buffer[j], &data[i], len - i);
}


/*
 * Add padding and return the message digest.
 */
static void SHA1Final(unsigned char *digest, SHA1Context *context){
    unsigned int i;
    unsigned char finalcount[8];

    for (i = 0; i < 8; i++) {
        finalcount[i] = (unsigned char)((context->count[(i >= 4 ? 0 : 1)]
         >> ((3-(i & 3)) * 8) ) & 255); /* Endian independent */
    }
................................................................................

    if (digest) {
        for (i = 0; i < 20; i++)
            digest[i] = (unsigned char)
                ((context->state[i>>2] >> ((3-(i & 3)) * 8) ) & 255);
    }
}
#endif /* Built-in SHA1 implemenation */


/*
** Convert a digest into base-16.  digest should be declared as
** "unsigned char digest[20]" in the calling function.  The SHA1
** digest is stored in the first 20 bytes.  zBuf should
** be "char zBuf[41]".
*/
................................................................................
** of computation.  The return pointer points to a static buffer that
** is overwritten by subsequent calls to this function.
*/
char *sha1sum_finish(Blob *pOut){
  unsigned char zResult[20];
  static char zOut[41];
  sha1sum_step_text(0,0);
  SHA1Final(zResult, &incrCtx);
  incrInit = 0;
  DigestToBase16(zResult, zOut);
  if( pOut ){
    blob_zero(pOut);
    blob_append(pOut, zOut, 40);
  }
  return zOut;
................................................................................
    n = fread(zBuf, 1, sizeof(zBuf), in);
    if( n<=0 ) break;
    SHA1Update(&ctx, (unsigned char*)zBuf, (unsigned)n);
  }
  fclose(in);
  blob_zero(pCksum);
  blob_resize(pCksum, 40);
  SHA1Final(zResult, &ctx);
  DigestToBase16(zResult, blob_buffer(pCksum));
  return 0;
}

/*
** Compute the SHA1 checksum of a blob in memory.  Store the resulting
** checksum in the blob pCksum.  pCksum is assumed to be either
................................................................................
  SHA1Update(&ctx, (unsigned char*)blob_buffer(pIn), blob_size(pIn));
  if( pIn==pCksum ){
    blob_reset(pCksum);
  }else{
    blob_zero(pCksum);
  }
  blob_resize(pCksum, 40);
  SHA1Final(zResult, &ctx);
  DigestToBase16(zResult, blob_buffer(pCksum));
  return 0;
}

/*
** Compute the SHA1 checksum of a zero-terminated string.  The
** result is held in memory obtained from mprintf().
................................................................................
char *sha1sum(const char *zIn){
  SHA1Context ctx;
  unsigned char zResult[20];
  char zDigest[41];

  SHA1Init(&ctx);
  SHA1Update(&ctx, (unsigned const char*)zIn, strlen(zIn));
  SHA1Final(zResult, &ctx);
  DigestToBase16(zResult, zDigest);
  return mprintf("%s", zDigest);
}

/*
** Convert a cleartext password for a specific user into a SHA1 hash.
**
................................................................................
    zProjCode = zProjectId;
  }
  SHA1Update(&ctx, (unsigned char*)zProjCode, strlen(zProjCode));
  SHA1Update(&ctx, (unsigned char*)"/", 1);
  SHA1Update(&ctx, (unsigned char*)zLogin, strlen(zLogin));
  SHA1Update(&ctx, (unsigned char*)"/", 1);
  SHA1Update(&ctx, (unsigned const char*)zPw, strlen(zPw));
  SHA1Final(zResult, &ctx);
  DigestToBase16(zResult, zDigest);
  return mprintf("%s", zDigest);
}

/*
** Implement the shared_secret() SQL function.  shared_secret() takes two or
** three arguments; the third argument is optional.

Added src/sha1hard.c.

























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
/*
** The code in this file is the concatenation of several files
** copied out of
**
**     https://github.com/cr-marcstevens/sha1collisiondetection
**
** The copy was made on 2017-03-01.  Some minor formatting changes
** were made but otherwise the code is unchanged.  All
** original copyright claims are preserved.
**
** The code here implements a version of the SHA1 hash function that
** is not vulnerable to crypto-analysis based attacks.  If an input
** is detected that looks like it might have been the result of a
** crypto-analysis attack, then the hash is perturbed to generate a
** completely different hash.  The authors claim that the chance of
** a false-positive is vanishingly small.
*/
/*MAKEHEADERS-STOP*/
#include "config.h"
#if FOSSIL_HARDENED_SHA1  /* Only do this code if requested */
/*************** File:  lib/sha1.c ****************/
/***
* Copyright 2017 Marc Stevens <marc@marc-stevens.nl>, Dan Shumow (danshu@microsoft.com)
* Distributed under the MIT Software License.
* See accompanying file LICENSE.txt or copy at
* https://opensource.org/licenses/MIT
***/
/*************** File:  LICENSE.txt ***************/
/*
** MIT License
**
** Copyright (c) 2017:
**     Marc Stevens
**     Cryptology Group
**     Centrum Wiskunde & Informatica
**     P.O. Box 94079, 1090 GB Amsterdam, Netherlands
**     marc@marc-stevens.nl
**
**     Dan Shumow
**     Microsoft Research
**     danshu@microsoft.com
**
** Permission is hereby granted, free of charge, to any person obtaining a copy
** of this software and associated documentation files (the "Software"), to deal
** in the Software without restriction, including without limitation the rights
** to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
** copies of the Software, and to permit persons to whom the Software is
** furnished to do so, subject to the following conditions:
**
** The above copyright notice and this permission notice shall be included in all
** copies or substantial portions of the Software.
**
** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
** SOFTWARE.
*/

#include <string.h>
#include <memory.h>
#include <stdio.h>

#define DVMASKSIZE 1
typedef struct { int dvType; int dvK; int dvB; int testt; int maski; int maskb; uint32_t dm[80]; } dv_info_t;
#define DOSTORESTATE58
#define DOSTORESTATE65
typedef void(*sha1_recompression_type)(uint32_t*, uint32_t*, const uint32_t*, const uint32_t*);
void sha1_message_expansion(uint32_t W[80]);
void sha1_compression(uint32_t ihv[5], const uint32_t m[16]);
void sha1_compression_W(uint32_t ihv[5], const uint32_t W[80]);
void sha1_compression_states(uint32_t ihv[5], const uint32_t W[80], uint32_t states[80][5]);
extern sha1_recompression_type sha1_recompression_step[80];
typedef void(*collision_block_callback)(uint64_t, const uint32_t*, const uint32_t*, const uint32_t*, const uint32_t*);
typedef struct {
	uint64_t total;
	uint32_t ihv[5];
	unsigned char buffer[64];
	int bigendian;
	int found_collision;
	int safe_hash;
	int detect_coll;
	int ubc_check;
	int reduced_round_coll;
	collision_block_callback callback;

	uint32_t ihv1[5];
	uint32_t ihv2[5];
	uint32_t m1[80];
	uint32_t m2[80];
	uint32_t states[80][5];
} SHA1_CTX;

/******************** File: lib/ubc_check.c **************************/
/***
* Copyright 2017 Marc Stevens <marc@marc-stevens.nl>, Dan Shumow <danshu@microsoft.com>
* Distributed under the MIT Software License.
* See accompanying file LICENSE.txt or copy at
* https://opensource.org/licenses/MIT
***/

/*
** this file was generated by the 'parse_bitrel' program in the tools section
** using the data files from directory 'tools/data/3565'
**
** sha1_dvs contains a list of SHA-1 Disturbance Vectors (DV) to check
** dvType, dvK and dvB define the DV: I(K,B) or II(K,B) (see the paper)
** dm[80] is the expanded message block XOR-difference defined by the DV
** testt is the step to do the recompression from for collision detection
** maski and maskb define the bit to check for each DV in the dvmask returned by ubc_check
**
** ubc_check takes as input an expanded message block and verifies the unavoidable bitconditions for all listed DVs
** it returns a dvmask where each bit belonging to a DV is set if all unavoidable bitconditions for that DV have been met
** thus one needs to do the recompression check for each DV that has its bit set
**
** ubc_check is programmatically generated and the unavoidable bitconditions have been hardcoded
** a directly verifiable version named ubc_check_verify can be found in ubc_check_verify.c
** ubc_check has been verified against ubc_check_verify using the 'ubc_check_test' program in the tools section
*/

static const uint32_t DV_I_43_0_bit   = (uint32_t)(1) << 0;
static const uint32_t DV_I_44_0_bit   = (uint32_t)(1) << 1;
static const uint32_t DV_I_45_0_bit   = (uint32_t)(1) << 2;
static const uint32_t DV_I_46_0_bit   = (uint32_t)(1) << 3;
static const uint32_t DV_I_46_2_bit   = (uint32_t)(1) << 4;
static const uint32_t DV_I_47_0_bit   = (uint32_t)(1) << 5;
static const uint32_t DV_I_47_2_bit   = (uint32_t)(1) << 6;
static const uint32_t DV_I_48_0_bit   = (uint32_t)(1) << 7;
static const uint32_t DV_I_48_2_bit   = (uint32_t)(1) << 8;
static const uint32_t DV_I_49_0_bit   = (uint32_t)(1) << 9;
static const uint32_t DV_I_49_2_bit   = (uint32_t)(1) << 10;
static const uint32_t DV_I_50_0_bit   = (uint32_t)(1) << 11;
static const uint32_t DV_I_50_2_bit   = (uint32_t)(1) << 12;
static const uint32_t DV_I_51_0_bit   = (uint32_t)(1) << 13;
static const uint32_t DV_I_51_2_bit   = (uint32_t)(1) << 14;
static const uint32_t DV_I_52_0_bit   = (uint32_t)(1) << 15;
static const uint32_t DV_II_45_0_bit   = (uint32_t)(1) << 16;
static const uint32_t DV_II_46_0_bit   = (uint32_t)(1) << 17;
static const uint32_t DV_II_46_2_bit   = (uint32_t)(1) << 18;
static const uint32_t DV_II_47_0_bit   = (uint32_t)(1) << 19;
static const uint32_t DV_II_48_0_bit   = (uint32_t)(1) << 20;
static const uint32_t DV_II_49_0_bit   = (uint32_t)(1) << 21;
static const uint32_t DV_II_49_2_bit   = (uint32_t)(1) << 22;
static const uint32_t DV_II_50_0_bit   = (uint32_t)(1) << 23;
static const uint32_t DV_II_50_2_bit   = (uint32_t)(1) << 24;
static const uint32_t DV_II_51_0_bit   = (uint32_t)(1) << 25;
static const uint32_t DV_II_51_2_bit   = (uint32_t)(1) << 26;
static const uint32_t DV_II_52_0_bit   = (uint32_t)(1) << 27;
static const uint32_t DV_II_53_0_bit   = (uint32_t)(1) << 28;
static const uint32_t DV_II_54_0_bit   = (uint32_t)(1) << 29;
static const uint32_t DV_II_55_0_bit   = (uint32_t)(1) << 30;
static const uint32_t DV_II_56_0_bit   = (uint32_t)(1) << 31;

dv_info_t sha1_dvs[] =
{
  {1,43,0,58,0,0, { 0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408,0x800000e6,0x8000004c,0x00000803,0x80000161,0x80000599 } }
, {1,44,0,58,0,1, { 0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408,0x800000e6,0x8000004c,0x00000803,0x80000161 } }
, {1,45,0,58,0,2, { 0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408,0x800000e6,0x8000004c,0x00000803 } }
, {1,46,0,58,0,3, { 0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408,0x800000e6,0x8000004c } }
, {1,46,2,58,0,4, { 0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a,0x00000060,0x00000590,0x00001020,0x0000039a,0x00000132 } }
, {1,47,0,58,0,5, { 0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408,0x800000e6 } }
, {1,47,2,58,0,6, { 0x20000043,0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a,0x00000060,0x00000590,0x00001020,0x0000039a } }
, {1,48,0,58,0,7, { 0xb800000a,0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164,0x00000408 } }
, {1,48,2,58,0,8, { 0xe000002a,0x20000043,0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a,0x00000060,0x00000590,0x00001020 } }
, {1,49,0,58,0,9, { 0x18000000,0xb800000a,0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018,0x00000164 } }
, {1,49,2,58,0,10, { 0x60000000,0xe000002a,0x20000043,0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a,0x00000060,0x00000590 } }
, {1,50,0,65,0,11, { 0x0800000c,0x18000000,0xb800000a,0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202,0x00000018 } }
, {1,50,2,65,0,12, { 0x20000030,0x60000000,0xe000002a,0x20000043,0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a,0x00000060 } }
, {1,51,0,65,0,13, { 0xe8000000,0x0800000c,0x18000000,0xb800000a,0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012,0x80000202 } }
, {1,51,2,65,0,14, { 0xa0000003,0x20000030,0x60000000,0xe000002a,0x20000043,0xb0000040,0xd0000053,0xd0000022,0x20000000,0x60000032,0x60000043,0x20000040,0xe0000042,0x60000002,0x80000001,0x00000020,0x00000003,0x40000052,0x40000040,0xe0000052,0xa0000000,0x80000040,0x20000001,0x20000060,0x80000001,0x40000042,0xc0000043,0x40000022,0x00000003,0x40000042,0xc0000043,0xc0000022,0x00000001,0x40000002,0xc0000043,0x40000062,0x80000001,0x40000042,0x40000042,0x40000002,0x00000002,0x00000040,0x80000002,0x80000000,0x80000002,0x80000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000000,0x00000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000101,0x00000009,0x00000012,0x00000202,0x0000001a,0x00000124,0x0000040c,0x00000026,0x0000004a,0x0000080a } }
, {1,52,0,65,0,15, { 0x04000010,0xe8000000,0x0800000c,0x18000000,0xb800000a,0xc8000010,0x2c000010,0xf4000014,0xb4000008,0x08000000,0x9800000c,0xd8000010,0x08000010,0xb8000010,0x98000000,0x60000000,0x00000008,0xc0000000,0x90000014,0x10000010,0xb8000014,0x28000000,0x20000010,0x48000000,0x08000018,0x60000000,0x90000010,0xf0000010,0x90000008,0xc0000000,0x90000010,0xf0000010,0xb0000008,0x40000000,0x90000000,0xf0000010,0x90000018,0x60000000,0x90000010,0x90000010,0x90000000,0x80000000,0x00000010,0xa0000000,0x20000000,0xa0000000,0x20000010,0x00000000,0x20000010,0x20000000,0x00000010,0x20000000,0x00000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000040,0x40000002,0x80000004,0x80000080,0x80000006,0x00000049,0x00000103,0x80000009,0x80000012 } }
, {2,45,0,58,0,16, { 0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b,0x8000016d,0x8000041a,0x000002e4,0x80000054,0x00000967 } }
, {2,46,0,58,0,17, { 0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b,0x8000016d,0x8000041a,0x000002e4,0x80000054 } }
, {2,46,2,58,0,18, { 0x90000070,0xb0000053,0x30000008,0x00000043,0xd0000072,0xb0000010,0xf0000062,0xc0000042,0x00000030,0xe0000042,0x20000060,0xe0000041,0x20000050,0xc0000041,0xe0000072,0xa0000003,0xc0000012,0x60000041,0xc0000032,0x20000001,0xc0000002,0xe0000042,0x60000042,0x80000002,0x00000000,0x00000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000001,0x00000060,0x80000003,0x40000002,0xc0000040,0xc0000002,0x80000000,0x80000000,0x80000002,0x00000040,0x00000002,0x80000000,0x80000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000105,0x00000089,0x00000016,0x0000020b,0x0000011b,0x0000012d,0x0000041e,0x00000224,0x00000050,0x0000092e,0x0000046c,0x000005b6,0x0000106a,0x00000b90,0x00000152 } }
, {2,47,0,58,0,19, { 0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b,0x8000016d,0x8000041a,0x000002e4 } }
, {2,48,0,58,0,20, { 0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b,0x8000016d,0x8000041a } }
, {2,49,0,58,0,21, { 0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b,0x8000016d } }
, {2,49,2,58,0,22, { 0xf0000010,0xf000006a,0x80000040,0x90000070,0xb0000053,0x30000008,0x00000043,0xd0000072,0xb0000010,0xf0000062,0xc0000042,0x00000030,0xe0000042,0x20000060,0xe0000041,0x20000050,0xc0000041,0xe0000072,0xa0000003,0xc0000012,0x60000041,0xc0000032,0x20000001,0xc0000002,0xe0000042,0x60000042,0x80000002,0x00000000,0x00000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000001,0x00000060,0x80000003,0x40000002,0xc0000040,0xc0000002,0x80000000,0x80000000,0x80000002,0x00000040,0x00000002,0x80000000,0x80000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000105,0x00000089,0x00000016,0x0000020b,0x0000011b,0x0000012d,0x0000041e,0x00000224,0x00000050,0x0000092e,0x0000046c,0x000005b6 } }
, {2,50,0,65,0,23, { 0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b,0x0000011b } }
, {2,50,2,65,0,24, { 0xd0000072,0xf0000010,0xf000006a,0x80000040,0x90000070,0xb0000053,0x30000008,0x00000043,0xd0000072,0xb0000010,0xf0000062,0xc0000042,0x00000030,0xe0000042,0x20000060,0xe0000041,0x20000050,0xc0000041,0xe0000072,0xa0000003,0xc0000012,0x60000041,0xc0000032,0x20000001,0xc0000002,0xe0000042,0x60000042,0x80000002,0x00000000,0x00000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000001,0x00000060,0x80000003,0x40000002,0xc0000040,0xc0000002,0x80000000,0x80000000,0x80000002,0x00000040,0x00000002,0x80000000,0x80000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000105,0x00000089,0x00000016,0x0000020b,0x0000011b,0x0000012d,0x0000041e,0x00000224,0x00000050,0x0000092e,0x0000046c } }
, {2,51,0,65,0,25, { 0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014,0x8000024b } }
, {2,51,2,65,0,26, { 0x00000043,0xd0000072,0xf0000010,0xf000006a,0x80000040,0x90000070,0xb0000053,0x30000008,0x00000043,0xd0000072,0xb0000010,0xf0000062,0xc0000042,0x00000030,0xe0000042,0x20000060,0xe0000041,0x20000050,0xc0000041,0xe0000072,0xa0000003,0xc0000012,0x60000041,0xc0000032,0x20000001,0xc0000002,0xe0000042,0x60000042,0x80000002,0x00000000,0x00000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000000,0x00000040,0x80000001,0x00000060,0x80000003,0x40000002,0xc0000040,0xc0000002,0x80000000,0x80000000,0x80000002,0x00000040,0x00000002,0x80000000,0x80000000,0x80000000,0x00000002,0x00000040,0x00000000,0x80000040,0x80000002,0x00000000,0x80000000,0x80000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000004,0x00000080,0x00000004,0x00000009,0x00000105,0x00000089,0x00000016,0x0000020b,0x0000011b,0x0000012d,0x0000041e,0x00000224,0x00000050,0x0000092e } }
, {2,52,0,65,0,27, { 0x0c000002,0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089,0x00000014 } }
, {2,53,0,65,0,28, { 0xcc000014,0x0c000002,0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107,0x00000089 } }
, {2,54,0,65,0,29, { 0x0400001c,0xcc000014,0x0c000002,0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b,0x80000107 } }
, {2,55,0,65,0,30, { 0x00000010,0x0400001c,0xcc000014,0x0c000002,0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046,0x4000004b } }
, {2,56,0,65,0,31, { 0x2600001a,0x00000010,0x0400001c,0xcc000014,0x0c000002,0xc0000010,0xb400001c,0x3c000004,0xbc00001a,0x20000010,0x2400001c,0xec000014,0x0c000002,0xc0000010,0xb400001c,0x2c000004,0xbc000018,0xb0000010,0x0000000c,0xb8000010,0x08000018,0x78000010,0x08000014,0x70000010,0xb800001c,0xe8000000,0xb0000004,0x58000010,0xb000000c,0x48000000,0xb0000000,0xb8000010,0x98000010,0xa0000000,0x00000000,0x00000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0x20000000,0x00000010,0x60000000,0x00000018,0xe0000000,0x90000000,0x30000010,0xb0000000,0x20000000,0x20000000,0xa0000000,0x00000010,0x80000000,0x20000000,0x20000000,0x20000000,0x80000000,0x00000010,0x00000000,0x20000010,0xa0000000,0x00000000,0x20000000,0x20000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000000,0x00000001,0x00000020,0x00000001,0x40000002,0x40000041,0x40000022,0x80000005,0xc0000082,0xc0000046 } }
, {0,0,0,0,0,0, {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}}
};
void ubc_check(const uint32_t W[80], uint32_t dvmask[1])
{
  uint32_t mask = ~((uint32_t)(0));
  mask &= (((((W[44]^W[45])>>29)&1)-1) | ~(DV_I_48_0_bit|DV_I_51_0_bit|DV_I_52_0_bit|DV_II_45_0_bit|DV_II_46_0_bit|DV_II_50_0_bit|DV_II_51_0_bit));
  mask &= (((((W[49]^W[50])>>29)&1)-1) | ~(DV_I_46_0_bit|DV_II_45_0_bit|DV_II_50_0_bit|DV_II_51_0_bit|DV_II_55_0_bit|DV_II_56_0_bit));
  mask &= (((((W[48]^W[49])>>29)&1)-1) | ~(DV_I_45_0_bit|DV_I_52_0_bit|DV_II_49_0_bit|DV_II_50_0_bit|DV_II_54_0_bit|DV_II_55_0_bit));
  mask &= ((((W[47]^(W[50]>>25))&(1<<4))-(1<<4)) | ~(DV_I_47_0_bit|DV_I_49_0_bit|DV_I_51_0_bit|DV_II_45_0_bit|DV_II_51_0_bit|DV_II_56_0_bit));
  mask &= (((((W[47]^W[48])>>29)&1)-1) | ~(DV_I_44_0_bit|DV_I_51_0_bit|DV_II_48_0_bit|DV_II_49_0_bit|DV_II_53_0_bit|DV_II_54_0_bit));
  mask &= (((((W[46]>>4)^(W[49]>>29))&1)-1) | ~(DV_I_46_0_bit|DV_I_48_0_bit|DV_I_50_0_bit|DV_I_52_0_bit|DV_II_50_0_bit|DV_II_55_0_bit));
  mask &= (((((W[46]^W[47])>>29)&1)-1) | ~(DV_I_43_0_bit|DV_I_50_0_bit|DV_II_47_0_bit|DV_II_48_0_bit|DV_II_52_0_bit|DV_II_53_0_bit));
  mask &= (((((W[45]>>4)^(W[48]>>29))&1)-1) | ~(DV_I_45_0_bit|DV_I_47_0_bit|DV_I_49_0_bit|DV_I_51_0_bit|DV_II_49_0_bit|DV_II_54_0_bit));
  mask &= (((((W[45]^W[46])>>29)&1)-1) | ~(DV_I_49_0_bit|DV_I_52_0_bit|DV_II_46_0_bit|DV_II_47_0_bit|DV_II_51_0_bit|DV_II_52_0_bit));
  mask &= (((((W[44]>>4)^(W[47]>>29))&1)-1) | ~(DV_I_44_0_bit|DV_I_46_0_bit|DV_I_48_0_bit|DV_I_50_0_bit|DV_II_48_0_bit|DV_II_53_0_bit));
  mask &= (((((W[43]>>4)^(W[46]>>29))&1)-1) | ~(DV_I_43_0_bit|DV_I_45_0_bit|DV_I_47_0_bit|DV_I_49_0_bit|DV_II_47_0_bit|DV_II_52_0_bit));
  mask &= (((((W[43]^W[44])>>29)&1)-1) | ~(DV_I_47_0_bit|DV_I_50_0_bit|DV_I_51_0_bit|DV_II_45_0_bit|DV_II_49_0_bit|DV_II_50_0_bit));
  mask &= (((((W[42]>>4)^(W[45]>>29))&1)-1) | ~(DV_I_44_0_bit|DV_I_46_0_bit|DV_I_48_0_bit|DV_I_52_0_bit|DV_II_46_0_bit|DV_II_51_0_bit));
  mask &= (((((W[41]>>4)^(W[44]>>29))&1)-1) | ~(DV_I_43_0_bit|DV_I_45_0_bit|DV_I_47_0_bit|DV_I_51_0_bit|DV_II_45_0_bit|DV_II_50_0_bit));
  mask &= (((((W[40]^W[41])>>29)&1)-1) | ~(DV_I_44_0_bit|DV_I_47_0_bit|DV_I_48_0_bit|DV_II_46_0_bit|DV_II_47_0_bit|DV_II_56_0_bit));
  mask &= (((((W[54]^W[55])>>29)&1)-1) | ~(DV_I_51_0_bit|DV_II_47_0_bit|DV_II_50_0_bit|DV_II_55_0_bit|DV_II_56_0_bit));
  mask &= (((((W[53]^W[54])>>29)&1)-1) | ~(DV_I_50_0_bit|DV_II_46_0_bit|DV_II_49_0_bit|DV_II_54_0_bit|DV_II_55_0_bit));
  mask &= (((((W[52]^W[53])>>29)&1)-1) | ~(DV_I_49_0_bit|DV_II_45_0_bit|DV_II_48_0_bit|DV_II_53_0_bit|DV_II_54_0_bit));
  mask &= ((((W[50]^(W[53]>>25))&(1<<4))-(1<<4)) | ~(DV_I_50_0_bit|DV_I_52_0_bit|DV_II_46_0_bit|DV_II_48_0_bit|DV_II_54_0_bit));
  mask &= (((((W[50]^W[51])>>29)&1)-1) | ~(DV_I_47_0_bit|DV_II_46_0_bit|DV_II_51_0_bit|DV_II_52_0_bit|DV_II_56_0_bit));
  mask &= ((((W[49]^(W[52]>>25))&(1<<4))-(1<<4)) | ~(DV_I_49_0_bit|DV_I_51_0_bit|DV_II_45_0_bit|DV_II_47_0_bit|DV_II_53_0_bit));
  mask &= ((((W[48]^(W[51]>>25))&(1<<4))-(1<<4)) | ~(DV_I_48_0_bit|DV_I_50_0_bit|DV_I_52_0_bit|DV_II_46_0_bit|DV_II_52_0_bit));
  mask &= (((((W[42]^W[43])>>29)&1)-1) | ~(DV_I_46_0_bit|DV_I_49_0_bit|DV_I_50_0_bit|DV_II_48_0_bit|DV_II_49_0_bit));
  mask &= (((((W[41]^W[42])>>29)&1)-1) | ~(DV_I_45_0_bit|DV_I_48_0_bit|DV_I_49_0_bit|DV_II_47_0_bit|DV_II_48_0_bit));
  mask &= (((((W[40]>>4)^(W[43]>>29))&1)-1) | ~(DV_I_44_0_bit|DV_I_46_0_bit|DV_I_50_0_bit|DV_II_49_0_bit|DV_II_56_0_bit));
  mask &= (((((W[39]>>4)^(W[42]>>29))&1)-1) | ~(DV_I_43_0_bit|DV_I_45_0_bit|DV_I_49_0_bit|DV_II_48_0_bit|DV_II_55_0_bit));
  if (mask & (DV_I_44_0_bit|DV_I_48_0_bit|DV_II_47_0_bit|DV_II_54_0_bit|DV_II_56_0_bit))
    mask &= (((((W[38]>>4)^(W[41]>>29))&1)-1) | ~(DV_I_44_0_bit|DV_I_48_0_bit|DV_II_47_0_bit|DV_II_54_0_bit|DV_II_56_0_bit));
  mask &= (((((W[37]>>4)^(W[40]>>29))&1)-1) | ~(DV_I_43_0_bit|DV_I_47_0_bit|DV_II_46_0_bit|DV_II_53_0_bit|DV_II_55_0_bit));
  if (mask & (DV_I_52_0_bit|DV_II_48_0_bit|DV_II_51_0_bit|DV_II_56_0_bit))
    mask &= (((((W[55]^W[56])>>29)&1)-1) | ~(DV_I_52_0_bit|DV_II_48_0_bit|DV_II_51_0_bit|DV_II_56_0_bit));
  if (mask & (DV_I_52_0_bit|DV_II_48_0_bit|DV_II_50_0_bit|DV_II_56_0_bit))
    mask &= ((((W[52]^(W[55]>>25))&(1<<4))-(1<<4)) | ~(DV_I_52_0_bit|DV_II_48_0_bit|DV_II_50_0_bit|DV_II_56_0_bit));
  if (mask & (DV_I_51_0_bit|DV_II_47_0_bit|DV_II_49_0_bit|DV_II_55_0_bit))
    mask &= ((((W[51]^(W[54]>>25))&(1<<4))-(1<<4)) | ~(DV_I_51_0_bit|DV_II_47_0_bit|DV_II_49_0_bit|DV_II_55_0_bit));
  if (mask & (DV_I_48_0_bit|DV_II_47_0_bit|DV_II_52_0_bit|DV_II_53_0_bit))
    mask &= (((((W[51]^W[52])>>29)&1)-1) | ~(DV_I_48_0_bit|DV_II_47_0_bit|DV_II_52_0_bit|DV_II_53_0_bit));
  if (mask & (DV_I_46_0_bit|DV_I_49_0_bit|DV_II_45_0_bit|DV_II_48_0_bit))
    mask &= (((((W[36]>>4)^(W[40]>>29))&1)-1) | ~(DV_I_46_0_bit|DV_I_49_0_bit|DV_II_45_0_bit|DV_II_48_0_bit));
  if (mask & (DV_I_52_0_bit|DV_II_48_0_bit|DV_II_49_0_bit))
    mask &= ((0-(((W[53]^W[56])>>29)&1)) | ~(DV_I_52_0_bit|DV_II_48_0_bit|DV_II_49_0_bit));
  if (mask & (DV_I_50_0_bit|DV_II_46_0_bit|DV_II_47_0_bit))
    mask &= ((0-(((W[51]^W[54])>>29)&1)) | ~(DV_I_50_0_bit|DV_II_46_0_bit|DV_II_47_0_bit));
  if (mask & (DV_I_49_0_bit|DV_I_51_0_bit|DV_II_45_0_bit))
    mask &= ((0-(((W[50]^W[52])>>29)&1)) | ~(DV_I_49_0_bit|DV_I_51_0_bit|DV_II_45_0_bit));
  if (mask & (DV_I_48_0_bit|DV_I_50_0_bit|DV_I_52_0_bit))
    mask &= ((0-(((W[49]^W[51])>>29)&1)) | ~(DV_I_48_0_bit|DV_I_50_0_bit|DV_I_52_0_bit));
  if (mask & (DV_I_47_0_bit|DV_I_49_0_bit|DV_I_51_0_bit))
    mask &= ((0-(((W[48]^W[50])>>29)&1)) | ~(DV_I_47_0_bit|DV_I_49_0_bit|DV_I_51_0_bit));
  if (mask & (DV_I_46_0_bit|DV_I_48_0_bit|DV_I_50_0_bit))
    mask &= ((0-(((W[47]^W[49])>>29)&1)) | ~(DV_I_46_0_bit|DV_I_48_0_bit|DV_I_50_0_bit));
  if (mask & (DV_I_45_0_bit|DV_I_47_0_bit|DV_I_49_0_bit))
    mask &= ((0-(((W[46]^W[48])>>29)&1)) | ~(DV_I_45_0_bit|DV_I_47_0_bit|DV_I_49_0_bit));
  mask &= ((((W[45]^W[47])&(1<<6))-(1<<6)) | ~(DV_I_47_2_bit|DV_I_49_2_bit|DV_I_51_2_bit));
  if (mask & (DV_I_44_0_bit|DV_I_46_0_bit|DV_I_48_0_bit))
    mask &= ((0-(((W[45]^W[47])>>29)&1)) | ~(DV_I_44_0_bit|DV_I_46_0_bit|DV_I_48_0_bit));
  mask &= (((((W[44]^W[46])>>6)&1)-1) | ~(DV_I_46_2_bit|DV_I_48_2_bit|DV_I_50_2_bit));
  if (mask & (DV_I_43_0_bit|DV_I_45_0_bit|DV_I_47_0_bit))
    mask &= ((0-(((W[44]^W[46])>>29)&1)) | ~(DV_I_43_0_bit|DV_I_45_0_bit|DV_I_47_0_bit));
  mask &= ((0-((W[41]^(W[42]>>5))&(1<<1))) | ~(DV_I_48_2_bit|DV_II_46_2_bit|DV_II_51_2_bit));
  mask &= ((0-((W[40]^(W[41]>>5))&(1<<1))) | ~(DV_I_47_2_bit|DV_I_51_2_bit|DV_II_50_2_bit));
  if (mask & (DV_I_44_0_bit|DV_I_46_0_bit|DV_II_56_0_bit))
    mask &= ((0-(((W[40]^W[42])>>4)&1)) | ~(DV_I_44_0_bit|DV_I_46_0_bit|DV_II_56_0_bit));
  mask &= ((0-((W[39]^(W[40]>>5))&(1<<1))) | ~(DV_I_46_2_bit|DV_I_50_2_bit|DV_II_49_2_bit));
  if (mask & (DV_I_43_0_bit|DV_I_45_0_bit|DV_II_55_0_bit))
    mask &= ((0-(((W[39]^W[41])>>4)&1)) | ~(DV_I_43_0_bit|DV_I_45_0_bit|DV_II_55_0_bit));
  if (mask & (DV_I_44_0_bit|DV_II_54_0_bit|DV_II_56_0_bit))
    mask &= ((0-(((W[38]^W[40])>>4)&1)) | ~(DV_I_44_0_bit|DV_II_54_0_bit|DV_II_56_0_bit));
  if (mask & (DV_I_43_0_bit|DV_II_53_0_bit|DV_II_55_0_bit))
    mask &= ((0-(((W[37]^W[39])>>4)&1)) | ~(DV_I_43_0_bit|DV_II_53_0_bit|DV_II_55_0_bit));
  mask &= ((0-((W[36]^(W[37]>>5))&(1<<1))) | ~(DV_I_47_2_bit|DV_I_50_2_bit|DV_II_46_2_bit));
  if (mask & (DV_I_45_0_bit|DV_I_48_0_bit|DV_II_47_0_bit))
    mask &= (((((W[35]>>4)^(W[39]>>29))&1)-1) | ~(DV_I_45_0_bit|DV_I_48_0_bit|DV_II_47_0_bit));
  if (mask & (DV_I_48_0_bit|DV_II_48_0_bit))
    mask &= ((0-((W[63]^(W[64]>>5))&(1<<0))) | ~(DV_I_48_0_bit|DV_II_48_0_bit));
  if (mask & (DV_I_45_0_bit|DV_II_45_0_bit))
    mask &= ((0-((W[63]^(W[64]>>5))&(1<<1))) | ~(DV_I_45_0_bit|DV_II_45_0_bit));
  if (mask & (DV_I_47_0_bit|DV_II_47_0_bit))
    mask &= ((0-((W[62]^(W[63]>>5))&(1<<0))) | ~(DV_I_47_0_bit|DV_II_47_0_bit));
  if (mask & (DV_I_46_0_bit|DV_II_46_0_bit))
    mask &= ((0-((W[61]^(W[62]>>5))&(1<<0))) | ~(DV_I_46_0_bit|DV_II_46_0_bit));
  mask &= ((0-((W[61]^(W[62]>>5))&(1<<2))) | ~(DV_I_46_2_bit|DV_II_46_2_bit));
  if (mask & (DV_I_45_0_bit|DV_II_45_0_bit))
    mask &= ((0-((W[60]^(W[61]>>5))&(1<<0))) | ~(DV_I_45_0_bit|DV_II_45_0_bit));
  if (mask & (DV_II_51_0_bit|DV_II_54_0_bit))
    mask &= (((((W[58]^W[59])>>29)&1)-1) | ~(DV_II_51_0_bit|DV_II_54_0_bit));
  if (mask & (DV_II_50_0_bit|DV_II_53_0_bit))
    mask &= (((((W[57]^W[58])>>29)&1)-1) | ~(DV_II_50_0_bit|DV_II_53_0_bit));
  if (mask & (DV_II_52_0_bit|DV_II_54_0_bit))
    mask &= ((((W[56]^(W[59]>>25))&(1<<4))-(1<<4)) | ~(DV_II_52_0_bit|DV_II_54_0_bit));
  if (mask & (DV_II_51_0_bit|DV_II_52_0_bit))
    mask &= ((0-(((W[56]^W[59])>>29)&1)) | ~(DV_II_51_0_bit|DV_II_52_0_bit));
  if (mask & (DV_II_49_0_bit|DV_II_52_0_bit))
    mask &= (((((W[56]^W[57])>>29)&1)-1) | ~(DV_II_49_0_bit|DV_II_52_0_bit));
  if (mask & (DV_II_51_0_bit|DV_II_53_0_bit))
    mask &= ((((W[55]^(W[58]>>25))&(1<<4))-(1<<4)) | ~(DV_II_51_0_bit|DV_II_53_0_bit));
  if (mask & (DV_II_50_0_bit|DV_II_52_0_bit))
    mask &= ((((W[54]^(W[57]>>25))&(1<<4))-(1<<4)) | ~(DV_II_50_0_bit|DV_II_52_0_bit));
  if (mask & (DV_II_49_0_bit|DV_II_51_0_bit))
    mask &= ((((W[53]^(W[56]>>25))&(1<<4))-(1<<4)) | ~(DV_II_49_0_bit|DV_II_51_0_bit));
  mask &= ((((W[51]^(W[50]>>5))&(1<<1))-(1<<1)) | ~(DV_I_50_2_bit|DV_II_46_2_bit));
  mask &= ((((W[48]^W[50])&(1<<6))-(1<<6)) | ~(DV_I_50_2_bit|DV_II_46_2_bit));
  if (mask & (DV_I_51_0_bit|DV_I_52_0_bit))
    mask &= ((0-(((W[48]^W[55])>>29)&1)) | ~(DV_I_51_0_bit|DV_I_52_0_bit));
  mask &= ((((W[47]^W[49])&(1<<6))-(1<<6)) | ~(DV_I_49_2_bit|DV_I_51_2_bit));
  mask &= ((((W[48]^(W[47]>>5))&(1<<1))-(1<<1)) | ~(DV_I_47_2_bit|DV_II_51_2_bit));
  mask &= ((((W[46]^W[48])&(1<<6))-(1<<6)) | ~(DV_I_48_2_bit|DV_I_50_2_bit));
  mask &= ((((W[47]^(W[46]>>5))&(1<<1))-(1<<1)) | ~(DV_I_46_2_bit|DV_II_50_2_bit));
  mask &= ((0-((W[44]^(W[45]>>5))&(1<<1))) | ~(DV_I_51_2_bit|DV_II_49_2_bit));
  mask &= ((((W[43]^W[45])&(1<<6))-(1<<6)) | ~(DV_I_47_2_bit|DV_I_49_2_bit));
  mask &= (((((W[42]^W[44])>>6)&1)-1) | ~(DV_I_46_2_bit|DV_I_48_2_bit));
  mask &= ((((W[43]^(W[42]>>5))&(1<<1))-(1<<1)) | ~(DV_II_46_2_bit|DV_II_51_2_bit));
  mask &= ((((W[42]^(W[41]>>5))&(1<<1))-(1<<1)) | ~(DV_I_51_2_bit|DV_II_50_2_bit));
  mask &= ((((W[41]^(W[40]>>5))&(1<<1))-(1<<1)) | ~(DV_I_50_2_bit|DV_II_49_2_bit));
  if (mask & (DV_I_52_0_bit|DV_II_51_0_bit))
    mask &= ((((W[39]^(W[43]>>25))&(1<<4))-(1<<4)) | ~(DV_I_52_0_bit|DV_II_51_0_bit));
  if (mask & (DV_I_51_0_bit|DV_II_50_0_bit))
    mask &= ((((W[38]^(W[42]>>25))&(1<<4))-(1<<4)) | ~(DV_I_51_0_bit|DV_II_50_0_bit));
  if (mask & (DV_I_48_2_bit|DV_I_51_2_bit))
    mask &= ((0-((W[37]^(W[38]>>5))&(1<<1))) | ~(DV_I_48_2_bit|DV_I_51_2_bit));
  if (mask & (DV_I_50_0_bit|DV_II_49_0_bit))
    mask &= ((((W[37]^(W[41]>>25))&(1<<4))-(1<<4)) | ~(DV_I_50_0_bit|DV_II_49_0_bit));
  if (mask & (DV_II_52_0_bit|DV_II_54_0_bit))
    mask &= ((0-((W[36]^W[38])&(1<<4))) | ~(DV_II_52_0_bit|DV_II_54_0_bit));
  mask &= ((0-((W[35]^(W[36]>>5))&(1<<1))) | ~(DV_I_46_2_bit|DV_I_49_2_bit));
  if (mask & (DV_I_51_0_bit|DV_II_47_0_bit))
    mask &= ((((W[35]^(W[39]>>25))&(1<<3))-(1<<3)) | ~(DV_I_51_0_bit|DV_II_47_0_bit));
if (mask) {

  if (mask & DV_I_43_0_bit)
     if (
          !((W[61]^(W[62]>>5)) & (1<<1))
       || !(!((W[59]^(W[63]>>25)) & (1<<5)))
       || !((W[58]^(W[63]>>30)) & (1<<0))
     )  mask &= ~DV_I_43_0_bit;
  if (mask & DV_I_44_0_bit)
     if (
          !((W[62]^(W[63]>>5)) & (1<<1))
       || !(!((W[60]^(W[64]>>25)) & (1<<5)))
       || !((W[59]^(W[64]>>30)) & (1<<0))
     )  mask &= ~DV_I_44_0_bit;
  if (mask & DV_I_46_2_bit)
    mask &= ((~((W[40]^W[42])>>2)) | ~DV_I_46_2_bit);
  if (mask & DV_I_47_2_bit)
     if (
          !((W[62]^(W[63]>>5)) & (1<<2))
       || !(!((W[41]^W[43]) & (1<<6)))
     )  mask &= ~DV_I_47_2_bit;
  if (mask & DV_I_48_2_bit)
     if (
          !((W[63]^(W[64]>>5)) & (1<<2))
       || !(!((W[48]^(W[49]<<5)) & (1<<6)))
     )  mask &= ~DV_I_48_2_bit;
  if (mask & DV_I_49_2_bit)
     if (
          !(!((W[49]^(W[50]<<5)) & (1<<6)))
       || !((W[42]^W[50]) & (1<<1))
       || !(!((W[39]^(W[40]<<5)) & (1<<6)))
       || !((W[38]^W[40]) & (1<<1))
     )  mask &= ~DV_I_49_2_bit;
  if (mask & DV_I_50_0_bit)
    mask &= ((((W[36]^W[37])<<7)) | ~DV_I_50_0_bit);
  if (mask & DV_I_50_2_bit)
    mask &= ((((W[43]^W[51])<<11)) | ~DV_I_50_2_bit);
  if (mask & DV_I_51_0_bit)
    mask &= ((((W[37]^W[38])<<9)) | ~DV_I_51_0_bit);
  if (mask & DV_I_51_2_bit)
     if (
          !(!((W[51]^(W[52]<<5)) & (1<<6)))
       || !(!((W[49]^W[51]) & (1<<6)))
       || !(!((W[37]^(W[37]>>5)) & (1<<1)))
       || !(!((W[35]^(W[39]>>25)) & (1<<5)))
     )  mask &= ~DV_I_51_2_bit;
  if (mask & DV_I_52_0_bit)
    mask &= ((((W[38]^W[39])<<11)) | ~DV_I_52_0_bit);
  if (mask & DV_II_46_2_bit)
    mask &= ((((W[47]^W[51])<<17)) | ~DV_II_46_2_bit);
  if (mask & DV_II_48_0_bit)
     if (
          !(!((W[36]^(W[40]>>25)) & (1<<3)))
       || !((W[35]^(W[40]<<2)) & (1<<30))
     )  mask &= ~DV_II_48_0_bit;
  if (mask & DV_II_49_0_bit)
     if (
          !(!((W[37]^(W[41]>>25)) & (1<<3)))
       || !((W[36]^(W[41]<<2)) & (1<<30))
     )  mask &= ~DV_II_49_0_bit;
  if (mask & DV_II_49_2_bit)
     if (
          !(!((W[53]^(W[54]<<5)) & (1<<6)))
       || !(!((W[51]^W[53]) & (1<<6)))
       || !((W[50]^W[54]) & (1<<1))
       || !(!((W[45]^(W[46]<<5)) & (1<<6)))
       || !(!((W[37]^(W[41]>>25)) & (1<<5)))
       || !((W[36]^(W[41]>>30)) & (1<<0))
     )  mask &= ~DV_II_49_2_bit;
  if (mask & DV_II_50_0_bit)
     if (
          !((W[55]^W[58]) & (1<<29))
       || !(!((W[38]^(W[42]>>25)) & (1<<3)))
       || !((W[37]^(W[42]<<2)) & (1<<30))
     )  mask &= ~DV_II_50_0_bit;
  if (mask & DV_II_50_2_bit)
     if (
          !(!((W[54]^(W[55]<<5)) & (1<<6)))
       || !(!((W[52]^W[54]) & (1<<6)))
       || !((W[51]^W[55]) & (1<<1))
       || !((W[45]^W[47]) & (1<<1))
       || !(!((W[38]^(W[42]>>25)) & (1<<5)))
       || !((W[37]^(W[42]>>30)) & (1<<0))
     )  mask &= ~DV_II_50_2_bit;
  if (mask & DV_II_51_0_bit)
     if (
          !(!((W[39]^(W[43]>>25)) & (1<<3)))
       || !((W[38]^(W[43]<<2)) & (1<<30))
     )  mask &= ~DV_II_51_0_bit;
  if (mask & DV_II_51_2_bit)
     if (
          !(!((W[55]^(W[56]<<5)) & (1<<6)))
       || !(!((W[53]^W[55]) & (1<<6)))
       || !((W[52]^W[56]) & (1<<1))
       || !((W[46]^W[48]) & (1<<1))
       || !(!((W[39]^(W[43]>>25)) & (1<<5)))
       || !((W[38]^(W[43]>>30)) & (1<<0))
     )  mask &= ~DV_II_51_2_bit;
  if (mask & DV_II_52_0_bit)
     if (
          !(!((W[59]^W[60]) & (1<<29)))
       || !(!((W[40]^(W[44]>>25)) & (1<<3)))
       || !(!((W[40]^(W[44]>>25)) & (1<<4)))
       || !((W[39]^(W[44]<<2)) & (1<<30))
     )  mask &= ~DV_II_52_0_bit;
  if (mask & DV_II_53_0_bit)
     if (
          !((W[58]^W[61]) & (1<<29))
       || !(!((W[57]^(W[61]>>25)) & (1<<4)))
       || !(!((W[41]^(W[45]>>25)) & (1<<3)))
       || !(!((W[41]^(W[45]>>25)) & (1<<4)))
     )  mask &= ~DV_II_53_0_bit;
  if (mask & DV_II_54_0_bit)
     if (
          !(!((W[58]^(W[62]>>25)) & (1<<4)))
       || !(!((W[42]^(W[46]>>25)) & (1<<3)))
       || !(!((W[42]^(W[46]>>25)) & (1<<4)))
     )  mask &= ~DV_II_54_0_bit;
  if (mask & DV_II_55_0_bit)
     if (
          !(!((W[59]^(W[63]>>25)) & (1<<4)))
       || !(!((W[57]^(W[59]>>25)) & (1<<4)))
       || !(!((W[43]^(W[47]>>25)) & (1<<3)))
       || !(!((W[43]^(W[47]>>25)) & (1<<4)))
     )  mask &= ~DV_II_55_0_bit;
  if (mask & DV_II_56_0_bit)
     if (
          !(!((W[60]^(W[64]>>25)) & (1<<4)))
       || !(!((W[44]^(W[48]>>25)) & (1<<3)))
       || !(!((W[44]^(W[48]>>25)) & (1<<4)))
     )  mask &= ~DV_II_56_0_bit;
}

  dvmask[0]=mask;
}
/******************** End Of File: lib/ubc_check.c *******************/
/******************** Continue with: lib/sha1.c **********************/


#define rotate_right(x,n) (((x)>>(n))|((x)<<(32-(n))))
#define rotate_left(x,n)  (((x)<<(n))|((x)>>(32-(n))))

#define sha1_f1(b,c,d) ((d)^((b)&((c)^(d))))
#define sha1_f2(b,c,d) ((b)^(c)^(d))
#define sha1_f3(b,c,d) (((b) & ((c)|(d))) | ((c)&(d)))
#define sha1_f4(b,c,d) ((b)^(c)^(d))

#define HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, m, t) \
  { e += rotate_left(a, 5) + sha1_f1(b,c,d) + 0x5A827999 + m[t]; b = rotate_left(b, 30); }
#define HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, m, t) \
  { e += rotate_left(a, 5) + sha1_f2(b,c,d) + 0x6ED9EBA1 + m[t]; b = rotate_left(b, 30); }
#define HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, m, t) \
  { e += rotate_left(a, 5) + sha1_f3(b,c,d) + 0x8F1BBCDC + m[t]; b = rotate_left(b, 30); }
#define HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, m, t) \
  { e += rotate_left(a, 5) + sha1_f4(b,c,d) + 0xCA62C1D6 + m[t]; b = rotate_left(b, 30); }

#define HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(a, b, c, d, e, m, t) \
  { b = rotate_right(b, 30); e -= rotate_left(a, 5) + sha1_f1(b,c,d) + 0x5A827999 + m[t]; }
#define HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(a, b, c, d, e, m, t) \
  { b = rotate_right(b, 30); e -= rotate_left(a, 5) + sha1_f2(b,c,d) + 0x6ED9EBA1 + m[t]; }
#define HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(a, b, c, d, e, m, t) \
  { b = rotate_right(b, 30); e -= rotate_left(a, 5) + sha1_f3(b,c,d) + 0x8F1BBCDC + m[t]; }
#define HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(a, b, c, d, e, m, t) \
  { b = rotate_right(b, 30); e -= rotate_left(a, 5) + sha1_f4(b,c,d) + 0xCA62C1D6 + m[t]; }

#define SHA1_STORE_STATE(i) states[i][0] = a; states[i][1] = b; states[i][2] = c; states[i][3] = d; states[i][4] = e;



void sha1_message_expansion(uint32_t W[80])
{
  unsigned i;
  for (i = 16; i < 80; ++i)
    W[i] = rotate_left(W[i - 3] ^ W[i - 8] ^ W[i - 14] ^ W[i - 16], 1);
}

void sha1_compression(uint32_t ihv[5], const uint32_t m[16])
{
  uint32_t W[80];
  uint32_t a,b,c,d,e;
  unsigned i;

  memcpy(W, m, 16 * 4);
  for (i = 16; i < 80; ++i)
    W[i] = rotate_left(W[i - 3] ^ W[i - 8] ^ W[i - 14] ^ W[i - 16], 1);

  a = ihv[0]; b = ihv[1]; c = ihv[2]; d = ihv[3]; e = ihv[4];

  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 0);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 1);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 2);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 3);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 4);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 5);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 6);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 7);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 8);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 9);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 10);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 11);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 12);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 13);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 14);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 15);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 16);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 17);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 18);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 19);

  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 20);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 21);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 22);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 23);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 24);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 25);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 26);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 27);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 28);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 29);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 30);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 31);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 32);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 33);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 34);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 35);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 36);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 37);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 38);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 39);

  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 40);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 41);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 42);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 43);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 44);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 45);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 46);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 47);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 48);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 49);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 50);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 51);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 52);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 53);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 54);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 55);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 56);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 57);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 58);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 59);

  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 60);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 61);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 62);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 63);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 64);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 65);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 66);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 67);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 68);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 69);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 70);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 71);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 72);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 73);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 74);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 75);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 76);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 77);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 78);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 79);

  ihv[0] += a; ihv[1] += b; ihv[2] += c; ihv[3] += d; ihv[4] += e;
}



void sha1_compression_W(uint32_t ihv[5], const uint32_t W[80])
{
  uint32_t a = ihv[0], b = ihv[1], c = ihv[2], d = ihv[3], e = ihv[4];

  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 0);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 1);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 2);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 3);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 4);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 5);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 6);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 7);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 8);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 9);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 10);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 11);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 12);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 13);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 14);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 15);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 16);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 17);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 18);
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 19);

  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 20);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 21);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 22);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 23);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 24);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 25);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 26);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 27);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 28);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 29);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 30);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 31);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 32);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 33);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 34);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 35);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 36);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 37);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 38);
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 39);

  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 40);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 41);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 42);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 43);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 44);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 45);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 46);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 47);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 48);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 49);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 50);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 51);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 52);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 53);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 54);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 55);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 56);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 57);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 58);
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 59);

  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 60);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 61);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 62);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 63);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 64);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 65);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 66);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 67);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 68);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 69);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 70);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 71);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 72);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 73);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 74);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 75);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 76);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 77);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 78);
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 79);

  ihv[0] += a; ihv[1] += b; ihv[2] += c; ihv[3] += d; ihv[4] += e;
}



void sha1_compression_states(uint32_t ihv[5], const uint32_t W[80], uint32_t states[80][5])
{
  uint32_t a = ihv[0], b = ihv[1], c = ihv[2], d = ihv[3], e = ihv[4];

#ifdef DOSTORESTATE00
  SHA1_STORE_STATE(0)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 0);

#ifdef DOSTORESTATE01
  SHA1_STORE_STATE(1)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 1);

#ifdef DOSTORESTATE02
  SHA1_STORE_STATE(2)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 2);

#ifdef DOSTORESTATE03
  SHA1_STORE_STATE(3)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 3);

#ifdef DOSTORESTATE04
  SHA1_STORE_STATE(4)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 4);

#ifdef DOSTORESTATE05
  SHA1_STORE_STATE(5)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 5);

#ifdef DOSTORESTATE06
  SHA1_STORE_STATE(6)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 6);

#ifdef DOSTORESTATE07
  SHA1_STORE_STATE(7)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 7);

#ifdef DOSTORESTATE08
  SHA1_STORE_STATE(8)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 8);

#ifdef DOSTORESTATE09
  SHA1_STORE_STATE(9)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 9);

#ifdef DOSTORESTATE10
  SHA1_STORE_STATE(10)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 10);

#ifdef DOSTORESTATE11
  SHA1_STORE_STATE(11)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 11);

#ifdef DOSTORESTATE12
  SHA1_STORE_STATE(12)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 12);

#ifdef DOSTORESTATE13
  SHA1_STORE_STATE(13)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 13);

#ifdef DOSTORESTATE14
  SHA1_STORE_STATE(14)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 14);

#ifdef DOSTORESTATE15
  SHA1_STORE_STATE(15)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, W, 15);

#ifdef DOSTORESTATE16
  SHA1_STORE_STATE(16)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, W, 16);

#ifdef DOSTORESTATE17
  SHA1_STORE_STATE(17)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, W, 17);

#ifdef DOSTORESTATE18
  SHA1_STORE_STATE(18)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, W, 18);

#ifdef DOSTORESTATE19
  SHA1_STORE_STATE(19)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, W, 19);



#ifdef DOSTORESTATE20
  SHA1_STORE_STATE(20)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 20);

#ifdef DOSTORESTATE21
  SHA1_STORE_STATE(21)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 21);

#ifdef DOSTORESTATE22
  SHA1_STORE_STATE(22)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 22);

#ifdef DOSTORESTATE23
  SHA1_STORE_STATE(23)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 23);

#ifdef DOSTORESTATE24
  SHA1_STORE_STATE(24)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 24);

#ifdef DOSTORESTATE25
  SHA1_STORE_STATE(25)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 25);

#ifdef DOSTORESTATE26
  SHA1_STORE_STATE(26)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 26);

#ifdef DOSTORESTATE27
  SHA1_STORE_STATE(27)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 27);

#ifdef DOSTORESTATE28
  SHA1_STORE_STATE(28)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 28);

#ifdef DOSTORESTATE29
  SHA1_STORE_STATE(29)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 29);

#ifdef DOSTORESTATE30
  SHA1_STORE_STATE(30)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 30);

#ifdef DOSTORESTATE31
  SHA1_STORE_STATE(31)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 31);

#ifdef DOSTORESTATE32
  SHA1_STORE_STATE(32)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 32);

#ifdef DOSTORESTATE33
  SHA1_STORE_STATE(33)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 33);

#ifdef DOSTORESTATE34
  SHA1_STORE_STATE(34)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 34);

#ifdef DOSTORESTATE35
  SHA1_STORE_STATE(35)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, W, 35);

#ifdef DOSTORESTATE36
  SHA1_STORE_STATE(36)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, W, 36);

#ifdef DOSTORESTATE37
  SHA1_STORE_STATE(37)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, W, 37);

#ifdef DOSTORESTATE38
  SHA1_STORE_STATE(38)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, W, 38);

#ifdef DOSTORESTATE39
  SHA1_STORE_STATE(39)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, W, 39);



#ifdef DOSTORESTATE40
  SHA1_STORE_STATE(40)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 40);

#ifdef DOSTORESTATE41
  SHA1_STORE_STATE(41)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 41);

#ifdef DOSTORESTATE42
  SHA1_STORE_STATE(42)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 42);

#ifdef DOSTORESTATE43
  SHA1_STORE_STATE(43)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 43);

#ifdef DOSTORESTATE44
  SHA1_STORE_STATE(44)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 44);

#ifdef DOSTORESTATE45
  SHA1_STORE_STATE(45)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 45);

#ifdef DOSTORESTATE46
  SHA1_STORE_STATE(46)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 46);

#ifdef DOSTORESTATE47
  SHA1_STORE_STATE(47)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 47);

#ifdef DOSTORESTATE48
  SHA1_STORE_STATE(48)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 48);

#ifdef DOSTORESTATE49
  SHA1_STORE_STATE(49)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 49);

#ifdef DOSTORESTATE50
  SHA1_STORE_STATE(50)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 50);

#ifdef DOSTORESTATE51
  SHA1_STORE_STATE(51)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 51);

#ifdef DOSTORESTATE52
  SHA1_STORE_STATE(52)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 52);

#ifdef DOSTORESTATE53
  SHA1_STORE_STATE(53)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 53);

#ifdef DOSTORESTATE54
  SHA1_STORE_STATE(54)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 54);

#ifdef DOSTORESTATE55
  SHA1_STORE_STATE(55)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, W, 55);

#ifdef DOSTORESTATE56
  SHA1_STORE_STATE(56)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, W, 56);

#ifdef DOSTORESTATE57
  SHA1_STORE_STATE(57)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, W, 57);

#ifdef DOSTORESTATE58
  SHA1_STORE_STATE(58)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, W, 58);

#ifdef DOSTORESTATE59
  SHA1_STORE_STATE(59)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, W, 59);




#ifdef DOSTORESTATE60
  SHA1_STORE_STATE(60)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 60);

#ifdef DOSTORESTATE61
  SHA1_STORE_STATE(61)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 61);

#ifdef DOSTORESTATE62
  SHA1_STORE_STATE(62)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 62);

#ifdef DOSTORESTATE63
  SHA1_STORE_STATE(63)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 63);

#ifdef DOSTORESTATE64
  SHA1_STORE_STATE(64)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 64);

#ifdef DOSTORESTATE65
  SHA1_STORE_STATE(65)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 65);

#ifdef DOSTORESTATE66
  SHA1_STORE_STATE(66)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 66);

#ifdef DOSTORESTATE67
  SHA1_STORE_STATE(67)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 67);

#ifdef DOSTORESTATE68
  SHA1_STORE_STATE(68)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 68);

#ifdef DOSTORESTATE69
  SHA1_STORE_STATE(69)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 69);

#ifdef DOSTORESTATE70
  SHA1_STORE_STATE(70)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 70);

#ifdef DOSTORESTATE71
  SHA1_STORE_STATE(71)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 71);

#ifdef DOSTORESTATE72
  SHA1_STORE_STATE(72)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 72);

#ifdef DOSTORESTATE73
  SHA1_STORE_STATE(73)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 73);

#ifdef DOSTORESTATE74
  SHA1_STORE_STATE(74)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 74);

#ifdef DOSTORESTATE75
  SHA1_STORE_STATE(75)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, W, 75);

#ifdef DOSTORESTATE76
  SHA1_STORE_STATE(76)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, W, 76);

#ifdef DOSTORESTATE77
  SHA1_STORE_STATE(77)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, W, 77);

#ifdef DOSTORESTATE78
  SHA1_STORE_STATE(78)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, W, 78);

#ifdef DOSTORESTATE79
  SHA1_STORE_STATE(79)
#endif
  HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, W, 79);



  ihv[0] += a; ihv[1] += b; ihv[2] += c; ihv[3] += d; ihv[4] += e;
}




#define SHA1_RECOMPRESS(t) \
void sha1recompress_fast_ ## t (uint32_t ihvin[5], uint32_t ihvout[5], const uint32_t me2[80], const uint32_t state[5]) \
{ \
  uint32_t a = state[0], b = state[1], c = state[2], d = state[3], e = state[4]; \
  if (t > 79) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(b, c, d, e, a, me2, 79); \
  if (t > 78) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(c, d, e, a, b, me2, 78); \
  if (t > 77) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(d, e, a, b, c, me2, 77); \
  if (t > 76) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(e, a, b, c, d, me2, 76); \
  if (t > 75) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(a, b, c, d, e, me2, 75); \
  if (t > 74) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(b, c, d, e, a, me2, 74); \
  if (t > 73) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(c, d, e, a, b, me2, 73); \
  if (t > 72) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(d, e, a, b, c, me2, 72); \
  if (t > 71) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(e, a, b, c, d, me2, 71); \
  if (t > 70) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(a, b, c, d, e, me2, 70); \
  if (t > 69) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(b, c, d, e, a, me2, 69); \
  if (t > 68) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(c, d, e, a, b, me2, 68); \
  if (t > 67) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(d, e, a, b, c, me2, 67); \
  if (t > 66) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(e, a, b, c, d, me2, 66); \
  if (t > 65) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(a, b, c, d, e, me2, 65); \
  if (t > 64) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(b, c, d, e, a, me2, 64); \
  if (t > 63) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(c, d, e, a, b, me2, 63); \
  if (t > 62) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(d, e, a, b, c, me2, 62); \
  if (t > 61) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(e, a, b, c, d, me2, 61); \
  if (t > 60) HASHCLASH_SHA1COMPRESS_ROUND4_STEP_BW(a, b, c, d, e, me2, 60); \
  if (t > 59) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(b, c, d, e, a, me2, 59); \
  if (t > 58) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(c, d, e, a, b, me2, 58); \
  if (t > 57) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(d, e, a, b, c, me2, 57); \
  if (t > 56) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(e, a, b, c, d, me2, 56); \
  if (t > 55) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(a, b, c, d, e, me2, 55); \
  if (t > 54) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(b, c, d, e, a, me2, 54); \
  if (t > 53) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(c, d, e, a, b, me2, 53); \
  if (t > 52) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(d, e, a, b, c, me2, 52); \
  if (t > 51) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(e, a, b, c, d, me2, 51); \
  if (t > 50) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(a, b, c, d, e, me2, 50); \
  if (t > 49) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(b, c, d, e, a, me2, 49); \
  if (t > 48) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(c, d, e, a, b, me2, 48); \
  if (t > 47) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(d, e, a, b, c, me2, 47); \
  if (t > 46) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(e, a, b, c, d, me2, 46); \
  if (t > 45) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(a, b, c, d, e, me2, 45); \
  if (t > 44) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(b, c, d, e, a, me2, 44); \
  if (t > 43) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(c, d, e, a, b, me2, 43); \
  if (t > 42) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(d, e, a, b, c, me2, 42); \
  if (t > 41) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(e, a, b, c, d, me2, 41); \
  if (t > 40) HASHCLASH_SHA1COMPRESS_ROUND3_STEP_BW(a, b, c, d, e, me2, 40); \
  if (t > 39) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(b, c, d, e, a, me2, 39); \
  if (t > 38) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(c, d, e, a, b, me2, 38); \
  if (t > 37) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(d, e, a, b, c, me2, 37); \
  if (t > 36) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(e, a, b, c, d, me2, 36); \
  if (t > 35) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(a, b, c, d, e, me2, 35); \
  if (t > 34) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(b, c, d, e, a, me2, 34); \
  if (t > 33) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(c, d, e, a, b, me2, 33); \
  if (t > 32) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(d, e, a, b, c, me2, 32); \
  if (t > 31) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(e, a, b, c, d, me2, 31); \
  if (t > 30) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(a, b, c, d, e, me2, 30); \
  if (t > 29) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(b, c, d, e, a, me2, 29); \
  if (t > 28) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(c, d, e, a, b, me2, 28); \
  if (t > 27) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(d, e, a, b, c, me2, 27); \
  if (t > 26) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(e, a, b, c, d, me2, 26); \
  if (t > 25) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(a, b, c, d, e, me2, 25); \
  if (t > 24) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(b, c, d, e, a, me2, 24); \
  if (t > 23) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(c, d, e, a, b, me2, 23); \
  if (t > 22) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(d, e, a, b, c, me2, 22); \
  if (t > 21) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(e, a, b, c, d, me2, 21); \
  if (t > 20) HASHCLASH_SHA1COMPRESS_ROUND2_STEP_BW(a, b, c, d, e, me2, 20); \
  if (t > 19) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(b, c, d, e, a, me2, 19); \
  if (t > 18) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(c, d, e, a, b, me2, 18); \
  if (t > 17) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(d, e, a, b, c, me2, 17); \
  if (t > 16) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(e, a, b, c, d, me2, 16); \
  if (t > 15) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(a, b, c, d, e, me2, 15); \
  if (t > 14) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(b, c, d, e, a, me2, 14); \
  if (t > 13) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(c, d, e, a, b, me2, 13); \
  if (t > 12) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(d, e, a, b, c, me2, 12); \
  if (t > 11) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(e, a, b, c, d, me2, 11); \
  if (t > 10) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(a, b, c, d, e, me2, 10); \
  if (t > 9) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(b, c, d, e, a, me2, 9); \
  if (t > 8) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(c, d, e, a, b, me2, 8); \
  if (t > 7) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(d, e, a, b, c, me2, 7); \
  if (t > 6) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(e, a, b, c, d, me2, 6); \
  if (t > 5) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(a, b, c, d, e, me2, 5); \
  if (t > 4) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(b, c, d, e, a, me2, 4); \
  if (t > 3) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(c, d, e, a, b, me2, 3); \
  if (t > 2) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(d, e, a, b, c, me2, 2); \
  if (t > 1) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(e, a, b, c, d, me2, 1); \
  if (t > 0) HASHCLASH_SHA1COMPRESS_ROUND1_STEP_BW(a, b, c, d, e, me2, 0); \
  ihvin[0] = a; ihvin[1] = b; ihvin[2] = c; ihvin[3] = d; ihvin[4] = e; \
  a = state[0]; b = state[1]; c = state[2]; d = state[3]; e = state[4]; \
  if (t <= 0) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, me2, 0); \
  if (t <= 1) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, me2, 1); \
  if (t <= 2) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, me2, 2); \
  if (t <= 3) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, me2, 3); \
  if (t <= 4) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, me2, 4); \
  if (t <= 5) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, me2, 5); \
  if (t <= 6) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, me2, 6); \
  if (t <= 7) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, me2, 7); \
  if (t <= 8) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, me2, 8); \
  if (t <= 9) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, me2, 9); \
  if (t <= 10) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, me2, 10); \
  if (t <= 11) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, me2, 11); \
  if (t <= 12) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, me2, 12); \
  if (t <= 13) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, me2, 13); \
  if (t <= 14) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, me2, 14); \
  if (t <= 15) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(a, b, c, d, e, me2, 15); \
  if (t <= 16) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(e, a, b, c, d, me2, 16); \
  if (t <= 17) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(d, e, a, b, c, me2, 17); \
  if (t <= 18) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(c, d, e, a, b, me2, 18); \
  if (t <= 19) HASHCLASH_SHA1COMPRESS_ROUND1_STEP(b, c, d, e, a, me2, 19); \
  if (t <= 20) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, me2, 20); \
  if (t <= 21) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, me2, 21); \
  if (t <= 22) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, me2, 22); \
  if (t <= 23) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, me2, 23); \
  if (t <= 24) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, me2, 24); \
  if (t <= 25) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, me2, 25); \
  if (t <= 26) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, me2, 26); \
  if (t <= 27) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, me2, 27); \
  if (t <= 28) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, me2, 28); \
  if (t <= 29) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, me2, 29); \
  if (t <= 30) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, me2, 30); \
  if (t <= 31) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, me2, 31); \
  if (t <= 32) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, me2, 32); \
  if (t <= 33) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, me2, 33); \
  if (t <= 34) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, me2, 34); \
  if (t <= 35) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(a, b, c, d, e, me2, 35); \
  if (t <= 36) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(e, a, b, c, d, me2, 36); \
  if (t <= 37) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(d, e, a, b, c, me2, 37); \
  if (t <= 38) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(c, d, e, a, b, me2, 38); \
  if (t <= 39) HASHCLASH_SHA1COMPRESS_ROUND2_STEP(b, c, d, e, a, me2, 39); \
  if (t <= 40) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, me2, 40); \
  if (t <= 41) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, me2, 41); \
  if (t <= 42) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, me2, 42); \
  if (t <= 43) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, me2, 43); \
  if (t <= 44) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, me2, 44); \
  if (t <= 45) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, me2, 45); \
  if (t <= 46) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, me2, 46); \
  if (t <= 47) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, me2, 47); \
  if (t <= 48) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, me2, 48); \
  if (t <= 49) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, me2, 49); \
  if (t <= 50) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, me2, 50); \
  if (t <= 51) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, me2, 51); \
  if (t <= 52) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, me2, 52); \
  if (t <= 53) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, me2, 53); \
  if (t <= 54) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, me2, 54); \
  if (t <= 55) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(a, b, c, d, e, me2, 55); \
  if (t <= 56) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(e, a, b, c, d, me2, 56); \
  if (t <= 57) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(d, e, a, b, c, me2, 57); \
  if (t <= 58) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(c, d, e, a, b, me2, 58); \
  if (t <= 59) HASHCLASH_SHA1COMPRESS_ROUND3_STEP(b, c, d, e, a, me2, 59); \
  if (t <= 60) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, me2, 60); \
  if (t <= 61) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, me2, 61); \
  if (t <= 62) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, me2, 62); \
  if (t <= 63) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, me2, 63); \
  if (t <= 64) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, me2, 64); \
  if (t <= 65) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, me2, 65); \
  if (t <= 66) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, me2, 66); \
  if (t <= 67) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, me2, 67); \
  if (t <= 68) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, me2, 68); \
  if (t <= 69) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, me2, 69); \
  if (t <= 70) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, me2, 70); \
  if (t <= 71) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, me2, 71); \
  if (t <= 72) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, me2, 72); \
  if (t <= 73) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, me2, 73); \
  if (t <= 74) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, me2, 74); \
  if (t <= 75) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(a, b, c, d, e, me2, 75); \
  if (t <= 76) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(e, a, b, c, d, me2, 76); \
  if (t <= 77) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(d, e, a, b, c, me2, 77); \
  if (t <= 78) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(c, d, e, a, b, me2, 78); \
  if (t <= 79) HASHCLASH_SHA1COMPRESS_ROUND4_STEP(b, c, d, e, a, me2, 79); \
  ihvout[0] = ihvin[0] + a; ihvout[1] = ihvin[1] + b; ihvout[2] = ihvin[2] + c; ihvout[3] = ihvin[3] + d; ihvout[4] = ihvin[4] + e; \
}

SHA1_RECOMPRESS(0)
SHA1_RECOMPRESS(1)
SHA1_RECOMPRESS(2)
SHA1_RECOMPRESS(3)
SHA1_RECOMPRESS(4)
SHA1_RECOMPRESS(5)
SHA1_RECOMPRESS(6)
SHA1_RECOMPRESS(7)
SHA1_RECOMPRESS(8)
SHA1_RECOMPRESS(9)

SHA1_RECOMPRESS(10)
SHA1_RECOMPRESS(11)
SHA1_RECOMPRESS(12)
SHA1_RECOMPRESS(13)
SHA1_RECOMPRESS(14)
SHA1_RECOMPRESS(15)
SHA1_RECOMPRESS(16)
SHA1_RECOMPRESS(17)
SHA1_RECOMPRESS(18)
SHA1_RECOMPRESS(19)

SHA1_RECOMPRESS(20)
SHA1_RECOMPRESS(21)
SHA1_RECOMPRESS(22)
SHA1_RECOMPRESS(23)
SHA1_RECOMPRESS(24)
SHA1_RECOMPRESS(25)
SHA1_RECOMPRESS(26)
SHA1_RECOMPRESS(27)
SHA1_RECOMPRESS(28)
SHA1_RECOMPRESS(29)

SHA1_RECOMPRESS(30)
SHA1_RECOMPRESS(31)
SHA1_RECOMPRESS(32)
SHA1_RECOMPRESS(33)
SHA1_RECOMPRESS(34)
SHA1_RECOMPRESS(35)
SHA1_RECOMPRESS(36)
SHA1_RECOMPRESS(37)
SHA1_RECOMPRESS(38)
SHA1_RECOMPRESS(39)

SHA1_RECOMPRESS(40)
SHA1_RECOMPRESS(41)
SHA1_RECOMPRESS(42)
SHA1_RECOMPRESS(43)
SHA1_RECOMPRESS(44)
SHA1_RECOMPRESS(45)
SHA1_RECOMPRESS(46)
SHA1_RECOMPRESS(47)
SHA1_RECOMPRESS(48)
SHA1_RECOMPRESS(49)

SHA1_RECOMPRESS(50)
SHA1_RECOMPRESS(51)
SHA1_RECOMPRESS(52)
SHA1_RECOMPRESS(53)
SHA1_RECOMPRESS(54)
SHA1_RECOMPRESS(55)
SHA1_RECOMPRESS(56)
SHA1_RECOMPRESS(57)
SHA1_RECOMPRESS(58)
SHA1_RECOMPRESS(59)

SHA1_RECOMPRESS(60)
SHA1_RECOMPRESS(61)
SHA1_RECOMPRESS(62)
SHA1_RECOMPRESS(63)
SHA1_RECOMPRESS(64)
SHA1_RECOMPRESS(65)
SHA1_RECOMPRESS(66)
SHA1_RECOMPRESS(67)
SHA1_RECOMPRESS(68)
SHA1_RECOMPRESS(69)

SHA1_RECOMPRESS(70)
SHA1_RECOMPRESS(71)
SHA1_RECOMPRESS(72)
SHA1_RECOMPRESS(73)
SHA1_RECOMPRESS(74)
SHA1_RECOMPRESS(75)
SHA1_RECOMPRESS(76)
SHA1_RECOMPRESS(77)
SHA1_RECOMPRESS(78)
SHA1_RECOMPRESS(79)

sha1_recompression_type sha1_recompression_step[80] =
{
  sha1recompress_fast_0, sha1recompress_fast_1, sha1recompress_fast_2, sha1recompress_fast_3, sha1recompress_fast_4, sha1recompress_fast_5, sha1recompress_fast_6, sha1recompress_fast_7, sha1recompress_fast_8, sha1recompress_fast_9,
  sha1recompress_fast_10, sha1recompress_fast_11, sha1recompress_fast_12, sha1recompress_fast_13, sha1recompress_fast_14, sha1recompress_fast_15, sha1recompress_fast_16, sha1recompress_fast_17, sha1recompress_fast_18, sha1recompress_fast_19,
  sha1recompress_fast_20, sha1recompress_fast_21, sha1recompress_fast_22, sha1recompress_fast_23, sha1recompress_fast_24, sha1recompress_fast_25, sha1recompress_fast_26, sha1recompress_fast_27, sha1recompress_fast_28, sha1recompress_fast_29,
  sha1recompress_fast_30, sha1recompress_fast_31, sha1recompress_fast_32, sha1recompress_fast_33, sha1recompress_fast_34, sha1recompress_fast_35, sha1recompress_fast_36, sha1recompress_fast_37, sha1recompress_fast_38, sha1recompress_fast_39,
  sha1recompress_fast_40, sha1recompress_fast_41, sha1recompress_fast_42, sha1recompress_fast_43, sha1recompress_fast_44, sha1recompress_fast_45, sha1recompress_fast_46, sha1recompress_fast_47, sha1recompress_fast_48, sha1recompress_fast_49,
  sha1recompress_fast_50, sha1recompress_fast_51, sha1recompress_fast_52, sha1recompress_fast_53, sha1recompress_fast_54, sha1recompress_fast_55, sha1recompress_fast_56, sha1recompress_fast_57, sha1recompress_fast_58, sha1recompress_fast_59,
  sha1recompress_fast_60, sha1recompress_fast_61, sha1recompress_fast_62, sha1recompress_fast_63, sha1recompress_fast_64, sha1recompress_fast_65, sha1recompress_fast_66, sha1recompress_fast_67, sha1recompress_fast_68, sha1recompress_fast_69,
  sha1recompress_fast_70, sha1recompress_fast_71, sha1recompress_fast_72, sha1recompress_fast_73, sha1recompress_fast_74, sha1recompress_fast_75, sha1recompress_fast_76, sha1recompress_fast_77, sha1recompress_fast_78, sha1recompress_fast_79,
};





void sha1_process(SHA1_CTX* ctx, const uint32_t block[16])
{
  unsigned i, j;
  uint32_t ubc_dv_mask[DVMASKSIZE];
  uint32_t ihvtmp[5];
  for (i=0; i < DVMASKSIZE; ++i)
    ubc_dv_mask[i]=0;
  ctx->ihv1[0] = ctx->ihv[0];
  ctx->ihv1[1] = ctx->ihv[1];
  ctx->ihv1[2] = ctx->ihv[2];
  ctx->ihv1[3] = ctx->ihv[3];
  ctx->ihv1[4] = ctx->ihv[4];
  memcpy(ctx->m1, block, 64);
  sha1_message_expansion(ctx->m1);
  if (ctx->detect_coll && ctx->ubc_check)
  {
    ubc_check(ctx->m1, ubc_dv_mask);
  }
  sha1_compression_states(ctx->ihv, ctx->m1, ctx->states);
  if (ctx->detect_coll)
  {
    for (i = 0; sha1_dvs[i].dvType != 0; ++i)
    {
      if ((0 == ctx->ubc_check) || (((uint32_t)(1) << sha1_dvs[i].maskb) & ubc_dv_mask[sha1_dvs[i].maski]))
      {
        for (j = 0; j < 80; ++j)
          ctx->m2[j] = ctx->m1[j] ^ sha1_dvs[i].dm[j];
        (sha1_recompression_step[sha1_dvs[i].testt])(ctx->ihv2, ihvtmp, ctx->m2, ctx->states[sha1_dvs[i].testt]);
        /* to verify SHA-1 collision detection code with collisions for reduced-step SHA-1 */
        if ((ihvtmp[0] == ctx->ihv[0] && ihvtmp[1] == ctx->ihv[1] && ihvtmp[2] == ctx->ihv[2] && ihvtmp[3] == ctx->ihv[3] && ihvtmp[4] == ctx->ihv[4])
          || (ctx->reduced_round_coll && ctx->ihv1[0] == ctx->ihv2[0] && ctx->ihv1[1] == ctx->ihv2[1] && ctx->ihv1[2] == ctx->ihv2[2] && ctx->ihv1[3] == ctx->ihv2[3] && ctx->ihv1[4] == ctx->ihv2[4]))
        {
          ctx->found_collision = 1;
          /* TODO: call callback */
          if (ctx->callback != NULL)
            ctx->callback(ctx->total - 64, ctx->ihv1, ctx->ihv2, ctx->m1, ctx->m2);

          if (ctx->safe_hash)
          {
            sha1_compression_W(ctx->ihv, ctx->m1);
            sha1_compression_W(ctx->ihv, ctx->m1);
          }

          break;
        }
      }
    }
  }
}





void swap_bytes(uint32_t val[16])
{
  unsigned i;
  for (i = 0; i < 16; ++i)
  {
    val[i] = ((val[i] << 8) & 0xFF00FF00) | ((val[i] >> 8) & 0xFF00FF);
    val[i] = (val[i] << 16) | (val[i] >> 16);
  }
}

void SHA1DCInit(SHA1_CTX* ctx)
{
  static const union { unsigned char bytes[4]; uint32_t value; } endianness = { { 0, 1, 2, 3 } };
  static const uint32_t littleendian = 0x03020100;
  ctx->total = 0;
  ctx->ihv[0] = 0x67452301;
  ctx->ihv[1] = 0xEFCDAB89;
  ctx->ihv[2] = 0x98BADCFE;
  ctx->ihv[3] = 0x10325476;
  ctx->ihv[4] = 0xC3D2E1F0;
  ctx->found_collision = 0;
  ctx->safe_hash = 1;
  ctx->ubc_check = 1;
  ctx->detect_coll = 1;
  ctx->reduced_round_coll = 0;
  ctx->bigendian = (endianness.value != littleendian);
  ctx->callback = NULL;
}

void SHA1DCSetSafeHash(SHA1_CTX* ctx, int safehash)
{
  if (safehash)
    ctx->safe_hash = 1;
  else
    ctx->safe_hash = 0;
}


void SHA1DCSetUseUBC(SHA1_CTX* ctx, int ubc_check)
{
  if (ubc_check)
    ctx->ubc_check = 1;
  else
    ctx->ubc_check = 0;
}

void SHA1DCSetUseDetectColl(SHA1_CTX* ctx, int detect_coll)
{
  if (detect_coll)
    ctx->detect_coll = 1;
  else
    ctx->detect_coll = 0;
}

void SHA1DCSetDetectReducedRoundCollision(SHA1_CTX* ctx, int reduced_round_coll)
{
  if (reduced_round_coll)
    ctx->reduced_round_coll = 1;
  else
    ctx->reduced_round_coll = 0;
}

void SHA1DCSetCallback(SHA1_CTX* ctx, collision_block_callback callback)
{
  ctx->callback = callback;
}

void SHA1DCUpdate(SHA1_CTX* ctx, const unsigned char* buf, unsigned len)
{
  unsigned left, fill;
  if (len == 0)
    return;

  left = ctx->total & 63;
  fill = 64 - left;

  if (left && len >= fill)
  {
    ctx->total += fill;
    memcpy(ctx->buffer + left, buf, fill);
    if (!ctx->bigendian)
      swap_bytes((uint32_t*)(ctx->buffer));
    sha1_process(ctx, (uint32_t*)(ctx->buffer));
    buf += fill;
    len -= fill;
    left = 0;
  }
  while (len >= 64)
  {
    ctx->total += 64;
    if (!ctx->bigendian)
    {
      memcpy(ctx->buffer, buf, 64);
      swap_bytes((uint32_t*)(ctx->buffer));
      sha1_process(ctx, (uint32_t*)(ctx->buffer));
    }
    else
      sha1_process(ctx, (uint32_t*)(buf));
    buf += 64;
    len -= 64;
  }
  if (len > 0)
  {
    ctx->total += len;
    memcpy(ctx->buffer + left, buf, len);
  }
}

static const unsigned char sha1_padding[64] =
{
  0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
};

int SHA1DCFinal(unsigned char output[20], SHA1_CTX *ctx)
{
  uint32_t last = ctx->total & 63;
  uint32_t padn = (last < 56) ? (56 - last) : (120 - last);
  uint64_t total;
  SHA1DCUpdate(ctx, sha1_padding, padn);

  total = ctx->total - padn;
  total <<= 3;
  ctx->buffer[56] = (unsigned char)(total >> 56);
  ctx->buffer[57] = (unsigned char)(total >> 48);
  ctx->buffer[58] = (unsigned char)(total >> 40);
  ctx->buffer[59] = (unsigned char)(total >> 32);
  ctx->buffer[60] = (unsigned char)(total >> 24);
  ctx->buffer[61] = (unsigned char)(total >> 16);
  ctx->buffer[62] = (unsigned char)(total >> 8);
  ctx->buffer[63] = (unsigned char)(total);
  if (!ctx->bigendian)
    swap_bytes((uint32_t*)(ctx->buffer));
  sha1_process(ctx, (uint32_t*)(ctx->buffer));
  output[0] = (unsigned char)(ctx->ihv[0] >> 24);
  output[1] = (unsigned char)(ctx->ihv[0] >> 16);
  output[2] = (unsigned char)(ctx->ihv[0] >> 8);
  output[3] = (unsigned char)(ctx->ihv[0]);
  output[4] = (unsigned char)(ctx->ihv[1] >> 24);
  output[5] = (unsigned char)(ctx->ihv[1] >> 16);
  output[6] = (unsigned char)(ctx->ihv[1] >> 8);
  output[7] = (unsigned char)(ctx->ihv[1]);
  output[8] = (unsigned char)(ctx->ihv[2] >> 24);
  output[9] = (unsigned char)(ctx->ihv[2] >> 16);
  output[10] = (unsigned char)(ctx->ihv[2] >> 8);
  output[11] = (unsigned char)(ctx->ihv[2]);
  output[12] = (unsigned char)(ctx->ihv[3] >> 24);
  output[13] = (unsigned char)(ctx->ihv[3] >> 16);
  output[14] = (unsigned char)(ctx->ihv[3] >> 8);
  output[15] = (unsigned char)(ctx->ihv[3]);
  output[16] = (unsigned char)(ctx->ihv[4] >> 24);
  output[17] = (unsigned char)(ctx->ihv[4] >> 16);
  output[18] = (unsigned char)(ctx->ihv[4] >> 8);
  output[19] = (unsigned char)(ctx->ihv[4]);
  return ctx->found_collision;
}
#endif /* FOSSIL_HARDENED_SHA1 */

Added src/sha3.c.



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
/*
** Copyright (c) 2017 D. Richard Hipp
**
** This program is free software; you can redistribute it and/or
** modify it under the terms of the Simplified BSD License (also
** known as the "2-Clause License" or "FreeBSD License".)
**
** This program is distributed in the hope that it will be useful,
** but without any warranty; without even the implied warranty of
** merchantability or fitness for a particular purpose.
**
** Author contact information:
**   drh@hwaci.com
**   http://www.hwaci.com/drh/
**
*******************************************************************************
**
** This file contains an implementation of SHA3 (Keccak) hashing.
*/
#include "config.h"
#include "sha3.h"

/*
** Macros to determine whether the machine is big or little endian,
** and whether or not that determination is run-time or compile-time.
**
** For best performance, an attempt is made to guess at the byte-order
** using C-preprocessor macros.  If that is unsuccessful, or if
** -DSHA3_BYTEORDER=0 is set, then byte-order is determined
** at run-time.
*/
#ifndef SHA3_BYTEORDER
# if defined(i386)     || defined(__i386__)   || defined(_M_IX86) ||    \
     defined(__x86_64) || defined(__x86_64__) || defined(_M_X64)  ||    \
     defined(_M_AMD64) || defined(_M_ARM)     || defined(__x86)   ||    \
     defined(__arm__)
#   define SHA3_BYTEORDER    1234
# elif defined(sparc)    || defined(__ppc__)
#   define SHA3_BYTEORDER    4321
# else
#   define SHA3_BYTEORDER 0
# endif
#endif


/*
** State structure for a SHA3 hash in progress
*/
typedef struct SHA3Context SHA3Context;
struct SHA3Context {
  union {
    u64 s[25];                /* Keccak state. 5x5 lines of 64 bits each */
    unsigned char x[1600];    /* ... or 1600 bytes */
  } u;
  unsigned nRate;        /* Bytes of input accepted per Keccak iteration */
  unsigned nLoaded;      /* Input bytes loaded into u.x[] so far this cycle */
  unsigned ixMask;       /* Insert next input into u.x[nLoaded^ixMask]. */
};

/*
** A single step of the Keccak mixing function for a 1600-bit state
*/
static void KeccakF1600Step(SHA3Context *p){
  int i;
  u64 B0, B1, B2, B3, B4;
  u64 C0, C1, C2, C3, C4;
  u64 D0, D1, D2, D3, D4;
  static const u64 RC[] = {
    0x0000000000000001ULL,  0x0000000000008082ULL,
    0x800000000000808aULL,  0x8000000080008000ULL,
    0x000000000000808bULL,  0x0000000080000001ULL,
    0x8000000080008081ULL,  0x8000000000008009ULL,
    0x000000000000008aULL,  0x0000000000000088ULL,
    0x0000000080008009ULL,  0x000000008000000aULL,
    0x000000008000808bULL,  0x800000000000008bULL,
    0x8000000000008089ULL,  0x8000000000008003ULL,
    0x8000000000008002ULL,  0x8000000000000080ULL,
    0x000000000000800aULL,  0x800000008000000aULL,
    0x8000000080008081ULL,  0x8000000000008080ULL,
    0x0000000080000001ULL,  0x8000000080008008ULL
  };
# define A00 (p->u.s[0])
# define A01 (p->u.s[1])
# define A02 (p->u.s[2])
# define A03 (p->u.s[3])
# define A04 (p->u.s[4])
# define A10 (p->u.s[5])
# define A11 (p->u.s[6])
# define A12 (p->u.s[7])
# define A13 (p->u.s[8])
# define A14 (p->u.s[9])
# define A20 (p->u.s[10])
# define A21 (p->u.s[11])
# define A22 (p->u.s[12])
# define A23 (p->u.s[13])
# define A24 (p->u.s[14])
# define A30 (p->u.s[15])
# define A31 (p->u.s[16])
# define A32 (p->u.s[17])
# define A33 (p->u.s[18])
# define A34 (p->u.s[19])
# define A40 (p->u.s[20])
# define A41 (p->u.s[21])
# define A42 (p->u.s[22])
# define A43 (p->u.s[23])
# define A44 (p->u.s[24])
# define ROL64(a,x) ((a<<x)|(a>>(64-x)))

  for(i=0; i<24; i+=4){
    C0 = A00^A10^A20^A30^A40;
    C1 = A01^A11^A21^A31^A41;
    C2 = A02^A12^A22^A32^A42;
    C3 = A03^A13^A23^A33^A43;
    C4 = A04^A14^A24^A34^A44;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A11^D1), 44);
    B2 = ROL64((A22^D2), 43);
    B3 = ROL64((A33^D3), 21);
    B4 = ROL64((A44^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i];
    A11 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A20^D0), 3);
    B3 = ROL64((A31^D1), 45);
    B4 = ROL64((A42^D2), 61);
    B0 = ROL64((A03^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A20 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A40^D0), 18);
    B0 = ROL64((A01^D1), 1);
    B1 = ROL64((A12^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A34^D4), 8);
    A40 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A10^D0), 36);
    B2 = ROL64((A21^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A43^D3), 56);
    B0 = ROL64((A04^D4), 27);
    A10 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A30^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A02^D2), 62);
    B1 = ROL64((A13^D3), 55);
    B2 = ROL64((A24^D4), 39);
    A30 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    C0 = A00^A20^A40^A10^A30;
    C1 = A11^A31^A01^A21^A41;
    C2 = A22^A42^A12^A32^A02;
    C3 = A33^A03^A23^A43^A13;
    C4 = A44^A14^A34^A04^A24;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A31^D1), 44);
    B2 = ROL64((A12^D2), 43);
    B3 = ROL64((A43^D3), 21);
    B4 = ROL64((A24^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+1];
    A31 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A40^D0), 3);
    B3 = ROL64((A21^D1), 45);
    B4 = ROL64((A02^D2), 61);
    B0 = ROL64((A33^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A40 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A30^D0), 18);
    B0 = ROL64((A11^D1), 1);
    B1 = ROL64((A42^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A04^D4), 8);
    A30 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A20^D0), 36);
    B2 = ROL64((A01^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A13^D3), 56);
    B0 = ROL64((A44^D4), 27);
    A20 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A10^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A22^D2), 62);
    B1 = ROL64((A03^D3), 55);
    B2 = ROL64((A34^D4), 39);
    A10 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    C0 = A00^A40^A30^A20^A10;
    C1 = A31^A21^A11^A01^A41;
    C2 = A12^A02^A42^A32^A22;
    C3 = A43^A33^A23^A13^A03;
    C4 = A24^A14^A04^A44^A34;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A21^D1), 44);
    B2 = ROL64((A42^D2), 43);
    B3 = ROL64((A13^D3), 21);
    B4 = ROL64((A34^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+2];
    A21 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A30^D0), 3);
    B3 = ROL64((A01^D1), 45);
    B4 = ROL64((A22^D2), 61);
    B0 = ROL64((A43^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A30 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A10^D0), 18);
    B0 = ROL64((A31^D1), 1);
    B1 = ROL64((A02^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A44^D4), 8);
    A10 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A40^D0), 36);
    B2 = ROL64((A11^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A03^D3), 56);
    B0 = ROL64((A24^D4), 27);
    A40 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A20^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A12^D2), 62);
    B1 = ROL64((A33^D3), 55);
    B2 = ROL64((A04^D4), 39);
    A20 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    C0 = A00^A30^A10^A40^A20;
    C1 = A21^A01^A31^A11^A41;
    C2 = A42^A22^A02^A32^A12;
    C3 = A13^A43^A23^A03^A33;
    C4 = A34^A14^A44^A24^A04;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A01^D1), 44);
    B2 = ROL64((A02^D2), 43);
    B3 = ROL64((A03^D3), 21);
    B4 = ROL64((A04^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+3];
    A01 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A10^D0), 3);
    B3 = ROL64((A11^D1), 45);
    B4 = ROL64((A12^D2), 61);
    B0 = ROL64((A13^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A10 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A20^D0), 18);
    B0 = ROL64((A21^D1), 1);
    B1 = ROL64((A22^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A24^D4), 8);
    A20 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A30^D0), 36);
    B2 = ROL64((A31^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A33^D3), 56);
    B0 = ROL64((A34^D4), 27);
    A30 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A40^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A42^D2), 62);
    B1 = ROL64((A43^D3), 55);
    B2 = ROL64((A44^D4), 39);
    A40 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );
  }
}

/*
** Initialize a new hash.  iSize determines the size of the hash
** in bits and should be one of 224, 256, 384, or 512.  Or iSize
** can be zero to use the default hash size of 256 bits.
*/
static void SHA3Init(SHA3Context *p, int iSize){
  memset(p, 0, sizeof(*p));
  if( iSize>=128 && iSize<=512 ){
    p->nRate = (1600 - ((iSize + 31)&~31)*2)/8;
  }else{
    p->nRate = (1600 - 2*256)/8;
  }
#if SHA3_BYTEORDER==1234
  /* Known to be little-endian at compile-time. No-op */
#elif SHA3_BYTEORDER==4321
  p->ixMask = 7;  /* Big-endian */
#else
  {
    static unsigned int one = 1;
    if( 1==*(unsigned char*)&one ){
      /* Little endian.  No byte swapping. */
      p->ixMask = 0;
    }else{
      /* Big endian.  Byte swap. */
      p->ixMask = 7;
    }
  }
#endif
}

/*
** Make consecutive calls to the SHA3Update function to add new content
** to the hash
*/
static void SHA3Update(
  SHA3Context *p,
  const unsigned char *aData,
  unsigned int nData
){
  unsigned int i = 0;
#if SHA3_BYTEORDER==1234
  if( (p->nLoaded % 8)==0 && ((aData - (const unsigned char*)0)&7)==0 ){
    for(; i+7<nData; i+=8){
      p->u.s[p->nLoaded/8] ^= *(u64*)&aData[i];
      p->nLoaded += 8;
      if( p->nLoaded>=p->nRate ){
        KeccakF1600Step(p);
        p->nLoaded = 0;
      }
    }
  }
#endif
  for(; i<nData; i++){
#if SHA3_BYTEORDER==1234
    p->u.x[p->nLoaded] ^= aData[i];
#elif SHA3_BYTEORDER==4321
    p->u.x[p->nLoaded^0x07] ^= aData[i];
#else
    p->u.x[p->nLoaded^p->ixMask] ^= aData[i];
#endif
    p->nLoaded++;
    if( p->nLoaded==p->nRate ){
      KeccakF1600Step(p);
      p->nLoaded = 0;
    }
  }
}

/*
** After all content has been added, invoke SHA3Final() to compute
** the final hash.  The function returns a pointer to the binary
** hash value.
*/
static unsigned char *SHA3Final(SHA3Context *p){
  unsigned int i;
  if( p->nLoaded==p->nRate-1 ){
    const unsigned char c1 = 0x86;
    SHA3Update(p, &c1, 1);
  }else{
    const unsigned char c2 = 0x06;
    const unsigned char c3 = 0x80;
    SHA3Update(p, &c2, 1);
    p->nLoaded = p->nRate - 1;
    SHA3Update(p, &c3, 1);
  }
  for(i=0; i<p->nRate; i++){
    p->u.x[i+p->nRate] = p->u.x[i^p->ixMask];
  }
  return &p->u.x[p->nRate];
}

/*
** Convert a digest into base-16.  digest should be declared as
** "unsigned char digest[20]" in the calling function.  The SHA3
** digest is stored in the first 20 bytes.  zBuf should
** be "char zBuf[41]".
*/
static void DigestToBase16(unsigned char *digest, char *zBuf, int nByte){
  static const char zEncode[] = "0123456789abcdef";
  int ix;

  for(ix=0; ix<nByte; ix++){
    *zBuf++ = zEncode[(*digest>>4)&0xf];
    *zBuf++ = zEncode[*digest++ & 0xf];
  }
  *zBuf = '\0';
}

/*
** The state of a incremental SHA3 checksum computation.  Only one
** such computation can be underway at a time, of course.
*/
static SHA3Context incrCtx;
static int incrInit = 0;

/*
** Initialize a new global SHA3 hash.
*/
void sha3sum_init(int iSize){
  assert( incrInit==0 );
  incrInit = iSize;
  SHA3Init(&incrCtx, incrInit);
}

/*
** Add more text to the incremental SHA3 checksum.
*/
void sha3sum_step_text(const char *zText, int nBytes){
  assert( incrInit );
  if( nBytes<=0 ){
    if( nBytes==0 ) return;
    nBytes = strlen(zText);
  }
  SHA3Update(&incrCtx, (unsigned char*)zText, nBytes);
}

/*
** Add the content of a blob to the incremental SHA3 checksum.
*/
void sha3sum_step_blob(Blob *p){
  assert( incrInit );
  SHA3Update(&incrCtx, (unsigned char*)blob_buffer(p), blob_size(p));
}

/*
** Finish the incremental SHA3 checksum.  Store the result in blob pOut
** if pOut!=0.  Also return a pointer to the result.
**
** This resets the incremental checksum preparing for the next round
** of computation.  The return pointer points to a static buffer that
** is overwritten by subsequent calls to this function.
*/
char *sha3sum_finish(Blob *pOut){
  static char zOut[132];
  DigestToBase16(SHA3Final(&incrCtx), zOut, incrInit/8);
  if( pOut ){
    blob_zero(pOut);
    blob_append(pOut, zOut, incrInit/4);
  }
  incrInit = 0;
  return zOut;
}


/*
** Compute the SHA3 checksum of a file on disk.  Store the resulting
** checksum in the blob pCksum.  pCksum is assumed to be initialized.
**
** Return the number of errors.
*/
int sha3sum_file(const char *zFilename, int iSize, Blob *pCksum){
  FILE *in;
  SHA3Context ctx;
  char zBuf[10240];

  if( file_wd_islink(zFilename) ){
    /* Instead of file content, return sha3 of link destination path */
    Blob destinationPath;
    int rc;

    blob_read_link(&destinationPath, zFilename);
    rc = sha3sum_blob(&destinationPath, iSize, pCksum);
    blob_reset(&destinationPath);
    return rc;
  }

  in = fossil_fopen(zFilename,"rb");
  if( in==0 ){
    return 1;
  }
  SHA3Init(&ctx, iSize);
  for(;;){
    int n;
    n = fread(zBuf, 1, sizeof(zBuf), in);
    if( n<=0 ) break;
    SHA3Update(&ctx, (unsigned char*)zBuf, (unsigned)n);
  }
  fclose(in);
  blob_zero(pCksum);
  blob_resize(pCksum, iSize/4);
  DigestToBase16(SHA3Final(&ctx), blob_buffer(pCksum), iSize/8);
  return 0;
}

/*
** Compute the SHA3 checksum of a blob in memory.  Store the resulting
** checksum in the blob pCksum.  pCksum is assumed to be either
** uninitialized or the same blob as pIn.
**
** Return the number of errors.
*/
int sha3sum_blob(const Blob *pIn, int iSize, Blob *pCksum){
  SHA3Context ctx;
  SHA3Init(&ctx, iSize);
  SHA3Update(&ctx, (unsigned char*)blob_buffer(pIn), blob_size(pIn));
  if( pIn==pCksum ){
    blob_reset(pCksum);
  }else{
    blob_zero(pCksum);
  }
  blob_resize(pCksum, iSize/4);
  DigestToBase16(SHA3Final(&ctx), blob_buffer(pCksum), iSize/8);
  return 0;
}

/*
** Compute the SHA3 checksum of a zero-terminated string.  The
** result is held in memory obtained from mprintf().
*/
char *sha3sum(const char *zIn, int iSize){
  SHA3Context ctx;
  char zDigest[132];

  SHA3Init(&ctx, iSize);
  SHA3Update(&ctx, (unsigned const char*)zIn, strlen(zIn));
  DigestToBase16(SHA3Final(&ctx), zDigest, iSize/4);
  return mprintf("%s", zDigest);
}

/*
** COMMAND: sha3sum*
**
** Usage: %fossil sha3sum FILE...
**
** Compute an SHA3 checksum of all files named on the command-line.
** If a file is named "-" then take its content from standard input.
**
** Options:
**
**    --224        Compute a SHA3-224 hash
**    --256        Compute a SHA3-256 hash (the default)
**    --384        Compute a SHA3-384 hash
**    --512        Compute a SHA3-512 hash
**    --size N     An N-bit hash.  N must be a multiple of 32 between 128
**                 and 512.
*/
void sha3sum_test(void){
  int i;
  Blob in;
  Blob cksum;
  int iSize = 256;

  if( find_option("224",0,0)!=0 ) iSize = 224;
  else if( find_option("256",0,0)!=0 ) iSize = 256;
  else if( find_option("384",0,0)!=0 ) iSize = 384;
  else if( find_option("512",0,0)!=0 ) iSize = 512;
  else{
    const char *zN = find_option("size",0,1);
    if( zN!=0 ){
      int n = atoi(zN);
      if( n%32!=0 || n<128 || n>512 ){
        fossil_fatal("--size must be a multiple of 64 between 128 and 512");
      }
      iSize = n;
    }
  }
  verify_all_options();

  for(i=2; i<g.argc; i++){
    blob_init(&cksum, "************** not found ***************", -1);
    if( g.argv[i][0]=='-' && g.argv[i][1]==0 ){
      blob_read_from_channel(&in, stdin, -1);
      sha3sum_blob(&in, iSize, &cksum);
    }else{
      sha3sum_file(g.argv[i], iSize, &cksum);
    }
    fossil_print("%s  %s\n", blob_str(&cksum), g.argv[i]);
    blob_reset(&cksum);
  }
}

Changes to src/shell.c.

450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
...
573
574
575
576
577
578
579





































































































































































































































































































































































































































































































































































































































































































































































































580
581
582
583
584
585
586
...
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
...
651
652
653
654
655
656
657
658
659
660











661
662
663
664
665
666
667
....
1290
1291
1292
1293
1294
1295
1296
1297
1298
































































1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319

1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
....
1482
1483
1484
1485
1486
1487
1488

























1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503

1504
1505
1506
1507
1508
1509
1510
1511

1512
1513
1514
1515
1516
1517

1518
1519
1520
1521
1522
1523

1524
1525
1526
1527
1528
1529

1530
1531
1532
1533
1534
1535
1536


1537
1538
1539
1540

1541
1542
1543
1544
1545
1546
1547

1548
1549

1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
....
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
....
2020
2021
2022
2023
2024
2025
2026







2027




2028

























































































































2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
....
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123












2124
2125
2126
2127
2128
2129
2130
....
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
....
2231
2232
2233
2234
2235
2236
2237

2238
2239
2240
2241
2242
2243
2244
....
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444








2445
2446
2447
2448
2449
2450
2451
....
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
....
2580
2581
2582
2583
2584
2585
2586











2587
2588
2589
2590
2591
2592
2593
....
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
....
3718
3719
3720
3721
3722
3723
3724























3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736



3737
3738
3739

3740
3741
3742
3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758
3759
3760


3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
....
4284
4285
4286
4287
4288
4289
4290

4291
4292

4293
4294


4295
4296
4297
4298
4299

4300
4301
4302
4303
4304
4305
4306
....
4335
4336
4337
4338
4339
4340
4341

4342
4343
4344
4345
4346
4347
4348
....
4564
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
4580
4581
4582
4583
4584
4585
4586
4587
4588
....
4605
4606
4607
4608
4609
4610
4611
4612
4613
4614
4615
4616
4617
4618
4619
....
4827
4828
4829
4830
4831
4832
4833

















































































































4834
4835
4836
4837
4838
4839
4840
....
4842
4843
4844
4845
4846
4847
4848




















































































































4849
4850
4851
4852
4853
4854
4855
....
4871
4872
4873
4874
4875
4876
4877
4878

4879
4880
4881
4882
4883
4884
4885
....
5183
5184
5185
5186
5187
5188
5189

5190
5191
5192
5193
5194
5195
5196
....
5219
5220
5221
5222
5223
5224
5225
5226
5227
5228
5229
5230
5231
5232
5233
....
5293
5294
5295
5296
5297
5298
5299
5300
5301
5302
5303
5304
5305
5306
5307
....
5436
5437
5438
5439
5440
5441
5442
5443
5444
5445
5446
5447
5448
5449
5450
....
5456
5457
5458
5459
5460
5461
5462
5463
5464
5465
5466
5467
5468
5469
5470
....
5499
5500
5501
5502
5503
5504
5505
5506
5507
5508
5509
5510
5511
5512
5513
5514
5515
5516
5517
....
5546
5547
5548
5549
5550
5551
5552
5553
5554
5555
5556
5557
5558
5559
5560
....
6015
6016
6017
6018
6019
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
....
6030
6031
6032
6033
6034
6035
6036
6037
6038
6039
6040
6041
6042
6043
6044
    if( *z=='+' || *z=='-' ) z++;
    if( !IsDigit(*z) ) return 0;
    while( IsDigit(*z) ){ z++; }
    if( realnum ) *realnum = 1;
  }
  return *z==0;
}

/*
** A global char* and an SQL function to access its current value
** from within an SQL statement. This program used to use the
** sqlite_exec_printf() API to substitue a string into an SQL statement.
** The correct way to do this with sqlite3 is to use the bind API, but
** since the shell is built around the callback paradigm it would be a lot
** of work. Instead just use this hack, which is quite harmless.
*/
static const char *zShellStatic = 0;
static void shellstaticFunc(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
  assert( 0==argc );
  assert( zShellStatic );
  UNUSED_PARAMETER(argc);
  UNUSED_PARAMETER(argv);
  sqlite3_result_text(context, zShellStatic, -1, SQLITE_STATIC);
}


/*
** Compute a string length that is limited to what can be stored in
** lower 30 bits of a 32-bit signed integer.
*/
static int strlen30(const char *z){
  const char *z2 = z;
................................................................................
    free(zPrior);
    zResult = shell_readline(zPrompt);
    if( zResult && *zResult ) shell_add_history(zResult);
#endif
  }
  return zResult;
}






































































































































































































































































































































































































































































































































































































































































































































































































#if defined(SQLITE_ENABLE_SESSION)
/*
** State information for a single open session
*/
typedef struct OpenSession OpenSession;
struct OpenSession {
................................................................................
/*
** State information about the database connection is contained in an
** instance of the following structure.
*/
typedef struct ShellState ShellState;
struct ShellState {
  sqlite3 *db;           /* The database */
  int echoOn;            /* True to echo input commands */
  int autoExplain;       /* Automatically turn on .explain mode */
  int autoEQP;           /* Run EXPLAIN QUERY PLAN prior to seach SQL stmt */
  int statsOn;           /* True to display memory stats before each finalize */
  int scanstatsOn;       /* True to display scan stats before each finalize */
  int countChanges;      /* True to display change counts */
  int backslashOn;       /* Resolve C-style \x escapes in SQL input text */
  int outCount;          /* Revert to stdout when reaching zero */
  int cnt;               /* Number of records displayed so far */
  FILE *out;             /* Write results here */
  FILE *traceOut;        /* Output for sqlite3_trace() */
  int nErr;              /* Number of errors seen */
  int mode;              /* An output mode setting */
  int cMode;             /* temporary output mode for the current query */
................................................................................
  OpenSession aSession[4];  /* Array of sessions.  [0] is in focus. */
#endif
};

/*
** These are the allowed shellFlgs values
*/
#define SHFLG_Scratch     0x00001     /* The --scratch option is used */
#define SHFLG_Pagecache   0x00002     /* The --pagecache option is used */
#define SHFLG_Lookaside   0x00004     /* Lookaside memory is used */












/*
** These are the allowed modes.
*/
#define MODE_Line     0  /* One column per line.  Blank line between records */
#define MODE_Column   1  /* One record per line in neat columns */
#define MODE_List     2  /* One record per line with a separator */
................................................................................
** This is the callback routine that the SQLite library
** invokes for each row of a query result.
*/
static int callback(void *pArg, int nArg, char **azArg, char **azCol){
  /* since we don't have type info, call the shell_callback with a NULL value */
  return shell_callback(pArg, nArg, azArg, azCol, NULL);
}

/*
































































** Set the destination table field of the ShellState structure to
** the name of the table given.  Escape any quote characters in the
** table name.
*/
static void set_table_name(ShellState *p, const char *zName){
  int i, n;
  int needQuote;
  char *z;

  if( p->zDestTable ){
    free(p->zDestTable);
    p->zDestTable = 0;
  }
  if( zName==0 ) return;
  needQuote = !isalpha((unsigned char)*zName) && *zName!='_';
  for(i=n=0; zName[i]; i++, n++){
    if( !isalnum((unsigned char)zName[i]) && zName[i]!='_' ){
      needQuote = 1;
      if( zName[i]=='\'' ) n++;
    }
  }

  if( needQuote ) n += 2;
  z = p->zDestTable = malloc( n+1 );
  if( z==0 ){
    raw_printf(stderr,"Error: out of memory\n");
    exit(1);
  }
  n = 0;
  if( needQuote ) z[n++] = '\'';
  for(i=0; zName[i]; i++){
    z[n++] = zName[i];
    if( zName[i]=='\'' ) z[n++] = '\'';
  }
  if( needQuote ) z[n++] = '\'';
  z[n] = 0;
}

/* zIn is either a pointer to a NULL-terminated string in memory obtained
** from malloc(), or a NULL pointer. The string pointed to by zAppend is
** added to zIn, and the result returned in memory obtained from malloc().
** zIn, if it was not NULL, is freed.
**
** If the third argument, quote, is not '\0', then it is used as a
** quote character for zAppend.
*/
static char *appendText(char *zIn, char const *zAppend, char quote){
  int len;
  int i;
  int nAppend = strlen30(zAppend);
  int nIn = (zIn?strlen30(zIn):0);

  len = nAppend+nIn+1;
  if( quote ){
    len += 2;
    for(i=0; i<nAppend; i++){
      if( zAppend[i]==quote ) len++;
    }
  }

  zIn = (char *)realloc(zIn, len);
  if( !zIn ){
    return 0;
  }

  if( quote ){
    char *zCsr = &zIn[nIn];
    *zCsr++ = quote;
    for(i=0; i<nAppend; i++){
      *zCsr++ = zAppend[i];
      if( zAppend[i]==quote ) *zCsr++ = quote;
    }
    *zCsr++ = quote;
    *zCsr++ = '\0';
    assert( (zCsr-zIn)==len );
  }else{
    memcpy(&zIn[nIn], zAppend, nAppend);
    zIn[len-1] = '\0';
  }

  return zIn;
}


/*
** Execute a query statement that will generate SQL output.  Print
** the result columns, comma-separated, on a line and then add a
** semicolon terminator to the end of that line.
**
** If the number of columns is 1 and that column contains text "--"
................................................................................
      }
    }
  }
  fclose(in);
}
#endif



























/*
** Display memory stats.
*/
static int display_stats(
  sqlite3 *db,                /* Database to query */
  ShellState *pArg,           /* Pointer to ShellState */
  int bReset                  /* True to reset the stats */
){
  int iCur;
  int iHiwtr;

  if( pArg && pArg->out ){

    iHiwtr = iCur = -1;

    sqlite3_status(SQLITE_STATUS_MEMORY_USED, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out,
            "Memory Used:                         %d (max %d) bytes\n",
            iCur, iHiwtr);
    iHiwtr = iCur = -1;
    sqlite3_status(SQLITE_STATUS_MALLOC_COUNT, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out, "Number of Outstanding Allocations:   %d (max %d)\n",
            iCur, iHiwtr);

    if( pArg->shellFlgs & SHFLG_Pagecache ){
      iHiwtr = iCur = -1;
      sqlite3_status(SQLITE_STATUS_PAGECACHE_USED, &iCur, &iHiwtr, bReset);
      raw_printf(pArg->out,
              "Number of Pcache Pages Used:         %d (max %d) pages\n",
              iCur, iHiwtr);

    }
    iHiwtr = iCur = -1;
    sqlite3_status(SQLITE_STATUS_PAGECACHE_OVERFLOW, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out,
            "Number of Pcache Overflow Bytes:     %d (max %d) bytes\n",
            iCur, iHiwtr);

    if( pArg->shellFlgs & SHFLG_Scratch ){
      iHiwtr = iCur = -1;
      sqlite3_status(SQLITE_STATUS_SCRATCH_USED, &iCur, &iHiwtr, bReset);
      raw_printf(pArg->out,
              "Number of Scratch Allocations Used:  %d (max %d)\n",
              iCur, iHiwtr);

    }
    iHiwtr = iCur = -1;
    sqlite3_status(SQLITE_STATUS_SCRATCH_OVERFLOW, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out,
            "Number of Scratch Overflow Bytes:    %d (max %d) bytes\n",
            iCur, iHiwtr);
    iHiwtr = iCur = -1;


    sqlite3_status(SQLITE_STATUS_MALLOC_SIZE, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out, "Largest Allocation:                  %d bytes\n",
            iHiwtr);
    iHiwtr = iCur = -1;

    sqlite3_status(SQLITE_STATUS_PAGECACHE_SIZE, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out, "Largest Pcache Allocation:           %d bytes\n",
            iHiwtr);
    iHiwtr = iCur = -1;
    sqlite3_status(SQLITE_STATUS_SCRATCH_SIZE, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out, "Largest Scratch Allocation:          %d bytes\n",
            iHiwtr);

#ifdef YYTRACKMAXSTACKDEPTH
    iHiwtr = iCur = -1;

    sqlite3_status(SQLITE_STATUS_PARSER_STACK, &iCur, &iHiwtr, bReset);
    raw_printf(pArg->out, "Deepest Parser Stack:                %d (max %d)\n",
            iCur, iHiwtr);
#endif
  }

  if( pArg && pArg->out && db ){
    if( pArg->shellFlgs & SHFLG_Lookaside ){
      iHiwtr = iCur = -1;
      sqlite3_db_status(db, SQLITE_DBSTATUS_LOOKASIDE_USED,
................................................................................
      /* save off the prepared statment handle and reset row count */
      if( pArg ){
        pArg->pStmt = pStmt;
        pArg->cnt = 0;
      }

      /* echo the sql statement if echo on */
      if( pArg && pArg->echoOn ){
        utf8_printf(pArg->out, "%s\n", zStmtSql ? zStmtSql : zSql);
      }

      /* Show the EXPLAIN QUERY PLAN if .eqp is on */
      if( pArg && pArg->autoEQP && sqlite3_strlike("EXPLAIN%",zStmtSql,0)!=0 ){
        sqlite3_stmt *pExplain;
        char *zEQP;
................................................................................
      }
    }
  } /* end while */

  return rc;
}













/*

























































































































** This is a different callback routine used for dumping the database.
** Each row received by this callback consists of a table name,
** the table type ("index" or "table") and SQL to create the table.
** This routine should print text sufficient to recreate the table.
*/
static int dump_callback(void *pArg, int nArg, char **azArg, char **azCol){
  int rc;
  const char *zTable;
  const char *zType;
  const char *zSql;
  const char *zPrepStmt = 0;
  ShellState *p = (ShellState *)pArg;

  UNUSED_PARAMETER(azCol);
  if( nArg!=3 ) return 1;
  zTable = azArg[0];
  zType = azArg[1];
  zSql = azArg[2];

  if( strcmp(zTable, "sqlite_sequence")==0 ){
    zPrepStmt = "DELETE FROM sqlite_sequence;\n";
  }else if( sqlite3_strglob("sqlite_stat?", zTable)==0 ){
    raw_printf(p->out, "ANALYZE sqlite_master;\n");
  }else if( strncmp(zTable, "sqlite_", 7)==0 ){
    return 0;
  }else if( strncmp(zSql, "CREATE VIRTUAL TABLE", 20)==0 ){
    char *zIns;
    if( !p->writableSchema ){
................................................................................
    sqlite3_free(zIns);
    return 0;
  }else{
    printSchemaLine(p->out, zSql, ";\n");
  }

  if( strcmp(zType, "table")==0 ){
    sqlite3_stmt *pTableInfo = 0;
    char *zSelect = 0;
    char *zTableInfo = 0;
    char *zTmp = 0;
    int nRow = 0;

    zTableInfo = appendText(zTableInfo, "PRAGMA table_info(", 0);
    zTableInfo = appendText(zTableInfo, zTable, '"');
    zTableInfo = appendText(zTableInfo, ");", 0);

    rc = sqlite3_prepare_v2(p->db, zTableInfo, -1, &pTableInfo, 0);
    free(zTableInfo);
    if( rc!=SQLITE_OK || !pTableInfo ){
      return 1;
    }

    zSelect = appendText(zSelect, "SELECT 'INSERT INTO ' || ", 0);
    /* Always quote the table name, even if it appears to be pure ascii,
    ** in case it is a keyword. Ex:  INSERT INTO "table" ... */
    zTmp = appendText(zTmp, zTable, '"');
    if( zTmp ){
      zSelect = appendText(zSelect, zTmp, '\'');
      free(zTmp);
    }
    zSelect = appendText(zSelect, " || ' VALUES(' || ", 0);
    rc = sqlite3_step(pTableInfo);
    while( rc==SQLITE_ROW ){
      const char *zText = (const char *)sqlite3_column_text(pTableInfo, 1);
      zSelect = appendText(zSelect, "quote(", 0);
      zSelect = appendText(zSelect, zText, '"');
      rc = sqlite3_step(pTableInfo);
      if( rc==SQLITE_ROW ){
        zSelect = appendText(zSelect, "), ", 0);
      }else{
        zSelect = appendText(zSelect, ") ", 0);
      }
      nRow++;
    }
    rc = sqlite3_finalize(pTableInfo);
    if( rc!=SQLITE_OK || nRow==0 ){
      free(zSelect);
      return 1;
    }
    zSelect = appendText(zSelect, "|| ')' FROM  ", 0);
    zSelect = appendText(zSelect, zTable, '"');

    rc = run_table_dump_query(p, zSelect, zPrepStmt);
    if( rc==SQLITE_CORRUPT ){
      zSelect = appendText(zSelect, " ORDER BY rowid DESC", 0);
      run_table_dump_query(p, zSelect, 0);
    }
    free(zSelect);












  }
  return 0;
}

/*
** Run zQuery.  Use dump_callback() as the callback routine so that
** the contents of the query are output as SQL statements.
................................................................................
  ".mode MODE ?TABLE?     Set output mode where MODE is one of:\n"
  "                         ascii    Columns/rows delimited by 0x1F and 0x1E\n"
  "                         csv      Comma-separated values\n"
  "                         column   Left-aligned columns.  (See .width)\n"
  "                         html     HTML <table> code\n"
  "                         insert   SQL insert statements for TABLE\n"
  "                         line     One value per line\n"
  "                         list     Values delimited by .separator strings\n"
  "                         quote    Escape answers as for SQL\n"
  "                         tabs     Tab-separated values\n"
  "                         tcl      TCL list elements\n"
  ".nullvalue STRING      Use STRING in place of NULL values\n"
  ".once FILENAME         Output for the next SQL command only to FILENAME\n"
  ".open ?--new? ?FILE?   Close existing database and reopen FILE\n"
  "                         The --new starts with an empty file\n"
................................................................................
  ".schema ?PATTERN?      Show the CREATE statements matching PATTERN\n"
  "                          Add --indent for pretty-printing\n"
  ".separator COL ?ROW?   Change the column separator and optionally the row\n"
  "                         separator for both the output mode and .import\n"
#if defined(SQLITE_ENABLE_SESSION)
  ".session CMD ...       Create or control sessions\n"
#endif

  ".shell CMD ARGS...     Run CMD ARGS... in a system shell\n"
  ".show                  Show the current values for various settings\n"
  ".stats ?on|off?        Show stats or turn stats on or off\n"
  ".system CMD ARGS...    Run CMD ARGS... in a system shell\n"
  ".tables ?TABLE?        List names of tables\n"
  "                         If TABLE specified, only list tables matching\n"
  "                         LIKE pattern TABLE.\n"
................................................................................
** the database fails to open, print an error message and exit.
*/
static void open_db(ShellState *p, int keepAlive){
  if( p->db==0 ){
    sqlite3_initialize();
    sqlite3_open(p->zDbFilename, &p->db);
    globalDb = p->db;
    if( p->db && sqlite3_errcode(p->db)==SQLITE_OK ){
      sqlite3_create_function(p->db, "shellstatic", 0, SQLITE_UTF8, 0,
          shellstaticFunc, 0, 0);
    }
    if( p->db==0 || SQLITE_OK!=sqlite3_errcode(p->db) ){
      utf8_printf(stderr,"Error: unable to open database \"%s\": %s\n",
          p->zDbFilename, sqlite3_errmsg(p->db));
      if( keepAlive ) return;
      exit(1);
    }
#ifndef SQLITE_OMIT_LOAD_EXTENSION
    sqlite3_enable_load_extension(p->db, 1);
#endif
    sqlite3_create_function(p->db, "readfile", 1, SQLITE_UTF8, 0,
                            readfileFunc, 0, 0);
    sqlite3_create_function(p->db, "writefile", 2, SQLITE_UTF8, 0,
                            writefileFunc, 0, 0);








  }
}

/*
** Do C-language style dequoting.
**
**    \a    -> alarm
................................................................................
  return isNeg? -v : v;
}

/*
** Interpret zArg as either an integer or a boolean value.  Return 1 or 0
** for TRUE and FALSE.  Return the integer value if appropriate.
*/
static int booleanValue(char *zArg){
  int i;
  if( zArg[0]=='0' && zArg[1]=='x' ){
    for(i=2; hexDigitValue(zArg[i])>=0; i++){}
  }else{
    for(i=0; zArg[i]>='0' && zArg[i]<='9'; i++){}
  }
  if( i>0 && zArg[i]==0 ) return (int)(integerValue(zArg) & 0xffffffff);
................................................................................
  if( sqlite3_stricmp(zArg, "off")==0 || sqlite3_stricmp(zArg,"no")==0 ){
    return 0;
  }
  utf8_printf(stderr, "ERROR: Not a boolean value: \"%s\". Assuming \"no\".\n",
          zArg);
  return 0;
}












/*
** Close an output file, assuming it is not stderr or stdout
*/
static void output_file_close(FILE *f){
  if( f && f!=stdout && f!=stderr ) fclose(f);
}
................................................................................
  */
  if( c=='b' && n>=3 && strncmp(azArg[0], "breakpoint", n)==0 ){
    test_breakpoint();
  }else

  if( c=='c' && n>=3 && strncmp(azArg[0], "changes", n)==0 ){
    if( nArg==2 ){
      p->countChanges = booleanValue(azArg[1]);
    }else{
      raw_printf(stderr, "Usage: .changes on|off\n");
      rc = 1;
    }
  }else

  /* Cancel output redirection, if it is currently set (by .testcase)
................................................................................
  }else

  if( c=='d' && strncmp(azArg[0], "dbinfo", n)==0 ){
    rc = shell_dbinfo_command(p, nArg, azArg);
  }else

  if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){























    open_db(p, 0);
    /* When playing back a "dump", the content might appear in an order
    ** which causes immediate foreign key constraints to be violated.
    ** So disable foreign-key constraint enforcement to prevent problems. */
    if( nArg!=1 && nArg!=2 ){
      raw_printf(stderr, "Usage: .dump ?LIKE-PATTERN?\n");
      rc = 1;
      goto meta_command_exit;
    }
    raw_printf(p->out, "PRAGMA foreign_keys=OFF;\n");
    raw_printf(p->out, "BEGIN TRANSACTION;\n");
    p->writableSchema = 0;



    sqlite3_exec(p->db, "SAVEPOINT dump; PRAGMA writable_schema=ON", 0, 0, 0);
    p->nErr = 0;
    if( nArg==1 ){

      run_schema_dump_query(p,
        "SELECT name, type, sql FROM sqlite_master "
        "WHERE sql NOT NULL AND type=='table' AND name!='sqlite_sequence'"
      );
      run_schema_dump_query(p,
        "SELECT name, type, sql FROM sqlite_master "
        "WHERE name=='sqlite_sequence'"
      );
      run_table_dump_query(p,
        "SELECT sql FROM sqlite_master "
        "WHERE sql NOT NULL AND type IN ('index','trigger','view')", 0
      );
    }else{
      int i;
      for(i=1; i<nArg; i++){
        zShellStatic = azArg[i];
        run_schema_dump_query(p,
          "SELECT name, type, sql FROM sqlite_master "
          "WHERE tbl_name LIKE shellstatic() AND type=='table'"
          "  AND sql NOT NULL");
        run_table_dump_query(p,


          "SELECT sql FROM sqlite_master "
          "WHERE sql NOT NULL"
          "  AND type IN ('index','trigger','view')"
          "  AND tbl_name LIKE shellstatic()", 0
        );
        zShellStatic = 0;
      }
    }
    if( p->writableSchema ){
      raw_printf(p->out, "PRAGMA writable_schema=OFF;\n");
      p->writableSchema = 0;
    }
    sqlite3_exec(p->db, "PRAGMA writable_schema=OFF;", 0, 0, 0);
    sqlite3_exec(p->db, "RELEASE dump;", 0, 0, 0);
    raw_printf(p->out, p->nErr ? "ROLLBACK; -- due to errors\n" : "COMMIT;\n");
  }else

  if( c=='e' && strncmp(azArg[0], "echo", n)==0 ){
    if( nArg==2 ){
      p->echoOn = booleanValue(azArg[1]);
    }else{
      raw_printf(stderr, "Usage: .echo on|off\n");
      rc = 1;
    }
  }else

  if( c=='e' && strncmp(azArg[0], "eqp", n)==0 ){
................................................................................

  if( c=='m' && strncmp(azArg[0], "mode", n)==0 ){
    const char *zMode = nArg>=2 ? azArg[1] : "";
    int n2 = (int)strlen(zMode);
    int c2 = zMode[0];
    if( c2=='l' && n2>2 && strncmp(azArg[1],"lines",n2)==0 ){
      p->mode = MODE_Line;

    }else if( c2=='c' && strncmp(azArg[1],"columns",n2)==0 ){
      p->mode = MODE_Column;

    }else if( c2=='l' && n2>2 && strncmp(azArg[1],"list",n2)==0 ){
      p->mode = MODE_List;


    }else if( c2=='h' && strncmp(azArg[1],"html",n2)==0 ){
      p->mode = MODE_Html;
    }else if( c2=='t' && strncmp(azArg[1],"tcl",n2)==0 ){
      p->mode = MODE_Tcl;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Space);

    }else if( c2=='c' && strncmp(azArg[1],"csv",n2)==0 ){
      p->mode = MODE_Csv;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Comma);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_CrLf);
    }else if( c2=='t' && strncmp(azArg[1],"tabs",n2)==0 ){
      p->mode = MODE_List;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Tab);
................................................................................
    char *zNewFilename;  /* Name of the database file to open */
    int iName = 1;       /* Index in azArg[] of the filename */
    int newFlag = 0;     /* True to delete file before opening */
    /* Close the existing database */
    session_close_all(p);
    sqlite3_close(p->db);
    p->db = 0;

    sqlite3_free(p->zFreeOnClose);
    p->zFreeOnClose = 0;
    /* Check for command-line arguments */
    for(iName=1; iName<nArg && azArg[iName][0]=='-'; iName++){
      const char *z = azArg[iName];
      if( optionMatch(z,"new") ){
        newFlag = 1;
................................................................................
                      ")";
        new_argv[1] = 0;
        new_colv[0] = "sql";
        new_colv[1] = 0;
        callback(&data, 1, new_argv, new_colv);
        rc = SQLITE_OK;
      }else{
        zShellStatic = azArg[1];
        rc = sqlite3_exec(p->db,
          "SELECT sql FROM "
          "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
          "     FROM sqlite_master UNION ALL"
          "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
          "WHERE lower(tbl_name) LIKE shellstatic()"
          "  AND type!='meta' AND sql NOTNULL "
          "ORDER BY rowid",
          callback, &data, &zErrMsg);
        zShellStatic = 0;
      }
    }else if( nArg==1 ){
      rc = sqlite3_exec(p->db,
         "SELECT sql FROM "
         "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
         "     FROM sqlite_master UNION ALL"
         "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
................................................................................
    }else{
      rc = 0;
    }
  }else

#if defined(SQLITE_DEBUG) && defined(SQLITE_ENABLE_SELECTTRACE)
  if( c=='s' && n==11 && strncmp(azArg[0], "selecttrace", n)==0 ){
    sqlite3SelectTrace = integerValue(azArg[1]);
  }else
#endif

#if defined(SQLITE_ENABLE_SESSION)
  if( c=='s' && strncmp(azArg[0],"session",n)==0 && n>=3 ){
    OpenSession *pSession = &p->aSession[0];
    char **azCmd = &azArg[1];
................................................................................
        v = integerValue(azArg[i]);
        sqlite3_snprintf(sizeof(zBuf),zBuf,"%s: %lld 0x%llx\n", azArg[i],v,v);
        utf8_printf(p->out, "%s", zBuf);
      }
    }
  }else
#endif


















































































































  if( c=='s' && strncmp(azArg[0], "separator", n)==0 ){
    if( nArg<2 || nArg>3 ){
      raw_printf(stderr, "Usage: .separator COL ?ROW?\n");
      rc = 1;
    }
    if( nArg>=2 ){
................................................................................
                       "%.*s", (int)ArraySize(p->colSeparator)-1, azArg[1]);
    }
    if( nArg>=3 ){
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator,
                       "%.*s", (int)ArraySize(p->rowSeparator)-1, azArg[2]);
    }
  }else





















































































































  if( c=='s'
   && (strncmp(azArg[0], "shell", n)==0 || strncmp(azArg[0],"system",n)==0)
  ){
    char *zCmd;
    int i, x;
    if( nArg<2 ){
................................................................................
    static const char *azBool[] = { "off", "on", "full", "unk" };
    int i;
    if( nArg!=1 ){
      raw_printf(stderr, "Usage: .show\n");
      rc = 1;
      goto meta_command_exit;
    }
    utf8_printf(p->out, "%12.12s: %s\n","echo", azBool[p->echoOn!=0]);

    utf8_printf(p->out, "%12.12s: %s\n","eqp", azBool[p->autoEQP&3]);
    utf8_printf(p->out, "%12.12s: %s\n","explain",
         p->mode==MODE_Explain ? "on" : p->autoExplain ? "auto" : "off");
    utf8_printf(p->out,"%12.12s: %s\n","headers", azBool[p->showHeader!=0]);
    utf8_printf(p->out, "%12.12s: %s\n","mode", modeDescr[p->mode]);
    utf8_printf(p->out, "%12.12s: ", "nullvalue");
      output_c_string(p->out, p->nullValue);
................................................................................
          utf8_printf(stderr,
                      "Error: CLI support for testctrl %s not implemented\n",
                      azArg[1]);
          break;
      }
    }
  }else


  if( c=='t' && n>4 && strncmp(azArg[0], "timeout", n)==0 ){
    open_db(p, 0);
    sqlite3_busy_timeout(p->db, nArg>=2 ? (int)integerValue(azArg[1]) : 0);
  }else

  if( c=='t' && n>=5 && strncmp(azArg[0], "timer", n)==0 ){
................................................................................
    if( p->traceOut==0 ){
      sqlite3_trace_v2(p->db, 0, 0, 0);
    }else{
      sqlite3_trace_v2(p->db, SQLITE_TRACE_STMT, sql_trace_callback,p->traceOut);
    }
#endif
  }else
#endif /* !defined(SQLITE_UNTESTABLE) */

#if SQLITE_USER_AUTHENTICATION
  if( c=='u' && strncmp(azArg[0], "user", n)==0 ){
    if( nArg<2 ){
      raw_printf(stderr, "Usage: .user SUBCOMMAND ...\n");
      rc = 1;
      goto meta_command_exit;
................................................................................
  if( c=='v' && strncmp(azArg[0], "version", n)==0 ){
    utf8_printf(p->out, "SQLite %s %s\n" /*extra-version-info*/,
        sqlite3_libversion(), sqlite3_sourceid());
  }else

  if( c=='v' && strncmp(azArg[0], "vfsinfo", n)==0 ){
    const char *zDbName = nArg==2 ? azArg[1] : "main";
    sqlite3_vfs *pVfs;
    if( p->db ){
      sqlite3_file_control(p->db, zDbName, SQLITE_FCNTL_VFS_POINTER, &pVfs);
      if( pVfs ){
        utf8_printf(p->out, "vfs.zName      = \"%s\"\n", pVfs->zName);
        raw_printf(p->out, "vfs.iVersion   = %d\n", pVfs->iVersion);
        raw_printf(p->out, "vfs.szOsFile   = %d\n", pVfs->szOsFile);
        raw_printf(p->out, "vfs.mxPathname = %d\n", pVfs->mxPathname);
................................................................................
** Run a single line of SQL
*/
static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
  int rc;
  char *zErrMsg = 0;

  open_db(p, 0);
  if( p->backslashOn ) resolve_backslashes(zSql);
  BEGIN_TIMER;
  rc = shell_exec(p->db, zSql, shell_callback, p, &zErrMsg);
  END_TIMER;
  if( rc || zErrMsg ){
    char zPrefix[100];
    if( in!=0 || !stdin_is_interactive ){
      sqlite3_snprintf(sizeof(zPrefix), zPrefix,
................................................................................
      utf8_printf(stderr, "%s %s\n", zPrefix, zErrMsg);
      sqlite3_free(zErrMsg);
      zErrMsg = 0;
    }else{
      utf8_printf(stderr, "%s %s\n", zPrefix, sqlite3_errmsg(p->db));
    }
    return 1;
  }else if( p->countChanges ){
    raw_printf(p->out, "changes: %3d   total_changes: %d\n",
            sqlite3_changes(p->db), sqlite3_total_changes(p->db));
  }
  return 0;
}


................................................................................
    }
    if( seenInterrupt ){
      if( in!=0 ) break;
      seenInterrupt = 0;
    }
    lineno++;
    if( nSql==0 && _all_whitespace(zLine) ){
      if( p->echoOn ) printf("%s\n", zLine);
      continue;
    }
    if( zLine && zLine[0]=='.' && nSql==0 ){
      if( p->echoOn ) printf("%s\n", zLine);
      rc = do_meta_command(zLine, p);
      if( rc==2 ){ /* exit requested */
        break;
      }else if( rc ){
        errCnt++;
      }
      continue;
................................................................................
      errCnt += runOneSqlLine(p, zSql, in, startline);
      nSql = 0;
      if( p->outCount ){
        output_reset(p);
        p->outCount = 0;
      }
    }else if( nSql && _all_whitespace(zSql) ){
      if( p->echoOn ) printf("%s\n", zSql);
      nSql = 0;
    }
  }
  if( nSql && !_all_whitespace(zSql) ){
    runOneSqlLine(p, zSql, in, startline);
  }
  free(zSql);
................................................................................
      sqlite3_snprintf(sizeof(data.nullValue), data.nullValue,
                       "%s",cmdline_option_value(argc,argv,++i));
    }else if( strcmp(z,"-header")==0 ){
      data.showHeader = 1;
    }else if( strcmp(z,"-noheader")==0 ){
      data.showHeader = 0;
    }else if( strcmp(z,"-echo")==0 ){
      data.echoOn = 1;
    }else if( strcmp(z,"-eqp")==0 ){
      data.autoEQP = 1;
    }else if( strcmp(z,"-eqpfull")==0 ){
      data.autoEQP = 2;
    }else if( strcmp(z,"-stats")==0 ){
      data.statsOn = 1;
    }else if( strcmp(z,"-scanstats")==0 ){
................................................................................
      data.scanstatsOn = 1;
    }else if( strcmp(z,"-backslash")==0 ){
      /* Undocumented command-line option: -backslash
      ** Causes C-style backslash escapes to be evaluated in SQL statements
      ** prior to sending the SQL into SQLite.  Useful for injecting
      ** crazy bytes in the middle of SQL statements for testing and debugging.
      */
      data.backslashOn = 1;
    }else if( strcmp(z,"-bail")==0 ){
      bail_on_error = 1;
    }else if( strcmp(z,"-version")==0 ){
      printf("%s %s\n", sqlite3_libversion(), sqlite3_sourceid());
      return 0;
    }else if( strcmp(z,"-interactive")==0 ){
      stdin_is_interactive = 1;







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







<




<
<







 







|
|
|
>
>
>
>
>
>
>
>
>
>
>







 









>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>






|







<
<
<
|
<
<
<
>
|






|


|

|



<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>













<
<
>
|
<
<
<
<
<
|
<
>

<
<
<
|
<
>

<
<
<
|
<
>

<
<
<
|
<
>

<
<
<
|
<
<
>
>
|
<
<
<
>
|
<
<
<
<
|
<
>

<
>
|
<
<







 







|







 







>
>
>
>
>
>
>
|
>
>
>
>

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>










<









|







 







|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>







 







|







 







>







 







<
<
<
<













>
>
>
>
>
>
>
>







 







|







 







>
>
>
>
>
>
>
>
>
>
>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>




<
<
<
<
<



>
>
>


<
>













|
|
<
<
|
|
|
|
>
>
|
|
|
|
|
|
<












|







 







>


>


>
>





>







 







>







 







|
|




|

|
|
|







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
>







 







>







 







<







 







|







 







|







 







|







 







|



|







 







|







 







|







 







|







450
451
452
453
454
455
456






















457
458
459
460
461
462
463
...
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
....
1357
1358
1359
1360
1361
1362
1363

1364
1365
1366
1367


1368
1369
1370
1371
1372
1373
1374
....
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
....
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135



2136



2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153













































2154
2155
2156
2157
2158
2159
2160
....
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299


2300
2301





2302

2303
2304



2305

2306
2307



2308

2309
2310



2311

2312
2313



2314


2315
2316
2317



2318
2319




2320

2321
2322

2323
2324


2325
2326
2327
2328
2329
2330
2331
....
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
....
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942

2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
....
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
....
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
....
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
....
3337
3338
3339
3340
3341
3342
3343




3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
....
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
....
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
....
4585
4586
4587
4588
4589
4590
4591
4592
4593
4594
4595
4596
4597
4598
4599
....
4649
4650
4651
4652
4653
4654
4655
4656
4657
4658
4659
4660
4661
4662
4663
4664
4665
4666
4667
4668
4669
4670
4671
4672
4673
4674
4675
4676
4677
4678
4679
4680
4681
4682





4683
4684
4685
4686
4687
4688
4689
4690

4691
4692
4693
4694
4695
4696
4697
4698
4699
4700
4701
4702
4703
4704
4705
4706


4707
4708
4709
4710
4711
4712
4713
4714
4715
4716
4717
4718

4719
4720
4721
4722
4723
4724
4725
4726
4727
4728
4729
4730
4731
4732
4733
4734
4735
4736
4737
4738
....
5235
5236
5237
5238
5239
5240
5241
5242
5243
5244
5245
5246
5247
5248
5249
5250
5251
5252
5253
5254
5255
5256
5257
5258
5259
5260
5261
5262
....
5291
5292
5293
5294
5295
5296
5297
5298
5299
5300
5301
5302
5303
5304
5305
....
5521
5522
5523
5524
5525
5526
5527
5528
5529
5530
5531
5532
5533
5534
5535
5536
5537
5538
5539
5540
5541
5542
5543
5544
5545
....
5562
5563
5564
5565
5566
5567
5568
5569
5570
5571
5572
5573
5574
5575
5576
....
5784
5785
5786
5787
5788
5789
5790
5791
5792
5793
5794
5795
5796
5797
5798
5799
5800
5801
5802
5803
5804
5805
5806
5807
5808
5809
5810
5811
5812
5813
5814
5815
5816
5817
5818
5819
5820
5821
5822
5823
5824
5825
5826
5827
5828
5829
5830
5831
5832
5833
5834
5835
5836
5837
5838
5839
5840
5841
5842
5843
5844
5845
5846
5847
5848
5849
5850
5851
5852
5853
5854
5855
5856
5857
5858
5859
5860
5861
5862
5863
5864
5865
5866
5867
5868
5869
5870
5871
5872
5873
5874
5875
5876
5877
5878
5879
5880
5881
5882
5883
5884
5885
5886
5887
5888
5889
5890
5891
5892
5893
5894
5895
5896
5897
5898
5899
5900
5901
5902
5903
5904
5905
5906
5907
5908
5909
5910
....
5912
5913
5914
5915
5916
5917
5918
5919
5920
5921
5922
5923
5924
5925
5926
5927
5928
5929
5930
5931
5932
5933
5934
5935
5936
5937
5938
5939
5940
5941
5942
5943
5944
5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955
5956
5957
5958
5959
5960
5961
5962
5963
5964
5965
5966
5967
5968
5969
5970
5971
5972
5973
5974
5975
5976
5977
5978
5979
5980
5981
5982
5983
5984
5985
5986
5987
5988
5989
5990
5991
5992
5993
5994
5995
5996
5997
5998
5999
6000
6001
6002
6003
6004
6005
6006
6007
6008
6009
6010
6011
6012
6013
6014
6015
6016
6017
6018
6019
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
6030
6031
6032
6033
6034
6035
6036
6037
6038
6039
6040
6041
....
6057
6058
6059
6060
6061
6062
6063
6064
6065
6066
6067
6068
6069
6070
6071
6072
....
6370
6371
6372
6373
6374
6375
6376
6377
6378
6379
6380
6381
6382
6383
6384
....
6407
6408
6409
6410
6411
6412
6413

6414
6415
6416
6417
6418
6419
6420
....
6480
6481
6482
6483
6484
6485
6486
6487
6488
6489
6490
6491
6492
6493
6494
....
6623
6624
6625
6626
6627
6628
6629
6630
6631
6632
6633
6634
6635
6636
6637
....
6643
6644
6645
6646
6647
6648
6649
6650
6651
6652
6653
6654
6655
6656
6657
....
6686
6687
6688
6689
6690
6691
6692
6693
6694
6695
6696
6697
6698
6699
6700
6701
6702
6703
6704
....
6733
6734
6735
6736
6737
6738
6739
6740
6741
6742
6743
6744
6745
6746
6747
....
7202
7203
7204
7205
7206
7207
7208
7209
7210
7211
7212
7213
7214
7215
7216
....
7217
7218
7219
7220
7221
7222
7223
7224
7225
7226
7227
7228
7229
7230
7231
    if( *z=='+' || *z=='-' ) z++;
    if( !IsDigit(*z) ) return 0;
    while( IsDigit(*z) ){ z++; }
    if( realnum ) *realnum = 1;
  }
  return *z==0;
}























/*
** Compute a string length that is limited to what can be stored in
** lower 30 bits of a 32-bit signed integer.
*/
static int strlen30(const char *z){
  const char *z2 = z;
................................................................................
    free(zPrior);
    zResult = shell_readline(zPrompt);
    if( zResult && *zResult ) shell_add_history(zResult);
#endif
  }
  return zResult;
}
/*
** A variable length string to which one can append text.
*/
typedef struct ShellText ShellText;
struct ShellText {
  char *z;
  int n;
  int nAlloc;
};

/*
** Initialize and destroy a ShellText object
*/
static void initText(ShellText *p){
  memset(p, 0, sizeof(*p));
}
static void freeText(ShellText *p){
  free(p->z);
  initText(p);
}

/* zIn is either a pointer to a NULL-terminated string in memory obtained
** from malloc(), or a NULL pointer. The string pointed to by zAppend is
** added to zIn, and the result returned in memory obtained from malloc().
** zIn, if it was not NULL, is freed.
**
** If the third argument, quote, is not '\0', then it is used as a
** quote character for zAppend.
*/
static void appendText(ShellText *p, char const *zAppend, char quote){
  int len;
  int i;
  int nAppend = strlen30(zAppend);

  len = nAppend+p->n+1;
  if( quote ){
    len += 2;
    for(i=0; i<nAppend; i++){
      if( zAppend[i]==quote ) len++;
    }
  }

  if( p->n+len>=p->nAlloc ){
    p->nAlloc = p->nAlloc*2 + len + 20;
    p->z = realloc(p->z, p->nAlloc);
    if( p->z==0 ){
      memset(p, 0, sizeof(*p));
      return;
    }
  }

  if( quote ){
    char *zCsr = p->z+p->n;
    *zCsr++ = quote;
    for(i=0; i<nAppend; i++){
      *zCsr++ = zAppend[i];
      if( zAppend[i]==quote ) *zCsr++ = quote;
    }
    *zCsr++ = quote;
    p->n = (int)(zCsr - p->z);
    *zCsr = '\0';
  }else{
    memcpy(p->z+p->n, zAppend, nAppend);
    p->n += nAppend;
    p->z[p->n] = '\0';
  }
}

/*
** Attempt to determine if identifier zName needs to be quoted, either
** because it contains non-alphanumeric characters, or because it is an
** SQLite keyword.  Be conservative in this estimate:  When in doubt assume
** that quoting is required.
**
** Return '"' if quoting is required.  Return 0 if no quoting is required.
*/
static char quoteChar(const char *zName){
  /* All SQLite keywords, in alphabetical order */
  static const char *azKeywords[] = {
    "ABORT", "ACTION", "ADD", "AFTER", "ALL", "ALTER", "ANALYZE", "AND", "AS",
    "ASC", "ATTACH", "AUTOINCREMENT", "BEFORE", "BEGIN", "BETWEEN", "BY",
    "CASCADE", "CASE", "CAST", "CHECK", "COLLATE", "COLUMN", "COMMIT",
    "CONFLICT", "CONSTRAINT", "CREATE", "CROSS", "CURRENT_DATE",
    "CURRENT_TIME", "CURRENT_TIMESTAMP", "DATABASE", "DEFAULT", "DEFERRABLE",
    "DEFERRED", "DELETE", "DESC", "DETACH", "DISTINCT", "DROP", "EACH",
    "ELSE", "END", "ESCAPE", "EXCEPT", "EXCLUSIVE", "EXISTS", "EXPLAIN",
    "FAIL", "FOR", "FOREIGN", "FROM", "FULL", "GLOB", "GROUP", "HAVING", "IF",
    "IGNORE", "IMMEDIATE", "IN", "INDEX", "INDEXED", "INITIALLY", "INNER",
    "INSERT", "INSTEAD", "INTERSECT", "INTO", "IS", "ISNULL", "JOIN", "KEY",
    "LEFT", "LIKE", "LIMIT", "MATCH", "NATURAL", "NO", "NOT", "NOTNULL",
    "NULL", "OF", "OFFSET", "ON", "OR", "ORDER", "OUTER", "PLAN", "PRAGMA",
    "PRIMARY", "QUERY", "RAISE", "RECURSIVE", "REFERENCES", "REGEXP",
    "REINDEX", "RELEASE", "RENAME", "REPLACE", "RESTRICT", "RIGHT",
    "ROLLBACK", "ROW", "SAVEPOINT", "SELECT", "SET", "TABLE", "TEMP",
    "TEMPORARY", "THEN", "TO", "TRANSACTION", "TRIGGER", "UNION", "UNIQUE",
    "UPDATE", "USING", "VACUUM", "VALUES", "VIEW", "VIRTUAL", "WHEN", "WHERE",
    "WITH", "WITHOUT",
  };
  int i, lwr, upr, mid, c;
  if( !isalpha((unsigned char)zName[0]) && zName[0]!='_' ) return '"';
  for(i=0; zName[i]; i++){
    if( !isalnum((unsigned char)zName[i]) && zName[i]!='_' ) return '"';
  }
  lwr = 0;
  upr = sizeof(azKeywords)/sizeof(azKeywords[0]) - 1;
  while( lwr<=upr ){
    mid = (lwr+upr)/2;
    c = sqlite3_stricmp(azKeywords[mid], zName);
    if( c==0 ) return '"';
    if( c<0 ){
      lwr = mid+1;
    }else{
      upr = mid-1;
    }
  }
  return 0;
}

/******************************************************************************
** SHA3 hash implementation copied from ../ext/misc/shathree.c
*/
typedef sqlite3_uint64 u64;
/*
** Macros to determine whether the machine is big or little endian,
** and whether or not that determination is run-time or compile-time.
**
** For best performance, an attempt is made to guess at the byte-order
** using C-preprocessor macros.  If that is unsuccessful, or if
** -DSHA3_BYTEORDER=0 is set, then byte-order is determined
** at run-time.
*/
#ifndef SHA3_BYTEORDER
# if defined(i386)     || defined(__i386__)   || defined(_M_IX86) ||    \
     defined(__x86_64) || defined(__x86_64__) || defined(_M_X64)  ||    \
     defined(_M_AMD64) || defined(_M_ARM)     || defined(__x86)   ||    \
     defined(__arm__)
#   define SHA3_BYTEORDER    1234
# elif defined(sparc)    || defined(__ppc__)
#   define SHA3_BYTEORDER    4321
# else
#   define SHA3_BYTEORDER 0
# endif
#endif


/*
** State structure for a SHA3 hash in progress
*/
typedef struct SHA3Context SHA3Context;
struct SHA3Context {
  union {
    u64 s[25];                /* Keccak state. 5x5 lines of 64 bits each */
    unsigned char x[1600];    /* ... or 1600 bytes */
  } u;
  unsigned nRate;        /* Bytes of input accepted per Keccak iteration */
  unsigned nLoaded;      /* Input bytes loaded into u.x[] so far this cycle */
  unsigned ixMask;       /* Insert next input into u.x[nLoaded^ixMask]. */
};

/*
** A single step of the Keccak mixing function for a 1600-bit state
*/
static void KeccakF1600Step(SHA3Context *p){
  int i;
  u64 B0, B1, B2, B3, B4;
  u64 C0, C1, C2, C3, C4;
  u64 D0, D1, D2, D3, D4;
  static const u64 RC[] = {
    0x0000000000000001ULL,  0x0000000000008082ULL,
    0x800000000000808aULL,  0x8000000080008000ULL,
    0x000000000000808bULL,  0x0000000080000001ULL,
    0x8000000080008081ULL,  0x8000000000008009ULL,
    0x000000000000008aULL,  0x0000000000000088ULL,
    0x0000000080008009ULL,  0x000000008000000aULL,
    0x000000008000808bULL,  0x800000000000008bULL,
    0x8000000000008089ULL,  0x8000000000008003ULL,
    0x8000000000008002ULL,  0x8000000000000080ULL,
    0x000000000000800aULL,  0x800000008000000aULL,
    0x8000000080008081ULL,  0x8000000000008080ULL,
    0x0000000080000001ULL,  0x8000000080008008ULL
  };
# define A00 (p->u.s[0])
# define A01 (p->u.s[1])
# define A02 (p->u.s[2])
# define A03 (p->u.s[3])
# define A04 (p->u.s[4])
# define A10 (p->u.s[5])
# define A11 (p->u.s[6])
# define A12 (p->u.s[7])
# define A13 (p->u.s[8])
# define A14 (p->u.s[9])
# define A20 (p->u.s[10])
# define A21 (p->u.s[11])
# define A22 (p->u.s[12])
# define A23 (p->u.s[13])
# define A24 (p->u.s[14])
# define A30 (p->u.s[15])
# define A31 (p->u.s[16])
# define A32 (p->u.s[17])
# define A33 (p->u.s[18])
# define A34 (p->u.s[19])
# define A40 (p->u.s[20])
# define A41 (p->u.s[21])
# define A42 (p->u.s[22])
# define A43 (p->u.s[23])
# define A44 (p->u.s[24])
# define ROL64(a,x) ((a<<x)|(a>>(64-x)))

  for(i=0; i<24; i+=4){
    C0 = A00^A10^A20^A30^A40;
    C1 = A01^A11^A21^A31^A41;
    C2 = A02^A12^A22^A32^A42;
    C3 = A03^A13^A23^A33^A43;
    C4 = A04^A14^A24^A34^A44;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A11^D1), 44);
    B2 = ROL64((A22^D2), 43);
    B3 = ROL64((A33^D3), 21);
    B4 = ROL64((A44^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i];
    A11 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A20^D0), 3);
    B3 = ROL64((A31^D1), 45);
    B4 = ROL64((A42^D2), 61);
    B0 = ROL64((A03^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A20 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A40^D0), 18);
    B0 = ROL64((A01^D1), 1);
    B1 = ROL64((A12^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A34^D4), 8);
    A40 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A10^D0), 36);
    B2 = ROL64((A21^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A43^D3), 56);
    B0 = ROL64((A04^D4), 27);
    A10 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A30^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A02^D2), 62);
    B1 = ROL64((A13^D3), 55);
    B2 = ROL64((A24^D4), 39);
    A30 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    C0 = A00^A20^A40^A10^A30;
    C1 = A11^A31^A01^A21^A41;
    C2 = A22^A42^A12^A32^A02;
    C3 = A33^A03^A23^A43^A13;
    C4 = A44^A14^A34^A04^A24;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A31^D1), 44);
    B2 = ROL64((A12^D2), 43);
    B3 = ROL64((A43^D3), 21);
    B4 = ROL64((A24^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+1];
    A31 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A40^D0), 3);
    B3 = ROL64((A21^D1), 45);
    B4 = ROL64((A02^D2), 61);
    B0 = ROL64((A33^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A40 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A30^D0), 18);
    B0 = ROL64((A11^D1), 1);
    B1 = ROL64((A42^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A04^D4), 8);
    A30 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A20^D0), 36);
    B2 = ROL64((A01^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A13^D3), 56);
    B0 = ROL64((A44^D4), 27);
    A20 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A10^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A22^D2), 62);
    B1 = ROL64((A03^D3), 55);
    B2 = ROL64((A34^D4), 39);
    A10 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    C0 = A00^A40^A30^A20^A10;
    C1 = A31^A21^A11^A01^A41;
    C2 = A12^A02^A42^A32^A22;
    C3 = A43^A33^A23^A13^A03;
    C4 = A24^A14^A04^A44^A34;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A21^D1), 44);
    B2 = ROL64((A42^D2), 43);
    B3 = ROL64((A13^D3), 21);
    B4 = ROL64((A34^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+2];
    A21 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A30^D0), 3);
    B3 = ROL64((A01^D1), 45);
    B4 = ROL64((A22^D2), 61);
    B0 = ROL64((A43^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A30 =   B0 ^((~B1)&  B2 );
    A01 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A10^D0), 18);
    B0 = ROL64((A31^D1), 1);
    B1 = ROL64((A02^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A44^D4), 8);
    A10 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A40^D0), 36);
    B2 = ROL64((A11^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A03^D3), 56);
    B0 = ROL64((A24^D4), 27);
    A40 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A20^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A12^D2), 62);
    B1 = ROL64((A33^D3), 55);
    B2 = ROL64((A04^D4), 39);
    A20 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    C0 = A00^A30^A10^A40^A20;
    C1 = A21^A01^A31^A11^A41;
    C2 = A42^A22^A02^A32^A12;
    C3 = A13^A43^A23^A03^A33;
    C4 = A34^A14^A44^A24^A04;
    D0 = C4^ROL64(C1, 1);
    D1 = C0^ROL64(C2, 1);
    D2 = C1^ROL64(C3, 1);
    D3 = C2^ROL64(C4, 1);
    D4 = C3^ROL64(C0, 1);

    B0 = (A00^D0);
    B1 = ROL64((A01^D1), 44);
    B2 = ROL64((A02^D2), 43);
    B3 = ROL64((A03^D3), 21);
    B4 = ROL64((A04^D4), 14);
    A00 =   B0 ^((~B1)&  B2 );
    A00 ^= RC[i+3];
    A01 =   B1 ^((~B2)&  B3 );
    A02 =   B2 ^((~B3)&  B4 );
    A03 =   B3 ^((~B4)&  B0 );
    A04 =   B4 ^((~B0)&  B1 );

    B2 = ROL64((A10^D0), 3);
    B3 = ROL64((A11^D1), 45);
    B4 = ROL64((A12^D2), 61);
    B0 = ROL64((A13^D3), 28);
    B1 = ROL64((A14^D4), 20);
    A10 =   B0 ^((~B1)&  B2 );
    A11 =   B1 ^((~B2)&  B3 );
    A12 =   B2 ^((~B3)&  B4 );
    A13 =   B3 ^((~B4)&  B0 );
    A14 =   B4 ^((~B0)&  B1 );

    B4 = ROL64((A20^D0), 18);
    B0 = ROL64((A21^D1), 1);
    B1 = ROL64((A22^D2), 6);
    B2 = ROL64((A23^D3), 25);
    B3 = ROL64((A24^D4), 8);
    A20 =   B0 ^((~B1)&  B2 );
    A21 =   B1 ^((~B2)&  B3 );
    A22 =   B2 ^((~B3)&  B4 );
    A23 =   B3 ^((~B4)&  B0 );
    A24 =   B4 ^((~B0)&  B1 );

    B1 = ROL64((A30^D0), 36);
    B2 = ROL64((A31^D1), 10);
    B3 = ROL64((A32^D2), 15);
    B4 = ROL64((A33^D3), 56);
    B0 = ROL64((A34^D4), 27);
    A30 =   B0 ^((~B1)&  B2 );
    A31 =   B1 ^((~B2)&  B3 );
    A32 =   B2 ^((~B3)&  B4 );
    A33 =   B3 ^((~B4)&  B0 );
    A34 =   B4 ^((~B0)&  B1 );

    B3 = ROL64((A40^D0), 41);
    B4 = ROL64((A41^D1), 2);
    B0 = ROL64((A42^D2), 62);
    B1 = ROL64((A43^D3), 55);
    B2 = ROL64((A44^D4), 39);
    A40 =   B0 ^((~B1)&  B2 );
    A41 =   B1 ^((~B2)&  B3 );
    A42 =   B2 ^((~B3)&  B4 );
    A43 =   B3 ^((~B4)&  B0 );
    A44 =   B4 ^((~B0)&  B1 );
  }
}

/*
** Initialize a new hash.  iSize determines the size of the hash
** in bits and should be one of 224, 256, 384, or 512.  Or iSize
** can be zero to use the default hash size of 256 bits.
*/
static void SHA3Init(SHA3Context *p, int iSize){
  memset(p, 0, sizeof(*p));
  if( iSize>=128 && iSize<=512 ){
    p->nRate = (1600 - ((iSize + 31)&~31)*2)/8;
  }else{
    p->nRate = (1600 - 2*256)/8;
  }
#if SHA3_BYTEORDER==1234
  /* Known to be little-endian at compile-time. No-op */
#elif SHA3_BYTEORDER==4321
  p->ixMask = 7;  /* Big-endian */
#else
  {
    static unsigned int one = 1;
    if( 1==*(unsigned char*)&one ){
      /* Little endian.  No byte swapping. */
      p->ixMask = 0;
    }else{
      /* Big endian.  Byte swap. */
      p->ixMask = 7;
    }
  }
#endif
}

/*
** Make consecutive calls to the SHA3Update function to add new content
** to the hash
*/
static void SHA3Update(
  SHA3Context *p,
  const unsigned char *aData,
  unsigned int nData
){
  unsigned int i = 0;
#if SHA3_BYTEORDER==1234
  if( (p->nLoaded % 8)==0 && ((aData - (const unsigned char*)0)&7)==0 ){
    for(; i+7<nData; i+=8){
      p->u.s[p->nLoaded/8] ^= *(u64*)&aData[i];
      p->nLoaded += 8;
      if( p->nLoaded>=p->nRate ){
        KeccakF1600Step(p);
        p->nLoaded = 0;
      }
    }
  }
#endif
  for(; i<nData; i++){
#if SHA3_BYTEORDER==1234
    p->u.x[p->nLoaded] ^= aData[i];
#elif SHA3_BYTEORDER==4321
    p->u.x[p->nLoaded^0x07] ^= aData[i];
#else
    p->u.x[p->nLoaded^p->ixMask] ^= aData[i];
#endif
    p->nLoaded++;
    if( p->nLoaded==p->nRate ){
      KeccakF1600Step(p);
      p->nLoaded = 0;
    }
  }
}

/*
** After all content has been added, invoke SHA3Final() to compute
** the final hash.  The function returns a pointer to the binary
** hash value.
*/
static unsigned char *SHA3Final(SHA3Context *p){
  unsigned int i;
  if( p->nLoaded==p->nRate-1 ){
    const unsigned char c1 = 0x86;
    SHA3Update(p, &c1, 1);
  }else{
    const unsigned char c2 = 0x06;
    const unsigned char c3 = 0x80;
    SHA3Update(p, &c2, 1);
    p->nLoaded = p->nRate - 1;
    SHA3Update(p, &c3, 1);
  }
  for(i=0; i<p->nRate; i++){
    p->u.x[i+p->nRate] = p->u.x[i^p->ixMask];
  }
  return &p->u.x[p->nRate];
}

/*
** Implementation of the sha3(X,SIZE) function.
**
** Return a BLOB which is the SIZE-bit SHA3 hash of X.  The default
** size is 256.  If X is a BLOB, it is hashed as is.  
** For all other non-NULL types of input, X is converted into a UTF-8 string
** and the string is hashed without the trailing 0x00 terminator.  The hash
** of a NULL value is NULL.
*/
static void sha3Func(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
  SHA3Context cx;
  int eType = sqlite3_value_type(argv[0]);
  int nByte = sqlite3_value_bytes(argv[0]);
  int iSize;
  if( argc==1 ){
    iSize = 256;
  }else{
    iSize = sqlite3_value_int(argv[1]);
    if( iSize!=224 && iSize!=256 && iSize!=384 && iSize!=512 ){
      sqlite3_result_error(context, "SHA3 size should be one of: 224 256 "
                                    "384 512", -1);
      return;
    }
  }
  if( eType==SQLITE_NULL ) return;
  SHA3Init(&cx, iSize);
  if( eType==SQLITE_BLOB ){
    SHA3Update(&cx, sqlite3_value_blob(argv[0]), nByte);
  }else{
    SHA3Update(&cx, sqlite3_value_text(argv[0]), nByte);
  }
  sqlite3_result_blob(context, SHA3Final(&cx), iSize/8, SQLITE_TRANSIENT);
}

/* Compute a string using sqlite3_vsnprintf() with a maximum length
** of 50 bytes and add it to the hash.
*/
static void hash_step_vformat(
  SHA3Context *p,                 /* Add content to this context */
  const char *zFormat,
  ...
){
  va_list ap;
  int n;
  char zBuf[50];
  va_start(ap, zFormat);
  sqlite3_vsnprintf(sizeof(zBuf),zBuf,zFormat,ap);
  va_end(ap);
  n = (int)strlen(zBuf);
  SHA3Update(p, (unsigned char*)zBuf, n);
}

/*
** Implementation of the sha3_query(SQL,SIZE) function.
**
** This function compiles and runs the SQL statement(s) given in the
** argument. The results are hashed using a SIZE-bit SHA3.  The default
** size is 256.
**
** The format of the byte stream that is hashed is summarized as follows:
**
**       S<n>:<sql>
**       R
**       N
**       I<int>
**       F<ieee-float>
**       B<size>:<bytes>
**       T<size>:<text>
**
** <sql> is the original SQL text for each statement run and <n> is
** the size of that text.  The SQL text is UTF-8.  A single R character
** occurs before the start of each row.  N means a NULL value.
** I mean an 8-byte little-endian integer <int>.  F is a floating point
** number with an 8-byte little-endian IEEE floating point value <ieee-float>.
** B means blobs of <size> bytes.  T means text rendered as <size>
** bytes of UTF-8.  The <n> and <size> values are expressed as an ASCII
** text integers.
**
** For each SQL statement in the X input, there is one S segment.  Each
** S segment is followed by zero or more R segments, one for each row in the
** result set.  After each R, there are one or more N, I, F, B, or T segments,
** one for each column in the result set.  Segments are concatentated directly
** with no delimiters of any kind.
*/
static void sha3QueryFunc(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
  sqlite3 *db = sqlite3_context_db_handle(context);
  const char *zSql = (const char*)sqlite3_value_text(argv[0]);
  sqlite3_stmt *pStmt = 0;
  int nCol;                   /* Number of columns in the result set */
  int i;                      /* Loop counter */
  int rc;
  int n;
  const char *z;
  SHA3Context cx;
  int iSize;

  if( argc==1 ){
    iSize = 256;
  }else{
    iSize = sqlite3_value_int(argv[1]);
    if( iSize!=224 && iSize!=256 && iSize!=384 && iSize!=512 ){
      sqlite3_result_error(context, "SHA3 size should be one of: 224 256 "
                                    "384 512", -1);
      return;
    }
  }
  if( zSql==0 ) return;
  SHA3Init(&cx, iSize);
  while( zSql[0] ){
    rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, &zSql);
    if( rc ){
      char *zMsg = sqlite3_mprintf("error SQL statement [%s]: %s",
                                   zSql, sqlite3_errmsg(db));
      sqlite3_finalize(pStmt);
      sqlite3_result_error(context, zMsg, -1);
      sqlite3_free(zMsg);
      return;
    }
    if( !sqlite3_stmt_readonly(pStmt) ){
      char *zMsg = sqlite3_mprintf("non-query: [%s]", sqlite3_sql(pStmt));
      sqlite3_finalize(pStmt);
      sqlite3_result_error(context, zMsg, -1);
      sqlite3_free(zMsg);
      return;
    }
    nCol = sqlite3_column_count(pStmt);
    z = sqlite3_sql(pStmt);
    if( z==0 ){
      sqlite3_finalize(pStmt);
      continue;
    }
    n = (int)strlen(z);
    hash_step_vformat(&cx,"S%d:",n);
    SHA3Update(&cx,(unsigned char*)z,n);

    /* Compute a hash over the result of the query */
    while( SQLITE_ROW==sqlite3_step(pStmt) ){
      SHA3Update(&cx,(const unsigned char*)"R",1);
      for(i=0; i<nCol; i++){
        switch( sqlite3_column_type(pStmt,i) ){
          case SQLITE_NULL: {
            SHA3Update(&cx, (const unsigned char*)"N",1);
            break;
          }
          case SQLITE_INTEGER: {
            sqlite3_uint64 u;
            int j;
            unsigned char x[9];
            sqlite3_int64 v = sqlite3_column_int64(pStmt,i);
            memcpy(&u, &v, 8);
            for(j=8; j>=1; j--){
              x[j] = u & 0xff;
              u >>= 8;
            }
            x[0] = 'I';
            SHA3Update(&cx, x, 9);
            break;
          }
          case SQLITE_FLOAT: {
            sqlite3_uint64 u;
            int j;
            unsigned char x[9];
            double r = sqlite3_column_double(pStmt,i);
            memcpy(&u, &r, 8);
            for(j=8; j>=1; j--){
              x[j] = u & 0xff;
              u >>= 8;
            }
            x[0] = 'F';
            SHA3Update(&cx,x,9);
            break;
          }
          case SQLITE_TEXT: {
            int n2 = sqlite3_column_bytes(pStmt, i);
            const unsigned char *z2 = sqlite3_column_text(pStmt, i);
            hash_step_vformat(&cx,"T%d:",n2);
            SHA3Update(&cx, z2, n2);
            break;
          }
          case SQLITE_BLOB: {
            int n2 = sqlite3_column_bytes(pStmt, i);
            const unsigned char *z2 = sqlite3_column_blob(pStmt, i);
            hash_step_vformat(&cx,"B%d:",n2);
            SHA3Update(&cx, z2, n2);
            break;
          }
        }
      }
    }
    sqlite3_finalize(pStmt);
  }
  sqlite3_result_blob(context, SHA3Final(&cx), iSize/8, SQLITE_TRANSIENT);
}
/* End of SHA3 hashing logic copy/pasted from ../ext/misc/shathree.c
********************************************************************************/

#if defined(SQLITE_ENABLE_SESSION)
/*
** State information for a single open session
*/
typedef struct OpenSession OpenSession;
struct OpenSession {
................................................................................
/*
** State information about the database connection is contained in an
** instance of the following structure.
*/
typedef struct ShellState ShellState;
struct ShellState {
  sqlite3 *db;           /* The database */

  int autoExplain;       /* Automatically turn on .explain mode */
  int autoEQP;           /* Run EXPLAIN QUERY PLAN prior to seach SQL stmt */
  int statsOn;           /* True to display memory stats before each finalize */
  int scanstatsOn;       /* True to display scan stats before each finalize */


  int outCount;          /* Revert to stdout when reaching zero */
  int cnt;               /* Number of records displayed so far */
  FILE *out;             /* Write results here */
  FILE *traceOut;        /* Output for sqlite3_trace() */
  int nErr;              /* Number of errors seen */
  int mode;              /* An output mode setting */
  int cMode;             /* temporary output mode for the current query */
................................................................................
  OpenSession aSession[4];  /* Array of sessions.  [0] is in focus. */
#endif
};

/*
** These are the allowed shellFlgs values
*/
#define SHFLG_Scratch        0x00000001 /* The --scratch option is used */
#define SHFLG_Pagecache      0x00000002 /* The --pagecache option is used */
#define SHFLG_Lookaside      0x00000004 /* Lookaside memory is used */
#define SHFLG_Backslash      0x00000008 /* The --backslash option is used */
#define SHFLG_PreserveRowid  0x00000010 /* .dump preserves rowid values */
#define SHFLG_CountChanges   0x00000020 /* .changes setting */
#define SHFLG_Echo           0x00000040 /* .echo or --echo setting */

/*
** Macros for testing and setting shellFlgs
*/
#define ShellHasFlag(P,X)    (((P)->shellFlgs & (X))!=0)
#define ShellSetFlag(P,X)    ((P)->shellFlgs|=(X))
#define ShellClearFlag(P,X)  ((P)->shellFlgs&=(~(X)))

/*
** These are the allowed modes.
*/
#define MODE_Line     0  /* One column per line.  Blank line between records */
#define MODE_Column   1  /* One record per line in neat columns */
#define MODE_List     2  /* One record per line with a separator */
................................................................................
** This is the callback routine that the SQLite library
** invokes for each row of a query result.
*/
static int callback(void *pArg, int nArg, char **azArg, char **azCol){
  /* since we don't have type info, call the shell_callback with a NULL value */
  return shell_callback(pArg, nArg, azArg, azCol, NULL);
}

/*
** This is the callback routine from sqlite3_exec() that appends all
** output onto the end of a ShellText object.
*/
static int captureOutputCallback(void *pArg, int nArg, char **azArg, char **az){
  ShellText *p = (ShellText*)pArg;
  int i;
  if( p->n ) appendText(p, "|", 0);
  for(i=0; i<nArg; i++){
    if( i ) appendText(p, ",", 0);
    if( azArg[i] ) appendText(p, azArg[i], 0);
  }
  return 0;
}

/*
** Generate an appropriate SELFTEST table in the main database.
*/
static void createSelftestTable(ShellState *p){
  char *zErrMsg = 0;
  sqlite3_exec(p->db,
    "SAVEPOINT selftest_init;\n"
    "CREATE TABLE IF NOT EXISTS selftest(\n"
    "  tno INTEGER PRIMARY KEY,\n"   /* Test number */
    "  op TEXT,\n"                   /* Operator:  memo run */
    "  cmd TEXT,\n"                  /* Command text */
    "  ans TEXT\n"                   /* Desired answer */
    ");"
    "CREATE TEMP TABLE [_shell$self](op,cmd,ans);\n"
    "INSERT INTO [_shell$self](rowid,op,cmd)\n"
    "  VALUES(coalesce((SELECT (max(tno)+100)/10 FROM selftest),10),\n"
    "         'memo','Tests generated by --init');\n"
    "INSERT INTO [_shell$self]\n"
    "  SELECT 'run',\n"
    "    'SELECT hex(sha3_query(''SELECT type,name,tbl_name,sql "
                                 "FROM sqlite_master ORDER BY 2'',224))',\n"
    "    hex(sha3_query('SELECT type,name,tbl_name,sql "
                          "FROM sqlite_master ORDER BY 2',224));\n"
    "INSERT INTO [_shell$self]\n"
    "  SELECT 'run',"
    "    'SELECT hex(sha3_query(''SELECT * FROM \"' ||"
    "        printf('%w',name) || '\" NOT INDEXED'',224))',\n"
    "    hex(sha3_query(printf('SELECT * FROM \"%w\" NOT INDEXED',name),224))\n"
    "  FROM (\n"
    "    SELECT name FROM sqlite_master\n"
    "     WHERE type='table'\n"
    "       AND name<>'selftest'\n"
    "       AND coalesce(rootpage,0)>0\n"
    "  )\n"
    " ORDER BY name;\n"
    "INSERT INTO [_shell$self]\n"
    "  VALUES('run','PRAGMA integrity_check','ok');\n"
    "INSERT INTO selftest(tno,op,cmd,ans)"
    "  SELECT rowid*10,op,cmd,ans FROM [_shell$self];\n"
    "DROP TABLE [_shell$self];"
    ,0,0,&zErrMsg);
  if( zErrMsg ){
    utf8_printf(stderr, "SELFTEST initialization failure: %s\n", zErrMsg);
    sqlite3_free(zErrMsg);
  }
  sqlite3_exec(p->db, "RELEASE selftest_init",0,0,0);
}


/*
** Set the destination table field of the ShellState structure to
** the name of the table given.  Escape any quote characters in the
** table name.
*/
static void set_table_name(ShellState *p, const char *zName){
  int i, n;
  int cQuote;
  char *z;

  if( p->zDestTable ){
    free(p->zDestTable);
    p->zDestTable = 0;
  }
  if( zName==0 ) return;



  cQuote = quoteChar(zName);



  n = strlen30(zName);
  if( cQuote ) n += 2;
  z = p->zDestTable = malloc( n+1 );
  if( z==0 ){
    raw_printf(stderr,"Error: out of memory\n");
    exit(1);
  }
  n = 0;
  if( cQuote ) z[n++] = cQuote;
  for(i=0; zName[i]; i++){
    z[n++] = zName[i];
    if( zName[i]==cQuote ) z[n++] = cQuote;
  }
  if( cQuote ) z[n++] = cQuote;
  z[n] = 0;
}















































/*
** Execute a query statement that will generate SQL output.  Print
** the result columns, comma-separated, on a line and then add a
** semicolon terminator to the end of that line.
**
** If the number of columns is 1 and that column contains text "--"
................................................................................
      }
    }
  }
  fclose(in);
}
#endif

/*
** Display a single line of status using 64-bit values.
*/
static void displayStatLine(
  ShellState *p,            /* The shell context */
  char *zLabel,             /* Label for this one line */
  char *zFormat,            /* Format for the result */
  int iStatusCtrl,          /* Which status to display */
  int bReset                /* True to reset the stats */
){
  sqlite3_int64 iCur = -1;
  sqlite3_int64 iHiwtr = -1;
  int i, nPercent;
  char zLine[200];
  sqlite3_status64(iStatusCtrl, &iCur, &iHiwtr, bReset);
  for(i=0, nPercent=0; zFormat[i]; i++){
    if( zFormat[i]=='%' ) nPercent++;
  }
  if( nPercent>1 ){
    sqlite3_snprintf(sizeof(zLine), zLine, zFormat, iCur, iHiwtr);
  }else{
    sqlite3_snprintf(sizeof(zLine), zLine, zFormat, iHiwtr);
  }
  raw_printf(p->out, "%-36s %s\n", zLabel, zLine);
}

/*
** Display memory stats.
*/
static int display_stats(
  sqlite3 *db,                /* Database to query */
  ShellState *pArg,           /* Pointer to ShellState */
  int bReset                  /* True to reset the stats */
){
  int iCur;
  int iHiwtr;

  if( pArg && pArg->out ){


    displayStatLine(pArg, "Memory Used:",
       "%lld (max %lld) bytes", SQLITE_STATUS_MEMORY_USED, bReset);





    displayStatLine(pArg, "Number of Outstanding Allocations:",

       "%lld (max %lld)", SQLITE_STATUS_MALLOC_COUNT, bReset);
    if( pArg->shellFlgs & SHFLG_Pagecache ){



      displayStatLine(pArg, "Number of Pcache Pages Used:",

         "%lld (max %lld) pages", SQLITE_STATUS_PAGECACHE_USED, bReset);
    }



    displayStatLine(pArg, "Number of Pcache Overflow Bytes:",

       "%lld (max %lld) bytes", SQLITE_STATUS_PAGECACHE_OVERFLOW, bReset);
    if( pArg->shellFlgs & SHFLG_Scratch ){



      displayStatLine(pArg, "Number of Scratch Allocations Used:",

         "%lld (max %lld)", SQLITE_STATUS_SCRATCH_USED, bReset);
    }



    displayStatLine(pArg, "Number of Scratch Overflow Bytes:",


       "%lld (max %lld) bytes", SQLITE_STATUS_SCRATCH_OVERFLOW, bReset);
    displayStatLine(pArg, "Largest Allocation:",
       "%lld bytes", SQLITE_STATUS_MALLOC_SIZE, bReset);



    displayStatLine(pArg, "Largest Pcache Allocation:",
       "%lld bytes", SQLITE_STATUS_PAGECACHE_SIZE, bReset);




    displayStatLine(pArg, "Largest Scratch Allocation:",

       "%lld bytes", SQLITE_STATUS_SCRATCH_SIZE, bReset);
#ifdef YYTRACKMAXSTACKDEPTH

    displayStatLine(pArg, "Deepest Parser Stack:",
       "%lld (max %lld)", SQLITE_STATUS_PARSER_STACK, bReset);


#endif
  }

  if( pArg && pArg->out && db ){
    if( pArg->shellFlgs & SHFLG_Lookaside ){
      iHiwtr = iCur = -1;
      sqlite3_db_status(db, SQLITE_DBSTATUS_LOOKASIDE_USED,
................................................................................
      /* save off the prepared statment handle and reset row count */
      if( pArg ){
        pArg->pStmt = pStmt;
        pArg->cnt = 0;
      }

      /* echo the sql statement if echo on */
      if( pArg && ShellHasFlag(pArg, SHFLG_Echo) ){
        utf8_printf(pArg->out, "%s\n", zStmtSql ? zStmtSql : zSql);
      }

      /* Show the EXPLAIN QUERY PLAN if .eqp is on */
      if( pArg && pArg->autoEQP && sqlite3_strlike("EXPLAIN%",zStmtSql,0)!=0 ){
        sqlite3_stmt *pExplain;
        char *zEQP;
................................................................................
      }
    }
  } /* end while */

  return rc;
}

/*
** Release memory previously allocated by tableColumnList().
*/
static void freeColumnList(char **azCol){
  int i;
  for(i=1; azCol[i]; i++){
    sqlite3_free(azCol[i]);
  }
  /* azCol[0] is a static string */
  sqlite3_free(azCol);
}

/*
** Return a list of pointers to strings which are the names of all
** columns in table zTab.   The memory to hold the names is dynamically
** allocated and must be released by the caller using a subsequent call
** to freeColumnList().
**
** The azCol[0] entry is usually NULL.  However, if zTab contains a rowid
** value that needs to be preserved, then azCol[0] is filled in with the
** name of the rowid column.
**
** The first regular column in the table is azCol[1].  The list is terminated
** by an entry with azCol[i]==0.
*/
static char **tableColumnList(ShellState *p, const char *zTab){
  char **azCol = 0;
  sqlite3_stmt *pStmt;
  char *zSql;
  int nCol = 0;
  int nAlloc = 0;
  int nPK = 0;       /* Number of PRIMARY KEY columns seen */
  int isIPK = 0;     /* True if one PRIMARY KEY column of type INTEGER */
  int preserveRowid = ShellHasFlag(p, SHFLG_PreserveRowid);
  int rc;

  zSql = sqlite3_mprintf("PRAGMA table_info=%Q", zTab);
  rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
  sqlite3_free(zSql);
  if( rc ) return 0;
  while( sqlite3_step(pStmt)==SQLITE_ROW ){
    if( nCol>=nAlloc-2 ){
      nAlloc = nAlloc*2 + nCol + 10;
      azCol = sqlite3_realloc(azCol, nAlloc*sizeof(azCol[0]));
      if( azCol==0 ){
        raw_printf(stderr, "Error: out of memory\n");
        exit(1);
      }
    }
    azCol[++nCol] = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 1));
    if( sqlite3_column_int(pStmt, 5) ){
      nPK++;
      if( nPK==1
       && sqlite3_stricmp((const char*)sqlite3_column_text(pStmt,2),
                          "INTEGER")==0 
      ){
        isIPK = 1;
      }else{
        isIPK = 0;
      }
    }
  }
  sqlite3_finalize(pStmt);
  azCol[0] = 0;
  azCol[nCol+1] = 0;

  /* The decision of whether or not a rowid really needs to be preserved
  ** is tricky.  We never need to preserve a rowid for a WITHOUT ROWID table
  ** or a table with an INTEGER PRIMARY KEY.  We are unable to preserve
  ** rowids on tables where the rowid is inaccessible because there are other
  ** columns in the table named "rowid", "_rowid_", and "oid".
  */
  if( preserveRowid && isIPK ){
    /* If a single PRIMARY KEY column with type INTEGER was seen, then it
    ** might be an alise for the ROWID.  But it might also be a WITHOUT ROWID
    ** table or a INTEGER PRIMARY KEY DESC column, neither of which are
    ** ROWID aliases.  To distinguish these cases, check to see if
    ** there is a "pk" entry in "PRAGMA index_list".  There will be
    ** no "pk" index if the PRIMARY KEY really is an alias for the ROWID.
    */
    zSql = sqlite3_mprintf("SELECT 1 FROM pragma_index_list(%Q)"
                           " WHERE origin='pk'", zTab);
    rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
    sqlite3_free(zSql);
    if( rc ){
      freeColumnList(azCol);
      return 0;
    }
    rc = sqlite3_step(pStmt);
    sqlite3_finalize(pStmt);
    preserveRowid = rc==SQLITE_ROW;
  }
  if( preserveRowid ){
    /* Only preserve the rowid if we can find a name to use for the
    ** rowid */
    static char *azRowid[] = { "rowid", "_rowid_", "oid" };
    int i, j;
    for(j=0; j<3; j++){
      for(i=1; i<=nCol; i++){
        if( sqlite3_stricmp(azRowid[j],azCol[i])==0 ) break;
      }
      if( i>nCol ){
        /* At this point, we know that azRowid[j] is not the name of any
        ** ordinary column in the table.  Verify that azRowid[j] is a valid
        ** name for the rowid before adding it to azCol[0].  WITHOUT ROWID
        ** tables will fail this last check */
        int rc;
        rc = sqlite3_table_column_metadata(p->db,0,zTab,azRowid[j],0,0,0,0,0);
        if( rc==SQLITE_OK ) azCol[0] = azRowid[j];
        break;
      }
    }
  }
  return azCol;
}

/*
** Toggle the reverse_unordered_selects setting.
*/
static void toggleSelectOrder(sqlite3 *db){
  sqlite3_stmt *pStmt = 0;
  int iSetting = 0;
  char zStmt[100];
  sqlite3_prepare_v2(db, "PRAGMA reverse_unordered_selects", -1, &pStmt, 0);
  if( sqlite3_step(pStmt)==SQLITE_ROW ){
    iSetting = sqlite3_column_int(pStmt, 0);
  }
  sqlite3_finalize(pStmt);
  sqlite3_snprintf(sizeof(zStmt), zStmt,
       "PRAGMA reverse_unordered_selects(%d)", !iSetting);
  sqlite3_exec(db, zStmt, 0, 0, 0);
}

/*
** This is a different callback routine used for dumping the database.
** Each row received by this callback consists of a table name,
** the table type ("index" or "table") and SQL to create the table.
** This routine should print text sufficient to recreate the table.
*/
static int dump_callback(void *pArg, int nArg, char **azArg, char **azCol){
  int rc;
  const char *zTable;
  const char *zType;
  const char *zSql;

  ShellState *p = (ShellState *)pArg;

  UNUSED_PARAMETER(azCol);
  if( nArg!=3 ) return 1;
  zTable = azArg[0];
  zType = azArg[1];
  zSql = azArg[2];

  if( strcmp(zTable, "sqlite_sequence")==0 ){
    raw_printf(p->out, "DELETE FROM sqlite_sequence;\n");
  }else if( sqlite3_strglob("sqlite_stat?", zTable)==0 ){
    raw_printf(p->out, "ANALYZE sqlite_master;\n");
  }else if( strncmp(zTable, "sqlite_", 7)==0 ){
    return 0;
  }else if( strncmp(zSql, "CREATE VIRTUAL TABLE", 20)==0 ){
    char *zIns;
    if( !p->writableSchema ){
................................................................................
    sqlite3_free(zIns);
    return 0;
  }else{
    printSchemaLine(p->out, zSql, ";\n");
  }

  if( strcmp(zType, "table")==0 ){
    ShellText sSelect;
    ShellText sTable;
    char **azCol;
    int i;
    char *savedDestTable;
    int savedMode;

    azCol = tableColumnList(p, zTable);
    if( azCol==0 ){
      p->nErr++;
      return 0;
    }

    /* Always quote the table name, even if it appears to be pure ascii,
    ** in case it is a keyword. Ex:  INSERT INTO "table" ... */
    initText(&sTable);
    appendText(&sTable, zTable, quoteChar(zTable));
    /* If preserving the rowid, add a column list after the table name.
    ** In other words:  "INSERT INTO tab(rowid,a,b,c,...) VALUES(...)"
    ** instead of the usual "INSERT INTO tab VALUES(...)".
    */
    if( azCol[0] ){
      appendText(&sTable, "(", 0);
      appendText(&sTable, azCol[0], 0);
      for(i=1; azCol[i]; i++){
        appendText(&sTable, ",", 0);
        appendText(&sTable, azCol[i], quoteChar(azCol[i]));
      }
      appendText(&sTable, ")", 0);
    }

    /* Build an appropriate SELECT statement */
    initText(&sSelect);
    appendText(&sSelect, "SELECT ", 0);
    if( azCol[0] ){
      appendText(&sSelect, azCol[0], 0);
      appendText(&sSelect, ",", 0);
    }
    for(i=1; azCol[i]; i++){
      appendText(&sSelect, azCol[i], quoteChar(azCol[i]));
      if( azCol[i+1] ){
        appendText(&sSelect, ",", 0);
      }
    }
    freeColumnList(azCol);
    appendText(&sSelect, " FROM ", 0);
    appendText(&sSelect, zTable, quoteChar(zTable));

    savedDestTable = p->zDestTable;
    savedMode = p->mode;
    p->zDestTable = sTable.z;
    p->mode = p->cMode = MODE_Insert;
    rc = shell_exec(p->db, sSelect.z, shell_callback, p, 0);
    if( (rc&0xff)==SQLITE_CORRUPT ){
      raw_printf(p->out, "/****** CORRUPTION ERROR *******/\n");
      toggleSelectOrder(p->db);
      shell_exec(p->db, sSelect.z, shell_callback, p, 0);
      toggleSelectOrder(p->db);
    }
    p->zDestTable = savedDestTable;
    p->mode = savedMode;
    freeText(&sTable);
    freeText(&sSelect);
    if( rc ) p->nErr++;
  }
  return 0;
}

/*
** Run zQuery.  Use dump_callback() as the callback routine so that
** the contents of the query are output as SQL statements.
................................................................................
  ".mode MODE ?TABLE?     Set output mode where MODE is one of:\n"
  "                         ascii    Columns/rows delimited by 0x1F and 0x1E\n"
  "                         csv      Comma-separated values\n"
  "                         column   Left-aligned columns.  (See .width)\n"
  "                         html     HTML <table> code\n"
  "                         insert   SQL insert statements for TABLE\n"
  "                         line     One value per line\n"
  "                         list     Values delimited by \"|\"\n"
  "                         quote    Escape answers as for SQL\n"
  "                         tabs     Tab-separated values\n"
  "                         tcl      TCL list elements\n"
  ".nullvalue STRING      Use STRING in place of NULL values\n"
  ".once FILENAME         Output for the next SQL command only to FILENAME\n"
  ".open ?--new? ?FILE?   Close existing database and reopen FILE\n"
  "                         The --new starts with an empty file\n"
................................................................................
  ".schema ?PATTERN?      Show the CREATE statements matching PATTERN\n"
  "                          Add --indent for pretty-printing\n"
  ".separator COL ?ROW?   Change the column separator and optionally the row\n"
  "                         separator for both the output mode and .import\n"
#if defined(SQLITE_ENABLE_SESSION)
  ".session CMD ...       Create or control sessions\n"
#endif
  ".sha3sum ?OPTIONS...?  Compute a SHA3 hash of database content\n"
  ".shell CMD ARGS...     Run CMD ARGS... in a system shell\n"
  ".show                  Show the current values for various settings\n"
  ".stats ?on|off?        Show stats or turn stats on or off\n"
  ".system CMD ARGS...    Run CMD ARGS... in a system shell\n"
  ".tables ?TABLE?        List names of tables\n"
  "                         If TABLE specified, only list tables matching\n"
  "                         LIKE pattern TABLE.\n"
................................................................................
** the database fails to open, print an error message and exit.
*/
static void open_db(ShellState *p, int keepAlive){
  if( p->db==0 ){
    sqlite3_initialize();
    sqlite3_open(p->zDbFilename, &p->db);
    globalDb = p->db;




    if( p->db==0 || SQLITE_OK!=sqlite3_errcode(p->db) ){
      utf8_printf(stderr,"Error: unable to open database \"%s\": %s\n",
          p->zDbFilename, sqlite3_errmsg(p->db));
      if( keepAlive ) return;
      exit(1);
    }
#ifndef SQLITE_OMIT_LOAD_EXTENSION
    sqlite3_enable_load_extension(p->db, 1);
#endif
    sqlite3_create_function(p->db, "readfile", 1, SQLITE_UTF8, 0,
                            readfileFunc, 0, 0);
    sqlite3_create_function(p->db, "writefile", 2, SQLITE_UTF8, 0,
                            writefileFunc, 0, 0);
    sqlite3_create_function(p->db, "sha3", 1, SQLITE_UTF8, 0,
                            sha3Func, 0, 0);
    sqlite3_create_function(p->db, "sha3", 2, SQLITE_UTF8, 0,
                            sha3Func, 0, 0);
    sqlite3_create_function(p->db, "sha3_query", 1, SQLITE_UTF8, 0,
                            sha3QueryFunc, 0, 0);
    sqlite3_create_function(p->db, "sha3_query", 2, SQLITE_UTF8, 0,
                            sha3QueryFunc, 0, 0);
  }
}

/*
** Do C-language style dequoting.
**
**    \a    -> alarm
................................................................................
  return isNeg? -v : v;
}

/*
** Interpret zArg as either an integer or a boolean value.  Return 1 or 0
** for TRUE and FALSE.  Return the integer value if appropriate.
*/
static int booleanValue(const char *zArg){
  int i;
  if( zArg[0]=='0' && zArg[1]=='x' ){
    for(i=2; hexDigitValue(zArg[i])>=0; i++){}
  }else{
    for(i=0; zArg[i]>='0' && zArg[i]<='9'; i++){}
  }
  if( i>0 && zArg[i]==0 ) return (int)(integerValue(zArg) & 0xffffffff);
................................................................................
  if( sqlite3_stricmp(zArg, "off")==0 || sqlite3_stricmp(zArg,"no")==0 ){
    return 0;
  }
  utf8_printf(stderr, "ERROR: Not a boolean value: \"%s\". Assuming \"no\".\n",
          zArg);
  return 0;
}

/*
** Set or clear a shell flag according to a boolean value.
*/
static void setOrClearFlag(ShellState *p, unsigned mFlag, const char *zArg){
  if( booleanValue(zArg) ){
    ShellSetFlag(p, mFlag);
  }else{
    ShellClearFlag(p, mFlag);
  }
}

/*
** Close an output file, assuming it is not stderr or stdout
*/
static void output_file_close(FILE *f){
  if( f && f!=stdout && f!=stderr ) fclose(f);
}
................................................................................
  */
  if( c=='b' && n>=3 && strncmp(azArg[0], "breakpoint", n)==0 ){
    test_breakpoint();
  }else

  if( c=='c' && n>=3 && strncmp(azArg[0], "changes", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_CountChanges, azArg[1]);
    }else{
      raw_printf(stderr, "Usage: .changes on|off\n");
      rc = 1;
    }
  }else

  /* Cancel output redirection, if it is currently set (by .testcase)
................................................................................
  }else

  if( c=='d' && strncmp(azArg[0], "dbinfo", n)==0 ){
    rc = shell_dbinfo_command(p, nArg, azArg);
  }else

  if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){
    const char *zLike = 0;
    int i;
    ShellClearFlag(p, SHFLG_PreserveRowid);
    for(i=1; i<nArg; i++){
      if( azArg[i][0]=='-' ){
        const char *z = azArg[i]+1;
        if( z[0]=='-' ) z++;
        if( strcmp(z,"preserve-rowids")==0 ){
          ShellSetFlag(p, SHFLG_PreserveRowid);
        }else
        {
          raw_printf(stderr, "Unknown option \"%s\" on \".dump\"\n", azArg[i]);
          rc = 1;
          goto meta_command_exit;
        }
      }else if( zLike ){
        raw_printf(stderr, "Usage: .dump ?--preserve-rowids? ?LIKE-PATTERN?\n");
        rc = 1;
        goto meta_command_exit;
      }else{
        zLike = azArg[i];
      }
    }
    open_db(p, 0);
    /* When playing back a "dump", the content might appear in an order
    ** which causes immediate foreign key constraints to be violated.
    ** So disable foreign-key constraint enforcement to prevent problems. */





    raw_printf(p->out, "PRAGMA foreign_keys=OFF;\n");
    raw_printf(p->out, "BEGIN TRANSACTION;\n");
    p->writableSchema = 0;
    /* Set writable_schema=ON since doing so forces SQLite to initialize
    ** as much of the schema as it can even if the sqlite_master table is
    ** corrupt. */
    sqlite3_exec(p->db, "SAVEPOINT dump; PRAGMA writable_schema=ON", 0, 0, 0);
    p->nErr = 0;

    if( zLike==0 ){
      run_schema_dump_query(p,
        "SELECT name, type, sql FROM sqlite_master "
        "WHERE sql NOT NULL AND type=='table' AND name!='sqlite_sequence'"
      );
      run_schema_dump_query(p,
        "SELECT name, type, sql FROM sqlite_master "
        "WHERE name=='sqlite_sequence'"
      );
      run_table_dump_query(p,
        "SELECT sql FROM sqlite_master "
        "WHERE sql NOT NULL AND type IN ('index','trigger','view')", 0
      );
    }else{
      char *zSql;
      zSql = sqlite3_mprintf(


        "SELECT name, type, sql FROM sqlite_master "
        "WHERE tbl_name LIKE %Q AND type=='table'"
        "  AND sql NOT NULL", zLike);
      run_schema_dump_query(p,zSql);
      sqlite3_free(zSql);
      zSql = sqlite3_mprintf(
        "SELECT sql FROM sqlite_master "
        "WHERE sql NOT NULL"
        "  AND type IN ('index','trigger','view')"
        "  AND tbl_name LIKE %Q", zLike);
      run_table_dump_query(p, zSql, 0);
      sqlite3_free(zSql);

    }
    if( p->writableSchema ){
      raw_printf(p->out, "PRAGMA writable_schema=OFF;\n");
      p->writableSchema = 0;
    }
    sqlite3_exec(p->db, "PRAGMA writable_schema=OFF;", 0, 0, 0);
    sqlite3_exec(p->db, "RELEASE dump;", 0, 0, 0);
    raw_printf(p->out, p->nErr ? "ROLLBACK; -- due to errors\n" : "COMMIT;\n");
  }else

  if( c=='e' && strncmp(azArg[0], "echo", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_Echo, azArg[1]);
    }else{
      raw_printf(stderr, "Usage: .echo on|off\n");
      rc = 1;
    }
  }else

  if( c=='e' && strncmp(azArg[0], "eqp", n)==0 ){
................................................................................

  if( c=='m' && strncmp(azArg[0], "mode", n)==0 ){
    const char *zMode = nArg>=2 ? azArg[1] : "";
    int n2 = (int)strlen(zMode);
    int c2 = zMode[0];
    if( c2=='l' && n2>2 && strncmp(azArg[1],"lines",n2)==0 ){
      p->mode = MODE_Line;
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
    }else if( c2=='c' && strncmp(azArg[1],"columns",n2)==0 ){
      p->mode = MODE_Column;
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
    }else if( c2=='l' && n2>2 && strncmp(azArg[1],"list",n2)==0 ){
      p->mode = MODE_List;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Column);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
    }else if( c2=='h' && strncmp(azArg[1],"html",n2)==0 ){
      p->mode = MODE_Html;
    }else if( c2=='t' && strncmp(azArg[1],"tcl",n2)==0 ){
      p->mode = MODE_Tcl;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Space);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
    }else if( c2=='c' && strncmp(azArg[1],"csv",n2)==0 ){
      p->mode = MODE_Csv;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Comma);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_CrLf);
    }else if( c2=='t' && strncmp(azArg[1],"tabs",n2)==0 ){
      p->mode = MODE_List;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Tab);
................................................................................
    char *zNewFilename;  /* Name of the database file to open */
    int iName = 1;       /* Index in azArg[] of the filename */
    int newFlag = 0;     /* True to delete file before opening */
    /* Close the existing database */
    session_close_all(p);
    sqlite3_close(p->db);
    p->db = 0;
    p->zDbFilename = 0;
    sqlite3_free(p->zFreeOnClose);
    p->zFreeOnClose = 0;
    /* Check for command-line arguments */
    for(iName=1; iName<nArg && azArg[iName][0]=='-'; iName++){
      const char *z = azArg[iName];
      if( optionMatch(z,"new") ){
        newFlag = 1;
................................................................................
                      ")";
        new_argv[1] = 0;
        new_colv[0] = "sql";
        new_colv[1] = 0;
        callback(&data, 1, new_argv, new_colv);
        rc = SQLITE_OK;
      }else{
        char *zSql;
        zSql = sqlite3_mprintf(
          "SELECT sql FROM "
          "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
          "     FROM sqlite_master UNION ALL"
          "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
          "WHERE lower(tbl_name) LIKE %Q"
          "  AND type!='meta' AND sql NOTNULL "
          "ORDER BY rowid", azArg[1]);
        rc = sqlite3_exec(p->db, zSql, callback, &data, &zErrMsg);
        sqlite3_free(zSql);
      }
    }else if( nArg==1 ){
      rc = sqlite3_exec(p->db,
         "SELECT sql FROM "
         "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
         "     FROM sqlite_master UNION ALL"
         "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
................................................................................
    }else{
      rc = 0;
    }
  }else

#if defined(SQLITE_DEBUG) && defined(SQLITE_ENABLE_SELECTTRACE)
  if( c=='s' && n==11 && strncmp(azArg[0], "selecttrace", n)==0 ){
    sqlite3SelectTrace = (int)integerValue(azArg[1]);
  }else
#endif

#if defined(SQLITE_ENABLE_SESSION)
  if( c=='s' && strncmp(azArg[0],"session",n)==0 && n>=3 ){
    OpenSession *pSession = &p->aSession[0];
    char **azCmd = &azArg[1];
................................................................................
        v = integerValue(azArg[i]);
        sqlite3_snprintf(sizeof(zBuf),zBuf,"%s: %lld 0x%llx\n", azArg[i],v,v);
        utf8_printf(p->out, "%s", zBuf);
      }
    }
  }else
#endif

  if( c=='s' && n>=4 && strncmp(azArg[0],"selftest",n)==0 ){
    int bIsInit = 0;         /* True to initialize the SELFTEST table */
    int bVerbose = 0;        /* Verbose output */
    int bSelftestExists;     /* True if SELFTEST already exists */
    char **azTest = 0;       /* Content of the SELFTEST table */
    int nRow = 0;            /* Number of rows in the SELFTEST table */
    int nCol = 4;            /* Number of columns in the SELFTEST table */
    int i;                   /* Loop counter */
    int nTest = 0;           /* Number of tests runs */
    int nErr = 0;            /* Number of errors seen */
    ShellText str;           /* Answer for a query */
    static char *azDefaultTest[] = {
       0, 0, 0, 0,
       "0", "memo", "Missing SELFTEST table - default checks only", "",
       "1", "run", "PRAGMA integrity_check", "ok"
    };
    static const int nDefaultRow = 2;

    open_db(p,0);
    for(i=1; i<nArg; i++){
      const char *z = azArg[i];
      if( z[0]=='-' && z[1]=='-' ) z++;
      if( strcmp(z,"-init")==0 ){
        bIsInit = 1;
      }else
      if( strcmp(z,"-v")==0 ){
        bVerbose++;
      }else
      {
        utf8_printf(stderr, "Unknown option \"%s\" on \"%s\"\n",
                    azArg[i], azArg[0]);
        raw_printf(stderr, "Should be one of: --init -v\n");
        rc = 1;
        goto meta_command_exit;
      }
    }
    if( sqlite3_table_column_metadata(p->db,"main","selftest",0,0,0,0,0,0)
           != SQLITE_OK ){
      bSelftestExists = 0;
    }else{
      bSelftestExists = 1;
    }
    if( bIsInit ){
      createSelftestTable(p);
      bSelftestExists = 1;
    }
    if( bSelftestExists ){
      rc = sqlite3_get_table(p->db, 
          "SELECT tno,op,cmd,ans FROM selftest ORDER BY tno",
          &azTest, &nRow, &nCol, 0);
      if( rc ){
        raw_printf(stderr, "Error querying the selftest table\n");
        rc = 1;
        sqlite3_free_table(azTest);
        goto meta_command_exit;
      }else if( nRow==0 ){
        sqlite3_free_table(azTest);
        azTest = azDefaultTest;
        nRow = nDefaultRow;
      }
    }else{
      azTest = azDefaultTest;
      nRow = nDefaultRow;
    }
    initText(&str);
    appendText(&str, "x", 0);
    for(i=1; i<=nRow; i++){
      int tno = atoi(azTest[i*nCol]);
      const char *zOp = azTest[i*nCol+1];
      const char *zSql = azTest[i*nCol+2];
      const char *zAns = azTest[i*nCol+3];
  
      if( bVerbose>0 ){
        char *zQuote = sqlite3_mprintf("%q", zSql);
        printf("%d: %s %s\n", tno, zOp, zSql);
        sqlite3_free(zQuote);
      }
      if( strcmp(zOp,"memo")==0 ){
        utf8_printf(p->out, "%s\n", zSql);
      }else
      if( strcmp(zOp,"run")==0 ){
        char *zErrMsg = 0;
        str.n = 0;
        str.z[0] = 0;
        rc = sqlite3_exec(p->db, zSql, captureOutputCallback, &str, &zErrMsg);
        nTest++;
        if( bVerbose ){
          utf8_printf(p->out, "Result: %s\n", str.z);
        }
        if( rc || zErrMsg ){
          nErr++;
          rc = 1;
          utf8_printf(p->out, "%d: error-code-%d: %s\n", tno, rc, zErrMsg);
          sqlite3_free(zErrMsg);
        }else if( strcmp(zAns,str.z)!=0 ){
          nErr++;
          rc = 1;
          utf8_printf(p->out, "%d: Expected: [%s]\n", tno, zAns);
          utf8_printf(p->out, "%d:      Got: [%s]\n", tno, str.z);
        }
      }else
      {
        utf8_printf(stderr,
          "Unknown operation \"%s\" on selftest line %d\n", zOp, tno);
        rc = 1;
        break;
      }
    }
    freeText(&str);
    if( azTest!=azDefaultTest ) sqlite3_free_table(azTest);
    utf8_printf(p->out, "%d errors out of %d tests\n", nErr, nTest);
  }else

  if( c=='s' && strncmp(azArg[0], "separator", n)==0 ){
    if( nArg<2 || nArg>3 ){
      raw_printf(stderr, "Usage: .separator COL ?ROW?\n");
      rc = 1;
    }
    if( nArg>=2 ){
................................................................................
                       "%.*s", (int)ArraySize(p->colSeparator)-1, azArg[1]);
    }
    if( nArg>=3 ){
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator,
                       "%.*s", (int)ArraySize(p->rowSeparator)-1, azArg[2]);
    }
  }else

  if( c=='s' && n>=4 && strncmp(azArg[0],"sha3sum",n)==0 ){
    const char *zLike = 0;   /* Which table to checksum. 0 means everything */
    int i;                   /* Loop counter */
    int bSchema = 0;         /* Also hash the schema */
    int bSeparate = 0;       /* Hash each table separately */
    int iSize = 224;         /* Hash algorithm to use */
    int bDebug = 0;          /* Only show the query that would have run */
    sqlite3_stmt *pStmt;     /* For querying tables names */
    char *zSql;              /* SQL to be run */
    char *zSep;              /* Separator */
    ShellText sSql;          /* Complete SQL for the query to run the hash */
    ShellText sQuery;        /* Set of queries used to read all content */
    open_db(p, 0);
    for(i=1; i<nArg; i++){
      const char *z = azArg[i];
      if( z[0]=='-' ){
        z++;
        if( z[0]=='-' ) z++;
        if( strcmp(z,"schema")==0 ){
          bSchema = 1;
        }else
        if( strcmp(z,"sha3-224")==0 || strcmp(z,"sha3-256")==0 
         || strcmp(z,"sha3-384")==0 || strcmp(z,"sha3-512")==0 
        ){
          iSize = atoi(&z[5]);
        }else
        if( strcmp(z,"debug")==0 ){
          bDebug = 1;
        }else
        {
          utf8_printf(stderr, "Unknown option \"%s\" on \"%s\"\n",
                      azArg[i], azArg[0]);
          raw_printf(stderr, "Should be one of: --schema"
                             " --sha3-224 --sha3-255 --sha3-384 --sha3-512\n");
          rc = 1;
          goto meta_command_exit;
        }
      }else if( zLike ){
        raw_printf(stderr, "Usage: .sha3sum ?OPTIONS? ?LIKE-PATTERN?\n");
        rc = 1;
        goto meta_command_exit;
      }else{
        zLike = z;
        bSeparate = 1;
        if( sqlite3_strlike("sqlite_%", zLike, 0)==0 ) bSchema = 1;
      }
    }
    if( bSchema ){
      zSql = "SELECT lower(name) FROM sqlite_master"
             " WHERE type='table' AND coalesce(rootpage,0)>1"
             " UNION ALL SELECT 'sqlite_master'"
             " ORDER BY 1 collate nocase";
    }else{
      zSql = "SELECT lower(name) FROM sqlite_master"
             " WHERE type='table' AND coalesce(rootpage,0)>1"
             " AND name NOT LIKE 'sqlite_%'"
             " ORDER BY 1 collate nocase";
    }
    sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
    initText(&sQuery);
    initText(&sSql);
    appendText(&sSql, "WITH [sha3sum$query](a,b) AS(",0);
    zSep = "VALUES(";
    while( SQLITE_ROW==sqlite3_step(pStmt) ){
      const char *zTab = (const char*)sqlite3_column_text(pStmt,0);
      if( zLike && sqlite3_strlike(zLike, zTab, 0)!=0 ) continue;
      if( strncmp(zTab, "sqlite_",7)!=0 ){
        appendText(&sQuery,"SELECT * FROM ", 0);
        appendText(&sQuery,zTab,'"');
        appendText(&sQuery," NOT INDEXED;", 0);
      }else if( strcmp(zTab, "sqlite_master")==0 ){
        appendText(&sQuery,"SELECT type,name,tbl_name,sql FROM sqlite_master"
                           " ORDER BY name;", 0);
      }else if( strcmp(zTab, "sqlite_sequence")==0 ){
        appendText(&sQuery,"SELECT name,seq FROM sqlite_sequence"
                           " ORDER BY name;", 0);
      }else if( strcmp(zTab, "sqlite_stat1")==0 ){
        appendText(&sQuery,"SELECT tbl,idx,stat FROM sqlite_stat1"
                           " ORDER BY tbl,idx;", 0);
      }else if( strcmp(zTab, "sqlite_stat3")==0
             || strcmp(zTab, "sqlite_stat4")==0 ){
        appendText(&sQuery, "SELECT * FROM ", 0);
        appendText(&sQuery, zTab, 0);
        appendText(&sQuery, " ORDER BY tbl, idx, rowid;\n", 0);
      }
      appendText(&sSql, zSep, 0);
      appendText(&sSql, sQuery.z, '\'');
      sQuery.n = 0;
      appendText(&sSql, ",", 0);
      appendText(&sSql, zTab, '\'');
      zSep = "),(";
    }
    sqlite3_finalize(pStmt);
    if( bSeparate ){
      zSql = sqlite3_mprintf(
          "%s))"
          " SELECT lower(hex(sha3_query(a,%d))) AS hash, b AS label"
          "   FROM [sha3sum$query]",
          sSql.z, iSize);
    }else{
      zSql = sqlite3_mprintf(
          "%s))"
          " SELECT lower(hex(sha3_query(group_concat(a,''),%d))) AS hash"
          "   FROM [sha3sum$query]",
          sSql.z, iSize);
    }
    freeText(&sQuery);
    freeText(&sSql);
    if( bDebug ){
      utf8_printf(p->out, "%s\n", zSql);
    }else{
      shell_exec(p->db, zSql, shell_callback, p, 0);
    }
    sqlite3_free(zSql);
  }else

  if( c=='s'
   && (strncmp(azArg[0], "shell", n)==0 || strncmp(azArg[0],"system",n)==0)
  ){
    char *zCmd;
    int i, x;
    if( nArg<2 ){
................................................................................
    static const char *azBool[] = { "off", "on", "full", "unk" };
    int i;
    if( nArg!=1 ){
      raw_printf(stderr, "Usage: .show\n");
      rc = 1;
      goto meta_command_exit;
    }
    utf8_printf(p->out, "%12.12s: %s\n","echo",
                                  azBool[ShellHasFlag(p, SHFLG_Echo)]);
    utf8_printf(p->out, "%12.12s: %s\n","eqp", azBool[p->autoEQP&3]);
    utf8_printf(p->out, "%12.12s: %s\n","explain",
         p->mode==MODE_Explain ? "on" : p->autoExplain ? "auto" : "off");
    utf8_printf(p->out,"%12.12s: %s\n","headers", azBool[p->showHeader!=0]);
    utf8_printf(p->out, "%12.12s: %s\n","mode", modeDescr[p->mode]);
    utf8_printf(p->out, "%12.12s: ", "nullvalue");
      output_c_string(p->out, p->nullValue);
................................................................................
          utf8_printf(stderr,
                      "Error: CLI support for testctrl %s not implemented\n",
                      azArg[1]);
          break;
      }
    }
  }else
#endif /* !defined(SQLITE_UNTESTABLE) */

  if( c=='t' && n>4 && strncmp(azArg[0], "timeout", n)==0 ){
    open_db(p, 0);
    sqlite3_busy_timeout(p->db, nArg>=2 ? (int)integerValue(azArg[1]) : 0);
  }else

  if( c=='t' && n>=5 && strncmp(azArg[0], "timer", n)==0 ){
................................................................................
    if( p->traceOut==0 ){
      sqlite3_trace_v2(p->db, 0, 0, 0);
    }else{
      sqlite3_trace_v2(p->db, SQLITE_TRACE_STMT, sql_trace_callback,p->traceOut);
    }
#endif
  }else


#if SQLITE_USER_AUTHENTICATION
  if( c=='u' && strncmp(azArg[0], "user", n)==0 ){
    if( nArg<2 ){
      raw_printf(stderr, "Usage: .user SUBCOMMAND ...\n");
      rc = 1;
      goto meta_command_exit;
................................................................................
  if( c=='v' && strncmp(azArg[0], "version", n)==0 ){
    utf8_printf(p->out, "SQLite %s %s\n" /*extra-version-info*/,
        sqlite3_libversion(), sqlite3_sourceid());
  }else

  if( c=='v' && strncmp(azArg[0], "vfsinfo", n)==0 ){
    const char *zDbName = nArg==2 ? azArg[1] : "main";
    sqlite3_vfs *pVfs = 0;
    if( p->db ){
      sqlite3_file_control(p->db, zDbName, SQLITE_FCNTL_VFS_POINTER, &pVfs);
      if( pVfs ){
        utf8_printf(p->out, "vfs.zName      = \"%s\"\n", pVfs->zName);
        raw_printf(p->out, "vfs.iVersion   = %d\n", pVfs->iVersion);
        raw_printf(p->out, "vfs.szOsFile   = %d\n", pVfs->szOsFile);
        raw_printf(p->out, "vfs.mxPathname = %d\n", pVfs->mxPathname);
................................................................................
** Run a single line of SQL
*/
static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
  int rc;
  char *zErrMsg = 0;

  open_db(p, 0);
  if( ShellHasFlag(p,SHFLG_Backslash) ) resolve_backslashes(zSql);
  BEGIN_TIMER;
  rc = shell_exec(p->db, zSql, shell_callback, p, &zErrMsg);
  END_TIMER;
  if( rc || zErrMsg ){
    char zPrefix[100];
    if( in!=0 || !stdin_is_interactive ){
      sqlite3_snprintf(sizeof(zPrefix), zPrefix,
................................................................................
      utf8_printf(stderr, "%s %s\n", zPrefix, zErrMsg);
      sqlite3_free(zErrMsg);
      zErrMsg = 0;
    }else{
      utf8_printf(stderr, "%s %s\n", zPrefix, sqlite3_errmsg(p->db));
    }
    return 1;
  }else if( ShellHasFlag(p, SHFLG_CountChanges) ){
    raw_printf(p->out, "changes: %3d   total_changes: %d\n",
            sqlite3_changes(p->db), sqlite3_total_changes(p->db));
  }
  return 0;
}


................................................................................
    }
    if( seenInterrupt ){
      if( in!=0 ) break;
      seenInterrupt = 0;
    }
    lineno++;
    if( nSql==0 && _all_whitespace(zLine) ){
      if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zLine);
      continue;
    }
    if( zLine && zLine[0]=='.' && nSql==0 ){
      if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zLine);
      rc = do_meta_command(zLine, p);
      if( rc==2 ){ /* exit requested */
        break;
      }else if( rc ){
        errCnt++;
      }
      continue;
................................................................................
      errCnt += runOneSqlLine(p, zSql, in, startline);
      nSql = 0;
      if( p->outCount ){
        output_reset(p);
        p->outCount = 0;
      }
    }else if( nSql && _all_whitespace(zSql) ){
      if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zSql);
      nSql = 0;
    }
  }
  if( nSql && !_all_whitespace(zSql) ){
    runOneSqlLine(p, zSql, in, startline);
  }
  free(zSql);
................................................................................
      sqlite3_snprintf(sizeof(data.nullValue), data.nullValue,
                       "%s",cmdline_option_value(argc,argv,++i));
    }else if( strcmp(z,"-header")==0 ){
      data.showHeader = 1;
    }else if( strcmp(z,"-noheader")==0 ){
      data.showHeader = 0;
    }else if( strcmp(z,"-echo")==0 ){
      ShellSetFlag(&data, SHFLG_Echo);
    }else if( strcmp(z,"-eqp")==0 ){
      data.autoEQP = 1;
    }else if( strcmp(z,"-eqpfull")==0 ){
      data.autoEQP = 2;
    }else if( strcmp(z,"-stats")==0 ){
      data.statsOn = 1;
    }else if( strcmp(z,"-scanstats")==0 ){
................................................................................
      data.scanstatsOn = 1;
    }else if( strcmp(z,"-backslash")==0 ){
      /* Undocumented command-line option: -backslash
      ** Causes C-style backslash escapes to be evaluated in SQL statements
      ** prior to sending the SQL into SQLite.  Useful for injecting
      ** crazy bytes in the middle of SQL statements for testing and debugging.
      */
      ShellSetFlag(&data, SHFLG_Backslash);
    }else if( strcmp(z,"-bail")==0 ){
      bail_on_error = 1;
    }else if( strcmp(z,"-version")==0 ){
      printf("%s %s\n", sqlite3_libversion(), sqlite3_sourceid());
      return 0;
    }else if( strcmp(z,"-interactive")==0 ){
      stdin_is_interactive = 1;

Changes to src/shun.c.

24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
..
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
...
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
...
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
...
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
/*
** Return true if the given artifact ID should be shunned.
*/
int uuid_is_shunned(const char *zUuid){
  static Stmt q;
  int rc;
  if( zUuid==0 || zUuid[0]==0 ) return 0;

  db_static_prepare(&q, "SELECT 1 FROM shun WHERE uuid=:uuid");
  db_bind_text(&q, ":uuid", zUuid);
  rc = db_step(&q);
  db_reset(&q);
  return rc==SQLITE_ROW;
}

/*
** WEBPAGE: shun
**
** View the SHA1 hashes of all shunned artifacts.  Add new hashes
** to the shun set.  Requires Admin privilege.
*/
void shun_page(void){
  Stmt q;
  int cnt = 0;
  const char *zUuid = P("uuid");
  const char *zShun = P("shun");
................................................................................
      }
      i++;
    }
    zCanonical[j+1] = zCanonical[j] = 0;
    p = zCanonical;
    while( *p ){
      int nUuid = strlen(p);
      if( nUuid!=UUID_SIZE || !validate16(p, nUuid) ){
        @ <p class="generalError">Error: Bad artifact IDs.</p>
        fossil_free(zCanonical);
        zCanonical = 0;
        break;
      }else{
        canonical16(p, UUID_SIZE);
        p += UUID_SIZE+1;
      }
    }
    zUuid = zCanonical;
  }
  style_header("Shunned Artifacts");
  if( zUuid && P("sub") ){
    const char *p = zUuid;
................................................................................
    login_verify_csrf_secret();
    while( *p ){
      db_multi_exec("DELETE FROM shun WHERE uuid=%Q", p);
      if( !db_exists("SELECT 1 FROM blob WHERE uuid=%Q", p) ){
        allExist = 0;
      }
      admin_log("Unshunned %Q", p);
      p += UUID_SIZE+1;
    }
    if( allExist ){
      @ <p class="noMoreShun">Artifact(s)<br />
      for( p = zUuid ; *p ; p += UUID_SIZE+1 ){
        @ <a href="%R/artifact/%s(p)">%s(p)</a><br />
      }
      @ are no longer being shunned.</p>
    }else{
      @ <p class="noMoreShun">Artifact(s)<br />
      for( p = zUuid ; *p ; p += UUID_SIZE+1 ){
        @ %s(p)<br />
      }
      @ will no longer be shunned.  But they may not exist in the repository.
      @ It may be necessary to rebuild the repository using the
      @ <b>fossil rebuild</b> command-line before the artifact content
      @ can pulled in from other repositories.</p>
    }
................................................................................
      tagid = db_int(0, "SELECT tagid FROM tag WHERE tagname='tkt-%q'", p);
      if( tagid ){
        db_multi_exec("DELETE FROM ticket WHERE tkt_uuid=%Q", p);
        db_multi_exec("DELETE FROM tag WHERE tagid=%d", tagid);
        db_multi_exec("DELETE FROM tagxref WHERE tagid=%d", tagid);
      }
      admin_log("Shunned %Q", p);
      p += UUID_SIZE+1;
    }
    @ <p class="shunned">Artifact(s)<br />
    for( p = zUuid ; *p ; p += UUID_SIZE+1 ){
      @ <a href="%R/artifact/%s(p)">%s(p)</a><br />
    }
    @ have been shunned.  They will no longer be pushed.
    @ They will be removed from the repository the next time the repository
    @ is rebuilt using the <b>fossil rebuild</b> command-line</p>
  }
  if( zRcvid ){
................................................................................
  }
  @ <p>A shunned artifact will not be pushed nor accepted in a pull and the
  @ artifact content will be purged from the repository the next time the
  @ repository is rebuilt.  A list of shunned artifacts can be seen at the
  @ bottom of this page.</p>
  @
  @ <a name="addshun"></a>
  @ <p>To shun artifacts, enter their artifact IDs (the 40-character SHA1
  @ hash of the artifacts) in the
  @ following box and press the "Shun" button.  This will cause the artifacts
  @ to be removed from the repository and will prevent the artifacts from being
  @ readded to the repository by subsequent sync operation.</p>
  @
  @ <p>Note that you must enter the full 40-character artifact IDs, not
  @ an abbreviation or a symbolic tag.</p>
  @
  @ <p>Warning:  Shunning should only be used to remove inappropriate content
  @ from the repository.  Inappropriate content includes such things as
  @ spam added to Wiki, files that violate copyright or patent agreements,
  @ or artifacts that by design or accident interfere with the processing
  @ of the repository.  Do not shun artifacts merely to remove them from
  @ sight - set the "hidden" tag on such artifacts instead.</p>







>










|







 







|





|
|







 







|



|





|







 







|


|







 







|
|




|
|







24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
..
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
...
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
...
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
...
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
/*
** Return true if the given artifact ID should be shunned.
*/
int uuid_is_shunned(const char *zUuid){
  static Stmt q;
  int rc;
  if( zUuid==0 || zUuid[0]==0 ) return 0;
  if( g.eHashPolicy==HPOLICY_SHUN_SHA1 && zUuid[HNAME_LEN_SHA1]==0 ) return 1;
  db_static_prepare(&q, "SELECT 1 FROM shun WHERE uuid=:uuid");
  db_bind_text(&q, ":uuid", zUuid);
  rc = db_step(&q);
  db_reset(&q);
  return rc==SQLITE_ROW;
}

/*
** WEBPAGE: shun
**
** View the hashes of all shunned artifacts.  Add new hashes
** to the shun set.  Requires Admin privilege.
*/
void shun_page(void){
  Stmt q;
  int cnt = 0;
  const char *zUuid = P("uuid");
  const char *zShun = P("shun");
................................................................................
      }
      i++;
    }
    zCanonical[j+1] = zCanonical[j] = 0;
    p = zCanonical;
    while( *p ){
      int nUuid = strlen(p);
      if( !hname_validate(p, nUuid) ){
        @ <p class="generalError">Error: Bad artifact IDs.</p>
        fossil_free(zCanonical);
        zCanonical = 0;
        break;
      }else{
        canonical16(p, nUuid);
        p += nUuid+1;
      }
    }
    zUuid = zCanonical;
  }
  style_header("Shunned Artifacts");
  if( zUuid && P("sub") ){
    const char *p = zUuid;
................................................................................
    login_verify_csrf_secret();
    while( *p ){
      db_multi_exec("DELETE FROM shun WHERE uuid=%Q", p);
      if( !db_exists("SELECT 1 FROM blob WHERE uuid=%Q", p) ){
        allExist = 0;
      }
      admin_log("Unshunned %Q", p);
      p += strlen(p)+1;
    }
    if( allExist ){
      @ <p class="noMoreShun">Artifact(s)<br />
      for( p = zUuid ; *p ; p += strlen(p)+1 ){
        @ <a href="%R/artifact/%s(p)">%s(p)</a><br />
      }
      @ are no longer being shunned.</p>
    }else{
      @ <p class="noMoreShun">Artifact(s)<br />
      for( p = zUuid ; *p ; p += strlen(p)+1 ){
        @ %s(p)<br />
      }
      @ will no longer be shunned.  But they may not exist in the repository.
      @ It may be necessary to rebuild the repository using the
      @ <b>fossil rebuild</b> command-line before the artifact content
      @ can pulled in from other repositories.</p>
    }
................................................................................
      tagid = db_int(0, "SELECT tagid FROM tag WHERE tagname='tkt-%q'", p);
      if( tagid ){
        db_multi_exec("DELETE FROM ticket WHERE tkt_uuid=%Q", p);
        db_multi_exec("DELETE FROM tag WHERE tagid=%d", tagid);
        db_multi_exec("DELETE FROM tagxref WHERE tagid=%d", tagid);
      }
      admin_log("Shunned %Q", p);
      p += strlen(p)+1;
    }
    @ <p class="shunned">Artifact(s)<br />
    for( p = zUuid ; *p ; p += strlen(p)+1 ){
      @ <a href="%R/artifact/%s(p)">%s(p)</a><br />
    }
    @ have been shunned.  They will no longer be pushed.
    @ They will be removed from the repository the next time the repository
    @ is rebuilt using the <b>fossil rebuild</b> command-line</p>
  }
  if( zRcvid ){
................................................................................
  }
  @ <p>A shunned artifact will not be pushed nor accepted in a pull and the
  @ artifact content will be purged from the repository the next time the
  @ repository is rebuilt.  A list of shunned artifacts can be seen at the
  @ bottom of this page.</p>
  @
  @ <a name="addshun"></a>
  @ <p>To shun artifacts, enter their artifact hashes (the 40- or
  @ 64-character lowercase hexadecimal hash of the artifact content) in the
  @ following box and press the "Shun" button.  This will cause the artifacts
  @ to be removed from the repository and will prevent the artifacts from being
  @ readded to the repository by subsequent sync operation.</p>
  @
  @ <p>Note that you must enter the full 40- or 64-character artifact hashes,
  @ not an abbreviation or a symbolic tag.</p>
  @
  @ <p>Warning:  Shunning should only be used to remove inappropriate content
  @ from the repository.  Inappropriate content includes such things as
  @ spam added to Wiki, files that violate copyright or patent agreements,
  @ or artifacts that by design or accident interfere with the processing
  @ of the repository.  Do not shun artifacts merely to remove them from
  @ sight - set the "hidden" tag on such artifacts instead.</p>

Changes to src/sitemap.c.

109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
  if( srchFlags ){
    @ <li>%z(href("%R/search"))Full-Text Search</a></li>
  }
  @ <li>%z(href("%R/login"))Login/Logout/Change Password</a></li>
  if( g.perm.Read ){
    @ <li>%z(href("%R/stat"))Repository Status</a>
    @   <ul>
    @   <li>%z(href("%R/hash-collisions"))Collisions on SHA1 hash
    @       prefixes</a></li>
    if( g.perm.Admin ){
      @   <li>%z(href("%R/urllist"))List of URLs used to access
      @       this repository</a></li>
    }
    @   <li>%z(href("%R/bloblist"))List of Artifacts</a></li>
    @   <li>%z(href("%R/timewarps"))List of "Timewarp" Check-ins</a></li>
    @   </ul>







|
<







109
110
111
112
113
114
115
116

117
118
119
120
121
122
123
  if( srchFlags ){
    @ <li>%z(href("%R/search"))Full-Text Search</a></li>
  }
  @ <li>%z(href("%R/login"))Login/Logout/Change Password</a></li>
  if( g.perm.Read ){
    @ <li>%z(href("%R/stat"))Repository Status</a>
    @   <ul>
    @   <li>%z(href("%R/hash-collisions"))Collisions on hash prefixes</a></li>

    if( g.perm.Admin ){
      @   <li>%z(href("%R/urllist"))List of URLs used to access
      @       this repository</a></li>
    }
    @   <li>%z(href("%R/bloblist"))List of Artifacts</a></li>
    @   <li>%z(href("%R/timewarps"))List of "Timewarp" Check-ins</a></li>
    @   </ul>

Changes to src/sqlcmd.c.

183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
...
210
211
212
213
214
215
216



217
218
219
220
221
222
223
**
** WARNING:  Careless use of this command can corrupt a Fossil repository
** in ways that are unrecoverable.  Be sure you know what you are doing before
** running any SQL commands that modify the repository database.
**
** The following extensions to the usual SQLite commands are provided:
**
**    content(X)                Return the content of artifact X.  X can be a
**                              SHA1 hash or prefix or a tag.
**
**    compress(X)               Compress text X.
**
**    decompress(X)             Decompress text X.  Undoes the work of
**                              compress(X).
**
**    checkin_mtime(X,Y)        Return the mtime for the file Y (a BLOB.RID)
................................................................................
**                              all files contained in check-in X.  Example:
**                                SELECT * FROM files_of_checkin('trunk');
*/
void cmd_sqlite3(void){
  int noRepository;
  const char *zConfigDb;
  extern int sqlite3_shell(int, char**);



  noRepository = find_option("no-repository", 0, 0)!=0;
  if( !noRepository ){
    db_find_and_open_repository(OPEN_ANY_SCHEMA, 0);
  }
  db_open_config(1,0);
  zConfigDb = g.zConfigDbName;
  fossil_close(1, noRepository);







|
|







 







>
>
>







183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
...
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
**
** WARNING:  Careless use of this command can corrupt a Fossil repository
** in ways that are unrecoverable.  Be sure you know what you are doing before
** running any SQL commands that modify the repository database.
**
** The following extensions to the usual SQLite commands are provided:
**
**    content(X)                Return the content of artifact X.  X can be an
**                              artifact hash or prefix or a tag.
**
**    compress(X)               Compress text X.
**
**    decompress(X)             Decompress text X.  Undoes the work of
**                              compress(X).
**
**    checkin_mtime(X,Y)        Return the mtime for the file Y (a BLOB.RID)
................................................................................
**                              all files contained in check-in X.  Example:
**                                SELECT * FROM files_of_checkin('trunk');
*/
void cmd_sqlite3(void){
  int noRepository;
  const char *zConfigDb;
  extern int sqlite3_shell(int, char**);
#ifdef FOSSIL_ENABLE_TH1_HOOKS
  g.fNoThHook = 1;
#endif
  noRepository = find_option("no-repository", 0, 0)!=0;
  if( !noRepository ){
    db_find_and_open_repository(OPEN_ANY_SCHEMA, 0);
  }
  db_open_config(1,0);
  zConfigDb = g.zConfigDbName;
  fossil_close(1, noRepository);

Changes to src/sqlite3.c.

1
2
3
4
5
6
7
8
9
10
...
200
201
202
203
204
205
206
207
208












209
210
211
212
213

214
215
216
217
218
219
220
221
222
223
224
225
226
...
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
....
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316




2317
2318









2319
2320
2321
2322

2323
2324
2325
2326
2327
2328
2329
2330
2331
....
2344
2345
2346
2347
2348
2349
2350










2351
2352
2353
2354
2355
2356
2357
....
8914
8915
8916
8917
8918
8919
8920
8921
8922
8923
8924
8925
8926
8927
8928
....
8932
8933
8934
8935
8936
8937
8938
8939
8940
8941
8942
8943
8944
8945
8946
....
8952
8953
8954
8955
8956
8957
8958
8959
8960
8961
8962
8963
8964
8965
8966
....
8981
8982
8983
8984
8985
8986
8987
8988
8989
8990
8991
8992
8993
8994
8995
....
9011
9012
9013
9014
9015
9016
9017
9018
9019
9020
9021
9022
9023
9024
9025
9026
9027
9028
9029
9030
9031
9032
9033
9034
9035
9036
9037
9038
9039
....
9138
9139
9140
9141
9142
9143
9144
9145
9146
9147
9148
9149
9150
9151
9152
....
9200
9201
9202
9203
9204
9205
9206
9207
9208
9209
9210
9211
9212
9213
9214
....
9236
9237
9238
9239
9240
9241
9242
9243
9244
9245
9246
9247
9248
9249
9250
....
9257
9258
9259
9260
9261
9262
9263
9264
9265
9266
9267
9268
9269
9270
9271
....
9292
9293
9294
9295
9296
9297
9298
9299
9300
9301
9302
9303
9304
9305
9306
....
9321
9322
9323
9324
9325
9326
9327
9328
9329
9330
9331
9332
9333
9334
9335
....
9349
9350
9351
9352
9353
9354
9355
9356
9357
9358
9359
9360
9361
9362
9363
....
9382
9383
9384
9385
9386
9387
9388
9389
9390
9391
9392
9393
9394
9395
9396
....
9412
9413
9414
9415
9416
9417
9418
9419
9420
9421
9422
9423
9424
9425
9426
....
9445
9446
9447
9448
9449
9450
9451
9452
9453
9454
9455
9456
9457
9458
9459
....
9472
9473
9474
9475
9476
9477
9478
9479
9480
9481
9482
9483
9484
9485
9486
....
9488
9489
9490
9491
9492
9493
9494
9495
9496
9497
9498
9499
9500
9501
9502
....
9521
9522
9523
9524
9525
9526
9527
9528
9529
9530
9531
9532
9533
9534
9535
....
9551
9552
9553
9554
9555
9556
9557
9558
9559
9560
9561
9562
9563
9564
9565
....
9580
9581
9582
9583
9584
9585
9586
9587
9588
9589
9590
9591
9592
9593
9594
....
9886
9887
9888
9889
9890
9891
9892
9893
9894
9895
9896
9897
9898
9899
9900
.....
10087
10088
10089
10090
10091
10092
10093
10094
10095
10096
10097
10098
10099
10100
10101
.....
10102
10103
10104
10105
10106
10107
10108
10109
10110
10111
10112
10113
10114
10115
10116
10117
10118
10119
10120
10121
10122
10123
10124
10125
10126
10127
10128
10129
10130
10131
10132
10133
10134
10135
10136
10137
10138
10139
10140
.....
11033
11034
11035
11036
11037
11038
11039

11040
11041
11042
11043
11044
11045
11046
.....
11847
11848
11849
11850
11851
11852
11853
11854
11855
11856

11857
11858
11859
11860
11861

11862

11863
11864

11865
11866
11867
11868
11869
11870
11871
11872
11873




11874
11875
11876
11877
11878
11879
11880
11881
11882
11883
11884
11885
11886
.....
12442
12443
12444
12445
12446
12447
12448

12449
12450
12451
12452
12453
12454
12455
.....
12742
12743
12744
12745
12746
12747
12748

12749
12750
12751
12752
12753
12754
12755
12756
12757
12758
12759
12760
12761
12762
12763
12764
12765
12766
12767
12768
12769
12770
12771
12772
12773
12774
12775
12776
12777
12778
12779
12780
12781
12782
12783
12784
12785
12786
12787
12788
12789
12790
12791

12792
12793
12794
12795
12796
12797
12798
12799
12800
12801
12802
12803
12804
12805
12806
12807
12808
12809
12810
12811
12812
12813
12814
12815
12816
12817
12818
12819
12820
12821
12822
12823
12824
12825
12826

12827
12828

12829
12830
12831
12832
12833
12834
12835
12836
12837
12838
12839
12840
12841
12842
12843
12844
12845
12846
12847
12848
12849
12850
12851
12852
12853
12854
12855
12856
12857
12858
12859
12860
12861
12862
.....
12868
12869
12870
12871
12872
12873
12874
12875
12876
12877
12878
12879
12880
12881
12882
12883
12884
12885
12886
12887
12888
12889
12890
12891
12892
12893
12894
12895
12896
12897
12898
12899
12900
12901
12902
12903
.....
14059
14060
14061
14062
14063
14064
14065

14066
14067
14068
14069
14070
14071
14072
.....
14574
14575
14576
14577
14578
14579
14580

14581
14582
14583
14584
14585
14586
14587
14588
14589
14590
14591
14592
14593
14594
14595
.....
14605
14606
14607
14608
14609
14610
14611
14612
14613
14614
14615
14616
14617
14618
14619
14620



14621
14622
14623
14624
14625
14626
14627
14628
14629
14630
14631
14632
14633
14634
14635
.....
14856
14857
14858
14859
14860
14861
14862

14863
14864
14865
14866
14867
14868
14869
.....
15166
15167
15168
15169
15170
15171
15172
15173
15174
15175
15176
15177
15178
15179
15180
.....
16489
16490
16491
16492
16493
16494
16495

16496
16497
16498
16499
16500
16501
16502
.....
17435
17436
17437
17438
17439
17440
17441
17442
17443
17444
17445
17446
17447
17448
17449
.....
19490
19491
19492
19493
19494
19495
19496

19497
19498
19499
19500
19501

19502
19503
19504
19505
19506
19507
19508
19509
19510
19511
19512
19513
19514
19515
.....
20691
20692
20693
20694
20695
20696
20697


20698
20699
20700
20701
20702
20703
20704
20705
20706
20707
20708
20709
20710
20711
20712
20713
20714
.....
23822
23823
23824
23825
23826
23827
23828
23829
23830
23831
23832
23833
23834
23835
23836
23837
.....
24034
24035
24036
24037
24038
24039
24040
24041
24042
24043
24044
24045
24046
24047
24048
24049
.....
24077
24078
24079
24080
24081
24082
24083
24084
24085
24086
24087
24088
24089
24090
24091
24092
.....
24106
24107
24108
24109
24110
24111
24112
24113
24114
24115
24116
24117
24118
24119
24120
24121
.....
24357
24358
24359
24360
24361
24362
24363
24364
24365
















24366
24367
24368
24369
24370
24371
24372
24373
24374
24375
24376
24377
24378
24379
24380
24381
24382
24383
24384
24385
24386
24387
24388
.....
24548
24549
24550
24551
24552
24553
24554
24555
24556
24557
24558
24559
24560
24561
24562
.....
24609
24610
24611
24612
24613
24614
24615
24616
24617
24618
24619
24620
24621
24622
24623
.....
24978
24979
24980
24981
24982
24983
24984
24985
24986
24987
24988
24989
24990
24991
24992
.....
24996
24997
24998
24999
25000
25001
25002

25003
25004
25005
25006
25007
25008
25009
25010
25011
.....
25021
25022
25023
25024
25025
25026
25027
25028
25029
25030
25031
25032
25033
25034
25035
25036
25037
25038
25039
25040
25041
25042
25043
25044
25045
25046
25047
25048
25049
25050
25051
25052
25053
25054
25055
25056
25057
25058
25059
25060
25061
25062
25063
25064
.....
25142
25143
25144
25145
25146
25147
25148
25149
25150
25151
25152
25153
25154
25155
25156
25157
25158
25159
25160
25161
25162
25163
.....
25192
25193
25194
25195
25196
25197
25198
25199
25200
25201
25202
25203
25204
25205
25206
25207
25208
25209

25210
25211
25212
25213
25214
25215
25216
.....
25272
25273
25274
25275
25276
25277
25278
25279
25280
25281
25282
25283
25284
25285
25286
25287
25288
25289
25290
25291
25292
.....
25296
25297
25298
25299
25300
25301
25302
25303
25304
25305
25306
25307
25308
25309
25310
25311
25312
25313
25314
25315
25316
25317
25318
25319
25320
25321
25322
25323
25324
25325



25326
25327
25328
25329
25330

25331
25332

25333

25334
25335
25336
25337
25338
25339
25340
25341
25342
25343
25344
25345
25346
25347
25348
25349
25350
25351
25352
25353

25354
25355

25356

25357
25358
25359
25360
25361
25362
25363
25364
25365
25366
25367
25368
25369
25370
25371
25372
25373
25374
25375
25376
25377
.....
25389
25390
25391
25392
25393
25394
25395
25396
25397















25398
25399
25400
25401
25402
25403
25404
.....
25417
25418
25419
25420
25421
25422
25423
25424
25425
25426
25427
25428
25429
25430
25431
25432
25433
.....
28672
28673
28674
28675
28676
28677
28678
28679
28680
28681
28682
28683
28684
28685
28686
28687
28688
28689
28690
28691
28692
28693
28694
28695
28696
28697
28698
28699
28700
28701
28702
28703
28704
28705
28706
28707
28708
28709
.....
28811
28812
28813
28814
28815
28816
28817
28818
28819
28820
28821
28822
28823
28824
28825
28826
.....
28832
28833
28834
28835
28836
28837
28838
28839
28840
28841
28842
28843
28844
28845
28846
28847
.....
28848
28849
28850
28851
28852
28853
28854
28855
28856
28857
28858
28859
28860
28861
28862
28863
.....
29447
29448
29449
29450
29451
29452
29453

29454
29455
29456
29457
29458
29459
29460
29461
29462
29463
29464
29465
29466
29467
29468
29469
29470
29471
29472
29473
29474
29475
29476
29477
29478
29479
29480
29481
29482
29483
29484
29485
29486
29487
29488
29489
29490
29491
29492
29493
29494
29495
29496

29497
29498
29499
29500
29501
29502
29503
29504
29505
29506
29507
29508
29509
29510
29511
29512
29513
29514
29515
29516
29517
29518
29519
29520
29521
29522
29523
29524
29525
29526
29527
29528
29529
29530
29531

29532
29533

29534
29535
29536
29537
29538
29539
29540
29541
29542
29543
29544
29545
29546
29547
29548
29549
29550
29551
29552
29553
29554
29555
29556
29557
29558
29559
29560
29561
29562
29563
29564
29565
29566
29567
.....
37935
37936
37937
37938
37939
37940
37941
37942



























37943
37944
37945
37946
37947
37948
37949
37950
37951
37952
37953
37954
37955
37956










37957
37958
37959
37960
37961
37962

37963
37964
37965
37966
37967
37968
37969
37970
37971
37972
37973
37974
37975
37976
37977
37978
37979
37980
37981
37982
37983
37984
.....
44067
44068
44069
44070
44071
44072
44073
44074
44075
44076
44077
44078
44079
44080
44081
.....
58538
58539
58540
58541
58542
58543
58544
58545
58546
58547
58548
58549
58550
58551
58552
58553
58554
58555
58556
.....
60162
60163
60164
60165
60166
60167
60168
60169
60170
60171
60172

60173
60174
60175
60176
60177
60178
60179
60180
60181
60182
60183
60184
60185
60186
.....
60187
60188
60189
60190
60191
60192
60193
60194
60195
60196
60197
60198
60199
60200
60201
60202
60203
60204
60205

60206















































60207
60208
60209
60210
60211
60212
60213
60214
60215
.....
60235
60236
60237
60238
60239
60240
60241






60242
60243
60244
60245
60246
60247
60248
60249
60250
60251
60252
60253
60254
60255
60256
60257
60258
.....
60382
60383
60384
60385
60386
60387
60388
60389
60390
60391
60392
60393
60394
60395
60396
60397
60398
60399
.....
63621
63622
63623
63624
63625
63626
63627
63628
63629
63630
63631
63632
63633
63634
63635
.....
64163
64164
64165
64166
64167
64168
64169
























64170
64171
64172
64173
64174
64175
64176
.....
65008
65009
65010
65011
65012
65013
65014
65015
65016
65017
65018
65019
65020
65021
65022
.....
66534
66535
66536
66537
66538
66539
66540
66541
66542
66543
66544
66545
66546
66547
66548
.....
71186
71187
71188
71189
71190
71191
71192

71193
71194
71195
71196
71197
71198
71199
.....
71214
71215
71216
71217
71218
71219
71220

71221
71222
71223
71224
71225
71226
71227
.....
73738
73739
73740
73741
73742
73743
73744



73745
73746
73747
73748
73749
73750
73751
73752
73753
73754
73755
73756
73757
73758
.....
74691
74692
74693
74694
74695
74696
74697
74698
74699
74700
74701
74702
74703
74704
74705
.....
74796
74797
74798
74799
74800
74801
74802
74803
74804
74805
74806
74807
74808
74809
74810
.....
75679
75680
75681
75682
75683
75684
75685
75686
75687
75688
75689
75690
75691
75692
75693
75694
.....
75950
75951
75952
75953
75954
75955
75956

75957
75958
75959
75960
75961
75962
75963
75964
.....
77054
77055
77056
77057
77058
77059
77060
77061
77062
77063
77064
77065
77066
77067
77068
77069
77070
.....
77319
77320
77321
77322
77323
77324
77325

77326
77327
77328

77329
77330
77331
77332
77333
77334
77335
77336
.....
79643
79644
79645
79646
79647
79648
79649
79650
79651
79652
79653
79654
79655
79656
79657
79658
79659
79660
79661
79662
79663
79664
79665
79666
79667
79668
79669
79670
79671
79672
79673
79674
79675
79676
79677
79678
79679
79680
79681
79682
79683
79684
79685
.....
80172
80173
80174
80175
80176
80177
80178
80179
80180
80181
80182
80183
80184
80185
80186
.....
82821
82822
82823
82824
82825
82826
82827



























82828
82829
82830
82831
82832
82833
82834
.....
83464
83465
83466
83467
83468
83469
83470












83471
83472
83473
83474
83475
83476
83477
.....
83585
83586
83587
83588
83589
83590
83591
83592
83593
83594
83595
83596
83597
83598
83599
.....
83618
83619
83620
83621
83622
83623
83624
83625
83626
83627
83628
83629
83630
83631
83632

83633
83634
83635
83636
83637
83638
83639
.....
88772
88773
88774
88775
88776
88777
88778
88779
88780
88781
88782
88783
88784
88785
88786
88787
88788
88789
88790
.....
89138
89139
89140
89141
89142
89143
89144
89145

89146
89147
89148
89149
89150
89151
89152
.....
89323
89324
89325
89326
89327
89328
89329
89330
89331
89332
89333
89334
89335
89336
89337
89338
89339
89340
89341
.....
89517
89518
89519
89520
89521
89522
89523
89524
89525
89526
89527
89528
89529
89530
89531




89532

89533
89534
89535
89536
89537
89538
89539




89540
89541
89542
89543
89544
89545
89546
89547
89548
89549
89550

89551
89552
89553
89554
89555
89556
89557
.....
90682
90683
90684
90685
90686
90687
90688
90689
90690
90691
90692
90693
90694
90695
90696
.....
92818
92819
92820
92821
92822
92823
92824
92825
92826
92827
92828
92829
92830
92831
92832
.....
93723
93724
93725
93726
93727
93728
93729
93730
93731
93732
93733
93734
93735
93736
93737
.....
95130
95131
95132
95133
95134
95135
95136











95137
95138
95139
95140
95141
95142
95143
.....
96683
96684
96685
96686
96687
96688
96689

96690
96691
96692
96693
96694
96695
96696
.....
96947
96948
96949
96950
96951
96952
96953







96954
96955
96956
96957
96958
96959
96960
.....
97044
97045
97046
97047
97048
97049
97050







97051

97052
97053
97054
97055
97056


97057
97058
97059
97060
97061
97062
97063
.....
97763
97764
97765
97766
97767
97768
97769
97770
97771
97772
97773
97774
97775
97776
97777
97778
97779
97780
97781
97782
97783
97784
97785
97786
97787
97788
97789
97790
97791
97792
97793
97794
97795
97796
97797
97798
97799
97800
97801
97802
97803

97804
97805
97806
97807
97808
97809
97810
.....
97925
97926
97927
97928
97929
97930
97931


97932


97933
97934
97935
97936
97937
97938
97939
97940

97941
97942
97943
97944
97945
97946
97947
.....
98228
98229
98230
98231
98232
98233
98234

98235
98236
98237
98238
98239
98240
98241




98242
98243

98244
98245
98246
98247
98248
98249
98250
.....
98259
98260
98261
98262
98263
98264
98265
98266
98267
98268
98269
98270
98271
98272
98273
98274
98275
98276
98277
98278
98279
98280
98281
98282
98283
98284
98285
......
100256
100257
100258
100259
100260
100261
100262

100263
100264
100265
100266
100267
100268
100269
......
102594
102595
102596
102597
102598
102599
102600



102601
102602
102603
102604
102605
102606
102607
......
109783
109784
109785
109786
109787
109788
109789



109790
109791
109792
109793
109794
109795
109796
......
109938
109939
109940
109941
109942
109943
109944
109945



109946
109947
109948
109949
109950
109951
109952
109953
......
110229
110230
110231
110232
110233
110234
110235
110236
110237
110238
110239
110240
110241
110242
110243
......
110291
110292
110293
110294
110295
110296
110297
110298
110299
110300
110301
110302
110303
110304
110305
......
110588
110589
110590
110591
110592
110593
110594
110595
110596
110597
110598
110599
110600
110601
110602
......
110609
110610
110611
110612
110613
110614
110615

110616
110617
110618
110619
110620
110621
110622
......
112079
112080
112081
112082
112083
112084
112085

112086
112087
112088
112089
112090
112091
112092
112093
112094
112095
112096
112097
112098
112099
112100
112101
112102
112103

112104
112105
112106
112107
112108
112109
112110
......
112120
112121
112122
112123
112124
112125
112126
112127
112128
112129
112130

112131
112132
112133

112134
112135
112136
112137
112138
112139

112140
112141
112142
112143
112144
112145
112146
112147
112148
112149
112150
112151
112152
112153
112154
112155
112156
112157

112158
112159
112160
112161
112162
112163

112164
112165
112166
112167
112168
112169
112170
112171
112172
112173
112174
......
112206
112207
112208
112209
112210
112211
112212
112213
112214
112215
112216
112217
112218
112219
112220
......
112243
112244
112245
112246
112247
112248
112249
112250
112251
112252
112253
112254
112255
112256
112257
......
112278
112279
112280
112281
112282
112283
112284
112285
112286
112287
112288
112289
112290
112291
112292
......
112315
112316
112317
112318
112319
112320
112321
112322
112323
112324
112325
112326
112327
112328
112329
112330
112331
112332
112333
112334
112335
112336
......
112385
112386
112387
112388
112389
112390
112391
112392
112393
112394
112395
112396
112397
112398
112399
112400
112401
112402
112403
112404
112405
112406
112407
112408
112409
......
112442
112443
112444
112445
112446
112447
112448
112449
112450
112451
112452
112453
112454
112455
112456
......
112461
112462
112463
112464
112465
112466
112467







112468
112469
112470
112471
112472
112473
112474
......
112559
112560
112561
112562
112563
112564
112565
112566
112567
112568
112569
112570
112571
112572
112573
112574
112575
112576
112577
......
112642
112643
112644
112645
112646
112647
112648
112649
112650
112651
112652
112653
112654
112655
112656
112657
112658
112659
112660
112661
112662
112663
112664
112665
112666
112667
......
112922
112923
112924
112925
112926
112927
112928
















112929
112930
112931
112932
112933
112934
112935
......
113726
113727
113728
113729
113730
113731
113732

113733
113734
113735
113736
113737
113738
113739
113740
113741
113742
113743
113744

113745
113746
113747
113748
113749
113750

113751
113752
113753
113754
113755

113756
113757
113758
113759
113760
113761
113762
......
114005
114006
114007
114008
114009
114010
114011







114012
114013
114014


114015
114016
114017
114018
114019
114020
114021
......
114038
114039
114040
114041
114042
114043
114044
114045
114046
114047
114048
114049
114050
114051
114052
......
114053
114054
114055
114056
114057
114058
114059
114060
114061
114062
114063
114064
114065
114066
114067
114068
114069
114070
......
114096
114097
114098
114099
114100
114101
114102
114103
114104
114105
114106
114107
114108
114109
114110
114111
114112
114113
114114
114115
114116
114117
114118
114119
114120
114121

114122
114123
114124
114125
114126
114127
114128
......
114129
114130
114131
114132
114133
114134
114135
114136
114137
114138
114139
114140
114141
114142
114143
114144
114145
114146
114147
114148
114149


114150









114151










114152
114153
114154
114155
114156
114157
114158
114159
114160
114161
114162
114163
114164
114165
114166
114167
114168
114169
114170
114171
114172
114173
114174
114175
114176
114177
114178
114179
114180
......
114187
114188
114189
114190
114191
114192
114193
114194
114195
114196
114197
114198
114199
114200
114201
114202
114203
114204

114205
114206
114207
114208
114209
114210
114211
114212
114213
114214
114215
114216
114217



114218
114219
114220
114221
114222
114223
114224
114225
114226
114227
114228
114229
114230
114231
114232
114233
114234
114235
114236
114237
114238
114239
114240
114241
......
114451
114452
114453
114454
114455
114456
114457
















































































































114458
114459
114460
114461
114462
114463
114464
......
123767
123768
123769
123770
123771
123772
123773






123774
123775












123776
123777
123778
123779
123780
123781
123782
......
124362
124363
124364
124365
124366
124367
124368
124369
124370
124371
124372
124373
124374
124375
124376
124377
......
124651
124652
124653
124654
124655
124656
124657
124658
124659
124660
124661
124662
124663
124664
124665
......
124721
124722
124723
124724
124725
124726
124727
124728
124729
124730
124731
124732
124733
124734
124735
......
124775
124776
124777
124778
124779
124780
124781
124782
124783
124784
124785
124786
124787
124788
124789
124790
124791
124792
124793
124794
124795
124796
124797
124798
124799
124800
124801
124802
124803
......
125078
125079
125080
125081
125082
125083
125084
125085
125086
125087
125088
125089
125090
125091
125092
......
125173
125174
125175
125176
125177
125178
125179
125180
125181
125182
125183
125184
125185
125186
125187
125188
......
125245
125246
125247
125248
125249
125250
125251
125252
125253
125254
125255
125256
125257
125258
125259
......
125684
125685
125686
125687
125688
125689
125690

125691
125692




125693
125694
125695
125696
125697
125698
125699
......
125700
125701
125702
125703
125704
125705
125706
125707
125708
125709
125710
125711
125712
125713
125714
......
126884
126885
126886
126887
126888
126889
126890

126891
126892
126893
126894
126895
126896
126897
......
126924
126925
126926
126927
126928
126929
126930





126931
126932
126933
126934
126935
126936
126937
......
127109
127110
127111
127112
127113
127114
127115
127116
127117
127118
127119
127120
127121
127122
127123
......
127755
127756
127757
127758
127759
127760
127761
127762
127763
127764
127765
127766
127767
127768
127769
......
128080
128081
128082
128083
128084
128085
128086
128087
128088
128089
128090
128091
128092
128093
128094
128095
128096
128097
128098
128099
128100
128101
128102
......
128105
128106
128107
128108
128109
128110
128111

















128112
128113
128114
128115
128116
128117
128118
......
128733
128734
128735
128736
128737
128738
128739
128740
128741
128742
128743
128744
128745
128746
128747
......
129524
129525
129526
129527
129528
129529
129530

129531
129532
129533
129534
129535
129536
129537
129538
......
129623
129624
129625
129626
129627
129628
129629

129630
129631
129632
129633
129634
129635
129636
......
131700
131701
131702
131703
131704
131705
131706





131707
131708
131709
131710
131711
131712
131713
......
132247
132248
132249
132250
132251
132252
132253

132254







132255
132256
132257
132258
132259
132260
132261
......
133427
133428
133429
133430
133431
133432
133433
133434
133435
133436
133437
133438
133439
133440
133441
133442
133443
......
133666
133667
133668
133669
133670
133671
133672
133673
133674
133675
133676
133677
133678
133679
133680
......
133742
133743
133744
133745
133746
133747
133748
133749
133750
133751
133752
133753
133754
133755
133756
......
133820
133821
133822
133823
133824
133825
133826
133827
133828
133829
133830
133831
133832
133833
133834
133835
133836
133837
133838
133839
133840
......
133902
133903
133904
133905
133906
133907
133908
133909
133910
133911
133912
133913
133914
133915
133916
133917
133918
133919
......
134688
134689
134690
134691
134692
134693
134694
134695
134696
134697
134698
134699
134700
134701
134702
134703
134704
134705
134706
134707
134708
134709
134710
134711
134712
134713
134714
134715
134716
134717
134718
134719
134720
134721
134722
134723
134724
134725
134726
134727
134728
134729
134730
134731
134732
134733
134734
134735
134736
134737
134738
134739
134740
134741
134742
134743
134744
134745
134746
134747
134748
134749
134750
134751
134752
134753
134754
134755
134756
134757
134758
134759
134760
134761
134762
134763
134764
134765
134766
134767
134768
134769
134770
134771
134772
134773
134774
134775
134776
134777
134778
134779
134780
134781
134782
134783
134784
134785
134786
134787
134788
134789
134790
134791
134792
134793
134794
134795
134796
134797
134798
134799
134800
134801
134802
134803
134804
134805
134806
134807
134808
134809
134810
134811
134812
134813
134814
134815
134816
134817
134818
134819
134820
134821
134822
134823
134824
134825
134826
134827
134828
134829
134830
134831
134832
134833
134834
134835
134836
134837
134838
134839
134840
134841
134842
134843
134844
134845
134846
134847
134848
134849
134850
134851
134852
134853
134854
134855
134856
134857
......
135096
135097
135098
135099
135100
135101
135102
135103
135104
135105
135106
135107
135108
135109
135110
135111
135112
135113
135114
135115
135116
135117
135118
135119
135120
135121
135122
135123
135124
135125
135126
135127
135128
135129
135130
135131
135132
135133
135134
135135
135136
135137
135138
135139
135140
135141
135142
135143
135144
135145
135146
135147
135148
135149
135150
135151
135152
135153
135154
......
135586
135587
135588
135589
135590
135591
135592
135593
135594
135595
135596
135597
135598
135599
135600
135601
135602
135603
135604
135605
135606
135607
135608
135609
135610
135611
135612
135613
135614
135615
135616
135617
135618
135619
135620
135621
135622
135623
135624
135625
135626
135627
135628
135629
135630
135631
135632
135633
135634
135635
135636
135637
135638
135639
135640
135641
135642
135643
135644
135645
135646
135647
135648
135649
135650
135651
135652
135653
135654
135655
135656
135657
135658
135659
135660
135661
135662
135663
135664
135665
135666
135667
135668
135669
135670
135671
135672
135673
135674
135675
135676
135677
135678
135679
135680
135681
135682
135683
135684
135685
135686
135687
135688
135689
135690
135691
135692
135693
135694
135695
135696
135697
135698
135699
135700
135701
135702
135703
135704
135705
135706
135707
135708
135709
135710
135711
135712
135713
135714
135715
135716
135717
135718
135719
135720
135721
135722
135723
135724
135725
135726
135727
135728
135729

135730
135731
135732
135733
135734
135735
135736
......
136359
136360
136361
136362
136363
136364
136365
136366
136367
136368
136369
136370
136371
136372
136373
......
136496
136497
136498
136499
136500
136501
136502

136503
136504
136505
136506
136507
136508
136509
......
136638
136639
136640
136641
136642
136643
136644
136645
136646
136647
136648
136649
136650
136651
136652
......
136782
136783
136784
136785
136786
136787
136788
136789
136790
136791
136792
136793
136794
136795
136796
136797
136798
......
136948
136949
136950
136951
136952
136953
136954
136955
136956
136957
136958
136959
136960
136961
136962
136963
136964
......
136976
136977
136978
136979
136980
136981
136982
136983
136984
136985
136986
136987
136988
136989
136990
136991
......
137060
137061
137062
137063
137064
137065
137066
137067
137068
137069
137070
137071
137072
137073
137074
137075
137076
137077
137078
137079
137080
137081
......
137214
137215
137216
137217
137218
137219
137220
137221
137222
137223
137224
137225
137226
137227
137228
......
137304
137305
137306
137307
137308
137309
137310
137311
137312
137313
137314
137315
137316
137317
137318
137319
137320
137321
137322
137323
137324
137325
137326
137327
137328
137329
137330
137331
137332
137333
137334
137335
137336
137337
137338
137339
137340
137341
137342
137343
137344
137345
137346
137347
137348
137349
137350
137351
137352
137353
137354
137355
137356
137357
137358
137359
137360
137361
137362
137363
137364
137365
137366
137367
137368
137369
137370
137371
137372
137373
137374
137375
137376
137377
137378
137379
137380
137381
137382
137383
137384
137385
137386
137387
137388
137389
137390
137391
137392
137393
137394
137395
137396
137397
......
137436
137437
137438
137439
137440
137441
137442
137443
137444
137445
137446
137447
137448
137449
137450
137451
137452
137453
137454
137455
137456
137457
137458
137459
137460
137461
137462
137463
137464
137465
137466
137467
137468
137469
137470
137471
137472
137473
137474
137475
137476
137477
137478
137479
137480
137481
137482
137483
137484
137485
137486
137487
137488
137489
137490
137491
137492
137493
137494
137495
137496
137497
137498
137499
137500
137501
137502
137503
137504
137505
137506
137507
137508
137509
137510
137511
137512
137513
137514
137515
137516
137517
137518
137519
137520
137521
137522
137523
137524
137525
137526
137527
137528
137529
137530
137531
137532
137533
137534
137535
137536
137537
137538
137539
137540
137541
137542
137543
137544
137545
137546
137547
137548
137549
137550
137551
137552
137553
137554
137555
137556
137557
137558
137559
137560
137561
137562
137563
137564
137565
137566
137567
137568
137569
137570
137571
137572
137573
137574
137575
137576
137577
137578
137579
137580
137581
137582
137583
137584
137585
137586
137587
137588
137589
137590
137591
137592
137593
137594
137595
137596
137597
137598
137599
137600
137601
137602
137603
137604
137605
137606
137607
137608
137609
137610
137611
137612
137613
137614
137615
137616
137617
137618
137619
137620
137621
137622
137623
137624
137625
137626
137627
137628
137629
137630
137631
137632
137633
137634
137635
137636
137637
137638
137639
137640
137641
137642
137643
137644
137645
137646
137647
137648
137649
137650
137651
137652
137653
137654
137655
137656
137657
137658
137659
137660
137661
137662
137663
137664
137665
137666
137667
137668
137669
137670
137671
137672
137673
137674
137675
137676
137677
137678
137679
137680
137681
137682
137683
137684
137685
137686
137687
137688
137689
137690
137691
137692
137693
137694
137695
137696
137697
137698
137699
137700
137701
137702
137703
137704
137705
137706
137707
137708
137709
137710
137711
137712
137713
137714
137715
137716
137717
137718
137719
137720
137721
137722
137723
137724
137725
137726
137727
137728
137729
137730
137731
137732
137733
137734
137735
137736
137737
137738
137739
137740
137741
137742
137743
137744
137745
137746
137747
137748
137749
137750
137751
137752
137753
137754
137755
137756
137757
137758
137759
137760
137761
137762
137763
137764
137765
137766
137767
137768
137769
137770
137771
137772
137773
137774
137775
137776
137777
137778
137779
137780
137781
137782
137783
137784
137785
137786
137787
137788
137789
137790
137791
137792

137793
137794
137795
137796
137797
137798
137799
......
138838
138839
138840
138841
138842
138843
138844
138845
138846

138847
138848
138849
138850
138851
138852
138853
......
138855
138856
138857
138858
138859
138860
138861
138862
138863
138864
138865
138866
138867
138868
138869
......
138873
138874
138875
138876
138877
138878
138879
138880
138881
138882
138883
138884
138885

138886
138887
138888
138889
138890
138891
138892
138893
138894
138895
138896
138897
138898

138899
138900
138901
138902
138903
138904
138905
138906
138907
138908
138909
138910

138911


138912
138913

138914
138915
138916
138917
138918
138919
138920
138921
138922
138923
138924
138925
......
140285
140286
140287
140288
140289
140290
140291















140292
140293
140294
140295
140296
140297
140298
......
144586
144587
144588
144589
144590
144591
144592

144593
144594
144595
144596
144597
144598
144599
......
144655
144656
144657
144658
144659
144660
144661

144662
144663
144664
144665
144666
144667
144668
......
145177
145178
145179
145180
145181
145182
145183

145184
145185
145186
145187
145188
145189
145190
......
146364
146365
146366
146367
146368
146369
146370




















146371
146372
146373
146374
146375
146376
146377
146378
146379
146380
146381
146382
146383
146384
146385
146386
......
146390
146391
146392
146393
146394
146395
146396
146397
146398
146399
146400
146401
146402
146403
146404




146405
146406
146407
146408
146409
146410


146411
146412
146413
146414
146415
146416
146417
146418
146419
146420
146421
146422
146423
146424
146425
146426
146427
146428
146429
146430
146431
......
147875
147876
147877
147878
147879
147880
147881
147882
147883
147884
147885
147886
147887
147888
147889
......
147943
147944
147945
147946
147947
147948
147949
147950
147951
147952
147953
147954
147955
147956
147957
......
148071
148072
148073
148074
148075
148076
148077
148078

148079

148080
148081
148082
148083
148084
148085
148086
......
148087
148088
148089
148090
148091
148092
148093

148094
148095
148096
148097
148098
148099
148100
......
162963
162964
162965
162966
162967
162968
162969

162970
162971
162972
162973
162974
162975
162976
......
163262
163263
163264
163265
163266
163267
163268
163269




163270
163271

163272
163273
163274
163275
163276
163277
163278
163279
163280
163281
163282
163283
163284
163285
163286
163287
163288
163289
163290
163291
163292
......
163298
163299
163300
163301
163302
163303
163304
163305
163306
163307
163308
163309
163310
163311
163312
163313
163314
163315
163316
163317
163318
163319
163320
163321
163322
163323
163324
163325
163326
163327
163328
......
163332
163333
163334
163335
163336
163337
163338
163339
163340
163341
163342
163343
163344
163345
163346
163347
163348
163349
163350
......
163353
163354
163355
163356
163357
163358
163359
163360
163361
163362
163363
163364
163365
163366
163367
......
163380
163381
163382
163383
163384
163385
163386
163387
163388
163389
163390
163391
163392
163393
163394
163395
163396
163397
163398
163399
163400
163401
163402
163403
163404
......
163409
163410
163411
163412
163413
163414
163415
163416
163417
163418
163419
163420
163421
163422
163423
......
163965
163966
163967
163968
163969
163970
163971
163972
163973
163974
163975
163976
163977
163978
163979
......
168432
168433
168434
168435
168436
168437
168438

168439
168440
168441
168442
168443
168444
168445
......
170696
170697
170698
170699
170700
170701
170702

















170703
170704
170705
170706
170707
170708
170709
......
171351
171352
171353
171354
171355
171356
171357












171358

171359
171360




171361
171362
171363
171364
171365
171366
171367
......
171793
171794
171795
171796
171797
171798
171799






171800
171801
171802
171803
171804
171805
171806
......
171917
171918
171919
171920
171921
171922
171923






171924
171925
171926
171927
171928
171929
171930
......
181049
181050
181051
181052
181053
181054
181055

181056

181057
181058
181059
181060
181061
181062
181063
......
186272
186273
186274
186275
186276
186277
186278
186279



186280
186281
186282
186283
186284
186285
186286
......
186303
186304
186305
186306
186307
186308
186309
186310



186311
186312
186313
186314
186315
186316
186317
......
186342
186343
186344
186345
186346
186347
186348


186349
186350
186351
186352
186353
186354
186355
......
186384
186385
186386
186387
186388
186389
186390



186391
186392
186393
186394
186395
186396
186397
......
197504
197505
197506
197507
197508
197509
197510
197511
197512
197513
197514
197515
197516
197517
197518
......
198167
198168
198169
198170
198171
198172
198173
198174
198175
198176
198177
198178
198179
198180
198181
198182
198183
198184
198185
......
198375
198376
198377
198378
198379
198380
198381
198382
198383
198384
198385
198386
198387
198388
198389
198390
198391
198392
198393
......
198714
198715
198716
198717
198718
198719
198720


198721
198722
198723
198724
198725

198726



198727
198728
198729
198730
198731
198732
198733
/******************************************************************************
** This file is an amalgamation of many separate C source files from SQLite
** version 3.17.0.  By combining all the individual C code files into this
** single large file, the entire code can be compiled as a single translation
** unit.  This allows many compilers to do optimizations that would not be
** possible if the files were compiled separately.  Performance improvements
** of 5% or more are commonly seen when SQLite is compiled as a single
** translation unit.
**
** This file is all you need to compile SQLite.  To use SQLite in other
................................................................................
# define _LARGE_FILE       1
# ifndef _FILE_OFFSET_BITS
#   define _FILE_OFFSET_BITS 64
# endif
# define _LARGEFILE_SOURCE 1
#endif

/* What version of GCC is being used.  0 means GCC is not being used */
#ifdef __GNUC__












# define GCC_VERSION (__GNUC__*1000000+__GNUC_MINOR__*1000+__GNUC_PATCHLEVEL__)
#else
# define GCC_VERSION 0
#endif


/* What version of CLANG is being used.  0 means CLANG is not being used */
#if defined(__clang__) && !defined(_WIN32)
# define CLANG_VERSION \
            (__clang_major__*1000000+__clang_minor__*1000+__clang_patchlevel__)
#else
# define CLANG_VERSION 0
#endif

/* Needed for various definitions... */
#if defined(__GNUC__) && !defined(_GNU_SOURCE)
# define _GNU_SOURCE
#endif

................................................................................
** string contains the date and time of the check-in (UTC) and an SHA1
** hash of the entire source tree.
**
** See also: [sqlite3_libversion()],
** [sqlite3_libversion_number()], [sqlite3_sourceid()],
** [sqlite_version()] and [sqlite_source_id()].
*/
#define SQLITE_VERSION        "3.17.0"
#define SQLITE_VERSION_NUMBER 3017000
#define SQLITE_SOURCE_ID      "2017-02-07 13:51:48 a136609c98ed3cc673c5a3c2578d49db3f2518d1"

/*
** CAPI3REF: Run-Time Library Version Numbers
** KEYWORDS: sqlite3_version sqlite3_sourceid
**
** These interfaces provide the same information as the [SQLITE_VERSION],
** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
** has a unique 64-bit signed
** integer key called the [ROWID | "rowid"]. ^The rowid is always available
** as an undeclared column named ROWID, OID, or _ROWID_ as long as those
** names are not also used by explicitly declared columns. ^If
** the table has a column of type [INTEGER PRIMARY KEY] then that column
** is another alias for the rowid.
**
** ^The sqlite3_last_insert_rowid(D) interface returns the [rowid] of the 
** most recent successful [INSERT] into a rowid table or [virtual table]
** on database connection D.
** ^Inserts into [WITHOUT ROWID] tables are not recorded.
** ^If no successful [INSERT]s into rowid tables
** have ever occurred on the database connection D, 




** then sqlite3_last_insert_rowid(D) returns zero.
**









** ^(If an [INSERT] occurs within a trigger or within a [virtual table]
** method, then this routine will return the [rowid] of the inserted
** row as long as the trigger or virtual table method is running.
** But once the trigger or virtual table method ends, the value returned 

** by this routine reverts to what it was before the trigger or virtual
** table method began.)^
**
** ^An [INSERT] that fails due to a constraint violation is not a
** successful [INSERT] and does not change the value returned by this
** routine.  ^Thus INSERT OR FAIL, INSERT OR IGNORE, INSERT OR ROLLBACK,
** and INSERT OR ABORT make no changes to the return value of this
** routine when their insertion fails.  ^(When INSERT OR REPLACE
** encounters a constraint violation, it does not fail.  The
................................................................................
** function is running and thus changes the last insert [rowid],
** then the value returned by [sqlite3_last_insert_rowid()] is
** unpredictable and might not equal either the old or the new
** last insert [rowid].
*/
SQLITE_API sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*);











/*
** CAPI3REF: Count The Number Of Rows Modified
** METHOD: sqlite3
**
** ^This function returns the number of rows modified, inserted or
** deleted by the most recently completed INSERT, UPDATE or DELETE
** statement on the database connection specified by the only parameter.
................................................................................
** either of these things are undefined.
**
** The session object will be used to create changesets for tables in
** database zDb, where zDb is either "main", or "temp", or the name of an
** attached database. It is not an error if database zDb is not attached
** to the database when the session object is created.
*/
int sqlite3session_create(
  sqlite3 *db,                    /* Database handle */
  const char *zDb,                /* Name of db (e.g. "main") */
  sqlite3_session **ppSession     /* OUT: New session object */
);

/*
** CAPI3REF: Delete A Session Object
................................................................................
** results of attempting to use pSession with any other session module
** function are undefined.
**
** Session objects must be deleted before the database handle to which they
** are attached is closed. Refer to the documentation for 
** [sqlite3session_create()] for details.
*/
void sqlite3session_delete(sqlite3_session *pSession);


/*
** CAPI3REF: Enable Or Disable A Session Object
**
** Enable or disable the recording of changes by a session object. When
** enabled, a session object records changes made to the database. When
................................................................................
** Passing zero to this function disables the session. Passing a value
** greater than zero enables it. Passing a value less than zero is a 
** no-op, and may be used to query the current state of the session.
**
** The return value indicates the final state of the session object: 0 if 
** the session is disabled, or 1 if it is enabled.
*/
int sqlite3session_enable(sqlite3_session *pSession, int bEnable);

/*
** CAPI3REF: Set Or Clear the Indirect Change Flag
**
** Each change recorded by a session object is marked as either direct or
** indirect. A change is marked as indirect if either:
**
................................................................................
** is set. Passing a value less than zero does not modify the current value
** of the indirect flag, and may be used to query the current state of the 
** indirect flag for the specified session object.
**
** The return value indicates the final state of the indirect flag: 0 if 
** it is clear, or 1 if it is set.
*/
int sqlite3session_indirect(sqlite3_session *pSession, int bIndirect);

/*
** CAPI3REF: Attach A Table To A Session Object
**
** If argument zTab is not NULL, then it is the name of a table to attach
** to the session object passed as the first argument. All subsequent changes 
** made to the table while the session object is enabled will be recorded. See 
................................................................................
**
** Changes are not recorded for individual rows that have NULL values stored
** in one or more of their PRIMARY KEY columns.
**
** SQLITE_OK is returned if the call completes without error. Or, if an error 
** occurs, an SQLite error code (e.g. SQLITE_NOMEM) is returned.
*/
int sqlite3session_attach(
  sqlite3_session *pSession,      /* Session object */
  const char *zTab                /* Table name */
);

/*
** CAPI3REF: Set a table filter on a Session Object.
**
** The second argument (xFilter) is the "filter callback". For changes to rows 
** in tables that are not attached to the Session object, the filter is called
** to determine whether changes to the table's rows should be tracked or not. 
** If xFilter returns 0, changes is not tracked. Note that once a table is 
** attached, xFilter will not be called again.
*/
void sqlite3session_table_filter(
  sqlite3_session *pSession,      /* Session object */
  int(*xFilter)(
    void *pCtx,                   /* Copy of third arg to _filter_table() */
    const char *zTab              /* Table name */
  ),
  void *pCtx                      /* First argument passed to xFilter */
);
................................................................................
** is inserted while a session object is enabled, then later deleted while 
** the same session object is disabled, no INSERT record will appear in the
** changeset, even though the delete took place while the session was disabled.
** Or, if one field of a row is updated while a session is disabled, and 
** another field of the same row is updated while the session is enabled, the
** resulting changeset will contain an UPDATE change that updates both fields.
*/
int sqlite3session_changeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
);

/*
** CAPI3REF: Lo