Fossil

Check-in [a253fba0]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merged trunk
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | commonmark-markdown
Files: files | file ages | folders
SHA3-256:a253fba0f7a7f2836ba799be912e643a624e513d2541f250a630bb57ef781621
User & Date: mjanssen 2018-06-06 10:39:12
Context
2018-06-06
10:39
Merged trunk Leaf check-in: a253fba0 user: mjanssen tags: commonmark-markdown
2018-06-05
23:06
Fix a bug in the pivot-finder introduced by check-in [917f1c21e52a29904] from 5 days ago. check-in: 83789c6e user: drh tags: trunk
2018-02-12
08:50
Merged trunk check-in: 270d2fb9 user: mjanssen tags: commonmark-markdown
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to Makefile.in.

    41     41   CFLAGS = @CFLAGS@
    42     42   LIB =	@LDFLAGS@ @EXTRA_LDFLAGS@ @LIBS@
    43     43   BCCFLAGS =	@CPPFLAGS@ @CFLAGS@
    44     44   TCCFLAGS =	@EXTRA_CFLAGS@ @CPPFLAGS@ @CFLAGS@ -DHAVE_AUTOCONFIG_H -D_HAVE_SQLITE_CONFIG_H
    45     45   INSTALLDIR = $(DESTDIR)@prefix@/bin
    46     46   USE_SYSTEM_SQLITE = @USE_SYSTEM_SQLITE@
    47     47   USE_LINENOISE = @USE_LINENOISE@
           48  +USE_MMAN_H = @USE_MMAN_H@
    48     49   USE_SEE = @USE_SEE@
    49     50   FOSSIL_ENABLE_MINIZ = @FOSSIL_ENABLE_MINIZ@
    50     51   
    51     52   include $(SRCDIR)/main.mk
    52     53   
    53     54   distclean: clean
    54     55   	rm -f autoconfig.h config.log Makefile

Added Makefile.osx-jaguar.

            1  +#!/usr/bin/make
            2  +#
            3  +# This is a specially modified version of the Makefile that will build
            4  +# Fossil on Mac OSX Jaguar (10.2) circa 2002.  This Makefile is used for
            5  +# testing on an old PPC iBook.  The use of this old platform helps to verify
            6  +# Fossil and SQLite running on big-endian hardware.
            7  +#
            8  +# To build with this Makefile, run:
            9  +#
           10  +#     make -f Makefile.osx-jaguar
           11  +#
           12  +#
           13  +# This is the top-level makefile for Fossil when the build is occurring
           14  +# on a unix platform.  This works out-of-the-box on most unix platforms.
           15  +# But you are free to vary some of the definitions if desired.
           16  +#
           17  +#### The toplevel directory of the source tree.  Fossil can be built
           18  +#    in a directory that is separate from the source tree.  Just change
           19  +#    the following to point from the build directory to the src/ folder.
           20  +#
           21  +SRCDIR = ./src
           22  +
           23  +#### The directory into which object code files should be written.
           24  +#    Having a "./" prefix in the value of this variable breaks our use of the
           25  +#    "makeheaders" tool when running make on the MinGW platform, apparently
           26  +#    due to some command line argument manipulation performed automatically
           27  +#    by the shell.
           28  +#
           29  +#
           30  +OBJDIR = bld
           31  +
           32  +#### C Compiler and options for use in building executables that
           33  +#    will run on the platform that is doing the build.  This is used
           34  +#    to compile code-generator programs as part of the build process.
           35  +#    See TCC below for the C compiler for building the finished binary.
           36  +#
           37  +BCC = cc
           38  +
           39  +#### The suffix to add to final executable file.  When cross-compiling
           40  +#    to windows, make this ".exe".  Otherwise leave it blank.
           41  +#
           42  +E = 
           43  +
           44  +TCC = cc
           45  +
           46  +#### Tcl shell for use in running the fossil testsuite.  If you do not
           47  +#    care about testing the end result, this can be blank.
           48  +#
           49  +TCLSH = tclsh
           50  +
           51  +# LIB =	  -lz
           52  +LIB = compat/zlib/libz.a
           53  +TCC +=	  -g -O0 -DHAVE_AUTOCONFIG_H
           54  +TCC += -Icompat/zlib
           55  +TCC += -DSQLITE_WITHOUT_ZONEMALLOC
           56  +TCC += -D_BSD_SOURCE=1
           57  +TCC += -DWITHOUT_ICONV
           58  +TCC += -Dsocklen_t=int
           59  +TCC += -DSQLITE_MAX_MMAP_SIZE=0
           60  +INSTALLDIR = $(DESTDIR)/usr/local/bin
           61  +USE_SYSTEM_SQLITE = 
           62  +USE_LINENOISE = 1
           63  +# FOSSIL_ENABLE_TCL = @FOSSIL_ENABLE_TCL@
           64  +FOSSIL_ENABLE_TCL = 0
           65  +FOSSIL_ENABLE_MINIZ = 0
           66  +
           67  +include $(SRCDIR)/main.mk
           68  +
           69  +distclean: clean
           70  +	rm -f autoconfig.h config.log Makefile

Changes to VERSION.

     1         -2.5
            1  +2.6

Changes to auto.def.

    13     13       with-legacy-mv-rm=0  => {Enable legacy behavior for mv/rm (skip checkout files)}
    14     14       with-th1-docs=0      => {Enable TH1 for embedded documentation pages}
    15     15       with-th1-hooks=0     => {Enable TH1 hooks for commands and web pages}
    16     16       with-tcl:path        => {Enable Tcl integration, with Tcl in the specified path}
    17     17       with-tcl-stubs=0     => {Enable Tcl integration via stubs library mechanism}
    18     18       with-tcl-private-stubs=0
    19     19                            => {Enable Tcl integration via private stubs mechanism}
           20  +    with-mman=0          => {Enable use of POSIX memory APIs from "sys/mman.h"}
    20     21       with-see=0           => {Enable the SQLite Encryption Extension (SEE)}
    21     22       internal-sqlite=1    => {Don't use the internal SQLite, use the system one}
    22     23       static=0             => {Link a static executable}
    23     24       fusefs=1             => {Disable the Fuse Filesystem}
    24     25       fossil-debug=0       => {Build with fossil debugging enabled}
    25     26       no-opt=0             => {Build without optimization}
    26     27       json=0               => {Build with fossil JSON API enabled}
................................................................................
    38     39   cc-check-progs tclsh
    39     40   
    40     41   define EXTRA_CFLAGS "-Wall"
    41     42   define EXTRA_LDFLAGS ""
    42     43   define USE_SYSTEM_SQLITE 0
    43     44   define USE_LINENOISE 0
    44     45   define FOSSIL_ENABLE_MINIZ 0
           46  +define USE_MMAN_H 0
    45     47   define USE_SEE 0
    46     48   
    47     49   # This procedure is a customized version of "cc-check-function-in-lib",
    48     50   # that does not modify the LIBS variable.  Its use prevents prematurely
    49     51   # pulling in libraries that will be added later anyhow (e.g. "-ldl").
    50     52   proc check-function-in-lib {function libs {otherlibs {}}} {
    51     53       if {[string length $otherlibs]} {
................................................................................
    87     89       # search for the system SQLite once with -ldl, and once without. If
    88     90       # the library can only be found with $extralibs set to -ldl, then
    89     91       # the code below will append -ldl to LIBS.
    90     92       #
    91     93       foreach extralibs {{} {-ldl}} {
    92     94   
    93     95         # Locate the system SQLite by searching for sqlite3_open(). Then check
    94         -      # if sqlite3_vtab_collation() can be found as well. If we can find open() but
    95         -      # not vtab_collation(), then the system SQLite is too old to link against
           96  +      # if sqlite3_keyword_check() can be found as well. If we can find open() but
           97  +      # not keyword_check(), then the system SQLite is too old to link against
    96     98         # fossil.
    97     99         #
    98    100         if {[check-function-in-lib sqlite3_open sqlite3 $extralibs]} {
    99         -        if {![check-function-in-lib sqlite3_vtab_collation sqlite3 $extralibs]} {
   100         -          user-error "system sqlite3 too old (require >= 3.22.0)"
          101  +        if {![check-function-in-lib sqlite3_keyword_check sqlite3 $extralibs]} {
          102  +          user-error "system sqlite3 too old (require >= 3.24.0)"
   101    103           }
   102    104   
   103    105           # Success. Update symbols and return.
   104    106           #
   105    107           define USE_SYSTEM_SQLITE 1
   106    108           define-append LIBS -lsqlite3
   107    109           define-append LIBS $extralibs
................................................................................
   139    141       msg-result "Debugging support enabled"
   140    142   }
   141    143   
   142    144   if {[opt-bool no-opt]} {
   143    145       define CFLAGS {-g -O0 -Wall}
   144    146       msg-result "Builting without compiler optimization"
   145    147   }
          148  +
          149  +if {[opt-bool with-mman]} {
          150  +    define-append EXTRA_CFLAGS -DUSE_MMAN_H
          151  +    define USE_MMAN_H 1
          152  +    msg-result "Enabling \"sys/mman.h\" support"
          153  +}
   146    154   
   147    155   if {[opt-bool with-see]} {
   148    156       define-append EXTRA_CFLAGS -DUSE_SEE
   149    157       define USE_SEE 1
   150    158       msg-result "Enabling encryption support"
   151    159   }
   152    160   

Changes to src/add.c.

   857    857   void mv_cmd(void){
   858    858     int i;
   859    859     int vid;
   860    860     int moveFiles;
   861    861     int dryRunFlag;
   862    862     int softFlag;
   863    863     int hardFlag;
          864  +  int origType;
          865  +  int destType;
   864    866     char *zDest;
   865    867     Blob dest;
   866    868     Stmt q;
   867    869   
   868    870     db_must_be_within_tree();
   869    871     dryRunFlag = find_option("dry-run","n",0)!=0;
   870    872     softFlag = find_option("soft",0,0)!=0;
................................................................................
   898    900     file_tree_name(zDest, &dest, 0, 1);
   899    901     db_multi_exec(
   900    902       "UPDATE vfile SET origname=pathname WHERE origname IS NULL;"
   901    903     );
   902    904     db_multi_exec(
   903    905       "CREATE TEMP TABLE mv(f TEXT UNIQUE ON CONFLICT IGNORE, t TEXT);"
   904    906     );
   905         -  if( file_isdir(zDest, RepoFILE)!=1 ){
          907  +  if( g.argc!=4 ){
          908  +    origType = -1;
          909  +  }else{
          910  +    origType = (file_isdir(g.argv[2], RepoFILE) == 1);
          911  +  }
          912  +  destType = file_isdir(zDest, RepoFILE);
          913  +  if( origType==-1 && destType!=1 ){
          914  +    usage("OLDNAME NEWNAME");
          915  +  }else if( origType==1 && destType==2 ){
          916  +    fossil_fatal("cannot rename '%s' to '%s' since another file named"
          917  +                 " '%s' exists", g.argv[2], zDest, zDest);
          918  +  }else if( origType==0 && destType!=1 ){
   906    919       Blob orig;
   907         -    if( g.argc!=4 ){
   908         -      usage("OLDNAME NEWNAME");
   909         -    }
   910    920       file_tree_name(g.argv[2], &orig, 0, 1);
   911    921       db_multi_exec(
   912    922         "INSERT INTO mv VALUES(%B,%B)", &orig, &dest
   913    923       );
   914    924     }else{
   915    925       if( blob_eq(&dest, ".") ){
   916    926         blob_reset(&dest);
................................................................................
   934    944         );
   935    945         while( db_step(&q)==SQLITE_ROW ){
   936    946           const char *zPath = db_column_text(&q, 0);
   937    947           int nPath = db_column_bytes(&q, 0);
   938    948           const char *zTail;
   939    949           if( nPath==nOrig ){
   940    950             zTail = file_tail(zPath);
          951  +        }else if( destType==1 ){
          952  +          zTail = &zPath[nOrig-strlen(file_tail(zOrig))];
   941    953           }else{
   942    954             zTail = &zPath[nOrig+1];
   943    955           }
   944    956           db_multi_exec(
   945    957             "INSERT INTO mv VALUES('%q','%q%q')",
   946    958             zPath, blob_str(&dest), zTail
   947    959           );

Changes to src/blob.c.

   112    112   }
   113    113   int fossil_isalpha(char c){
   114    114     return (c>='a' && c<='z') || (c>='A' && c<='Z');
   115    115   }
   116    116   int fossil_isalnum(char c){
   117    117     return (c>='a' && c<='z') || (c>='A' && c<='Z') || (c>='0' && c<='9');
   118    118   }
          119  +
          120  +/* Return true if and only if the entire string consists of only
          121  +** alphanumeric characters.
          122  +*/
          123  +int fossil_no_strange_characters(const char *z){
          124  +  while( z && (fossil_isalnum(z[0]) || z[0]=='_' || z[0]=='-') ) z++;
          125  +  return z[0]==0;
          126  +}
   119    127   
   120    128   
   121    129   /*
   122    130   ** COMMAND: test-isspace
   123    131   **
   124    132   ** Verify that the fossil_isspace() routine is working correctly by
   125    133   ** testing it on all possible inputs.

Changes to src/cache.c.

   163    163     sqlite3_bind_text(pStmt, 1, zKey, -1, SQLITE_STATIC);
   164    164     sqlite3_bind_int(pStmt, 2, blob_size(pContent));
   165    165     sqlite3_bind_int(pStmt, 3, sqlite3_last_insert_rowid(db));
   166    166     if( sqlite3_step(pStmt)!=SQLITE_DONE) goto cache_write_end;
   167    167     rc = sqlite3_changes(db);
   168    168   
   169    169     /* If the write was successful, truncate the cache to keep at most
   170         -  ** max-cache-entry entries in the cache */
          170  +  ** max-cache-entry entries in the cache.
          171  +  **
          172  +  ** The cache entry replacement algorithm is approximately LRU
          173  +  ** (least recently used).  However, each access of an entry buys
          174  +  ** that entry an extra hour of grace, so that more commonly accessed
          175  +  ** entries are held in cache longer.  The extra "grace" allotted to
          176  +  ** an entry is limited to 2 days worth.
          177  +  */
   171    178     if( rc ){
   172    179       nKeep = db_get_int("max-cache-entry",10);
   173    180       sqlite3_finalize(pStmt);
   174    181       pStmt = cacheStmt(db,
   175    182                    "DELETE FROM cache WHERE rowid IN ("
   176         -                    "SELECT rowid FROM cache ORDER BY tm DESC"
          183  +                    "SELECT rowid FROM cache"
          184  +                    " ORDER BY (tm + 3600*min(nRef,48)) DESC"
   177    185                       " LIMIT -1 OFFSET ?1)");
   178    186       if( pStmt ){
   179    187         sqlite3_bind_int(pStmt, 1, nKeep);
   180    188         sqlite3_step(pStmt);
   181    189       }
   182    190     }
   183    191   
................................................................................
   289    297       if( db ){
   290    298         sqlite3_exec(db, "DELETE FROM cache; DELETE FROM blob; VACUUM;",0,0,0);
   291    299         sqlite3_close(db);
   292    300         fossil_print("cache cleared\n");
   293    301       }else{
   294    302         fossil_print("nothing to clear; cache does not exist\n");
   295    303       }
   296         -  }else if(( strncmp(zCmd, "list", nCmd)==0 ) || ( strncmp(zCmd, "ls", nCmd)==0 )){
          304  +  }else if(( strncmp(zCmd, "list", nCmd)==0 )
          305  +             || ( strncmp(zCmd, "ls", nCmd)==0 )){
   297    306       db = cacheOpen(0);
   298    307       if( db==0 ){
   299    308         fossil_print("cache does not exist\n");
   300    309       }else{
   301    310         int nEntry = 0;
   302    311         char *zDbName = cacheName();
   303    312         cache_register_sizename(db);
................................................................................
   348    357       @ The web-page cache is disabled for this repository
   349    358     }else{
   350    359       char *zDbName = cacheName();
   351    360       cache_register_sizename(db);
   352    361       pStmt = cacheStmt(db,
   353    362            "SELECT key, sizename(sz), nRef, datetime(tm,'unixepoch')"
   354    363            "  FROM cache"
   355         -         " ORDER BY tm DESC"
          364  +         " ORDER BY (tm + 3600*min(nRef,48)) DESC"
   356    365       );
   357    366       if( pStmt ){
   358    367         @ <ol>
   359    368         while( sqlite3_step(pStmt)==SQLITE_ROW ){
   360    369           const unsigned char *zName = sqlite3_column_text(pStmt,0);
   361    370           @ <li><p>%z(href("%R/cacheget?key=%T",zName))%h(zName)</a><br />
   362    371           @ size: %s(sqlite3_column_text(pStmt,1))

Changes to src/cgi.c.

   215    215       zPath = g.zTop;
   216    216       if( zPath[0]==0 ) zPath = "/";
   217    217     }
   218    218     if( g.zBaseURL!=0 && strncmp(g.zBaseURL, "https:", 6)==0 ){
   219    219       zSecure = " secure;";
   220    220     }
   221    221     if( lifetime>0 ){
   222         -    lifetime += (int)time(0);
   223    222       blob_appendf(&extraHeader,
   224         -       "Set-Cookie: %s=%t; Path=%s; expires=%z; HttpOnly;%s Version=1\r\n",
   225         -        zName, zValue, zPath, cgi_rfc822_datestamp(lifetime), zSecure);
          223  +       "Set-Cookie: %s=%t; Path=%s; max-age=%d; HttpOnly;%s Version=1\r\n",
          224  +        zName, zValue, zPath, lifetime, zSecure);
   226    225     }else{
   227    226       blob_appendf(&extraHeader,
   228    227          "Set-Cookie: %s=%t; Path=%s; HttpOnly;%s Version=1\r\n",
   229    228          zName, zValue, zPath, zSecure);
   230    229     }
   231    230   }
   232    231   
   233         -#if 0
   234         -/*
   235         -** Add an ETag header line
   236         -*/
   237         -static char *cgi_add_etag(char *zTxt, int nLen){
   238         -  MD5Context ctx;
   239         -  unsigned char digest[16];
   240         -  int i, j;
   241         -  char zETag[64];
   242         -
   243         -  MD5Init(&ctx);
   244         -  MD5Update(&ctx,zTxt,nLen);
   245         -  MD5Final(digest,&ctx);
   246         -  for(j=i=0; i<16; i++,j+=2){
   247         -    bprintf(&zETag[j],sizeof(zETag)-j,"%02x",(int)digest[i]);
   248         -  }
   249         -  blob_appendf(&extraHeader, "ETag: %s\r\n", zETag);
   250         -  return fossil_strdup(zETag);
   251         -}
   252         -
   253         -/*
   254         -** Do some cache control stuff. First, we generate an ETag and include it in
   255         -** the response headers. Second, we do whatever is necessary to determine if
   256         -** the request was asking about caching and whether we need to send back the
   257         -** response body. If we shouldn't send a body, return non-zero.
   258         -**
   259         -** Currently, we just check the ETag against any If-None-Match header.
   260         -**
   261         -** FIXME: In some cases (attachments, file contents) we could check
   262         -** If-Modified-Since headers and always include Last-Modified in responses.
   263         -*/
   264         -static int check_cache_control(void){
   265         -  /* FIXME: there's some gotchas wth cookies and some headers. */
   266         -  char *zETag = cgi_add_etag(blob_buffer(&cgiContent),blob_size(&cgiContent));
   267         -  char *zMatch = P("HTTP_IF_NONE_MATCH");
   268         -
   269         -  if( zETag!=0 && zMatch!=0 ) {
   270         -    char *zBuf = fossil_strdup(zMatch);
   271         -    if( zBuf!=0 ){
   272         -      char *zTok = 0;
   273         -      char *zPos;
   274         -      for( zTok = strtok_r(zBuf, ",\"",&zPos);
   275         -           zTok && fossil_stricmp(zTok,zETag);
   276         -           zTok =  strtok_r(0, ",\"",&zPos)){}
   277         -      fossil_free(zBuf);
   278         -      if(zTok) return 1;
   279         -    }
   280         -  }
   281         -
   282         -  return 0;
   283         -}
   284         -#endif
   285    232   
   286    233   /*
   287    234   ** Return true if the response should be sent with Content-Encoding: gzip.
   288    235   */
   289    236   static int is_gzippable(void){
   290    237     if( strstr(PD("HTTP_ACCEPT_ENCODING", ""), "gzip")==0 ) return 0;
   291    238     return strncmp(zContentType, "text/", 5)==0
................................................................................
   299    246   void cgi_reply(void){
   300    247     int total_size;
   301    248     if( iReplyStatus<=0 ){
   302    249       iReplyStatus = 200;
   303    250       zReplyStatus = "OK";
   304    251     }
   305    252   
   306         -#if 0
   307         -  if( iReplyStatus==200 && check_cache_control() ) {
   308         -    /* change the status to "unchanged" and we can skip sending the
   309         -    ** actual response body. Obviously we only do this when we _have_ a
   310         -    ** body (code 200).
   311         -    */
   312         -    iReplyStatus = 304;
   313         -    zReplyStatus = "Not Modified";
   314         -  }
   315         -#endif
   316         -
   317    253     if( g.fullHttpReply ){
   318    254       fprintf(g.httpOut, "HTTP/1.0 %d %s\r\n", iReplyStatus, zReplyStatus);
   319    255       fprintf(g.httpOut, "Date: %s\r\n", cgi_rfc822_datestamp(time(0)));
   320    256       fprintf(g.httpOut, "Connection: close\r\n");
   321    257       fprintf(g.httpOut, "X-UA-Compatible: IE=edge\r\n");
   322    258     }else{
   323    259       fprintf(g.httpOut, "Status: %d %s\r\n", iReplyStatus, zReplyStatus);
   324    260     }
          261  +  if( g.isConst ){
          262  +    /* isConst means that the reply is guaranteed to be invariant, even
          263  +    ** after configuration changes and/or Fossil binary recompiles. */
          264  +    fprintf(g.httpOut, "Cache-Control: max-age=31536000\r\n");
          265  +  }else if( etag_tag()!=0 ){
          266  +    fprintf(g.httpOut, "ETag: %s\r\n", etag_tag());
          267  +    fprintf(g.httpOut, "Cache-Control: max-age=%d\r\n", etag_maxage());
          268  +  }else{
          269  +    fprintf(g.httpOut, "Cache-control: no-cache\r\n");
          270  +  }
          271  +  if( etag_mtime()>0 ){
          272  +    fprintf(g.httpOut, "Last-Modified: %s\r\n",
          273  +            cgi_rfc822_datestamp(etag_mtime()));
          274  +  }
   325    275   
   326    276     if( blob_size(&extraHeader)>0 ){
   327    277       fprintf(g.httpOut, "%s", blob_buffer(&extraHeader));
   328    278     }
   329    279   
   330    280     /* Add headers to turn on useful security options in browsers. */
   331    281     fprintf(g.httpOut, "X-Frame-Options: SAMEORIGIN\r\n");
................................................................................
   341    291     ** deliberate inclusion of external resources, such as JavaScript syntax
   342    292     ** highlighter scripts.
   343    293     **
   344    294     ** These headers are probably best added by the web server hosting fossil as
   345    295     ** a CGI script.
   346    296     */
   347    297   
   348         -  if( g.isConst ){
   349         -    /* constant means that the input URL will _never_ generate anything
   350         -    ** else. In the case of attachments, the contents won't change because
   351         -    ** an attempt to change them generates a new attachment number. In the
   352         -    ** case of most /getfile calls for specific versions, the only way the
   353         -    ** content changes is if someone breaks the SCM. And if that happens, a
   354         -    ** stale cache is the least of the problem. So we provide an Expires
   355         -    ** header set to a reasonable period (default: one week).
   356         -    */
   357         -    fprintf(g.httpOut, "Cache-control: max-age=28800\r\n");
   358         -  }else{
   359         -    fprintf(g.httpOut, "Cache-control: no-cache\r\n");
   360         -  }
   361         -
   362    298     /* Content intended for logged in users should only be cached in
   363    299     ** the browser, not some shared location.
   364    300     */
   365    301     fprintf(g.httpOut, "Content-Type: %s; charset=utf-8\r\n", zContentType);
   366    302     if( fossil_strcmp(zContentType,"application/x-fossil")==0 ){
   367    303       cgi_combine_header_and_body();
   368    304       blob_compress(&cgiContent[0], &cgiContent[0]);
................................................................................
   617    553   **
   618    554   **      *  cookies and query parameters that have uppercase names
   619    555   **         are ignored.
   620    556   **
   621    557   **      *  it is impossible for a cookie or query parameter to
   622    558   **         override the value of an environment variable since
   623    559   **         environment variables always have uppercase names.
          560  +**
          561  +** 2018-03-29:  Also ignore the entry if NAME that contains any characters
          562  +** other than [a-zA-Z0-9_].  There are no known exploits involving unusual
          563  +** names that contain characters outside that set, but it never hurts to
          564  +** be extra cautious when sanitizing inputs.
   624    565   **
   625    566   ** Parameters are separated by the "terminator" character.  Whitespace
   626    567   ** before the NAME is ignored.
   627    568   **
   628    569   ** The input string "z" is modified but no copies is made.  "z"
   629    570   ** should not be deallocated or changed again after this routine
   630    571   ** returns or it will corrupt the parameter table.
................................................................................
   647    588           z++;
   648    589         }
   649    590         dehttpize(zValue);
   650    591       }else{
   651    592         if( *z ){ *z++ = 0; }
   652    593         zValue = "";
   653    594       }
   654         -    if( fossil_islower(zName[0]) ){
          595  +    if( fossil_islower(zName[0]) && fossil_no_strange_characters(zName+1) ){
   655    596         cgi_set_parameter_nocopy(zName, zValue, isQP);
   656    597       }
   657    598   #ifdef FOSSIL_ENABLE_JSON
   658    599       json_setenv( zName, cson_value_new_string(zValue,strlen(zValue)) );
   659    600   #endif /* FOSSIL_ENABLE_JSON */
   660    601     }
   661    602   }
................................................................................
  1975   1916   ** Parse an RFC822-formatted timestamp as we'd expect from HTTP and return
  1976   1917   ** a Unix epoch time. <= zero is returned on failure.
  1977   1918   **
  1978   1919   ** Note that this won't handle all the _allowed_ HTTP formats, just the
  1979   1920   ** most popular one (the one generated by cgi_rfc822_datestamp(), actually).
  1980   1921   */
  1981   1922   time_t cgi_rfc822_parsedate(const char *zDate){
  1982         -  struct tm t;
  1983         -  char zIgnore[16];
  1984         -  char zMonth[16];
  1985         -
  1986         -  memset(&t, 0, sizeof(t));
  1987         -  if( 7==sscanf(zDate, "%12[A-Za-z,] %d %12[A-Za-z] %d %d:%d:%d", zIgnore,
  1988         -                       &t.tm_mday, zMonth, &t.tm_year, &t.tm_hour, &t.tm_min,
  1989         -                       &t.tm_sec)){
  1990         -
  1991         -    if( t.tm_year > 1900 ) t.tm_year -= 1900;
  1992         -    for(t.tm_mon=0; azMonths[t.tm_mon]; t.tm_mon++){
  1993         -      if( !fossil_strnicmp( azMonths[t.tm_mon], zMonth, 3 )){
  1994         -        return mkgmtime(&t);
         1923  +  int mday, mon, year, yday, hour, min, sec;
         1924  +  char zIgnore[4];
         1925  +  char zMonth[4];
         1926  +  static const char *const azMonths[] =
         1927  +    {"Jan", "Feb", "Mar", "Apr", "May", "Jun",
         1928  +     "Jul", "Aug", "Sep", "Oct", "Nov", "Dec", 0};
         1929  +  if( 7==sscanf(zDate, "%3[A-Za-z], %d %3[A-Za-z] %d %d:%d:%d", zIgnore,
         1930  +                       &mday, zMonth, &year, &hour, &min, &sec)){
         1931  +    if( year > 1900 ) year -= 1900;
         1932  +    for(mon=0; azMonths[mon]; mon++){
         1933  +      if( !strncmp( azMonths[mon], zMonth, 3 )){
         1934  +        int nDay;
         1935  +        int isLeapYr;
         1936  +        static int priorDays[] =
         1937  +         {  0, 31, 59, 90,120,151,181,212,243,273,304,334 };
         1938  +        if( mon<0 ){
         1939  +          int nYear = (11 - mon)/12;
         1940  +          year -= nYear;
         1941  +          mon += nYear*12;
         1942  +        }else if( mon>11 ){
         1943  +          year += mon/12;
         1944  +          mon %= 12;
         1945  +        }
         1946  +        isLeapYr = year%4==0 && (year%100!=0 || (year+300)%400==0);
         1947  +        yday = priorDays[mon] + mday - 1;
         1948  +        if( isLeapYr && mon>1 ) yday++;
         1949  +        nDay = (year-70)*365 + (year-69)/4 - year/100 + (year+300)/400 + yday;
         1950  +        return ((time_t)(nDay*24 + hour)*60 + min)*60 + sec;
  1995   1951         }
  1996   1952       }
  1997   1953     }
  1998         -
  1999   1954     return 0;
  2000   1955   }
  2001   1956   
  2002         -/*
  2003         -** Convert a struct tm* that represents a moment in UTC into the number
  2004         -** of seconds in 1970, UTC.
  2005         -*/
  2006         -time_t mkgmtime(struct tm *p){
  2007         -  time_t t;
  2008         -  int nDay;
  2009         -  int isLeapYr;
  2010         -  /* Days in each month:       31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 */
  2011         -  static int priorDays[]   = {  0, 31, 59, 90,120,151,181,212,243,273,304,334 };
  2012         -  if( p->tm_mon<0 ){
  2013         -    int nYear = (11 - p->tm_mon)/12;
  2014         -    p->tm_year -= nYear;
  2015         -    p->tm_mon += nYear*12;
  2016         -  }else if( p->tm_mon>11 ){
  2017         -    p->tm_year += p->tm_mon/12;
  2018         -    p->tm_mon %= 12;
  2019         -  }
  2020         -  isLeapYr = p->tm_year%4==0 && (p->tm_year%100!=0 || (p->tm_year+300)%400==0);
  2021         -  p->tm_yday = priorDays[p->tm_mon] + p->tm_mday - 1;
  2022         -  if( isLeapYr && p->tm_mon>1 ) p->tm_yday++;
  2023         -  nDay = (p->tm_year-70)*365 + (p->tm_year-69)/4 -p->tm_year/100 +
  2024         -         (p->tm_year+300)/400 + p->tm_yday;
  2025         -  t = ((nDay*24 + p->tm_hour)*60 + p->tm_min)*60 + p->tm_sec;
  2026         -  return t;
  2027         -}
  2028         -
  2029   1957   /*
  2030   1958   ** Check the objectTime against the If-Modified-Since request header. If the
  2031   1959   ** object time isn't any newer than the header, we immediately send back
  2032   1960   ** a 304 reply and exit.
  2033   1961   */
  2034   1962   void cgi_modified_since(time_t objectTime){
  2035   1963     const char *zIf = P("HTTP_IF_MODIFIED_SINCE");

Changes to src/checkin.c.

   470    470     int showHdr = command==CHANGES && find_option("header", 0, 0);
   471    471     int verboseFlag = command==CHANGES && find_option("verbose", "v", 0);
   472    472     const char *zIgnoreFlag = find_option("ignore", 0, 1);
   473    473     unsigned scanFlags = 0;
   474    474     unsigned flags = 0;
   475    475     int vid, i;
   476    476   
   477         -  fossil_pledge("stdio rpath wpath cpath id flock tty chown");
          477  +  fossil_pledge("stdio rpath wpath cpath fattr id flock tty chown");
   478    478   
   479    479     /* Load affirmative flag options. */
   480    480     for( i=0; i<count(flagDefs); ++i ){
   481    481       if( (command==CHANGES || !(flagDefs[i].mask & C_CLASSIFY))
   482    482        && find_option(flagDefs[i].option, 0, 0) ){
   483    483         flags |= flagDefs[i].mask;
   484    484       }
................................................................................
   555    555     }
   556    556   
   557    557     /* Find and print all requested changes. */
   558    558     blob_zero(&report);
   559    559     status_report(&report, flags);
   560    560     if( blob_size(&report) ){
   561    561       if( showHdr ){
   562         -      fossil_print("Changes for %s at %s:\n", db_get("project-name", "???"),
   563         -                   g.zLocalRoot);
          562  +      fossil_print(
          563  +        "Changes for %s at %s:\n", db_get("project-name", "<unnamed>"),
          564  +        g.zLocalRoot);
   564    565       }
   565    566       blob_write_to_file(&report, "-");
   566    567     }else if( verboseFlag ){
   567    568       fossil_print("  (none)\n");
   568    569     }
   569    570     blob_reset(&report);
   570    571   
................................................................................
   863    864     locate_unmanaged_files(g.argc-2, g.argv+2, scanFlags, pIgnore);
   864    865     glob_free(pIgnore);
   865    866   
   866    867     blob_zero(&report);
   867    868     status_report(&report, flags);
   868    869     if( blob_size(&report) ){
   869    870       if( showHdr ){
   870         -      fossil_print("Extras for %s at %s:\n", db_get("project-name","???"),
          871  +      fossil_print("Extras for %s at %s:\n", db_get("project-name","<unnamed>"),
   871    872                      g.zLocalRoot);
   872    873       }
   873    874       blob_write_to_file(&report, "-");
   874    875     }
   875    876     blob_reset(&report);
   876    877   }
   877    878   

Changes to src/clone.c.

   315    315       }else{
   316    316         @ Contact the site administrator and ask them to give
   317    317         @ you "Download Zip" privileges.
   318    318       }
   319    319     }else{
   320    320       const char *zDLTag = db_get("download-tag","trunk");
   321    321       const char *zNm = db_get("short-project-name","download");
   322         -    char *zUrl = href("%R/zip/%t.zip?uuid=%t", zNm, zDLTag);
          322  +    char *zUrl = href("%R/zip/%t/%t.zip", zDLTag, zNm);
   323    323       @ <p>ZIP Archive: %z(zUrl)%h(zNm).zip</a>
   324         -    zUrl = href("%R/tarball/%t.tar.gz?uuid=%t", zNm, zDLTag);
          324  +    zUrl = href("%R/tarball/%t/%t.tar.gz", zDLTag, zNm);
   325    325       @ <p>Tarball: %z(zUrl)%h(zNm).tar.gz</a>
          326  +    zUrl = href("%R/sqlar/%t/%t.sqlar", zDLTag, zNm);
          327  +    @ <p>SQLite Archive: %z(zUrl)%h(zNm).sqlar</a>
   326    328     }
   327    329     if( !g.perm.Clone ){
   328    330       @ <p>You are not authorized to clone this repository.
   329    331       if( g.zLogin==0 || g.zLogin[0]==0 ){
   330    332         @ Maybe you would be able to clone if you
   331    333         @ <a href="../login">logged in</a>.
   332    334       }else{

Changes to src/cookies.c.

   171    171     cookie_readwrite(zQP, zPName, zDflt, COOKIE_READ|COOKIE_WRITE);
   172    172   }
   173    173   
   174    174   /* Update the user preferences cookie, if necessary, and shut down this
   175    175   ** module
   176    176   */
   177    177   void cookie_render(void){
   178         -  if( cookies.bChanged ){
          178  +  if( cookies.bChanged && P("udc")!=0 ){
   179    179       Blob new;
   180    180       int i;
   181    181       blob_init(&new, 0, 0);
   182    182       for(i=0;i<cookies.nParam;i++){
   183    183         if( i>0 ) blob_append(&new, ",", 1);
   184    184         blob_appendf(&new, "%s=%T",
   185    185             cookies.aParam[i].zPName, cookies.aParam[i].zPValue);

Changes to src/cson_amalgamation.c.

   320    320   #      pragma warning(disable:4127) /* conditional expression is constant */
   321    321   #   endif
   322    322   #endif
   323    323   
   324    324   
   325    325   #define true  1
   326    326   #define false 0
   327         -#define __   -1     /* the universal error code */
          327  +#define XX   -1     /* the universal error code */
   328    328   
   329    329   /* values chosen so that the object size is approx equal to one page (4K) */
   330    330   #ifndef JSON_PARSER_STACK_SIZE
   331    331   #   define JSON_PARSER_STACK_SIZE 128
   332    332   #endif
   333    333   
   334    334   #ifndef JSON_PARSER_PARSE_BUFFER_SIZE
................................................................................
   412    412   
   413    413   static const signed char ascii_class[128] = {
   414    414   /*
   415    415       This array maps the 128 ASCII characters into character classes.
   416    416       The remaining Unicode characters should be mapped to C_ETC.
   417    417       Non-whitespace control characters are errors.
   418    418   */
   419         -    __,      __,      __,      __,      __,      __,      __,      __,
   420         -    __,      C_WHITE, C_WHITE, __,      __,      C_WHITE, __,      __,
   421         -    __,      __,      __,      __,      __,      __,      __,      __,
   422         -    __,      __,      __,      __,      __,      __,      __,      __,
          419  +    XX,      XX,      XX,      XX,      XX,      XX,      XX,      XX,
          420  +    XX,      C_WHITE, C_WHITE, XX,      XX,      C_WHITE, XX,      XX,
          421  +    XX,      XX,      XX,      XX,      XX,      XX,      XX,      XX,
          422  +    XX,      XX,      XX,      XX,      XX,      XX,      XX,      XX,
   423    423   
   424    424       C_SPACE, C_ETC,   C_QUOTE, C_ETC,   C_ETC,   C_ETC,   C_ETC,   C_ETC,
   425    425       C_ETC,   C_ETC,   C_STAR,   C_PLUS, C_COMMA, C_MINUS, C_POINT, C_SLASH,
   426    426       C_ZERO,  C_DIGIT, C_DIGIT, C_DIGIT, C_DIGIT, C_DIGIT, C_DIGIT, C_DIGIT,
   427    427       C_DIGIT, C_DIGIT, C_COLON, C_ETC,   C_ETC,   C_ETC,   C_ETC,   C_ETC,
   428    428   
   429    429       C_ETC,   C_ABCDF, C_ABCDF, C_ABCDF, C_ABCDF, C_E,     C_ABCDF, C_ETC,
................................................................................
   446    446       OK,  /* ok       */
   447    447       OB,  /* object   */
   448    448       KE,  /* key      */
   449    449       CO,  /* colon    */
   450    450       VA,  /* value    */
   451    451       AR,  /* array    */
   452    452       ST,  /* string   */
   453         -    ES,  /* escape   */
          453  +    ESC,  /* escape   */
   454    454       U1,  /* u1       */
   455    455       U2,  /* u2       */
   456    456       U3,  /* u3       */
   457    457       U4,  /* u4       */
   458    458       MI,  /* minus    */
   459    459       ZE,  /* zero     */
   460    460       IT,  /* integer  */
................................................................................
   504    504       The state transition table takes the current state and the current symbol,
   505    505       and returns either a new state or an action. An action is represented as a
   506    506       negative number. A JSON text is accepted if at the end of the text the
   507    507       state is OK and if the mode is MODE_DONE.
   508    508   
   509    509                    white                                      1-9                                   ABCDF  etc
   510    510                space |  {  }  [  ]  :  ,  "  \  /  +  -  .  0  |  a  b  c  d  e  f  l  n  r  s  t  u  |  E  |  * */
   511         -/*start  GO*/ {GO,GO,-6,__,-5,__,__,__,__,__,CB,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   512         -/*ok     OK*/ {OK,OK,__,-8,__,-7,__,-3,__,__,CB,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   513         -/*object OB*/ {OB,OB,__,-9,__,__,__,__,SB,__,CB,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   514         -/*key    KE*/ {KE,KE,__,__,__,__,__,__,SB,__,CB,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   515         -/*colon  CO*/ {CO,CO,__,__,__,__,-2,__,__,__,CB,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   516         -/*value  VA*/ {VA,VA,-6,__,-5,__,__,__,SB,__,CB,__,MX,__,ZX,IX,__,__,__,__,__,FA,__,NU,__,__,TR,__,__,__,__,__},
   517         -/*array  AR*/ {AR,AR,-6,__,-5,-7,__,__,SB,__,CB,__,MX,__,ZX,IX,__,__,__,__,__,FA,__,NU,__,__,TR,__,__,__,__,__},
   518         -/*string ST*/ {ST,__,ST,ST,ST,ST,ST,ST,-4,EX,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST},
   519         -/*escape ES*/ {__,__,__,__,__,__,__,__,ST,ST,ST,__,__,__,__,__,__,ST,__,__,__,ST,__,ST,ST,__,ST,U1,__,__,__,__},
   520         -/*u1     U1*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,U2,U2,U2,U2,U2,U2,U2,U2,__,__,__,__,__,__,U2,U2,__,__},
   521         -/*u2     U2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,U3,U3,U3,U3,U3,U3,U3,U3,__,__,__,__,__,__,U3,U3,__,__},
   522         -/*u3     U3*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,U4,U4,U4,U4,U4,U4,U4,U4,__,__,__,__,__,__,U4,U4,__,__},
   523         -/*u4     U4*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,UC,UC,UC,UC,UC,UC,UC,UC,__,__,__,__,__,__,UC,UC,__,__},
   524         -/*minus  MI*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,ZE,IT,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   525         -/*zero   ZE*/ {OK,OK,__,-8,__,-7,__,-3,__,__,CB,__,__,DF,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   526         -/*int    IT*/ {OK,OK,__,-8,__,-7,__,-3,__,__,CB,__,__,DF,IT,IT,__,__,__,__,DE,__,__,__,__,__,__,__,__,DE,__,__},
   527         -/*frac   FR*/ {OK,OK,__,-8,__,-7,__,-3,__,__,CB,__,__,__,FR,FR,__,__,__,__,E1,__,__,__,__,__,__,__,__,E1,__,__},
   528         -/*e      E1*/ {__,__,__,__,__,__,__,__,__,__,__,E2,E2,__,E3,E3,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   529         -/*ex     E2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,E3,E3,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   530         -/*exp    E3*/ {OK,OK,__,-8,__,-7,__,-3,__,__,__,__,__,__,E3,E3,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   531         -/*tr     T1*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,T2,__,__,__,__,__,__,__},
   532         -/*tru    T2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,T3,__,__,__,__},
   533         -/*true   T3*/ {__,__,__,__,__,__,__,__,__,__,CB,__,__,__,__,__,__,__,__,__,OK,__,__,__,__,__,__,__,__,__,__,__},
   534         -/*fa     F1*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,F2,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   535         -/*fal    F2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,F3,__,__,__,__,__,__,__,__,__},
   536         -/*fals   F3*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,F4,__,__,__,__,__,__},
   537         -/*false  F4*/ {__,__,__,__,__,__,__,__,__,__,CB,__,__,__,__,__,__,__,__,__,OK,__,__,__,__,__,__,__,__,__,__,__},
   538         -/*nu     N1*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,N2,__,__,__,__},
   539         -/*nul    N2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,N3,__,__,__,__,__,__,__,__,__},
   540         -/*null   N3*/ {__,__,__,__,__,__,__,__,__,__,CB,__,__,__,__,__,__,__,__,__,__,__,OK,__,__,__,__,__,__,__,__,__},
   541         -/*/      C1*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,C2},
          511  +/*start  GO*/ {GO,GO,-6,XX,-5,XX,XX,XX,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          512  +/*ok     OK*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          513  +/*object OB*/ {OB,OB,XX,-9,XX,XX,XX,XX,SB,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          514  +/*key    KE*/ {KE,KE,XX,XX,XX,XX,XX,XX,SB,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          515  +/*colon  CO*/ {CO,CO,XX,XX,XX,XX,-2,XX,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          516  +/*value  VA*/ {VA,VA,-6,XX,-5,XX,XX,XX,SB,XX,CB,XX,MX,XX,ZX,IX,XX,XX,XX,XX,XX,FA,XX,NU,XX,XX,TR,XX,XX,XX,XX,XX},
          517  +/*array  AR*/ {AR,AR,-6,XX,-5,-7,XX,XX,SB,XX,CB,XX,MX,XX,ZX,IX,XX,XX,XX,XX,XX,FA,XX,NU,XX,XX,TR,XX,XX,XX,XX,XX},
          518  +/*string ST*/ {ST,XX,ST,ST,ST,ST,ST,ST,-4,EX,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST,ST},
          519  +/*escape ES*/ {XX,XX,XX,XX,XX,XX,XX,XX,ST,ST,ST,XX,XX,XX,XX,XX,XX,ST,XX,XX,XX,ST,XX,ST,ST,XX,ST,U1,XX,XX,XX,XX},
          520  +/*u1     U1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,U2,U2,U2,U2,U2,U2,U2,U2,XX,XX,XX,XX,XX,XX,U2,U2,XX,XX},
          521  +/*u2     U2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,U3,U3,U3,U3,U3,U3,U3,U3,XX,XX,XX,XX,XX,XX,U3,U3,XX,XX},
          522  +/*u3     U3*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,U4,U4,U4,U4,U4,U4,U4,U4,XX,XX,XX,XX,XX,XX,U4,U4,XX,XX},
          523  +/*u4     U4*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,UC,UC,UC,UC,UC,UC,UC,UC,XX,XX,XX,XX,XX,XX,UC,UC,XX,XX},
          524  +/*minus  MI*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,ZE,IT,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          525  +/*zero   ZE*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,CB,XX,XX,DF,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          526  +/*int    IT*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,CB,XX,XX,DF,IT,IT,XX,XX,XX,XX,DE,XX,XX,XX,XX,XX,XX,XX,XX,DE,XX,XX},
          527  +/*frac   FR*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,CB,XX,XX,XX,FR,FR,XX,XX,XX,XX,E1,XX,XX,XX,XX,XX,XX,XX,XX,E1,XX,XX},
          528  +/*e      E1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,E2,E2,XX,E3,E3,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          529  +/*ex     E2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,E3,E3,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          530  +/*exp    E3*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,XX,XX,XX,XX,E3,E3,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          531  +/*tr     T1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,T2,XX,XX,XX,XX,XX,XX,XX},
          532  +/*tru    T2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,T3,XX,XX,XX,XX},
          533  +/*true   T3*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,OK,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          534  +/*fa     F1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,F2,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          535  +/*fal    F2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,F3,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          536  +/*fals   F3*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,F4,XX,XX,XX,XX,XX,XX},
          537  +/*false  F4*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,OK,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          538  +/*nu     N1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,N2,XX,XX,XX,XX},
          539  +/*nul    N2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,N3,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          540  +/*null   N3*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,CB,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,OK,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          541  +/*/      C1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,C2},
   542    542   /*/star  C2*/ {C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C3},
   543    543   /**      C3*/ {C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,CE,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C2,C3},
   544         -/*_.     FX*/ {OK,OK,__,-8,__,-7,__,-3,__,__,__,__,__,__,FR,FR,__,__,__,__,E1,__,__,__,__,__,__,__,__,E1,__,__},
   545         -/*\      D1*/ {__,__,__,__,__,__,__,__,__,D2,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__},
   546         -/*\      D2*/ {__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,__,U1,__,__,__,__},
          544  +/*_.     FX*/ {OK,OK,XX,-8,XX,-7,XX,-3,XX,XX,XX,XX,XX,XX,FR,FR,XX,XX,XX,XX,E1,XX,XX,XX,XX,XX,XX,XX,XX,E1,XX,XX},
          545  +/*\      D1*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,D2,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX},
          546  +/*\      D2*/ {XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,XX,U1,XX,XX,XX,XX},
   547    547   };
   548    548   
   549    549   
   550    550   /*
   551    551       These modes can be pushed on the stack.
   552    552   */
   553    553   enum modes {
................................................................................
  1048   1048           jc->error = JSON_E_INVALID_CHAR;
  1049   1049           return false;
  1050   1050       }
  1051   1051       if (next_char >= 128) {
  1052   1052           next_class = C_ETC;
  1053   1053       } else {
  1054   1054           next_class = ascii_class[next_char];
  1055         -        if (next_class <= __) {
         1055  +        if (next_class <= XX) {
  1056   1056               set_error(jc);
  1057   1057               return false;
  1058   1058           }
  1059   1059       }
  1060   1060   
  1061   1061       if (!add_char_to_parse_buffer(jc, next_char, next_class)) {
  1062   1062           return false;
................................................................................
  1088   1088               } else {
  1089   1089                   jc->state = ST;
  1090   1090               }
  1091   1091               break;
  1092   1092   /* escaped char */
  1093   1093           case EX:
  1094   1094               jc->escaped = 1;
  1095         -            jc->state = ES;
         1095  +            jc->state = ESC;
  1096   1096               break;
  1097   1097   /* integer detected by minus */
  1098   1098           case MX:
  1099   1099               jc->type = JSON_T_INTEGER;
  1100   1100               jc->state = MI;
  1101   1101               break;
  1102   1102   /* integer detected by zero */
................................................................................
  1399   1399   
  1400   1400           config->depth = JSON_PARSER_STACK_SIZE - 1;
  1401   1401           config->malloc = malloc;
  1402   1402           config->free = free;
  1403   1403       }
  1404   1404   }
  1405   1405   
         1406  +#undef XX
         1407  +#undef COUNTOF
         1408  +#undef parse_buffer_clear
         1409  +#undef parse_buffer_pop_back_char
  1406   1410   /* end file parser/JSON_parser.c */
  1407   1411   /* begin file ./cson.c */
  1408   1412   #include <assert.h>
  1409   1413   #include <stdlib.h> /* malloc()/free() */
  1410   1414   #include <string.h>
  1411   1415   #include <errno.h>
  1412   1416   

Changes to src/cson_amalgamation.h.

  1285   1285   /**
  1286   1286      Returns the special JSON "null" value. When outputing JSON,
  1287   1287      its string representation is "null" (without the quotes).
  1288   1288      
  1289   1289      See cson_value_new_bool() for notes regarding the returned
  1290   1290      value's memory.
  1291   1291   */
  1292         -cson_value * cson_value_null();
         1292  +cson_value * cson_value_null( void );
  1293   1293   
  1294   1294   /**
  1295   1295      Equivalent to cson_value_new_bool(1).
  1296   1296   */
  1297         -cson_value * cson_value_true();
         1297  +cson_value * cson_value_true( void );
  1298   1298   
  1299   1299   /**
  1300   1300      Equivalent to cson_value_new_bool(0).
  1301   1301   */
  1302         -cson_value * cson_value_false();
         1302  +cson_value * cson_value_false( void );
  1303   1303   
  1304   1304   /**
  1305   1305      Semantically the same as cson_value_new_bool(), but for integers.
  1306   1306   */
  1307   1307   cson_value * cson_value_new_integer( cson_int_t v );
  1308   1308   
  1309   1309   /**
................................................................................
  1344   1344      Returns NULL on allocation error.
  1345   1345   
  1346   1346      Post-conditions: cson_value_is_object(value) will return true.
  1347   1347   
  1348   1348      @see cson_value_new_array()
  1349   1349      @see cson_value_free()
  1350   1350   */
  1351         -cson_value * cson_value_new_object();
         1351  +cson_value * cson_value_new_object( void );
  1352   1352   
  1353   1353   /**
  1354   1354      This works like cson_value_new_object() but returns an Object
  1355   1355      handle directly.
  1356   1356   
  1357   1357      The value handle for the returned object can be fetched with
  1358   1358      cson_object_value(theObject).
................................................................................
  1359   1359      
  1360   1360      Ownership is transfered to the caller, who must eventually free it
  1361   1361      by passing the Value handle (NOT the Object handle) to
  1362   1362      cson_value_free() or passing ownership to a parent container.
  1363   1363   
  1364   1364      Returns NULL on error (out of memory).
  1365   1365   */
  1366         -cson_object * cson_new_object();
         1366  +cson_object * cson_new_object( void );
  1367   1367   
  1368   1368   /**
  1369   1369      Identical to cson_new_object() except that it creates
  1370   1370      an Array.
  1371   1371   */
  1372         -cson_array * cson_new_array();
         1372  +cson_array * cson_new_array( void );
  1373   1373   
  1374   1374   /**
  1375   1375      Identical to cson_new_object() except that it creates
  1376   1376      a String.
  1377   1377   */
  1378   1378   cson_string * cson_new_string(char const * val, unsigned int len);
  1379   1379   
................................................................................
  1401   1401      Returns NULL on allocation error.
  1402   1402   
  1403   1403      Post-conditions: cson_value_is_array(value) will return true.
  1404   1404   
  1405   1405      @see cson_value_new_object()
  1406   1406      @see cson_value_free()
  1407   1407   */
  1408         -cson_value * cson_value_new_array();
         1408  +cson_value * cson_value_new_array( void );
  1409   1409   
  1410   1410   /**
  1411   1411      Frees any resources owned by v, then frees v. If v is a container
  1412   1412      type (object or array) its children are also freed (recursively).
  1413   1413   
  1414   1414      If v is NULL, this is a no-op.
  1415   1415   

Changes to src/db.c.

  1090   1090   ** connection.  An error results in process abort.
  1091   1091   */
  1092   1092   LOCAL sqlite3 *db_open(const char *zDbName){
  1093   1093     int rc;
  1094   1094     sqlite3 *db;
  1095   1095   
  1096   1096     if( g.fSqlTrace ) fossil_trace("-- sqlite3_open: [%s]\n", zDbName);
         1097  +  if( strcmp(zDbName, g.nameOfExe)==0 ){
         1098  +    extern int sqlite3_appendvfs_init(
         1099  +      sqlite3 *, char **, const sqlite3_api_routines *
         1100  +    );
         1101  +    sqlite3_appendvfs_init(0,0,0);
         1102  +    g.zVfsName = "apndvfs";
         1103  +  }
  1097   1104     rc = sqlite3_open_v2(
  1098   1105          zDbName, &db,
  1099   1106          SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE,
  1100   1107          g.zVfsName
  1101   1108     );
  1102   1109     if( rc!=SQLITE_OK ){
  1103   1110       db_err("[%s]: %s", zDbName, sqlite3_errmsg(db));
................................................................................
  2839   2846   ** object to which the symbolic link points.
  2840   2847   */
  2841   2848   #endif
  2842   2849   /*
  2843   2850   ** SETTING: auto-captcha    boolean default=on variable=autocaptcha
  2844   2851   ** If enabled, the /login page provides a button that will automatically
  2845   2852   ** fill in the captcha password.  This makes things easier for human users,
  2846         -** at the expense of also making logins easier for malecious robots.
         2853  +** at the expense of also making logins easier for malicious robots.
  2847   2854   */
  2848   2855   /*
  2849   2856   ** SETTING: auto-hyperlink  boolean default=on
  2850   2857   ** Use javascript to enable hyperlinks on web pages
  2851   2858   ** for all users (regardless of the "h" privilege) if the
  2852   2859   ** User-Agent string in the HTTP header look like it came
  2853   2860   ** from real person, not a spider or bot.
................................................................................
  2912   2919   ** The value is a comma or newline-separated list of GLOB patterns for
  2913   2920   ** text files in which it is ok to have CR, CR+LF or mixed
  2914   2921   ** line endings. Set to "*" to disable CR+LF checking.
  2915   2922   ** The crnl-glob setting is a compatibility alias.
  2916   2923   */
  2917   2924   /*
  2918   2925   ** SETTING: crnl-glob       width=40 versionable block-text
  2919         -** This is an alias for the crlf-glob setting
         2926  +** This is an alias for the crlf-glob setting.
  2920   2927   */
  2921   2928   /*
  2922   2929   ** SETTING: default-perms   width=16 default=u
  2923   2930   ** Permissions given automatically to new users.  For more
  2924   2931   ** information on permissions see the Users page in Server
  2925   2932   ** Administration of the HTTP UI.
  2926   2933   */
................................................................................
  3001   3008   ** SETTING: http-port        width=16 default=8080
  3002   3009   ** The default TCP/IP port number to use by the "server"
  3003   3010   ** and "ui" commands.
  3004   3011   */
  3005   3012   /*
  3006   3013   ** SETTING: https-login      boolean default=off
  3007   3014   ** If true, then the Fossil web server will redirect unencrypted
  3008         -** login screeen requests to HTTPS.
         3015  +** login screen requests to HTTPS.
  3009   3016   */
  3010   3017   /*
  3011   3018   ** SETTING: ignore-glob      width=40 versionable block-text
  3012   3019   ** The value is a comma or newline-separated list of GLOB
  3013   3020   ** patterns specifying files that the "add", "addremove",
  3014   3021   ** "clean", and "extras" commands will ignore.
  3015   3022   **
  3016   3023   ** Example:  *.log customCode.c notes.txt
  3017   3024   */
  3018   3025   /*
  3019   3026   ** SETTING: keep-glob        width=40 versionable block-text
  3020   3027   ** The value is a comma or newline-separated list of GLOB
  3021         -** patterns specifying files that the "clean" command will keep
         3028  +** patterns specifying files that the "clean" command will keep.
  3022   3029   */
  3023   3030   /*
  3024   3031   ** SETTING: localauth        boolean default=off
  3025   3032   ** If enabled, require that HTTP connections from
  3026   3033   ** 127.0.0.1 be authenticated by password.  If
  3027   3034   ** false, all HTTP requests from localhost have
  3028   3035   ** unrestricted access to the repository.

Changes to src/diff.c.

  2348   2348   ** check-in).  /blame and /praise also show the user who made the check-in.
  2349   2349   **
  2350   2350   ** Reverse Annotations:  Normally, these web pages look at versions of
  2351   2351   ** FILENAME moving backwards in time back toward the root check-in.  However,
  2352   2352   ** if the origin= query parameter is used to specify some future check-in
  2353   2353   ** (example: "origin=trunk") then these pages show changes moving towards
  2354   2354   ** that alternative origin.  Thus using "origin=trunk" on an historical
  2355         -** version of the file shows the first time each line in the file was been
  2356         -** changed in subsequent check-ins.
         2355  +** version of the file shows the first time each line in the file was changed
         2356  +** or removed by any subsequent check-in.
  2357   2357   **
  2358   2358   ** Query parameters:
  2359   2359   **
  2360   2360   **    checkin=ID          The check-in at which to start the annotation
  2361   2361   **    filename=FILENAME   The filename.
  2362   2362   **    filevers=BOOLEAN    Show file versions rather than check-in versions
  2363   2363   **    limit=LIMIT         Limit the amount of analysis:
................................................................................
  2366   2366   **                           "N"     N versions
  2367   2367   **    log=BOOLEAN         Show a log of versions analyzed
  2368   2368   **    origin=ID           The origin checkin.  If unspecified, the root
  2369   2369   **                           check-in over the entire repository is used.
  2370   2370   **                           Specify "origin=trunk" or similar for a reverse
  2371   2371   **                           annotation
  2372   2372   **    w=BOOLEAN           Ignore whitespace
  2373         -**
  2374   2373   */
  2375   2374   void annotation_page(void){
  2376   2375     int i;
  2377   2376     const char *zLimit;    /* Depth limit */
  2378   2377     u64 annFlags = DIFF_STRIP_EOLCR;
  2379   2378     int showLog;           /* True to display the log */
  2380   2379     int fileVers;          /* Show file version instead of check-in versions */
................................................................................
  2536   2535   **
  2537   2536   ** Reverse Annotations:  Normally, these commands look at versions of
  2538   2537   ** FILENAME moving backwards in time back toward the root check-in, and
  2539   2538   ** thus the output shows the most recent change to each line.  However,
  2540   2539   ** if the -o|--origin option is used to specify some future check-in
  2541   2540   ** (example: "-o trunk") then these commands show changes moving towards
  2542   2541   ** that alternative origin.  Thus using "-o trunk" on an historical version
  2543         -** of the file shows the first time each line in the file was been changed
  2544         -** by subsequent check-ins.
         2542  +** of the file shows the first time each line in the file was changed or
         2543  +** removed by any subsequent check-in.
  2545   2544   **
  2546   2545   ** Options:
  2547   2546   **   --filevers                  Show file version numbers rather than
  2548   2547   **                               check-in versions
  2549   2548   **   -r|--revision VERSION       The specific check-in containing the file
  2550   2549   **   -l|--log                    List all versions analyzed
  2551   2550   **   -n|--limit LIMIT            Limit the amount of analysis:

Changes to src/doc.c.

   637    637             goto doc_not_found;
   638    638           }
   639    639         }else{
   640    640           goto doc_not_found;
   641    641         }
   642    642       }
   643    643       if( isUV ){
   644         -      if( db_table_exists("repository","unversioned")
   645         -       && unversioned_content(zName, &filebody)==0
   646         -      ){
   647         -        rid = 1;
   648         -        zDfltTitle = zName;
          644  +      if( db_table_exists("repository","unversioned") ){
          645  +        Stmt q;
          646  +        db_prepare(&q, "SELECT hash, mtime FROM unversioned"
          647  +                       " WHERE name=%Q", zName);
          648  +        if( db_step(&q)==SQLITE_ROW ){
          649  +          etag_check(ETAG_HASH, db_column_text(&q,0));
          650  +          etag_last_modified(db_column_int64(&q,1));
          651  +        }
          652  +        db_finalize(&q);
          653  +        if( unversioned_content(zName, &filebody)==0 ){
          654  +          rid = 1;
          655  +          zDfltTitle = zName;
          656  +        }
   649    657         }
   650    658       }else if( fossil_strcmp(zCheckin,"ckout")==0 ){
   651    659         /* Read from the local checkout */
   652    660         char *zFullpath;
   653    661         db_must_be_within_tree();
   654    662         zFullpath = mprintf("%s/%s", g.zLocalRoot, zName);
   655    663         if( file_isfile(zFullpath, RepoFILE)
................................................................................
   837    845   ** the login page.  It is designed for use in the upper left-hand corner
   838    846   ** of the header.
   839    847   */
   840    848   void logo_page(void){
   841    849     Blob logo;
   842    850     char *zMime;
   843    851   
          852  +  etag_check(ETAG_CONFIG, 0);
   844    853     zMime = db_get("logo-mimetype", "image/gif");
   845    854     blob_zero(&logo);
   846    855     db_blob(&logo, "SELECT value FROM config WHERE name='logo-image'");
   847    856     if( blob_size(&logo)==0 ){
   848    857       blob_init(&logo, (char*)aLogo, sizeof(aLogo));
   849    858     }
   850    859     cgi_set_content_type(zMime);
   851    860     cgi_set_content(&logo);
   852         -  g.isConst = 1;
   853    861   }
   854    862   
   855    863   /*
   856    864   ** The default background image:  a 16x16 white GIF
   857    865   */
   858    866   static const unsigned char aBackground[] = {
   859    867       71,  73,  70,  56,  57,  97,  16,   0,  16,   0,
................................................................................
   871    879   ** Return the background image.  If no background image is defined, a
   872    880   ** built-in 16x16 pixel white GIF is returned.
   873    881   */
   874    882   void background_page(void){
   875    883     Blob bgimg;
   876    884     char *zMime;
   877    885   
          886  +  etag_check(ETAG_CONFIG, 0);
   878    887     zMime = db_get("background-mimetype", "image/gif");
   879    888     blob_zero(&bgimg);
   880    889     db_blob(&bgimg, "SELECT value FROM config WHERE name='background-image'");
   881    890     if( blob_size(&bgimg)==0 ){
   882    891       blob_init(&bgimg, (char*)aBackground, sizeof(aBackground));
   883    892     }
   884    893     cgi_set_content_type(zMime);
   885    894     cgi_set_content(&bgimg);
   886         -  g.isConst = 1;
   887    895   }
   888    896   
   889    897   
   890    898   /*
   891    899   ** WEBPAGE: docsrch
   892    900   **
   893    901   ** Search for documents that match a user-supplied full-text search pattern.

Added src/etag.c.

            1  +/*
            2  +** Copyright (c) 2018 D. Richard Hipp
            3  +**
            4  +** This program is free software; you can redistribute it and/or
            5  +** modify it under the terms of the Simplified BSD License (also
            6  +** known as the "2-Clause License" or "FreeBSD License".)
            7  +**
            8  +** This program is distributed in the hope that it will be useful,
            9  +** but without any warranty; without even the implied warranty of
           10  +** merchantability or fitness for a particular purpose.
           11  +**
           12  +** Author contact information:
           13  +**   drh@hwaci.com
           14  +**   http://www.hwaci.com/drh/
           15  +**
           16  +*******************************************************************************
           17  +**
           18  +** This file implements ETags: cache control for Fossil
           19  +**
           20  +** An ETag is a hash that encodes attributes which must be the same for
           21  +** the page to continue to be valid.  Attributes that might be contained
           22  +** in the ETag include:
           23  +**
           24  +**   (1)  The mtime on the Fossil executable
           25  +**   (2)  The last change to the CONFIG table
           26  +**   (3)  The last change to the EVENT table
           27  +**   (4)  The value of the display cookie
           28  +**   (5)  A hash value supplied by the page generator
           29  +**
           30  +** Item (1) is always included in the ETag.  The other elements are
           31  +** optional.  Because (1) is always included as part of the ETag, all
           32  +** outstanding ETags can be invalidated by touching the fossil executable.
           33  +**
           34  +** A page generator routine invokes etag_check() exactly once, with
           35  +** arguments that indicates which of the above elements to include in the
           36  +** hash.  If the request contained an If-None-Match header which matches
           37  +** the generated ETag, then a 304 Not Modified reply is generated and
           38  +** the process exits.  In other words, etag_check() never returns.  But
           39  +** if there is no If-None_Match header or if the ETag does not match,
           40  +** then etag_check() returns normally.  Later, during reply generation,
           41  +** the cgi.c module will invoke etag_tag() to recover the generated tag
           42  +** and include it in the reply header.
           43  +**
           44  +** 2018-02-25:
           45  +**
           46  +** Also support Last-Modified: and If-Modified-Since:.  The
           47  +** etag_last_modified(mtime) API records a timestamp for the page in
           48  +** seconds since 1970.  This causes a Last-Modified: header to be
           49  +** issued in the reply.  Or, if the request contained If-Modified-Since:
           50  +** and the new mtime is not greater than the mtime associated with
           51  +** If-Modified-Since, then a 304 Not Modified reply is generated and
           52  +** the etag_last_modified() API never returns.
           53  +*/
           54  +#include "config.h"
           55  +#include "etag.h"
           56  +
           57  +#if INTERFACE
           58  +/*
           59  +** Things to monitor
           60  +*/
           61  +#define ETAG_CONFIG   0x01 /* Output depends on the CONFIG table */
           62  +#define ETAG_DATA     0x02 /* Output depends on the EVENT table */
           63  +#define ETAG_COOKIE   0x04 /* Output depends on a display cookie value */
           64  +#define ETAG_HASH     0x08 /* Output depends on a hash */
           65  +#endif
           66  +
           67  +static char zETag[33];      /* The generated ETag */
           68  +static int iMaxAge = 0;     /* The max-age parameter in the reply */
           69  +static sqlite3_int64 iEtagMtime = 0;  /* Last-Modified time */
           70  +
           71  +/*
           72  +** Generate an ETag
           73  +*/
           74  +void etag_check(unsigned eFlags, const char *zHash){
           75  +  sqlite3_int64 mtime;
           76  +  const char *zIfNoneMatch;
           77  +  char zBuf[50];
           78  +  assert( zETag[0]==0 );  /* Only call this routine once! */
           79  +
           80  +  iMaxAge = 86400;
           81  +  md5sum_init();
           82  +
           83  +  /* Always include the mtime of the executable as part of the hash */
           84  +  mtime = file_mtime(g.nameOfExe, ExtFILE);
           85  +  sqlite3_snprintf(sizeof(zBuf),zBuf,"mtime: %lld\n", mtime);
           86  +  md5sum_step_text(zBuf, -1);
           87  +  
           88  +  if( (eFlags & ETAG_HASH)!=0 && zHash ){
           89  +    md5sum_step_text("hash: ", -1);
           90  +    md5sum_step_text(zHash, -1);
           91  +    md5sum_step_text("\n", 1);
           92  +    iMaxAge = 0;
           93  +  }else if( eFlags & ETAG_DATA ){
           94  +    int iKey = db_int(0, "SELECT max(rcvid) FROM rcvfrom");
           95  +    sqlite3_snprintf(sizeof(zBuf),zBuf,"%d",iKey);
           96  +    md5sum_step_text("data: ", -1);
           97  +    md5sum_step_text(zBuf, -1);
           98  +    md5sum_step_text("\n", 1);
           99  +    iMaxAge = 60;
          100  +  }else if( eFlags & ETAG_CONFIG ){
          101  +    int iKey = db_int(0, "SELECT value FROM config WHERE name='cfgcnt'");
          102  +    sqlite3_snprintf(sizeof(zBuf),zBuf,"%d",iKey);
          103  +    md5sum_step_text("data: ", -1);
          104  +    md5sum_step_text(zBuf, -1);
          105  +    md5sum_step_text("\n", 1);
          106  +    iMaxAge = 3600;
          107  +  }
          108  +
          109  +  /* Include the display cookie */
          110  +  if( eFlags & ETAG_COOKIE ){
          111  +    md5sum_step_text("display-cookie: ", -1);
          112  +    md5sum_step_text(PD(DISPLAY_SETTINGS_COOKIE,""), -1);
          113  +    md5sum_step_text("\n", 1);
          114  +    iMaxAge = 0;
          115  +  }
          116  +
          117  +  /* Generate the ETag */
          118  +  memcpy(zETag, md5sum_finish(0), 33);
          119  +
          120  +  /* Check to see if the generated ETag matches If-None-Match and
          121  +  ** generate a 304 reply if it does. */
          122  +  zIfNoneMatch = P("HTTP_IF_NONE_MATCH");
          123  +  if( zIfNoneMatch==0 ) return;
          124  +  if( strcmp(zIfNoneMatch,zETag)!=0 ) return;
          125  +
          126  +  /* If we get this far, it means that the content has
          127  +  ** not changed and we can do a 304 reply */
          128  +  cgi_reset_content();
          129  +  cgi_set_status(304, "Not Modified");
          130  +  cgi_reply();
          131  +  fossil_exit(0);
          132  +}
          133  +
          134  +/*
          135  +** Accept a new Last-Modified time.  This routine should be called by
          136  +** page generators that know a valid last-modified time.  This routine
          137  +** might generate a 304 Not Modified reply and exit(), never returning.
          138  +** Or, if not, it will cause a Last-Modified: header to be included in the
          139  +** reply.
          140  +*/
          141  +void etag_last_modified(sqlite3_int64 mtime){
          142  +  const char *zIfModifiedSince;
          143  +  sqlite3_int64 x;
          144  +  assert( iEtagMtime==0 );   /* Only call this routine once */
          145  +  assert( mtime>0 );         /* Only call with a valid mtime */
          146  +  iEtagMtime = mtime;
          147  +
          148  +  /* Check to see the If-Modified-Since constraint is satisfied */
          149  +  zIfModifiedSince = P("HTTP_IF_MODIFIED_SINCE");
          150  +  if( zIfModifiedSince==0 ) return;
          151  +  x = cgi_rfc822_parsedate(zIfModifiedSince);
          152  +  if( x<=0 || x>mtime ) return;
          153  +
          154  +#if 0  
          155  +  /* If the Fossil executable is more recent than If-Modified-Since,
          156  +  ** go ahead and regenerate the resource. */
          157  +  if( file_mtime(g.nameOfExe, ExtFILE)>x ) return;
          158  +#endif
          159  +
          160  +  /* If we reach this point, it means that the resource has not changed
          161  +  ** and that we should generate a 304 Not Modified reply */
          162  +  cgi_reset_content();
          163  +  cgi_set_status(304, "Not Modified");
          164  +  cgi_reply();
          165  +  fossil_exit(0);
          166  +}
          167  +
          168  +/* Return the ETag, if there is one.
          169  +*/
          170  +const char *etag_tag(void){
          171  +  return zETag;
          172  +}
          173  +
          174  +/* Return the recommended max-age
          175  +*/
          176  +int etag_maxage(void){
          177  +  return iMaxAge;
          178  +}
          179  +
          180  +/* Return the last-modified time in seconds since 1970.  Or return 0 if
          181  +** there is no last-modified time.
          182  +*/
          183  +sqlite3_int64 etag_mtime(void){
          184  +  return iEtagMtime;
          185  +}
          186  +
          187  +/* 
          188  +** COMMAND: test-etag
          189  +**
          190  +** Usage:  fossil test-etag -key KEY-NUMBER  -hash HASH
          191  +**
          192  +** Generate an etag given a KEY-NUMBER and/or a HASH.
          193  +**
          194  +** KEY-NUMBER is some combination of:
          195  +**
          196  +**    1   ETAG_CONFIG   The config table version number
          197  +**    2   ETAG_DATA     The event table version number
          198  +**    4   ETAG_COOKIE   The display cookie
          199  +*/
          200  +void test_etag_cmd(void){
          201  +  const char *zHash = 0;
          202  +  const char *zKey;
          203  +  int iKey = 0;
          204  +  db_find_and_open_repository(0, 0);
          205  +  zKey = find_option("key",0,1);
          206  +  zHash = find_option("hash",0,1);
          207  +  if( zKey ) iKey = atoi(zKey);
          208  +  etag_check(iKey, zHash);
          209  +  fossil_print("%s\n", etag_tag());
          210  +}

Changes to src/export.c.

   598    598     }
   599    599     db_finalize(&q);
   600    600     db_finalize(&q2);
   601    601     db_finalize(&q3);
   602    602   
   603    603     /* Output the commit records.
   604    604     */
          605  +  topological_sort_checkins(0);
   605    606     db_prepare(&q,
   606    607       "SELECT strftime('%%s',mtime), objid, coalesce(ecomment,comment),"
   607    608       "       coalesce(euser,user),"
   608    609       "       (SELECT value FROM tagxref WHERE rid=objid AND tagid=%d)"
   609         -    "  FROM event"
   610         -    " WHERE type='ci' AND NOT EXISTS (SELECT 1 FROM oldcommit WHERE objid=rid)"
   611         -    " ORDER BY mtime ASC",
          610  +    "  FROM toponode, event"
          611  +    " WHERE toponode.tid=event.objid"
          612  +    "   AND event.type='ci'"
          613  +    "   AND NOT EXISTS (SELECT 1 FROM oldcommit WHERE toponode.tid=rid)"
          614  +    " ORDER BY toponode.tseq ASC",
   612    615       TAG_BRANCH
   613    616     );
   614    617     db_prepare(&q2, "INSERT INTO oldcommit VALUES (:rid)");
   615    618     while( db_step(&q)==SQLITE_ROW ){
   616    619       Stmt q4;
   617    620       const char *zSecondsSince1970 = db_column_text(&q, 0);
   618    621       int ckinId = db_column_int(&q, 1);
................................................................................
   621    624       const char *zBranch = db_column_text(&q, 4);
   622    625       char *zMark;
   623    626   
   624    627       bag_insert(&vers, ckinId);
   625    628       db_bind_int(&q2, ":rid", ckinId);
   626    629       db_step(&q2);
   627    630       db_reset(&q2);
   628         -    if( zBranch==0 || fossil_strcmp(zBranch, "trunk")==0 ) zBranch = gexport.zTrunkName;
          631  +    if( zBranch==0 || fossil_strcmp(zBranch, "trunk")==0 ){
          632  +      zBranch = gexport.zTrunkName;
          633  +    }
   629    634       zMark = mark_name_from_rid(ckinId, &unused_mark);
   630    635       printf("commit refs/heads/");
   631    636       print_ref(zBranch);
   632    637       printf("\nmark %s\n", zMark);
   633    638       free(zMark);
   634    639       printf("committer");
   635    640       print_person(zUser);
................................................................................
   735    740       if( ferror(f)!=0 || fclose(f)!=0 ){
   736    741         fossil_fatal("error while writing %s", markfile_out);
   737    742       }
   738    743     }
   739    744     bag_clear(&blobs);
   740    745     bag_clear(&vers);
   741    746   }
          747  +
          748  +/*
          749  +** Construct the temporary table toposort as follows:
          750  +**
          751  +**     CREATE TEMP TABLE toponode(
          752  +**        tid INTEGER PRIMARY KEY,   -- Check-in id
          753  +**        tseq INT                   -- integer total order on check-ins.
          754  +**     );
          755  +**
          756  +** This table contains all check-ins of the repository in topological
          757  +** order.  "Topological order" means that every parent check-in comes
          758  +** before all of its children.  Topological order is *almost* the same
          759  +** thing as "ORDER BY event.mtime".  Differences only arrise when there
          760  +** are timewarps.  In as much as Git hates timewarps, we have to compute
          761  +** a correct topological order when doing an export.
          762  +**
          763  +** Since mtime is a usually already nearly in topological order, the
          764  +** algorithm is to start with mtime, then make adjustments as necessary
          765  +** for timewarps.  This is not a great algorithm for the general case,
          766  +** but it is very fast for the overwhelmingly common case where there
          767  +** are few timewarps.
          768  +*/
          769  +int topological_sort_checkins(int bVerbose){
          770  +  int nChange = 0;
          771  +  Stmt q1;
          772  +  Stmt chng;
          773  +  db_multi_exec(
          774  +    "CREATE TEMP TABLE toponode(\n"
          775  +    "  tid INTEGER PRIMARY KEY,\n"
          776  +    "  tseq INT\n"
          777  +    ");\n"
          778  +    "INSERT INTO toponode(tid,tseq) "
          779  +    " SELECT objid, CAST(mtime*8640000 AS int) FROM event WHERE type='ci';\n"
          780  +    "CREATE TEMP TABLE topolink(\n"
          781  +    "  tparent INT,\n"
          782  +    "  tchild INT,\n"
          783  +    "  PRIMARY KEY(tparent,tchild)\n"
          784  +    ") WITHOUT ROWID;"
          785  +    "INSERT INTO topolink(tparent,tchild)"
          786  +    "  SELECT pid, cid FROM plink;\n"
          787  +    "CREATE INDEX topolink_child ON topolink(tchild);\n"
          788  +  );
          789  +
          790  +  /* Find a timewarp instance */
          791  +  db_prepare(&q1,
          792  +    "SELECT P.tseq, C.tid, C.tseq\n"
          793  +    "  FROM toponode P, toponode C, topolink X\n"
          794  +    " WHERE X.tparent=P.tid\n"
          795  +    "   AND X.tchild=C.tid\n"
          796  +    "   AND P.tseq>=C.tseq;"
          797  +  );
          798  +
          799  +  /* Update the timestamp on :tid to have value :tseq */
          800  +  db_prepare(&chng,
          801  +    "UPDATE toponode SET tseq=:tseq WHERE tid=:tid"
          802  +  );
          803  +
          804  +  while( db_step(&q1)==SQLITE_ROW ){
          805  +    i64 iParentTime = db_column_int64(&q1, 0);
          806  +    int iChild = db_column_int(&q1, 1);
          807  +    i64 iChildTime = db_column_int64(&q1, 2);
          808  +    nChange++;
          809  +    if( nChange>10000 ){
          810  +      fossil_fatal("failed to fix all timewarps after 100000 attempts");
          811  +    }
          812  +    db_reset(&q1);
          813  +    db_bind_int64(&chng, ":tid", iChild);
          814  +    db_bind_int64(&chng, ":tseq", iParentTime+1);
          815  +    db_step(&chng);
          816  +    db_reset(&chng);
          817  +    if( bVerbose ){
          818  +      fossil_print("moving %d from %lld to %lld\n",
          819  +                   iChild, iChildTime, iParentTime+1);
          820  +    }
          821  +  }
          822  +
          823  +  db_finalize(&q1);
          824  +  db_finalize(&chng);
          825  +  return nChange;
          826  +}
          827  +
          828  +/*
          829  +** COMMAND: test-topological-sort
          830  +**
          831  +** Invoke the topological_sort_checkins() interface for testing
          832  +** purposes.
          833  +*/
          834  +void test_topological_sort(void){
          835  +  int n;
          836  +  db_find_and_open_repository(0, 0);
          837  +  n = topological_sort_checkins(1);
          838  +  fossil_print("%d reorderings required\n", n);
          839  +}

Changes to src/graph.c.

    35     35   */
    36     36   struct GraphRow {
    37     37     int rid;                    /* The rid for the check-in */
    38     38     i8 nParent;                 /* Number of parents.  -1 for technote lines */
    39     39     int *aParent;               /* Array of parents.  0 element is primary .*/
    40     40     char *zBranch;              /* Branch name */
    41     41     char *zBgClr;               /* Background Color */
    42         -  char zUuid[41];             /* Check-in for file ID */
           42  +  char zUuid[HNAME_MAX+1];    /* Check-in for file ID */
    43     43   
    44     44     GraphRow *pNext;            /* Next row down in the list of all rows */
    45     45     GraphRow *pPrev;            /* Previous row */
    46     46   
    47     47     int idx;                    /* Row index.  First is 1.  0 used for "none" */
    48     48     int idxTop;                 /* Direct descendent highest up on the graph */
    49     49     GraphRow *pChild;           /* Child immediately above this node */

Changes to src/import.c.

   515    515     }
   516    516     zName[i] = 0;
   517    517   }
   518    518   
   519    519   
   520    520   static struct{
   521    521     const char *zMasterName;    /* Name of master branch */
          522  +  int authorFlag;             /* Use author as checkin committer */
   522    523   } ggit;
   523    524   
   524    525   /*
   525    526   ** Read the git-fast-import format from pIn and insert the corresponding
   526    527   ** content into the database.
   527    528   */
   528    529   static void git_fast_import(FILE *pIn){
................................................................................
   617    618   	    gg.aData[got-1] = '\0';
   618    619             gg.zComment = gg.aData;
   619    620             gg.aData = 0;
   620    621             gg.nData = 0;
   621    622           }
   622    623         }
   623    624       }else
   624         -    if( strncmp(zLine, "author ", 7)==0 ){
          625  +    if( (!ggit.authorFlag && strncmp(zLine, "author ", 7)==0)
          626  +        || (ggit.authorFlag && strncmp(zLine, "committer ",10)==0
          627  +            && gg.zUser!=NULL) ){
   625    628         /* No-op */
   626    629       }else
   627    630       if( strncmp(zLine, "mark ", 5)==0 ){
   628    631         trim_newline(&zLine[5]);
   629    632         fossil_free(gg.zMark);
   630    633         gg.zMark = fossil_strdup(&zLine[5]);
   631    634       }else
   632         -    if( strncmp(zLine, "tagger ", 7)==0 || strncmp(zLine, "committer ",10)==0 ){
          635  +    if( strncmp(zLine, "tagger ", 7)==0
          636  +        || (ggit.authorFlag && strncmp(zLine, "author ", 7)==0)
          637  +        || strncmp(zLine, "committer ",10)==0 ){
   633    638         sqlite3_int64 secSince1970;
   634    639         z = strchr(zLine, ' ');
   635    640         while( fossil_isspace(*z) ) z++;
   636    641         if( (zTo=strchr(z, '>'))==NULL ) goto malformed_line;
   637    642         *(++zTo) = '\0';
   638    643         /* Lookup user by contact info. */
   639    644         fossil_free(gg.zUser);
................................................................................
  1601   1606   ** The following formats are currently understood by this command
  1602   1607   **
  1603   1608   **   --git        Import from the git-fast-export file format (default)
  1604   1609   **                Options:
  1605   1610   **                  --import-marks  FILE Restore marks table from FILE
  1606   1611   **                  --export-marks  FILE Save marks table to FILE
  1607   1612   **                  --rename-master NAME Renames the master branch to NAME
         1613  +**                  --use-author    Uses author as the committer
  1608   1614   **
  1609   1615   **   --svn        Import from the svnadmin-dump file format.  The default
  1610   1616   **                behaviour (unless overridden by --flat) is to treat 3
  1611   1617   **                folders in the SVN root as special, following the
  1612   1618   **                common layout of SVN repositories.  These are (by
  1613   1619   **                default) trunk/, branches/ and tags/.  The SVN --deltas
  1614   1620   **                format is supported but not required.
................................................................................
  1725   1731                   || (incrFlag && !find_option("no-rev-tags", 0, 0));
  1726   1732     }else if( gitFlag ){
  1727   1733       markfile_in = find_option("import-marks", 0, 1);
  1728   1734       markfile_out = find_option("export-marks", 0, 1);
  1729   1735       if( !(ggit.zMasterName = find_option("rename-master", 0, 1)) ){
  1730   1736         ggit.zMasterName = "master";
  1731   1737       }
         1738  +    ggit.authorFlag = find_option("use-author", 0, 0)!=0;
  1732   1739     }
  1733   1740     verify_all_options();
  1734   1741   
  1735   1742     if( g.argc!=3 && g.argc!=4 ){
  1736   1743       usage("--git|--svn ?OPTIONS? NEW-REPOSITORY ?INPUT-FILE?");
  1737   1744     }
  1738   1745     if( g.argc==4 ){

Changes to src/info.c.

   712    712         blob_trim(&projName);
   713    713         zPJ = blob_str(&projName);
   714    714         for(jj=0; zPJ[jj]; jj++){
   715    715           if( (zPJ[jj]>0 && zPJ[jj]<' ') || strchr("\"*/:<>?\\|", zPJ[jj]) ){
   716    716             zPJ[jj] = '_';
   717    717           }
   718    718         }
   719         -      zUrl = mprintf("%R/tarball/%t-%S.tar.gz?r=%s", zPJ, zUuid, zUuid);
          719  +      zUrl = mprintf("%R/tarball/%S/%t-%S.tar.gz", zUuid, zPJ, zUuid);
   720    720         @ <tr><th>Downloads:</th><td>
   721    721         @ %z(href("%s",zUrl))Tarball</a>
   722         -      @ | %z(href("%R/zip/%t-%S.zip?r=%!S",zPJ,zUuid,zUuid))ZIP archive</a>
   723         -      @ | %z(href("%R/sqlar/%t-%S.sqlar?r=%!S",zPJ,zUuid,zUuid))\
          722  +      @ | %z(href("%R/zip/%S/%t-%S.zip",zUuid, zPJ,zUuid))ZIP archive</a>
          723  +      @ | %z(href("%R/sqlar/%S/%t-%S.sqlar",zUuid,zPJ,zUuid))\
   724    724         @ SQL archive</a></td></tr>
   725    725         fossil_free(zUrl);
   726    726         blob_reset(&projName);
   727    727       }
   728    728   
   729    729       @ <tr><th>Timelines:</th><td>
   730    730       @   %z(href("%R/timeline?f=%!S&unhide",zUuid))family</a>
................................................................................
  1462   1462       "  FROM attachment"
  1463   1463       " WHERE src=(SELECT uuid FROM blob WHERE rid=%d)"
  1464   1464       " ORDER BY mtime DESC /*sort*/",
  1465   1465       rid
  1466   1466     );
  1467   1467     while( db_step(&q)==SQLITE_ROW ){
  1468   1468       const char *zTarget = db_column_text(&q, 0);
  1469         -    int nTarget = db_column_bytes(&q, 0);
  1470   1469       const char *zFilename = db_column_text(&q, 1);
  1471   1470       const char *zDate = db_column_text(&q, 2);
  1472   1471       const char *zUser = db_column_text(&q, 3);
  1473   1472       /* const char *zSrc = db_column_text(&q, 4); */
  1474   1473       if( cnt>0 ){
  1475   1474         @ Also attachment "%h(zFilename)" to
  1476   1475       }else{
  1477   1476         @ Attachment "%h(zFilename)" to
  1478   1477       }
  1479   1478       objType |= OBJTYPE_ATTACHMENT;
  1480         -    if( nTarget==UUID_SIZE && validate16(zTarget,UUID_SIZE) ){
         1479  +    if( fossil_is_uuid(zTarget) ){
  1481   1480         if ( db_exists("SELECT 1 FROM tag WHERE tagname='tkt-%q'",
  1482   1481               zTarget)
  1483   1482         ){
  1484   1483           if( g.perm.Hyperlink && g.anon.RdTkt ){
  1485   1484             @ ticket [%z(href("%R/tktview?name=%!S",zTarget))%S(zTarget)</a>]
  1486   1485           }else{
  1487   1486             @ ticket [%S(zTarget)]
................................................................................
  2183   2182   **
  2184   2183   ** Show the details of a ticket change control artifact.
  2185   2184   */
  2186   2185   void tinfo_page(void){
  2187   2186     int rid;
  2188   2187     char *zDate;
  2189   2188     const char *zUuid;
  2190         -  char zTktName[UUID_SIZE+1];
         2189  +  char zTktName[HNAME_MAX+1];
  2191   2190     Manifest *pTktChng;
  2192   2191     int modPending;
  2193   2192     const char *zModAction;
  2194   2193     char *zTktTitle;
  2195   2194     login_check_credentials();
  2196   2195     if( !g.perm.RdTkt ){ login_needed(g.anon.RdTkt); return; }
  2197   2196     rid = name_to_rid_www("name");
................................................................................
  2204   2203       }else{
  2205   2204         style_submenu_element("Shun", "%s/shun?shun=%s#addshun", g.zTop, zUuid);
  2206   2205       }
  2207   2206     }
  2208   2207     pTktChng = manifest_get(rid, CFTYPE_TICKET, 0);
  2209   2208     if( pTktChng==0 ) fossil_redirect_home();
  2210   2209     zDate = db_text(0, "SELECT datetime(%.12f)", pTktChng->rDate);
  2211         -  memcpy(zTktName, pTktChng->zTicketUuid, UUID_SIZE+1);
         2210  +  sqlite3_snprintf(sizeof(zTktName), zTktName, "%s", pTktChng->zTicketUuid);
  2212   2211     if( g.perm.ModTkt && (zModAction = P("modaction"))!=0 ){
  2213   2212       if( strcmp(zModAction,"delete")==0 ){
  2214   2213         moderation_disapprove(rid);
  2215   2214         /*
  2216   2215         ** Next, check if the ticket still exists; if not, we cannot
  2217   2216         ** redirect to it.
  2218   2217         */
................................................................................
  2616   2615     zNewColorFlag = P("newclr") ? " checked" : "";
  2617   2616     zNewTagFlag = P("newtag") ? " checked" : "";
  2618   2617     zNewTag = PDT("tagname","");
  2619   2618     zNewBrFlag = P("newbr") ? " checked" : "";
  2620   2619     zNewBranch = PDT("brname","");
  2621   2620     zCloseFlag = P("close") ? " checked" : "";
  2622   2621     zHideFlag = P("hide") ? " checked" : "";
  2623         -  if( P("apply") ){
         2622  +  if( P("apply") && cgi_csrf_safe(1) ){
  2624   2623       Blob ctrl;
  2625   2624       char *zNow;
  2626   2625   
  2627   2626       login_verify_csrf_secret();
  2628   2627       blob_zero(&ctrl);
  2629   2628       zNow = date_in_standard_format(zChngTime ? zChngTime : "now");
  2630   2629       blob_appendf(&ctrl, "D %s\n", zNow);

Changes to src/json_timeline.c.

   316    316   ** or 0 for defaults.
   317    317   */
   318    318   cson_value * json_get_changed_files(int rid, int flags){
   319    319     cson_value * rowsV = NULL;
   320    320     cson_array * rows = NULL;
   321    321     Stmt q = empty_Stmt;
   322    322     db_prepare(&q,
   323         -           "SELECT (pid==0) AS isnew,"
   324         -           "       (fid==0) AS isdel,"
   325         -           "       (SELECT name FROM filename WHERE fnid=mlink.fnid) AS name,"
   326         -           "       blob.uuid as uuid,"
   327         -           "       (SELECT uuid FROM blob WHERE rid=pid) as parent,"
   328         -           "       blob.size as size"
   329         -           "  FROM mlink, blob"
   330         -           " WHERE mid=%d AND pid!=fid"
   331         -           " AND blob.rid=fid AND NOT mlink.isaux"
   332         -           " ORDER BY name /*sort*/",
   333         -             rid
   334         -             );
          323  +         "SELECT (pid<=0) AS isnew,"
          324  +         "       (fid==0) AS isdel,"
          325  +         "       (SELECT name FROM filename WHERE fnid=mlink.fnid) AS name,"
          326  +         "       (SELECT uuid FROM blob WHERE rid=fid) as uuid,"
          327  +         "       (SELECT uuid FROM blob WHERE rid=pid) as parent,"
          328  +         "       blob.size as size"
          329  +         "  FROM mlink"
          330  +         " LEFT JOIN blob ON blob.rid=fid"
          331  +         " WHERE mid=%d AND pid!=fid"
          332  +         " AND NOT mlink.isaux"
          333  +         " ORDER BY name /*sort*/",
          334  +         rid
          335  +         );
   335    336     while( (SQLITE_ROW == db_step(&q)) ){
   336    337       cson_value * rowV = cson_value_new_object();
   337    338       cson_object * row = cson_value_get_object(rowV);
   338    339       int const isNew = db_column_int(&q,0);
   339    340       int const isDel = db_column_int(&q,1);
   340    341       char * zDownload = NULL;
   341    342       if(!rowsV){

Changes to src/login.c.

   500    500     char *zSha1Pw;
   501    501     const char *zIpAddr;         /* IP address of requestor */
   502    502     const char *zReferer;
   503    503   
   504    504     login_check_credentials();
   505    505     if( login_wants_https_redirect() ){
   506    506       const char *zQS = P("QUERY_STRING");
          507  +    if( P("redir")!=0 ){
          508  +      style_header("Insecure Connection");
          509  +      @ <h1>Unable To Establish An Encrypted Connection</h1>
          510  +      @ <p>This website requires that login credentials be sent over
          511  +      @ an encrypted connection.  The current connection is not encrypted
          512  +      @ across the entire route between your browser and the server.
          513  +      @ An attempt was made to redirect to %h(g.zHttpsURL) but
          514  +      @ the connection is still insecure even after the redirect.</p>
          515  +      @ <p>This is probably some kind of configuration problem.  Please
          516  +      @ contact your sysadmin.</p>
          517  +      @ <p>Sorry it did not work out.</p>
          518  +      style_footer();
          519  +      return;
          520  +    }
   507    521       if( zQS==0 ){
   508         -      zQS = "";
          522  +      zQS = "?redir=1";
   509    523       }else if( zQS[0]!=0 ){
   510         -      zQS = mprintf("?%s", zQS);
          524  +      zQS = mprintf("?%s&redir=1", zQS);
   511    525       }
   512    526       cgi_redirectf("%s%s%s", g.zHttpsURL, P("PATH_INFO"), zQS);
   513    527       return;
   514    528     }
   515    529     sqlite3_create_function(g.db, "constant_time_cmp", 2, SQLITE_UTF8, 0,
   516    530                     constant_time_cmp_function, 0, 0);
   517    531     zUsername = P("u");
................................................................................
   659    673     @ <table class="login_out">
   660    674     @ <tr>
   661    675     @   <td class="login_out_label">User ID:</td>
   662    676     if( anonFlag ){
   663    677       @ <td><input type="text" id="u" name="u" value="anonymous" size="30" /></td>
   664    678     }else{
   665    679       @ <td><input type="text" id="u" name="u" value="" size="30" /></td>
          680  +  }
          681  +  if( P("HTTPS")==0 ){
          682  +    @ <td width="15"><td rowspan="3">
          683  +    @ <p class='securityWarning'>
          684  +    @ Warning: Your password will be sent in the clear over an
          685  +    @ unencrypted connection.
          686  +    if( g.sslNotAvailable ){
          687  +      @ No encrypted connection is available on this server.
          688  +    }else{
          689  +      @ Consider logging in at
          690  +      @ <a href='%s(g.zHttpsURL)'>%h(g.zHttpsURL)</a> instead.
          691  +    }
          692  +    @ </p>
   666    693     }
   667    694     @ </tr>
   668    695     @ <tr>
   669    696     @  <td class="login_out_label">Password:</td>
   670    697     @   <td><input type="password" id="p" name="p" value="" size="30" /></td>
   671    698     @ </tr>
   672    699     if( g.zLogin==0 && (anonFlag || zGoto==0) ){
................................................................................
  1335   1362     }else
  1336   1363   #endif /* FOSSIL_ENABLE_JSON */
  1337   1364     {
  1338   1365       const char *zUrl = PD("REQUEST_URI", "index");
  1339   1366       const char *zQS = P("QUERY_STRING");
  1340   1367       Blob redir;
  1341   1368       blob_init(&redir, 0, 0);
  1342         -    if( login_wants_https_redirect() ){
         1369  +    if( login_wants_https_redirect() && !g.sslNotAvailable ){
  1343   1370         blob_appendf(&redir, "%s/login?g=%T", g.zHttpsURL, zUrl);
  1344   1371       }else{
  1345   1372         blob_appendf(&redir, "%R/login?g=%T", zUrl);
  1346   1373       }
  1347   1374       if( anonOk ) blob_append(&redir, "&anon", 5);
  1348   1375       if( zQS && zQS[0] ){
  1349   1376         blob_appendf(&redir, "&%s", zQS);

Changes to src/main.c.

    47     47   #  include "tcl.h"
    48     48   #endif
    49     49   #ifdef FOSSIL_ENABLE_JSON
    50     50   #  include "cson_amalgamation.h" /* JSON API. */
    51     51   #  include "json_detail.h"
    52     52   #endif
    53     53   
    54         -/*
    55         -** Size of a UUID in characters.   A UUID is a randomly generated
    56         -** lower-case hexadecimal number used to identify tickets.
    57         -**
    58         -** In Fossil 1.x, UUID also referred to a SHA1 artifact hash.  But that
    59         -** usage is now obsolete.  The term UUID should now mean only a very large
    60         -** random number used as a unique identifier for tickets or other objects.
    61         -*/
    62         -#define UUID_SIZE 40
    63         -
    64     54   /*
    65     55   ** Maximum number of auxiliary parameters on reports
    66     56   */
    67     57   #define MX_AUX  5
    68     58   
    69     59   /*
    70     60   ** Holds flags for fossil user permissions.
................................................................................
   514    504   /* Error logs from SQLite */
   515    505   static void fossil_sqlite_log(void *notUsed, int iCode, const char *zErrmsg){
   516    506   #ifdef __APPLE__
   517    507     /* Disable the file alias warning on apple products because Time Machine
   518    508     ** creates lots of aliases and the warning alarms people. */
   519    509     if( iCode==SQLITE_WARNING ) return;
   520    510   #endif
          511  +#ifndef FOSSIL_DEBUG
          512  +  /* Disable the automatic index warning except in FOSSIL_DEBUG builds. */
          513  +  if( iCode==SQLITE_WARNING_AUTOINDEX ) return;
          514  +#endif
   521    515     if( iCode==SQLITE_SCHEMA ) return;
   522    516     if( g.dbIgnoreErrors ) return;
   523    517   #ifdef SQLITE_READONLY_DIRECTORY
   524    518     if( iCode==SQLITE_READONLY_DIRECTORY ){
   525    519       zErrmsg = "database is in a read-only directory";
   526    520     }
   527    521   #endif
................................................................................
   538    532     const char *zValue = find_option("comfmtflags", 0, 1);
   539    533     if( zValue ){
   540    534       g.comFmtFlags = atoi(zValue);
   541    535     }else{
   542    536       g.comFmtFlags = COMMENT_PRINT_DEFAULT;
   543    537     }
   544    538   }
          539  +
          540  +/*
          541  +** Check to see if the Fossil binary contains an appended repository
          542  +** file using the appendvfs extension.  If so, change command-line arguments
          543  +** to cause Fossil to launch with "fossil ui" on that repo.
          544  +*/
          545  +static int fossilExeHasAppendedRepo(void){
          546  +  extern int deduceDatabaseType(const char*,int);
          547  +  if( 2==deduceDatabaseType(g.nameOfExe,0) ){
          548  +    static char *azAltArgv[] = { 0, "ui", 0, 0 };
          549  +    azAltArgv[0] = g.nameOfExe;
          550  +    azAltArgv[2] = g.nameOfExe;
          551  +    g.argv = azAltArgv;
          552  +    g.argc = 3;
          553  +    return 1;
          554  +  }else{
          555  +    return 0;
          556  +  }
          557  +}
   545    558   
   546    559   /*
   547    560   ** This procedure runs first.
   548    561   */
   549    562   #if defined(_WIN32) && !defined(BROKEN_MINGW_CMDLINE)
   550    563   int _dowildcard = -1; /* This turns on command-line globbing in MinGW-w64 */
   551    564   int wmain(int argc, wchar_t **argv)
................................................................................
   602    615       }else{
   603    616         fossil_fatal("no such VFS: \"%s\"", g.zVfsName);
   604    617       }
   605    618     }
   606    619     if( fossil_getenv("GATEWAY_INTERFACE")!=0 && !find_option("nocgi", 0, 0)){
   607    620       zCmdName = "cgi";
   608    621       g.isHTTP = 1;
   609         -  }else if( g.argc<2 ){
          622  +  }else if( g.argc<2 && !fossilExeHasAppendedRepo() ){
   610    623       fossil_print(
   611    624          "Usage: %s COMMAND ...\n"
   612    625          "   or: %s help           -- for a list of common commands\n"
   613    626          "   or: %s help COMMAND   -- for help with the named command\n",
   614    627          g.argv[0], g.argv[0], g.argv[0]);
   615    628       fossil_print(
   616    629         "\nCommands and filenames may be passed on to fossil from a file\n"
................................................................................
   928    941     int bVerbose                /* Non-zero for full information. */
   929    942   ){
   930    943   #if defined(FOSSIL_ENABLE_TCL)
   931    944     int rc;
   932    945     const char *zRc;
   933    946   #endif
   934    947     Stmt q;
          948  +  size_t pageSize = 0;
   935    949     blob_zero(pOut);
   936    950     blob_appendf(pOut, "This is fossil version %s\n", get_version());
   937    951     if( !bVerbose ) return;
   938    952     blob_appendf(pOut, "Compiled on %s %s using %s (%d-bit)\n",
   939    953                  __DATE__, __TIME__, COMPILER_NAME, sizeof(void*)*8);
   940    954     blob_appendf(pOut, "Schema version %s\n", AUX_SCHEMA_MAX);
          955  +  fossil_get_page_size(&pageSize);
          956  +  blob_appendf(pOut, "Detected memory page size is %lu bytes\n",
          957  +               (unsigned long)pageSize);
   941    958   #if defined(FOSSIL_ENABLE_MINIZ)
   942    959     blob_appendf(pOut, "miniz %s, loaded %s\n", MZ_VERSION, mz_version());
   943    960   #else
   944    961     blob_appendf(pOut, "zlib %s, loaded %s\n", ZLIB_VERSION, zlibVersion());
   945    962   #endif
   946    963   #if FOSSIL_HARDENED_SHA1
   947    964     blob_appendf(pOut, "hardened-SHA1 by Marc Stevens and Dan Shumow\n");
................................................................................
  1000   1017     blob_append(pOut, "FOSSIL_DYNAMIC_BUILD\n", -1);
  1001   1018   #else
  1002   1019     blob_append(pOut, "FOSSIL_STATIC_BUILD\n", -1);
  1003   1020   #endif
  1004   1021   #if defined(HAVE_PLEDGE)
  1005   1022     blob_append(pOut, "HAVE_PLEDGE\n", -1);
  1006   1023   #endif
         1024  +#if defined(USE_MMAN_H)
         1025  +  blob_append(pOut, "USE_MMAN_H\n", -1);
         1026  +#endif
  1007   1027   #if defined(USE_SEE)
  1008   1028     blob_append(pOut, "USE_SEE\n", -1);
  1009   1029   #endif
  1010   1030   #if defined(FOSSIL_ALLOW_OUT_OF_ORDER_DATES)
  1011   1031     blob_append(pOut, "FOSSIL_ALLOW_OUT_OF_ORDER_DATES\n");
  1012   1032   #endif
  1013   1033     blob_appendf(pOut, "SQLite %s %.30s\n", sqlite3_libversion(),
................................................................................
  1282   1302     @ <base href="%s(g.zBaseURL)/" />
  1283   1303     @ <title>Repository List</title>
  1284   1304     @ </head>
  1285   1305     @ <body>
  1286   1306     n = db_int(0, "SELECT count(*) FROM sfile");
  1287   1307     if( n>0 ){
  1288   1308       Stmt q;
  1289         -    @ <h1>Available Repositories:</h1>
  1290         -    @ <ol>
         1309  +    sqlite3_int64 iNow, iMTime;
         1310  +    @ <h1 align="center">Fossil Repositories</h1>
         1311  +    @ <table border="0" class="sortable" data-init-sort="1" \
         1312  +    @ data-column-types="tnk"><thead>
         1313  +    @ <tr><th>Filename<th width="20"><th>Last Modified</tr>
         1314  +    @ </thead><tbody>
  1291   1315       db_prepare(&q, "SELECT pathname"
  1292   1316                      " FROM sfile ORDER BY pathname COLLATE nocase;");
         1317  +    iNow = db_int64(0, "SELECT strftime('%%s','now')");
  1293   1318       while( db_step(&q)==SQLITE_ROW ){
  1294   1319         const char *zName = db_column_text(&q, 0);
  1295   1320         int nName = (int)strlen(zName);
  1296   1321         char *zUrl;
         1322  +      char *zAge;
         1323  +      char *zFull;
  1297   1324         if( nName<7 ) continue;
  1298   1325         zUrl = sqlite3_mprintf("%.*s", nName-7, zName);
         1326  +      if( zName[0]=='/'
         1327  +#ifdef _WIN32
         1328  +          || sqlite3_strglob("[a-zA-Z]:/*", zName)==0
         1329  +#endif
         1330  +      ){
         1331  +        zFull = mprintf("%s", zName);
         1332  +      }else if ( allRepo ){
         1333  +        zFull = mprintf("/%s", zName);
         1334  +      }else{
         1335  +        zFull = mprintf("%s/%s", g.zRepositoryName, zName);
         1336  +      }
         1337  +      iMTime = file_mtime(zFull, ExtFILE);
         1338  +      fossil_free(zFull);
         1339  +      if( iMTime<=0 ){
         1340  +        zAge = mprintf("...");
         1341  +      }else{
         1342  +        zAge = human_readable_age((iNow - iMTime)/86400.0);
         1343  +      }
  1299   1344         if( sqlite3_strglob("*.fossil", zName)!=0 ){
  1300   1345           /* The "fossil server DIRECTORY" and "fossil ui DIRECTORY" commands
  1301   1346           ** do not work for repositories whose names do not end in ".fossil".
  1302   1347           ** So do not hyperlink those cases. */
  1303         -        @ <li>%h(zName)</li>
         1348  +        @ <tr><td>%h(zName)
  1304   1349         } else if( sqlite3_strglob("*/.*", zName)==0 ){
  1305   1350           /* Do not show hidden repos */
  1306         -        @ <li>%h(zName) (hidden)</li>
         1351  +        @ <tr><td>%h(zName) (hidden)
  1307   1352         } else if( allRepo && sqlite3_strglob("[a-zA-Z]:/?*", zName)!=0 ){
  1308         -        @ <li><a href="%R/%T(zUrl)/home" target="_blank">/%h(zName)</a></li>
         1353  +        @ <tr><td><a href="%R/%T(zUrl)/home" target="_blank">/%h(zName)</a>
  1309   1354         }else{
  1310         -        @ <li><a href="%R/%T(zUrl)/home" target="_blank">%h(zName)</a></li>
         1355  +        @ <tr><td><a href="%R/%T(zUrl)/home" target="_blank">%h(zName)</a>
  1311   1356         }
         1357  +      @ <td></td><td data-sortkey='%010llx(iNow - iMTime)'>%h(zAge)</tr>
         1358  +      fossil_free(zAge);
  1312   1359         sqlite3_free(zUrl);
  1313   1360       }
  1314         -    @ </ol>
         1361  +    @ </tbody></table>
  1315   1362     }else{
  1316   1363       @ <h1>No Repositories Found</h1>
  1317   1364     }
         1365  +  @ <script>%s(builtin_text("sorttable.js"))</script>
  1318   1366     @ </body>
  1319   1367     @ </html>
  1320   1368     cgi_reply();
  1321   1369     sqlite3_close(g.db);
  1322   1370     g.db = 0;
  1323   1371     return n;
  1324   1372   }
         1373  +
         1374  +/*
         1375  +** COMMAND: test-list-page
         1376  +**
         1377  +** Usage: %fossil test-list-page DIRECTORY
         1378  +**
         1379  +** Show all repositories underneath DIRECTORY.  Or if DIRECTORY is "/"
         1380  +** show all repositories in the ~/.fossil file.
         1381  +*/
         1382  +void test_list_page(void){
         1383  +  if( g.argc<3 ){
         1384  +    g.zRepositoryName = "/";
         1385  +  }else{
         1386  +    g.zRepositoryName = g.argv[2];
         1387  +  }
         1388  +  g.httpOut = stdout;
         1389  +  repo_list_page();
         1390  +}
  1325   1391   
  1326   1392   /*
  1327   1393   ** Preconditions:
  1328   1394   **
  1329   1395   **  * Environment variables are set up according to the CGI standard.
  1330   1396   **
  1331   1397   ** If the repository is known, it has already been opened.  If unknown,
................................................................................
  2458   2524     if( find_option("scgi", 0, 0)!=0 ) flags |= HTTP_SERVER_SCGI;
  2459   2525     if( zAltBase ){
  2460   2526       set_base_url(zAltBase);
  2461   2527     }
  2462   2528     g.sslNotAvailable = find_option("nossl", 0, 0)!=0;
  2463   2529     if( find_option("https",0,0)!=0 ){
  2464   2530       cgi_replace_parameter("HTTPS","on");
  2465         -  }else{
  2466         -    /* without --https, defaults to not available. */
  2467         -    g.sslNotAvailable = 1;
  2468   2531     }
  2469   2532     if( find_option("localhost", 0, 0)!=0 ){
  2470   2533       flags |= HTTP_SERVER_LOCALHOST;
  2471   2534     }
  2472   2535   
  2473   2536   #if defined(_WIN32) && USE_SEE
  2474   2537     zPidKey = find_option("usepidkey", 0, 1);

Changes to src/main.mk.

    41     41     $(SRCDIR)/deltacmd.c \
    42     42     $(SRCDIR)/descendants.c \
    43     43     $(SRCDIR)/diff.c \
    44     44     $(SRCDIR)/diffcmd.c \
    45     45     $(SRCDIR)/dispatch.c \
    46     46     $(SRCDIR)/doc.c \
    47     47     $(SRCDIR)/encode.c \
           48  +  $(SRCDIR)/etag.c \
    48     49     $(SRCDIR)/event.c \
    49     50     $(SRCDIR)/export.c \
    50     51     $(SRCDIR)/file.c \
    51     52     $(SRCDIR)/finfo.c \
    52     53     $(SRCDIR)/foci.c \
    53     54     $(SRCDIR)/fshell.c \
    54     55     $(SRCDIR)/fusefs.c \
................................................................................
   241    242     $(OBJDIR)/deltacmd_.c \
   242    243     $(OBJDIR)/descendants_.c \
   243    244     $(OBJDIR)/diff_.c \
   244    245     $(OBJDIR)/diffcmd_.c \
   245    246     $(OBJDIR)/dispatch_.c \
   246    247     $(OBJDIR)/doc_.c \
   247    248     $(OBJDIR)/encode_.c \
          249  +  $(OBJDIR)/etag_.c \
   248    250     $(OBJDIR)/event_.c \
   249    251     $(OBJDIR)/export_.c \
   250    252     $(OBJDIR)/file_.c \
   251    253     $(OBJDIR)/finfo_.c \
   252    254     $(OBJDIR)/foci_.c \
   253    255     $(OBJDIR)/fshell_.c \
   254    256     $(OBJDIR)/fusefs_.c \
................................................................................
   370    372    $(OBJDIR)/deltacmd.o \
   371    373    $(OBJDIR)/descendants.o \
   372    374    $(OBJDIR)/diff.o \
   373    375    $(OBJDIR)/diffcmd.o \
   374    376    $(OBJDIR)/dispatch.o \
   375    377    $(OBJDIR)/doc.o \
   376    378    $(OBJDIR)/encode.o \
          379  + $(OBJDIR)/etag.o \
   377    380    $(OBJDIR)/event.o \
   378    381    $(OBJDIR)/export.o \
   379    382    $(OBJDIR)/file.o \
   380    383    $(OBJDIR)/finfo.o \
   381    384    $(OBJDIR)/foci.o \
   382    385    $(OBJDIR)/fshell.o \
   383    386    $(OBJDIR)/fusefs.o \
................................................................................
   550    553                    -DSQLITE_ENABLE_EXPLAIN_COMMENTS \
   551    554                    -DSQLITE_ENABLE_FTS4 \
   552    555                    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
   553    556                    -DSQLITE_ENABLE_DBSTAT_VTAB \
   554    557                    -DSQLITE_ENABLE_JSON1 \
   555    558                    -DSQLITE_ENABLE_FTS5 \
   556    559                    -DSQLITE_ENABLE_STMTVTAB \
   557         -                 -DSQLITE_USE_ZLIB \
          560  +                 -DSQLITE_HAVE_ZLIB \
   558    561                    -DSQLITE_INTROSPECTION_PRAGMAS \
   559    562                    -DSQLITE_ENABLE_DBPAGE_VTAB
   560    563   
   561    564   # Setup the options used to compile the included SQLite shell.
   562         -SHELL_OPTIONS = -Dmain=sqlite3_shell \
          565  +SHELL_OPTIONS = -DNDEBUG=1 \
          566  +                -DSQLITE_THREADSAFE=0 \
          567  +                -DSQLITE_DEFAULT_MEMSTATUS=0 \
          568  +                -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1 \
          569  +                -DSQLITE_LIKE_DOESNT_MATCH_BLOBS \
          570  +                -DSQLITE_OMIT_DECLTYPE \
          571  +                -DSQLITE_OMIT_DEPRECATED \
          572  +                -DSQLITE_OMIT_GET_TABLE \
          573  +                -DSQLITE_OMIT_PROGRESS_CALLBACK \
          574  +                -DSQLITE_OMIT_SHARED_CACHE \
          575  +                -DSQLITE_OMIT_LOAD_EXTENSION \
          576  +                -DSQLITE_MAX_EXPR_DEPTH=0 \
          577  +                -DSQLITE_USE_ALLOCA \
          578  +                -DSQLITE_ENABLE_LOCKING_STYLE=0 \
          579  +                -DSQLITE_DEFAULT_FILE_FORMAT=4 \
          580  +                -DSQLITE_ENABLE_EXPLAIN_COMMENTS \
          581  +                -DSQLITE_ENABLE_FTS4 \
          582  +                -DSQLITE_ENABLE_FTS3_PARENTHESIS \
          583  +                -DSQLITE_ENABLE_DBSTAT_VTAB \
          584  +                -DSQLITE_ENABLE_JSON1 \
          585  +                -DSQLITE_ENABLE_FTS5 \
          586  +                -DSQLITE_ENABLE_STMTVTAB \
          587  +                -DSQLITE_HAVE_ZLIB \
          588  +                -DSQLITE_INTROSPECTION_PRAGMAS \
          589  +                -DSQLITE_ENABLE_DBPAGE_VTAB \
          590  +                -Dmain=sqlite3_shell \
   563    591                   -DSQLITE_SHELL_IS_UTF8=1 \
   564    592                   -DSQLITE_OMIT_LOAD_EXTENSION=1 \
   565    593                   -DUSE_SYSTEM_SQLITE=$(USE_SYSTEM_SQLITE) \
   566         -                -DSQLITE_SHELL_DBNAME_PROC=fossil_open
          594  +                -DSQLITE_SHELL_DBNAME_PROC=sqlcmd_get_dbname \
          595  +                -DSQLITE_SHELL_INIT_PROC=sqlcmd_init_proc
   567    596   
   568    597   # Setup the options used to compile the included miniz library.
   569    598   MINIZ_OPTIONS = -DMINIZ_NO_STDIO \
   570    599                   -DMINIZ_NO_TIME \
   571    600                   -DMINIZ_NO_ARCHIVE_APIS
   572    601   
   573    602   # The USE_SYSTEM_SQLITE variable may be undefined, set to 0, or set
................................................................................
   672    701   	$(OBJDIR)/deltacmd_.c:$(OBJDIR)/deltacmd.h \
   673    702   	$(OBJDIR)/descendants_.c:$(OBJDIR)/descendants.h \
   674    703   	$(OBJDIR)/diff_.c:$(OBJDIR)/diff.h \
   675    704   	$(OBJDIR)/diffcmd_.c:$(OBJDIR)/diffcmd.h \
   676    705   	$(OBJDIR)/dispatch_.c:$(OBJDIR)/dispatch.h \
   677    706   	$(OBJDIR)/doc_.c:$(OBJDIR)/doc.h \
   678    707   	$(OBJDIR)/encode_.c:$(OBJDIR)/encode.h \
          708  +	$(OBJDIR)/etag_.c:$(OBJDIR)/etag.h \
   679    709   	$(OBJDIR)/event_.c:$(OBJDIR)/event.h \
   680    710   	$(OBJDIR)/export_.c:$(OBJDIR)/export.h \
   681    711   	$(OBJDIR)/file_.c:$(OBJDIR)/file.h \
   682    712   	$(OBJDIR)/finfo_.c:$(OBJDIR)/finfo.h \
   683    713   	$(OBJDIR)/foci_.c:$(OBJDIR)/foci.h \
   684    714   	$(OBJDIR)/fshell_.c:$(OBJDIR)/fshell.h \
   685    715   	$(OBJDIR)/fusefs_.c:$(OBJDIR)/fusefs.h \
................................................................................
  1015   1045   $(OBJDIR)/encode_.c:	$(SRCDIR)/encode.c $(OBJDIR)/translate
  1016   1046   	$(OBJDIR)/translate $(SRCDIR)/encode.c >$@
  1017   1047   
  1018   1048   $(OBJDIR)/encode.o:	$(OBJDIR)/encode_.c $(OBJDIR)/encode.h $(SRCDIR)/config.h
  1019   1049   	$(XTCC) -o $(OBJDIR)/encode.o -c $(OBJDIR)/encode_.c
  1020   1050   
  1021   1051   $(OBJDIR)/encode.h:	$(OBJDIR)/headers
         1052  +
         1053  +$(OBJDIR)/etag_.c:	$(SRCDIR)/etag.c $(OBJDIR)/translate
         1054  +	$(OBJDIR)/translate $(SRCDIR)/etag.c >$@
         1055  +
         1056  +$(OBJDIR)/etag.o:	$(OBJDIR)/etag_.c $(OBJDIR)/etag.h $(SRCDIR)/config.h
         1057  +	$(XTCC) -o $(OBJDIR)/etag.o -c $(OBJDIR)/etag_.c
         1058  +
         1059  +$(OBJDIR)/etag.h:	$(OBJDIR)/headers
  1022   1060   
  1023   1061   $(OBJDIR)/event_.c:	$(SRCDIR)/event.c $(OBJDIR)/translate
  1024   1062   	$(OBJDIR)/translate $(SRCDIR)/event.c >$@
  1025   1063   
  1026   1064   $(OBJDIR)/event.o:	$(OBJDIR)/event_.c $(OBJDIR)/event.h $(SRCDIR)/config.h
  1027   1065   	$(XTCC) -o $(OBJDIR)/event.o -c $(OBJDIR)/event_.c
  1028   1066   

Changes to src/makeheaders.c.

  1470   1470       *pReset = ';';
  1471   1471       return 0;
  1472   1472     }
  1473   1473   
  1474   1474     for(pEnd=pName->pNext; pEnd && pEnd->eType!=TT_Braces; pEnd=pEnd->pNext){
  1475   1475       switch( pEnd->zText[0] ){
  1476   1476         case '(':
         1477  +      case ')':
  1477   1478         case '*':
  1478   1479         case '[':
  1479   1480         case '=':
  1480   1481         case ';':
  1481   1482           return 0;
  1482   1483       }
  1483   1484     }

Changes to src/makemake.tcl.

    53     53     deltacmd
    54     54     descendants
    55     55     diff
    56     56     diffcmd
    57     57     dispatch
    58     58     doc
    59     59     encode
           60  +  etag
    60     61     event
    61     62     export
    62     63     file
    63     64     finfo
    64     65     foci
    65     66     fshell
    66     67     fusefs
................................................................................
   187    188     -DSQLITE_ENABLE_EXPLAIN_COMMENTS
   188    189     -DSQLITE_ENABLE_FTS4
   189    190     -DSQLITE_ENABLE_FTS3_PARENTHESIS
   190    191     -DSQLITE_ENABLE_DBSTAT_VTAB
   191    192     -DSQLITE_ENABLE_JSON1
   192    193     -DSQLITE_ENABLE_FTS5
   193    194     -DSQLITE_ENABLE_STMTVTAB
   194         -  -DSQLITE_USE_ZLIB
          195  +  -DSQLITE_HAVE_ZLIB
   195    196     -DSQLITE_INTROSPECTION_PRAGMAS
   196    197     -DSQLITE_ENABLE_DBPAGE_VTAB
   197    198   }
   198    199   #lappend SQLITE_OPTIONS -DSQLITE_ENABLE_FTS3=1
   199    200   #lappend SQLITE_OPTIONS -DSQLITE_ENABLE_STAT4
   200    201   #lappend SQLITE_OPTIONS -DSQLITE_WIN32_NO_ANSI
   201    202   #lappend SQLITE_OPTIONS -DSQLITE_WINNT_MAX_PATH_CHARS=4096
   202    203   
   203    204   # Options used to compile the included SQLite shell.
   204    205   #
   205         -set SHELL_OPTIONS {
          206  +set SHELL_OPTIONS [concat $SQLITE_OPTIONS {
   206    207     -Dmain=sqlite3_shell
   207    208     -DSQLITE_SHELL_IS_UTF8=1
   208    209     -DSQLITE_OMIT_LOAD_EXTENSION=1
   209    210     -DUSE_SYSTEM_SQLITE=$(USE_SYSTEM_SQLITE)
   210         -  -DSQLITE_SHELL_DBNAME_PROC=fossil_open
   211         -}
          211  +  -DSQLITE_SHELL_DBNAME_PROC=sqlcmd_get_dbname
          212  +  -DSQLITE_SHELL_INIT_PROC=sqlcmd_init_proc
          213  +}]
   212    214   
   213    215   # miniz (libz drop-in alternative) precompiler flags.
   214    216   #
   215    217   set MINIZ_OPTIONS {
   216    218     -DMINIZ_NO_STDIO
   217    219     -DMINIZ_NO_TIME
   218    220     -DMINIZ_NO_ARCHIVE_APIS
................................................................................
   620    622   #### Load Tcl using the private stubs mechanism
   621    623   #
   622    624   # FOSSIL_ENABLE_TCL_PRIVATE_STUBS = 1
   623    625   
   624    626   #### Use 'system' SQLite
   625    627   #
   626    628   # USE_SYSTEM_SQLITE = 1
          629  +
          630  +#### Use POSIX memory APIs from "sys/mman.h"
          631  +#
          632  +# USE_MMAN_H = 1
   627    633   
   628    634   #### Use the SQLite Encryption Extension
   629    635   #
   630    636   # USE_SEE = 1
   631    637   
   632    638   #### Use the miniz compression library
   633    639   #
................................................................................
   694    700   endif
   695    701   
   696    702   #### The directories where the OpenSSL include and library files are located.
   697    703   #    The recommended usage here is to use the Sysinternals junction tool
   698    704   #    to create a hard link between an "openssl-1.x" sub-directory of the
   699    705   #    Fossil source code directory and the target OpenSSL source directory.
   700    706   #
   701         -OPENSSLDIR = $(SRCDIR)/../compat/openssl-1.0.2n
          707  +OPENSSLDIR = $(SRCDIR)/../compat/openssl-1.0.2o
   702    708   OPENSSLINCDIR = $(OPENSSLDIR)/include
   703    709   OPENSSLLIBDIR = $(OPENSSLDIR)
   704    710   
   705    711   #### Either the directory where the Tcl library is installed or the Tcl
   706    712   #    source code directory resides (depending on the value of the macro
   707    713   #    FOSSIL_TCL_SOURCE).  If this points to the Tcl install directory,
   708    714   #    this directory must have "include" and "lib" sub-directories.  If
................................................................................
   857    863   endif
   858    864   
   859    865   # With JSON support
   860    866   ifdef FOSSIL_ENABLE_JSON
   861    867   TCC += -DFOSSIL_ENABLE_JSON=1
   862    868   RCC += -DFOSSIL_ENABLE_JSON=1
   863    869   endif
          870  +
          871  +# With "sys/mman.h" support
          872  +ifdef USE_MMAN_H
          873  +TCC += -DUSE_MMAN_H=1
          874  +RCC += -DUSE_MMAN_H=1
          875  +endif
   864    876   
   865    877   # With SQLite Encryption Extension support
   866    878   ifdef USE_SEE
   867    879   TCC += -DUSE_SEE=1
   868    880   RCC += -DUSE_SEE=1
   869    881   endif
   870    882   
................................................................................
  1540   1552   
  1541   1553   # Enable support for the SQLite Encryption Extension?
  1542   1554   !ifndef USE_SEE
  1543   1555   USE_SEE = 0
  1544   1556   !endif
  1545   1557   
  1546   1558   !if $(FOSSIL_ENABLE_SSL)!=0
  1547         -SSLDIR    = $(B)\compat\openssl-1.0.2n
         1559  +SSLDIR    = $(B)\compat\openssl-1.0.2o
  1548   1560   SSLINCDIR = $(SSLDIR)\inc32
  1549   1561   !if $(FOSSIL_DYNAMIC_BUILD)!=0
  1550   1562   SSLLIBDIR = $(SSLDIR)\out32dll
  1551   1563   !else
  1552   1564   SSLLIBDIR = $(SSLDIR)\out32
  1553   1565   !endif
  1554   1566   SSLLFLAGS = /nologo /opt:ref /debug

Changes to src/manifest.c.

   605    605         **
   606    606         ** A K-line gives the UUID for the ticket which this control file
   607    607         ** is amending.
   608    608         */
   609    609         case 'K': {
   610    610           if( p->zTicketUuid!=0 ) SYNTAX("more than one K-card");
   611    611           p->zTicketUuid = next_token(&x, &sz);
   612         -        if( sz!=UUID_SIZE ) SYNTAX("K-card UUID is the wrong size");
   613         -        if( !validate16(p->zTicketUuid, UUID_SIZE) ){
          612  +        if( sz!=HNAME_LEN_SHA1 ) SYNTAX("K-card UUID is the wrong size");
          613  +        if( !validate16(p->zTicketUuid, sz) ){
   614    614             SYNTAX("invalid K-card UUID");
   615    615           }
   616    616           break;
   617    617         }
   618    618   
   619    619         /*
   620    620         **     L <wikititle>
................................................................................
  1265   1265     Manifest *p,             /* Manifest to search */
  1266   1266     const char *zName,       /* Name of the file we are looking for */
  1267   1267     int bBest                /* 0: exact match only.  1: closest match */
  1268   1268   ){
  1269   1269     int lwr, upr;
  1270   1270     int c;
  1271   1271     int i;
         1272  +  if( p->aFile==0 ){
         1273  +    return 0;
         1274  +  }
  1272   1275     lwr = 0;
  1273   1276     upr = p->nFile - 1;
  1274   1277     if( p->iFile>=lwr && p->iFile<upr ){
  1275   1278       c = fossil_strcmp(p->aFile[p->iFile+1].zName, zName);
  1276   1279       if( c==0 ){
  1277   1280         return &p->aFile[++p->iFile];
  1278   1281       }else if( c>0 ){

Changes to src/markdown.c.

  1924   1924       /* render the header row */
  1925   1925       head = new_work_buffer(rndr);
  1926   1926       if( head ){
  1927   1927         parse_table_row(head, rndr, data, head_end, 0, 0, MKD_CELL_HEAD);
  1928   1928       }
  1929   1929   
  1930   1930       /* parse alignments if provided */
  1931         -    if( col && (aligns=malloc(align_size * sizeof *aligns))!=0 ){
         1931  +    if( col && (aligns=fossil_malloc(align_size * sizeof *aligns))!=0 ){
  1932   1932         for(i=0; i<align_size; i++) aligns[i] = 0;
  1933   1933         col = 0;
  1934   1934         i = head_end+1;
  1935   1935   
  1936   1936         /* skip initial white space and optional separator */
  1937   1937         while( i<size && (data[i]==' ' || data[i]=='\t') ){ i++; }
  1938   1938         if( data[i]=='|' ) i++;
................................................................................
  1966   1966   
  1967   1967     /* render the full table */
  1968   1968     rndr->make.table(ob, head, rows, rndr->make.opaque);
  1969   1969   
  1970   1970     /* cleanup */
  1971   1971     if( head ) release_work_buffer(rndr, head);
  1972   1972     if( rows!=&fallback ) release_work_buffer(rndr, rows);
  1973         -  free(aligns);
         1973  +  fossil_free(aligns);
  1974   1974     return i;
  1975   1975   }
  1976   1976   
  1977   1977   
  1978   1978   /* parse_block -- parsing of one block, returning next char to parse */
  1979   1979   static void parse_block(
  1980   1980     struct Blob *ob,        /* output blob */

Changes to src/merge.c.

   285    285       ** leaf that is (1) not the version currently checked out and (2)
   286    286       ** has not already been merged into the current checkout and (3)
   287    287       ** the leaf is not closed and (4) the leaf is in the same branch
   288    288       ** as the current checkout.
   289    289       */
   290    290       Stmt q;
   291    291       if( pickFlag || backoutFlag || integrateFlag){
   292         -      fossil_fatal("cannot use --backout, --cherrypick or --integrate with a fork merge");
          292  +      fossil_fatal("cannot use --backout, --cherrypick or --integrate "
          293  +                   "with a fork merge");
   293    294       }
   294    295       mid = fossil_find_nearest_fork(vid, db_open_local(0));
   295    296       if( mid==0 ){
   296    297         fossil_fatal("no unmerged forks of branch \"%s\"",
   297    298           db_text(0, "SELECT value FROM tagxref"
   298    299                      " WHERE tagid=%d AND rid=%d AND tagtype>0",
   299    300                      TAG_BRANCH, vid)
................................................................................
   323    324   
   324    325     if( zPivot ){
   325    326       pid = name_to_typed_rid(zPivot, "ci");
   326    327       if( pid==0 || !is_a_version(pid) ){
   327    328         fossil_fatal("not a version: %s", zPivot);
   328    329       }
   329    330       if( pickFlag ){
   330         -      fossil_fatal("incompatible options: --cherrypick & --baseline");
          331  +      fossil_fatal("incompatible options: --cherrypick and --baseline");
   331    332       }
   332    333     }
   333    334     if( pickFlag || backoutFlag ){
   334    335       if( integrateFlag ){
   335         -      fossil_fatal("incompatible options: --integrate & --cherrypick or --backout");
          336  +      fossil_fatal("incompatible options: --integrate and --cherrypick "
          337  +                   "with --backout");
   336    338       }
   337    339       pid = db_int(0, "SELECT pid FROM plink WHERE cid=%d AND isprim", mid);
   338    340       if( pid<=0 ){
   339    341         fossil_fatal("cannot find an ancestor for %s", g.argv[2]);
   340    342       }
   341    343     }else{
   342    344       if( !zPivot ){
................................................................................
   377    379       return;
   378    380     }
   379    381     if( integrateFlag && !is_a_leaf(mid)){
   380    382       fossil_warning("ignoring --integrate: %s is not a leaf", g.argv[2]);
   381    383       integrateFlag = 0;
   382    384     }
   383    385     if( verboseFlag ){
   384         -    print_checkin_description(mid, 12, integrateFlag?"integrate:":"merge-from:");
          386  +    print_checkin_description(mid, 12,
          387  +              integrateFlag ? "integrate:" : "merge-from:");
   385    388       print_checkin_description(pid, 12, "baseline:");
   386    389     }
   387    390     vfile_check_signature(vid, CKSIG_ENOTFILE);
   388    391     db_begin_transaction();
   389    392     if( !dryRunFlag ) undo_begin();
   390    393     if( load_vfile_from_rid(mid) && !forceMissingFlag ){
   391    394       fossil_fatal("missing content, unable to merge");

Changes to src/path.c.

   545    545       g.argv += 2;
   546    546       g.argc -= 2;
   547    547     }
   548    548   }
   549    549   
   550    550   /* Query to extract all rename operations */
   551    551   static const char zRenameQuery[] =
          552  +@ CREATE TEMP TABLE renames AS
          553  +@ SELECT
          554  +@     datetime(event.mtime) AS date,
          555  +@     F.name AS old_name,
          556  +@     T.name AS new_name,
          557  +@     blob.uuid AS checkin
          558  +@   FROM mlink, filename F, filename T, event, blob
          559  +@  WHERE coalesce(mlink.pfnid,0)!=0 AND mlink.pfnid!=mlink.fnid
          560  +@    AND F.fnid=mlink.pfnid
          561  +@    AND T.fnid=mlink.fnid
          562  +@    AND event.objid=mlink.mid
          563  +@    AND event.type='ci'
          564  +@    AND blob.rid=mlink.mid;
          565  +;
          566  +
          567  +/* Query to extract distinct rename operations */
          568  +static const char zDistinctRenameQuery[] =
          569  +@ CREATE TEMP TABLE renames AS
   552    570   @ SELECT
   553         -@     datetime(event.mtime),
          571  +@     min(datetime(event.mtime)) AS date,
   554    572   @     F.name AS old_name,
   555    573   @     T.name AS new_name,
   556         -@     blob.uuid
          574  +@     blob.uuid AS checkin
   557    575   @   FROM mlink, filename F, filename T, event, blob
   558    576   @  WHERE coalesce(mlink.pfnid,0)!=0 AND mlink.pfnid!=mlink.fnid
   559    577   @    AND F.fnid=mlink.pfnid
   560    578   @    AND T.fnid=mlink.fnid
   561    579   @    AND event.objid=mlink.mid
   562    580   @    AND event.type='ci'
   563    581   @    AND blob.rid=mlink.mid
   564         -@  ORDER BY 1 DESC, 2;
          582  +@  GROUP BY 2, 3;
   565    583   ;
   566    584   
   567    585   /*
   568    586   ** WEBPAGE: test-rename-list
   569    587   **
   570    588   ** Print a list of all file rename operations throughout history.
   571    589   ** This page is intended for for testing purposes only and may change
   572    590   ** or be discontinued without notice.
   573    591   */
   574    592   void test_rename_list_page(void){
   575    593     Stmt q;
          594  +  int nRename;
          595  +  int nCheckin;
   576    596   
   577    597     login_check_credentials();
   578    598     if( !g.perm.Read ){ login_needed(g.anon.Read); return; }
   579         -  style_header("List Of File Name Changes");
   580         -  @ <h3>NB: Experimental Page</h3>
   581         -  @ <table border="1" width="100%%">
   582         -  @ <tr><th>Date &amp; Time</th>
          599  +  if( P("all")!=0 ){
          600  +    style_header("List Of All Filename Changes");
          601  +    db_multi_exec("%s", zRenameQuery/*safe-for-%s*/);
          602  +    style_submenu_element("Distinct", "%R/test-rename-list");
          603  +  }else{
          604  +    style_header("List Of Distinct Filename Changes");
          605  +    db_multi_exec("%s", zDistinctRenameQuery/*safe-for-%s*/);
          606  +    style_submenu_element("All", "%R/test-rename-list?all");
          607  +  }
          608  +  nRename = db_int(0, "SELECT count(*) FROM renames;");
          609  +  nCheckin = db_int(0, "SELECT count(DISTINCT checkin) FROM renames;");
          610  +  db_prepare(&q, "SELECT date, old_name, new_name, checkin FROM renames"
          611  +                 " ORDER BY date DESC, old_name ASC");
          612  +  @ <h1>%d(nRename) filename changes in %d(nCheckin) check-ins</h1>
          613  +  @ <table class='sortable' data-column-types='tttt' data-init-sort='1'\
          614  +  @  border="1" cellpadding="2" cellspacing="0">
          615  +  @ <thead><tr><th>Date &amp; Time</th>
   583    616     @ <th>Old Name</th>
   584    617     @ <th>New Name</th>
   585         -  @ <th>Check-in</th></tr>
   586         -  db_prepare(&q, "%s", zRenameQuery/*safe-for-%s*/);
          618  +  @ <th>Check-in</th></tr></thead><tbody>
   587    619     while( db_step(&q)==SQLITE_ROW ){
   588    620       const char *zDate = db_column_text(&q, 0);
   589    621       const char *zOld = db_column_text(&q, 1);
   590    622       const char *zNew = db_column_text(&q, 2);
   591    623       const char *zUuid = db_column_text(&q, 3);
   592    624       @ <tr>
   593    625       @ <td>%z(href("%R/timeline?c=%t",zDate))%s(zDate)</a></td>
   594    626       @ <td>%z(href("%R/finfo?name=%t",zOld))%h(zOld)</a></td>
   595    627       @ <td>%z(href("%R/finfo?name=%t",zNew))%h(zNew)</a></td>
   596    628       @ <td>%z(href("%R/info/%!S",zUuid))%S(zUuid)</a></td></tr>
   597    629     }
   598         -  @ </table>
          630  +  @ </tbody></table>
   599    631     db_finalize(&q);
          632  +  style_table_sorter();
   600    633     style_footer();
   601    634   }

Changes to src/pivot.c.

    36     36   **
    37     37   ** The act of setting the primary resets the pivot-finding algorithm.
    38     38   */
    39     39   void pivot_set_primary(int rid){
    40     40     /* Set up table used to do the search */
    41     41     db_multi_exec(
    42     42       "CREATE TEMP TABLE IF NOT EXISTS aqueue("
    43         -    "  rid INTEGER PRIMARY KEY,"  /* The record id for this version */
           43  +    "  rid INTEGER,"              /* The record id for this version */
    44     44       "  mtime REAL,"               /* Time when this version was created */
    45     45       "  pending BOOLEAN,"          /* True if we have not check this one yet */
    46         -    "  src BOOLEAN"               /* 1 for primary.  0 for others */
    47         -    ");"
           46  +    "  src BOOLEAN,"               /* 1 for primary.  0 for others */
           47  +    "  PRIMARY KEY(rid,src)"
           48  +    ") WITHOUT ROWID;"
    48     49       "DELETE FROM aqueue;"
    49     50       "CREATE INDEX IF NOT EXISTS aqueue_idx1 ON aqueue(pending, mtime);"
    50     51     );
    51     52   
    52     53     /* Insert the primary record */
    53     54     db_multi_exec(
    54     55       "INSERT INTO aqueue(rid, mtime, pending, src)"
................................................................................
   116    117     db_prepare(&u1,
   117    118       "UPDATE aqueue SET pending=0 WHERE rid=:rid"
   118    119     );
   119    120   
   120    121     /* Add to the queue all ancestors of :rid.
   121    122     */
   122    123     db_prepare(&i1,
   123         -    "INSERT OR IGNORE INTO aqueue "
          124  +    "REPLACE INTO aqueue "
   124    125       "SELECT plink.pid,"
   125         -    "       coalesce((SELECT mtime FROM plink X WHERE X.cid=plink.pid), 0.0),"
          126  +    "       coalesce((SELECT mtime FROM event X WHERE X.objid=plink.pid), 0.0),"
   126    127       "       1,"
   127    128       "       aqueue.src "
   128    129       "  FROM plink, aqueue"
   129    130       " WHERE plink.cid=:rid"
   130    131       "   AND aqueue.rid=:rid %s",
   131    132       ignoreMerges ? "AND plink.isprim" : ""
   132    133     );

Changes to src/search.c.

  1773   1773        { 1,  "reindex"  },
  1774   1774        { 2,  "index"    },
  1775   1775        { 3,  "disable"  },
  1776   1776        { 4,  "enable"   },
  1777   1777        { 5,  "stemmer"  },
  1778   1778     };
  1779   1779     static const struct { char *zSetting; char *zName; char *zSw; } aSetng[] = {
  1780         -     { "search-ckin",     "check-in search:",  "c" },
         1780  +     { "search-ci",       "check-in search:",  "c" },
  1781   1781        { "search-doc",      "document search:",  "d" },
  1782   1782        { "search-tkt",      "ticket search:",    "t" },
  1783   1783        { "search-wiki",     "wiki search:",      "w" },
  1784   1784        { "search-technote", "tech note search:", "e" },
  1785   1785     };
  1786   1786     char *zSubCmd = 0;
  1787   1787     int i, j, n;

Changes to src/setup.c.

    16     16   *******************************************************************************
    17     17   **
    18     18   ** Implementation of the Setup page
    19     19   */
    20     20   #include "config.h"
    21     21   #include <assert.h>
    22     22   #include "setup.h"
           23  +
           24  +/*
           25  +** Increment the "cfgcnt" variable, so that ETags will know that
           26  +** the configuration has changed.
           27  +*/
           28  +void setup_incr_cfgcnt(void){
           29  +  static int once = 1;
           30  +  if( once ){
           31  +    once = 0;
           32  +    db_multi_exec("UPDATE config SET value=value+1 WHERE name='cfgcnt'");
           33  +    if( db_changes()==0 ){
           34  +      db_multi_exec("INSERT INTO config(name,value) VALUES('cfgcnt',1)");
           35  +    }
           36  +  }
           37  +}
    23     38   
    24     39   /*
    25     40   ** Output a single entry for a menu generated using an HTML table.
    26     41   ** If zLink is not NULL or an empty string, then it is the page that
    27     42   ** the menu entry will hyperlink to.  If zLink is NULL or "", then
    28     43   ** the menu entry has no hyperlink - it is disabled.
    29     44   */
................................................................................
   494    509       }
   495    510       login_verify_csrf_secret();
   496    511       db_multi_exec(
   497    512          "REPLACE INTO user(uid,login,info,pw,cap,mtime) "
   498    513          "VALUES(nullif(%d,0),%Q,%Q,%Q,%Q,now())",
   499    514         uid, zLogin, P("info"), zPw, zCap
   500    515       );
          516  +    setup_incr_cfgcnt();
   501    517       admin_log( "Updated user [%q] with capabilities [%q].",
   502    518                  zLogin, zCap );
   503    519       if( atoi(PD("all","0"))>0 ){
   504    520         Blob sql;
   505    521         char *zErr = 0;
   506    522         blob_zero(&sql);
   507    523         if( zOldLogin==0 ){

Changes to src/shell.c.

   146    146   #if defined(_WIN32) || defined(WIN32)
   147    147   # include <io.h>
   148    148   # include <fcntl.h>
   149    149   # define isatty(h) _isatty(h)
   150    150   # ifndef access
   151    151   #  define access(f,m) _access((f),(m))
   152    152   # endif
          153  +# ifndef unlink
          154  +#  define unlink _unlink
          155  +# endif
   153    156   # undef popen
   154    157   # define popen _popen
   155    158   # undef pclose
   156    159   # define pclose _pclose
   157    160   #else
   158    161    /* Make sure isatty() has a prototype. */
   159    162    extern int isatty(int);
................................................................................
   446    449   /*
   447    450   ** Render output like fprintf().  This should not be used on anything that
   448    451   ** includes string formatting (e.g. "%s").
   449    452   */
   450    453   #if !defined(raw_printf)
   451    454   # define raw_printf fprintf
   452    455   #endif
          456  +
          457  +/* Indicate out-of-memory and exit. */
          458  +static void shell_out_of_memory(void){
          459  +  raw_printf(stderr,"Error: out of memory\n");
          460  +  exit(1);
          461  +}
   453    462   
   454    463   /*
   455    464   ** Write I/O traces to the following stream.
   456    465   */
   457    466   #ifdef SQLITE_ENABLE_IOTRACE
   458    467   static FILE *iotrace = 0;
   459    468   #endif
................................................................................
   779    788   ** because it contains non-alphanumeric characters, or because it is an
   780    789   ** SQLite keyword.  Be conservative in this estimate:  When in doubt assume
   781    790   ** that quoting is required.
   782    791   **
   783    792   ** Return '"' if quoting is required.  Return 0 if no quoting is required.
   784    793   */
   785    794   static char quoteChar(const char *zName){
   786         -  /* All SQLite keywords, in alphabetical order */
   787         -  static const char *azKeywords[] = {
   788         -    "ABORT", "ACTION", "ADD", "AFTER", "ALL", "ALTER", "ANALYZE", "AND", "AS",
   789         -    "ASC", "ATTACH", "AUTOINCREMENT", "BEFORE", "BEGIN", "BETWEEN", "BY",
   790         -    "CASCADE", "CASE", "CAST", "CHECK", "COLLATE", "COLUMN", "COMMIT",
   791         -    "CONFLICT", "CONSTRAINT", "CREATE", "CROSS", "CURRENT_DATE",
   792         -    "CURRENT_TIME", "CURRENT_TIMESTAMP", "DATABASE", "DEFAULT", "DEFERRABLE",
   793         -    "DEFERRED", "DELETE", "DESC", "DETACH", "DISTINCT", "DROP", "EACH",
   794         -    "ELSE", "END", "ESCAPE", "EXCEPT", "EXCLUSIVE", "EXISTS", "EXPLAIN",
   795         -    "FAIL", "FOR", "FOREIGN", "FROM", "FULL", "GLOB", "GROUP", "HAVING", "IF",
   796         -    "IGNORE", "IMMEDIATE", "IN", "INDEX", "INDEXED", "INITIALLY", "INNER",
   797         -    "INSERT", "INSTEAD", "INTERSECT", "INTO", "IS", "ISNULL", "JOIN", "KEY",
   798         -    "LEFT", "LIKE", "LIMIT", "MATCH", "NATURAL", "NO", "NOT", "NOTNULL",
   799         -    "NULL", "OF", "OFFSET", "ON", "OR", "ORDER", "OUTER", "PLAN", "PRAGMA",
   800         -    "PRIMARY", "QUERY", "RAISE", "RECURSIVE", "REFERENCES", "REGEXP",
   801         -    "REINDEX", "RELEASE", "RENAME", "REPLACE", "RESTRICT", "RIGHT",
   802         -    "ROLLBACK", "ROW", "SAVEPOINT", "SELECT", "SET", "TABLE", "TEMP",
   803         -    "TEMPORARY", "THEN", "TO", "TRANSACTION", "TRIGGER", "UNION", "UNIQUE",
   804         -    "UPDATE", "USING", "VACUUM", "VALUES", "VIEW", "VIRTUAL", "WHEN", "WHERE",
   805         -    "WITH", "WITHOUT",
   806         -  };
   807         -  int i, lwr, upr, mid, c;
          795  +  int i;
   808    796     if( !isalpha((unsigned char)zName[0]) && zName[0]!='_' ) return '"';
   809    797     for(i=0; zName[i]; i++){
   810    798       if( !isalnum((unsigned char)zName[i]) && zName[i]!='_' ) return '"';
   811    799     }
   812         -  lwr = 0;
   813         -  upr = sizeof(azKeywords)/sizeof(azKeywords[0]) - 1;
   814         -  while( lwr<=upr ){
   815         -    mid = (lwr+upr)/2;
   816         -    c = sqlite3_stricmp(azKeywords[mid], zName);
   817         -    if( c==0 ) return '"';
   818         -    if( c<0 ){
   819         -      lwr = mid+1;
   820         -    }else{
   821         -      upr = mid-1;
   822         -    }
   823         -  }
   824         -  return 0;
          800  +  return sqlite3_keyword_check(zName, i) ? '"' : 0;
   825    801   }
   826    802   
   827    803   /*
   828    804   ** Construct a fake object name and column list to describe the structure
   829    805   ** of the view, virtual table, or table valued function zSchema.zName.
   830    806   */
   831    807   static char *shellFakeSchema(
................................................................................
   993    969   #if defined(_WIN32) && defined(_MSC_VER) && !defined(SQLITE_WINDIRENT_H)
   994    970   #define SQLITE_WINDIRENT_H
   995    971   
   996    972   /*
   997    973   ** We need several data types from the Windows SDK header.
   998    974   */
   999    975   
          976  +#ifndef WIN32_LEAN_AND_MEAN
  1000    977   #define WIN32_LEAN_AND_MEAN
          978  +#endif
          979  +
  1001    980   #include "windows.h"
  1002    981   
  1003    982   /*
  1004    983   ** We need several support functions from the SQLite core.
  1005    984   */
  1006    985   
  1007    986   
................................................................................
  2135   2114   #  include <sys/time.h>
  2136   2115   #else
  2137   2116   #  include "windows.h"
  2138   2117   #  include <io.h>
  2139   2118   #  include <direct.h>
  2140   2119   /* #  include "test_windirent.h" */
  2141   2120   #  define dirent DIRENT
         2121  +#  ifndef chmod
         2122  +#    define chmod _chmod
         2123  +#  endif
  2142   2124   #  ifndef stat
  2143   2125   #    define stat _stat
  2144   2126   #  endif
  2145   2127   #  define mkdir(path,mode) _mkdir(path)
  2146   2128   #  define lstat(path,buf) stat(path,buf)
  2147   2129   #endif
  2148   2130   #include <time.h>
................................................................................
  2200   2182     va_list ap;
  2201   2183     va_start(ap, zFmt);
  2202   2184     zMsg = sqlite3_vmprintf(zFmt, ap);
  2203   2185     sqlite3_result_error(ctx, zMsg, -1);
  2204   2186     sqlite3_free(zMsg);
  2205   2187     va_end(ap);
  2206   2188   }
         2189  +
         2190  +#if defined(_WIN32)
         2191  +/*
         2192  +** This function is designed to convert a Win32 FILETIME structure into the
         2193  +** number of seconds since the Unix Epoch (1970-01-01 00:00:00 UTC).
         2194  +*/
         2195  +static sqlite3_uint64 fileTimeToUnixTime(
         2196  +  LPFILETIME pFileTime
         2197  +){
         2198  +  SYSTEMTIME epochSystemTime;
         2199  +  ULARGE_INTEGER epochIntervals;
         2200  +  FILETIME epochFileTime;
         2201  +  ULARGE_INTEGER fileIntervals;
         2202  +
         2203  +  memset(&epochSystemTime, 0, sizeof(SYSTEMTIME));
         2204  +  epochSystemTime.wYear = 1970;
         2205  +  epochSystemTime.wMonth = 1;
         2206  +  epochSystemTime.wDay = 1;
         2207  +  SystemTimeToFileTime(&epochSystemTime, &epochFileTime);
         2208  +  epochIntervals.LowPart = epochFileTime.dwLowDateTime;
         2209  +  epochIntervals.HighPart = epochFileTime.dwHighDateTime;
         2210  +
         2211  +  fileIntervals.LowPart = pFileTime->dwLowDateTime;
         2212  +  fileIntervals.HighPart = pFileTime->dwHighDateTime;
         2213  +
         2214  +  return (fileIntervals.QuadPart - epochIntervals.QuadPart) / 10000000;
         2215  +}
         2216  +
         2217  +/*
         2218  +** This function attempts to normalize the time values found in the stat()
         2219  +** buffer to UTC.  This is necessary on Win32, where the runtime library
         2220  +** appears to return these values as local times.
         2221  +*/
         2222  +static void statTimesToUtc(
         2223  +  const char *zPath,
         2224  +  struct stat *pStatBuf
         2225  +){
         2226  +  HANDLE hFindFile;
         2227  +  WIN32_FIND_DATAW fd;
         2228  +  LPWSTR zUnicodeName;
         2229  +  extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
         2230  +  zUnicodeName = sqlite3_win32_utf8_to_unicode(zPath);
         2231  +  if( zUnicodeName ){
         2232  +    memset(&fd, 0, sizeof(WIN32_FIND_DATA));
         2233  +    hFindFile = FindFirstFileW(zUnicodeName, &fd);
         2234  +    if( hFindFile!=NULL ){
         2235  +      pStatBuf->st_ctime = (time_t)fileTimeToUnixTime(&fd.ftCreationTime);
         2236  +      pStatBuf->st_atime = (time_t)fileTimeToUnixTime(&fd.ftLastAccessTime);
         2237  +      pStatBuf->st_mtime = (time_t)fileTimeToUnixTime(&fd.ftLastWriteTime);
         2238  +      FindClose(hFindFile);
         2239  +    }
         2240  +    sqlite3_free(zUnicodeName);
         2241  +  }
         2242  +}
         2243  +#endif
         2244  +
         2245  +/*
         2246  +** This function is used in place of stat().  On Windows, special handling
         2247  +** is required in order for the included time to be returned as UTC.  On all
         2248  +** other systems, this function simply calls stat().
         2249  +*/
         2250  +static int fileStat(
         2251  +  const char *zPath,
         2252  +  struct stat *pStatBuf
         2253  +){
         2254  +#if defined(_WIN32)
         2255  +  int rc = stat(zPath, pStatBuf);
         2256  +  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
         2257  +  return rc;
         2258  +#else
         2259  +  return stat(zPath, pStatBuf);
         2260  +#endif
         2261  +}
         2262  +
         2263  +/*
         2264  +** This function is used in place of lstat().  On Windows, special handling
         2265  +** is required in order for the included time to be returned as UTC.  On all
         2266  +** other systems, this function simply calls lstat().
         2267  +*/
         2268  +static int fileLinkStat(
         2269  +  const char *zPath,
         2270  +  struct stat *pStatBuf
         2271  +){
         2272  +#if defined(_WIN32)
         2273  +  int rc = lstat(zPath, pStatBuf);
         2274  +  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
         2275  +  return rc;
         2276  +#else
         2277  +  return lstat(zPath, pStatBuf);
         2278  +#endif
         2279  +}
  2207   2280   
  2208   2281   /*
  2209   2282   ** Argument zFile is the name of a file that will be created and/or written
  2210   2283   ** by SQL function writefile(). This function ensures that the directory
  2211   2284   ** zFile will be written to exists, creating it if required. The permissions
  2212   2285   ** for any path components created by this function are set to (mode&0777).
  2213   2286   **
................................................................................
  2232   2305         struct stat sStat;
  2233   2306         int rc2;
  2234   2307   
  2235   2308         for(; zCopy[i]!='/' && i<nCopy; i++);
  2236   2309         if( i==nCopy ) break;
  2237   2310         zCopy[i] = '\0';
  2238   2311   
  2239         -      rc2 = stat(zCopy, &sStat);
         2312  +      rc2 = fileStat(zCopy, &sStat);
  2240   2313         if( rc2!=0 ){
  2241   2314           if( mkdir(zCopy, mode & 0777) ) rc = SQLITE_ERROR;
  2242   2315         }else{
  2243   2316           if( !S_ISDIR(sStat.st_mode) ) rc = SQLITE_ERROR;
  2244   2317         }
  2245   2318         zCopy[i] = '/';
  2246   2319         i++;
................................................................................
  2274   2347         if( mkdir(zFile, mode) ){
  2275   2348           /* The mkdir() call to create the directory failed. This might not
  2276   2349           ** be an error though - if there is already a directory at the same
  2277   2350           ** path and either the permissions already match or can be changed
  2278   2351           ** to do so using chmod(), it is not an error.  */
  2279   2352           struct stat sStat;
  2280   2353           if( errno!=EEXIST
  2281         -         || 0!=stat(zFile, &sStat)
         2354  +         || 0!=fileStat(zFile, &sStat)
  2282   2355            || !S_ISDIR(sStat.st_mode)
  2283   2356            || ((sStat.st_mode&0777)!=(mode&0777) && 0!=chmod(zFile, mode&0777))
  2284   2357           ){
  2285   2358             return 1;
  2286   2359           }
  2287   2360         }
  2288   2361       }else{
................................................................................
  2312   2385   #if defined(_WIN32)
  2313   2386       /* Windows */
  2314   2387       FILETIME lastAccess;
  2315   2388       FILETIME lastWrite;
  2316   2389       SYSTEMTIME currentTime;
  2317   2390       LONGLONG intervals;
  2318   2391       HANDLE hFile;
         2392  +    LPWSTR zUnicodeName;
         2393  +    extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
         2394  +
  2319   2395       GetSystemTime(&currentTime);
  2320   2396       SystemTimeToFileTime(&currentTime, &lastAccess);
  2321   2397       intervals = Int32x32To64(mtime, 10000000) + 116444736000000000;
  2322   2398       lastWrite.dwLowDateTime = (DWORD)intervals;
  2323   2399       lastWrite.dwHighDateTime = intervals >> 32;
  2324         -    hFile = CreateFile(
  2325         -      zFile, FILE_WRITE_ATTRIBUTES, 0, NULL, OPEN_EXISTING,
         2400  +    zUnicodeName = sqlite3_win32_utf8_to_unicode(zFile);
         2401  +    if( zUnicodeName==0 ){
         2402  +      return 1;
         2403  +    }
         2404  +    hFile = CreateFileW(
         2405  +      zUnicodeName, FILE_WRITE_ATTRIBUTES, 0, NULL, OPEN_EXISTING,
  2326   2406         FILE_FLAG_BACKUP_SEMANTICS, NULL
  2327   2407       );
         2408  +    sqlite3_free(zUnicodeName);
  2328   2409       if( hFile!=INVALID_HANDLE_VALUE ){
  2329   2410         BOOL bResult = SetFileTime(hFile, NULL, &lastAccess, &lastWrite);
  2330   2411         CloseHandle(hFile);
  2331   2412         return !bResult;
  2332   2413       }else{
  2333   2414         return 1;
  2334   2415       }
  2335         -#elif defined(AT_FDCWD) && 0 /* utimensat() is not univerally available */
         2416  +#elif defined(AT_FDCWD) && 0 /* utimensat() is not universally available */
  2336   2417       /* Recent unix */
  2337   2418       struct timespec times[2];
  2338   2419       times[0].tv_nsec = times[1].tv_nsec = 0;
  2339   2420       times[0].tv_sec = time(0);
  2340   2421       times[1].tv_sec = mtime;
  2341   2422       if( utimensat(AT_FDCWD, zFile, times, AT_SYMLINK_NOFOLLOW) ){
  2342   2423         return 1;
................................................................................
  2528   2609     int i;
  2529   2610     for(i=0; i<=pCur->iLvl; i++){
  2530   2611       FsdirLevel *pLvl = &pCur->aLvl[i];
  2531   2612       if( pLvl->pDir ) closedir(pLvl->pDir);
  2532   2613       sqlite3_free(pLvl->zDir);
  2533   2614     }
  2534   2615     sqlite3_free(pCur->zPath);
         2616  +  sqlite3_free(pCur->aLvl);
  2535   2617     pCur->aLvl = 0;
  2536   2618     pCur->zPath = 0;
  2537   2619     pCur->zBase = 0;
  2538   2620     pCur->nBase = 0;
         2621  +  pCur->nLvl = 0;
  2539   2622     pCur->iLvl = -1;
  2540   2623     pCur->iRowid = 1;
  2541   2624   }
  2542   2625   
  2543   2626   /*
  2544   2627   ** Destructor for an fsdir_cursor.
  2545   2628   */
  2546   2629   static int fsdirClose(sqlite3_vtab_cursor *cur){
  2547   2630     fsdir_cursor *pCur = (fsdir_cursor*)cur;
  2548   2631   
  2549   2632     fsdirResetCursor(pCur);
  2550         -  sqlite3_free(pCur->aLvl);
  2551   2633     sqlite3_free(pCur);
  2552   2634     return SQLITE_OK;
  2553   2635   }
  2554   2636   
  2555   2637   /*
  2556   2638   ** Set the error message for the virtual table associated with cursor
  2557   2639   ** pCur to the results of vprintf(zFmt, ...).
................................................................................
  2604   2686         if( pEntry->d_name[0]=='.' ){
  2605   2687          if( pEntry->d_name[1]=='.' && pEntry->d_name[2]=='\0' ) continue;
  2606   2688          if( pEntry->d_name[1]=='\0' ) continue;
  2607   2689         }
  2608   2690         sqlite3_free(pCur->zPath);
  2609   2691         pCur->zPath = sqlite3_mprintf("%s/%s", pLvl->zDir, pEntry->d_name);
  2610   2692         if( pCur->zPath==0 ) return SQLITE_NOMEM;
  2611         -      if( lstat(pCur->zPath, &pCur->sStat) ){
         2693  +      if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
  2612   2694           fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
  2613   2695           return SQLITE_ERROR;
  2614   2696         }
  2615   2697         return SQLITE_OK;
  2616   2698       }
  2617   2699       closedir(pLvl->pDir);
  2618   2700       sqlite3_free(pLvl->zDir);
................................................................................
  2738   2820     }else{
  2739   2821       pCur->zPath = sqlite3_mprintf("%s", zDir);
  2740   2822     }
  2741   2823   
  2742   2824     if( pCur->zPath==0 ){
  2743   2825       return SQLITE_NOMEM;
  2744   2826     }
  2745         -  if( lstat(pCur->zPath, &pCur->sStat) ){
         2827  +  if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
  2746   2828       fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
  2747   2829       return SQLITE_ERROR;
  2748   2830     }
  2749   2831   
  2750   2832     return SQLITE_OK;
  2751   2833   }
  2752   2834   
................................................................................
  2929   3011   struct completion_cursor {
  2930   3012     sqlite3_vtab_cursor base;  /* Base class - must be first */
  2931   3013     sqlite3 *db;               /* Database connection for this cursor */
  2932   3014     int nPrefix, nLine;        /* Number of bytes in zPrefix and zLine */
  2933   3015     char *zPrefix;             /* The prefix for the word we want to complete */
  2934   3016     char *zLine;               /* The whole that we want to complete */
  2935   3017     const char *zCurrentRow;   /* Current output row */
         3018  +  int szRow;                 /* Length of the zCurrentRow string */
  2936   3019     sqlite3_stmt *pStmt;       /* Current statement */
  2937   3020     sqlite3_int64 iRowid;      /* The rowid */
  2938   3021     int ePhase;                /* Current phase */
  2939   3022     int j;                     /* inter-phase counter */
  2940   3023   };
  2941   3024   
  2942   3025   /* Values for ePhase:
................................................................................
  2945   3028   #define COMPLETION_KEYWORDS      1
  2946   3029   #define COMPLETION_PRAGMAS       2
  2947   3030   #define COMPLETION_FUNCTIONS     3
  2948   3031   #define COMPLETION_COLLATIONS    4
  2949   3032   #define COMPLETION_INDEXES       5
  2950   3033   #define COMPLETION_TRIGGERS      6
  2951   3034   #define COMPLETION_DATABASES     7
  2952         -#define COMPLETION_TABLES        8
         3035  +#define COMPLETION_TABLES        8    /* Also VIEWs and TRIGGERs */
  2953   3036   #define COMPLETION_COLUMNS       9
  2954   3037   #define COMPLETION_MODULES       10
  2955   3038   #define COMPLETION_EOF           11
  2956   3039   
  2957   3040   /*
  2958   3041   ** The completionConnect() method is invoked to create a new
  2959   3042   ** completion_vtab that describes the completion virtual table.
................................................................................
  3041   3124   */
  3042   3125   static int completionClose(sqlite3_vtab_cursor *cur){
  3043   3126     completionCursorReset((completion_cursor*)cur);
  3044   3127     sqlite3_free(cur);
  3045   3128     return SQLITE_OK;
  3046   3129   }
  3047   3130   
  3048         -/*
  3049         -** All SQL keywords understood by SQLite
  3050         -*/
  3051         -static const char *completionKwrds[] = {
  3052         -  "ABORT", "ACTION", "ADD", "AFTER", "ALL", "ALTER", "ANALYZE", "AND", "AS",
  3053         -  "ASC", "ATTACH", "AUTOINCREMENT", "BEFORE", "BEGIN", "BETWEEN", "BY",
  3054         -  "CASCADE", "CASE", "CAST", "CHECK", "COLLATE", "COLUMN", "COMMIT",
  3055         -  "CONFLICT", "CONSTRAINT", "CREATE", "CROSS", "CURRENT_DATE",
  3056         -  "CURRENT_TIME", "CURRENT_TIMESTAMP", "DATABASE", "DEFAULT", "DEFERRABLE",
  3057         -  "DEFERRED", "DELETE", "DESC", "DETACH", "DISTINCT", "DROP", "EACH",
  3058         -  "ELSE", "END", "ESCAPE", "EXCEPT", "EXCLUSIVE", "EXISTS", "EXPLAIN",
  3059         -  "FAIL", "FOR", "FOREIGN", "FROM", "FULL", "GLOB", "GROUP", "HAVING", "IF",
  3060         -  "IGNORE", "IMMEDIATE", "IN", "INDEX", "INDEXED", "INITIALLY", "INNER",
  3061         -  "INSERT", "INSTEAD", "INTERSECT", "INTO", "IS", "ISNULL", "JOIN", "KEY",
  3062         -  "LEFT", "LIKE", "LIMIT", "MATCH", "NATURAL", "NO", "NOT", "NOTNULL",
  3063         -  "NULL", "OF", "OFFSET", "ON", "OR", "ORDER", "OUTER", "PLAN", "PRAGMA",
  3064         -  "PRIMARY", "QUERY", "RAISE", "RECURSIVE", "REFERENCES", "REGEXP",
  3065         -  "REINDEX", "RELEASE", "RENAME", "REPLACE", "RESTRICT", "RIGHT",
  3066         -  "ROLLBACK", "ROW", "SAVEPOINT", "SELECT", "SET", "TABLE", "TEMP",
  3067         -  "TEMPORARY", "THEN", "TO", "TRANSACTION", "TRIGGER", "UNION", "UNIQUE",
  3068         -  "UPDATE", "USING", "VACUUM", "VALUES", "VIEW", "VIRTUAL", "WHEN", "WHERE",
  3069         -  "WITH", "WITHOUT",
  3070         -};
  3071         -#define completionKwCount \
  3072         -   (int)(sizeof(completionKwrds)/sizeof(completionKwrds[0]))
  3073         -
  3074   3131   /*
  3075   3132   ** Advance a completion_cursor to its next row of output.
  3076   3133   **
  3077   3134   ** The ->ePhase, ->j, and ->pStmt fields of the completion_cursor object
  3078   3135   ** record the current state of the scan.  This routine sets ->zCurrentRow
  3079   3136   ** to the current row of output and then returns.  If no more rows remain,
  3080   3137   ** then ->ePhase is set to COMPLETION_EOF which will signal the virtual
................................................................................
  3089   3146     completion_cursor *pCur = (completion_cursor*)cur;
  3090   3147     int eNextPhase = 0;  /* Next phase to try if current phase reaches end */
  3091   3148     int iCol = -1;       /* If >=0, step pCur->pStmt and use the i-th column */
  3092   3149     pCur->iRowid++;
  3093   3150     while( pCur->ePhase!=COMPLETION_EOF ){
  3094   3151       switch( pCur->ePhase ){
  3095   3152         case COMPLETION_KEYWORDS: {
  3096         -        if( pCur->j >= completionKwCount ){
         3153  +        if( pCur->j >= sqlite3_keyword_count() ){
  3097   3154             pCur->zCurrentRow = 0;
  3098   3155             pCur->ePhase = COMPLETION_DATABASES;
  3099   3156           }else{
  3100         -          pCur->zCurrentRow = completionKwrds[pCur->j++];
         3157  +          sqlite3_keyword_name(pCur->j++, &pCur->zCurrentRow, &pCur->szRow);
  3101   3158           }
  3102   3159           iCol = -1;
  3103   3160           break;
  3104   3161         }
  3105   3162         case COMPLETION_DATABASES: {
  3106   3163           if( pCur->pStmt==0 ){
  3107   3164             sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1,
................................................................................
  3117   3174             char *zSql = 0;
  3118   3175             const char *zSep = "";
  3119   3176             sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0);
  3120   3177             while( sqlite3_step(pS2)==SQLITE_ROW ){
  3121   3178               const char *zDb = (const char*)sqlite3_column_text(pS2, 1);
  3122   3179               zSql = sqlite3_mprintf(
  3123   3180                  "%z%s"
  3124         -               "SELECT name FROM \"%w\".sqlite_master"
  3125         -               " WHERE type='table'",
         3181  +               "SELECT name FROM \"%w\".sqlite_master",
  3126   3182                  zSql, zSep, zDb
  3127   3183               );
  3128   3184               if( zSql==0 ) return SQLITE_NOMEM;
  3129   3185               zSep = " UNION ";
  3130   3186             }
  3131   3187             sqlite3_finalize(pS2);
  3132   3188             sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
................................................................................
  3166   3222       if( iCol<0 ){
  3167   3223         /* This case is when the phase presets zCurrentRow */
  3168   3224         if( pCur->zCurrentRow==0 ) continue;
  3169   3225       }else{
  3170   3226         if( sqlite3_step(pCur->pStmt)==SQLITE_ROW ){
  3171   3227           /* Extract the next row of content */
  3172   3228           pCur->zCurrentRow = (const char*)sqlite3_column_text(pCur->pStmt, iCol);
         3229  +        pCur->szRow = sqlite3_column_bytes(pCur->pStmt, iCol);
  3173   3230         }else{
  3174   3231           /* When all rows are finished, advance to the next phase */
  3175   3232           sqlite3_finalize(pCur->pStmt);
  3176   3233           pCur->pStmt = 0;
  3177   3234           pCur->ePhase = eNextPhase;
  3178   3235           continue;
  3179   3236         }
  3180   3237       }
  3181   3238       if( pCur->nPrefix==0 ) break;
  3182         -    if( sqlite3_strnicmp(pCur->zPrefix, pCur->zCurrentRow, pCur->nPrefix)==0 ){
         3239  +    if( pCur->nPrefix<=pCur->szRow
         3240  +     && sqlite3_strnicmp(pCur->zPrefix, pCur->zCurrentRow, pCur->nPrefix)==0
         3241  +    ){
  3183   3242         break;
  3184   3243       }
  3185   3244     }
  3186   3245   
  3187   3246     return SQLITE_OK;
  3188   3247   }
  3189   3248   
................................................................................
  3195   3254     sqlite3_vtab_cursor *cur,   /* The cursor */
  3196   3255     sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
  3197   3256     int i                       /* Which column to return */
  3198   3257   ){
  3199   3258     completion_cursor *pCur = (completion_cursor*)cur;
  3200   3259     switch( i ){
  3201   3260       case COMPLETION_COLUMN_CANDIDATE: {
  3202         -      sqlite3_result_text(ctx, pCur->zCurrentRow, -1, SQLITE_TRANSIENT);
         3261  +      sqlite3_result_text(ctx, pCur->zCurrentRow, pCur->szRow,SQLITE_TRANSIENT);
  3203   3262         break;
  3204   3263       }
  3205   3264       case COMPLETION_COLUMN_PREFIX: {
  3206   3265         sqlite3_result_text(ctx, pCur->zPrefix, -1, SQLITE_TRANSIENT);
  3207   3266         break;
  3208   3267       }
  3209   3268       case COMPLETION_COLUMN_WHOLELINE: {
................................................................................
  3990   4049   **    *  Only the "inflate/deflate" (zlib) compression method is supported
  3991   4050   */
  3992   4051   SQLITE_EXTENSION_INIT1
  3993   4052   #include <stdio.h>
  3994   4053   #include <string.h>
  3995   4054   #include <assert.h>
  3996   4055   
  3997         -#include <sys/types.h>
  3998         -#include <sys/stat.h>
  3999         -#include <fcntl.h>
  4000         -#if !defined(_WIN32) && !defined(WIN32)
  4001         -#  include <unistd.h>
  4002         -#  include <dirent.h>
  4003         -#  include <utime.h>
  4004         -#else
  4005         -#  include <io.h>
  4006         -#endif
  4007         -#include <time.h>
  4008         -#include <errno.h>
  4009         -
  4010   4056   #include <zlib.h>
  4011   4057   
  4012   4058   #ifndef SQLITE_OMIT_VIRTUALTABLE
  4013   4059   
  4014   4060   #ifndef SQLITE_AMALGAMATION
         4061  +
  4015   4062   /* typedef sqlite3_int64 i64; */
  4016   4063   /* typedef unsigned char u8; */
  4017   4064   typedef unsigned short u16;
  4018   4065   typedef unsigned long u32;
  4019   4066   #define MIN(a,b) ((a)<(b) ? (a) : (b))
         4067  +
         4068  +#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
         4069  +# define ALWAYS(X)      (1)
         4070  +# define NEVER(X)       (0)
         4071  +#elif !defined(NDEBUG)
         4072  +# define ALWAYS(X)      ((X)?1:(assert(0),0))
         4073  +# define NEVER(X)       ((X)?(assert(0),1):0)
         4074  +#else
         4075  +# define ALWAYS(X)      (X)
         4076  +# define NEVER(X)       (X)
         4077  +#endif
         4078  +
         4079  +#endif   /* SQLITE_AMALGAMATION */
         4080  +
         4081  +/*
         4082  +** Definitions for mode bitmasks S_IFDIR, S_IFREG and S_IFLNK.
         4083  +**
         4084  +** In some ways it would be better to obtain these values from system 
         4085  +** header files. But, the dependency is undesirable and (a) these
         4086  +** have been stable for decades, (b) the values are part of POSIX and
         4087  +** are also made explicit in [man stat], and (c) are part of the 
         4088  +** file format for zip archives.
         4089  +*/
         4090  +#ifndef S_IFDIR
         4091  +# define S_IFDIR 0040000
         4092  +#endif
         4093  +#ifndef S_IFREG
         4094  +# define S_IFREG 0100000
         4095  +#endif
         4096  +#ifndef S_IFLNK
         4097  +# define S_IFLNK 0120000
  4020   4098   #endif
  4021   4099   
  4022   4100   static const char ZIPFILE_SCHEMA[] = 
  4023   4101     "CREATE TABLE y("
  4024   4102       "name PRIMARY KEY,"  /* 0: Name of file in zip archive */
  4025   4103       "mode,"              /* 1: POSIX mode for file */
  4026   4104       "mtime,"             /* 2: Last modification time (secs since 1970)*/
................................................................................
  4053   4131   **   11 means "utf-8 filename and comment".
  4054   4132   **
  4055   4133   ** ZIPFILE_SIGNATURE_CDS:
  4056   4134   **   First 4 bytes of a valid CDS record.
  4057   4135   **
  4058   4136   ** ZIPFILE_SIGNATURE_LFH:
  4059   4137   **   First 4 bytes of a valid LFH record.
         4138  +**
         4139  +** ZIPFILE_SIGNATURE_EOCD
         4140  +**   First 4 bytes of a valid EOCD record.
  4060   4141   */
  4061   4142   #define ZIPFILE_EXTRA_TIMESTAMP   0x5455
  4062   4143   #define ZIPFILE_NEWENTRY_MADEBY   ((3<<8) + 30)
  4063   4144   #define ZIPFILE_NEWENTRY_REQUIRED 20
  4064   4145   #define ZIPFILE_NEWENTRY_FLAGS    0x800
  4065   4146   #define ZIPFILE_SIGNATURE_CDS     0x02014b50
  4066   4147   #define ZIPFILE_SIGNATURE_LFH     0x04034b50
  4067   4148   #define ZIPFILE_SIGNATURE_EOCD    0x06054b50
         4149  +
         4150  +/*
         4151  +** The sizes of the fixed-size part of each of the three main data 
         4152  +** structures in a zip archive.
         4153  +*/
  4068   4154   #define ZIPFILE_LFH_FIXED_SZ      30
  4069         -
  4070         -/*
  4071         -** Set the error message contained in context ctx to the results of
  4072         -** vprintf(zFmt, ...).
  4073         -*/
  4074         -static void zipfileCtxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
  4075         -  char *zMsg = 0;
  4076         -  va_list ap;
  4077         -  va_start(ap, zFmt);
  4078         -  zMsg = sqlite3_vmprintf(zFmt, ap);
  4079         -  sqlite3_result_error(ctx, zMsg, -1);
  4080         -  sqlite3_free(zMsg);
  4081         -  va_end(ap);
  4082         -}
  4083         -
         4155  +#define ZIPFILE_EOCD_FIXED_SZ     22
         4156  +#define ZIPFILE_CDS_FIXED_SZ      46
  4084   4157   
  4085   4158   /*
  4086   4159   *** 4.3.16  End of central directory record:
  4087   4160   ***
  4088   4161   ***   end of central dir signature    4 bytes  (0x06054b50)
  4089   4162   ***   number of this disk             2 bytes
  4090   4163   ***   number of the disk with the
................................................................................
  4182   4255     u32 szUncompressed;
  4183   4256     u16 nFile;
  4184   4257     u16 nExtra;
  4185   4258   };
  4186   4259   
  4187   4260   typedef struct ZipfileEntry ZipfileEntry;
  4188   4261   struct ZipfileEntry {
  4189         -  char *zPath;               /* Path of zipfile entry */
  4190         -  u8 *aCdsEntry;             /* Buffer containing entire CDS entry */
  4191         -  int nCdsEntry;             /* Size of buffer aCdsEntry[] in bytes */
  4192         -  int bDeleted;              /* True if entry has been deleted */
         4262  +  ZipfileCDS cds;            /* Parsed CDS record */
         4263  +  u32 mUnixTime;             /* Modification time, in UNIX format */
         4264  +  u8 *aExtra;                /* cds.nExtra+cds.nComment bytes of extra data */
         4265  +  i64 iDataOff;              /* Offset to data in file (if aData==0) */
         4266  +  u8 *aData;                 /* cds.szCompressed bytes of compressed data */
  4193   4267     ZipfileEntry *pNext;       /* Next element in in-memory CDS */
  4194   4268   };
  4195   4269   
  4196   4270   /* 
  4197         -** Cursor type for recursively iterating through a directory structure.
         4271  +** Cursor type for zipfile tables.
  4198   4272   */
  4199   4273   typedef struct ZipfileCsr ZipfileCsr;
  4200   4274   struct ZipfileCsr {
  4201   4275     sqlite3_vtab_cursor base;  /* Base class - must be first */
  4202   4276     i64 iId;                   /* Cursor ID */
  4203         -  int bEof;                  /* True when at EOF */
         4277  +  u8 bEof;                   /* True when at EOF */
         4278  +  u8 bNoop;                  /* If next xNext() call is no-op */
  4204   4279   
  4205   4280     /* Used outside of write transactions */
  4206   4281     FILE *pFile;               /* Zip file */
  4207   4282     i64 iNextOff;              /* Offset of next record in central directory */
  4208   4283     ZipfileEOCD eocd;          /* Parse of central directory record */
  4209   4284   
  4210         -  /* Used inside write transactions */
  4211         -  ZipfileEntry *pCurrent;
  4212         -
  4213         -  ZipfileCDS cds;            /* Central Directory Structure */
  4214         -  ZipfileLFH lfh;            /* Local File Header for current entry */
  4215         -  i64 iDataOff;              /* Offset in zipfile to data */
  4216         -  u32 mTime;                 /* Extended mtime value */
  4217         -  int flags;                 /* Flags byte (see below for bits) */
         4285  +  ZipfileEntry *pFreeEntry;  /* Free this list when cursor is closed or reset */
         4286  +  ZipfileEntry *pCurrent;    /* Current entry */
  4218   4287     ZipfileCsr *pCsrNext;      /* Next cursor on same virtual table */
  4219   4288   };
  4220   4289   
  4221         -/*
  4222         -** Values for ZipfileCsr.flags.
  4223         -*/
  4224         -#define ZIPFILE_MTIME_VALID 0x0001
  4225         -
  4226   4290   typedef struct ZipfileTab ZipfileTab;
  4227   4291   struct ZipfileTab {
  4228   4292     sqlite3_vtab base;         /* Base class - must be first */
  4229   4293     char *zFile;               /* Zip file this table accesses (may be NULL) */
         4294  +  sqlite3 *db;               /* Host database connection */
  4230   4295     u8 *aBuffer;               /* Temporary buffer used for various tasks */
  4231   4296   
  4232   4297     ZipfileCsr *pCsrList;      /* List of cursors */
  4233   4298     i64 iNextCsrid;
  4234   4299   
  4235   4300     /* The following are used by write transactions only */
  4236   4301     ZipfileEntry *pFirstEntry; /* Linked list of all files (if pWriteFd!=0) */
  4237   4302     ZipfileEntry *pLastEntry;  /* Last element in pFirstEntry list */
  4238   4303     FILE *pWriteFd;            /* File handle open on zip archive */
  4239   4304     i64 szCurrent;             /* Current size of zip archive */
  4240   4305     i64 szOrig;                /* Size of archive at start of transaction */
  4241   4306   };
  4242   4307   
         4308  +/*
         4309  +** Set the error message contained in context ctx to the results of
         4310  +** vprintf(zFmt, ...).
         4311  +*/
         4312  +static void zipfileCtxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
         4313  +  char *zMsg = 0;
         4314  +  va_list ap;
         4315  +  va_start(ap, zFmt);
         4316  +  zMsg = sqlite3_vmprintf(zFmt, ap);
         4317  +  sqlite3_result_error(ctx, zMsg, -1);
         4318  +  sqlite3_free(zMsg);
         4319  +  va_end(ap);
         4320  +}
         4321  +
         4322  +/*
         4323  +** If string zIn is quoted, dequote it in place. Otherwise, if the string
         4324  +** is not quoted, do nothing.
         4325  +*/
  4243   4326   static void zipfileDequote(char *zIn){
  4244   4327     char q = zIn[0];
  4245   4328     if( q=='"' || q=='\'' || q=='`' || q=='[' ){
  4246         -    char c;
  4247   4329       int iIn = 1;
  4248   4330       int iOut = 0;
  4249   4331       if( q=='[' ) q = ']';
  4250         -    while( (c = zIn[iIn++]) ){
  4251         -      if( c==q ){
  4252         -        if( zIn[iIn++]!=q ) break;
  4253         -      }
         4332  +    while( ALWAYS(zIn[iIn]) ){
         4333  +      char c = zIn[iIn++];
         4334  +      if( c==q && zIn[iIn++]!=q ) break;
  4254   4335         zIn[iOut++] = c;
  4255   4336       }
  4256   4337       zIn[iOut] = '\0';
  4257   4338     }
  4258   4339   }
  4259   4340   
  4260   4341   /*
................................................................................
  4273   4354     char **pzErr
  4274   4355   ){
  4275   4356     int nByte = sizeof(ZipfileTab) + ZIPFILE_BUFFER_SIZE;
  4276   4357     int nFile = 0;
  4277   4358     const char *zFile = 0;
  4278   4359     ZipfileTab *pNew = 0;
  4279   4360     int rc;
         4361  +
         4362  +  /* If the table name is not "zipfile", require that the argument be
         4363  +  ** specified. This stops zipfile tables from being created as:
         4364  +  **
         4365  +  **   CREATE VIRTUAL TABLE zzz USING zipfile();
         4366  +  **
         4367  +  ** It does not prevent:
         4368  +  **
         4369  +  **   CREATE VIRTUAL TABLE zipfile USING zipfile();
         4370  +  */
         4371  +  assert( 0==sqlite3_stricmp(argv[0], "zipfile") );
         4372  +  if( (0!=sqlite3_stricmp(argv[2], "zipfile") && argc<4) || argc>4 ){
         4373  +    *pzErr = sqlite3_mprintf("zipfile constructor requires one argument");
         4374  +    return SQLITE_ERROR;
         4375  +  }
  4280   4376   
  4281   4377     if( argc>3 ){
  4282   4378       zFile = argv[3];
  4283   4379       nFile = (int)strlen(zFile)+1;
  4284   4380     }
  4285   4381   
  4286   4382     rc = sqlite3_declare_vtab(db, ZIPFILE_SCHEMA);
  4287   4383     if( rc==SQLITE_OK ){
  4288   4384       pNew = (ZipfileTab*)sqlite3_malloc(nByte+nFile);
  4289   4385       if( pNew==0 ) return SQLITE_NOMEM;
  4290   4386       memset(pNew, 0, nByte+nFile);
         4387  +    pNew->db = db;
  4291   4388       pNew->aBuffer = (u8*)&pNew[1];
  4292   4389       if( zFile ){
  4293   4390         pNew->zFile = (char*)&pNew->aBuffer[ZIPFILE_BUFFER_SIZE];
  4294   4391         memcpy(pNew->zFile, zFile, nFile);
  4295   4392         zipfileDequote(pNew->zFile);
  4296   4393       }
  4297   4394     }
  4298   4395     *ppVtab = (sqlite3_vtab*)pNew;
  4299   4396     return rc;
  4300   4397   }
         4398  +
         4399  +/*
         4400  +** Free the ZipfileEntry structure indicated by the only argument.
         4401  +*/
         4402  +static void zipfileEntryFree(ZipfileEntry *p){
         4403  +  if( p ){
         4404  +    sqlite3_free(p->cds.zFile);
         4405  +    sqlite3_free(p);
         4406  +  }
         4407  +}
         4408  +
         4409  +/*
         4410  +** Release resources that should be freed at the end of a write 
         4411  +** transaction.
         4412  +*/
         4413  +static void zipfileCleanupTransaction(ZipfileTab *pTab){
         4414  +  ZipfileEntry *pEntry;
         4415  +  ZipfileEntry *pNext;
         4416  +
         4417  +  if( pTab->pWriteFd ){
         4418  +    fclose(pTab->pWriteFd);
         4419  +    pTab->pWriteFd = 0;
         4420  +  }
         4421  +  for(pEntry=pTab->pFirstEntry; pEntry; pEntry=pNext){
         4422  +    pNext = pEntry->pNext;
         4423  +    zipfileEntryFree(pEntry);
         4424  +  }
         4425  +  pTab->pFirstEntry = 0;
         4426  +  pTab->pLastEntry = 0;
         4427  +  pTab->szCurrent = 0;
         4428  +  pTab->szOrig = 0;
         4429  +}
  4301   4430   
  4302   4431   /*
  4303   4432   ** This method is the destructor for zipfile vtab objects.
  4304   4433   */
  4305   4434   static int zipfileDisconnect(sqlite3_vtab *pVtab){
         4435  +  zipfileCleanupTransaction((ZipfileTab*)pVtab);
  4306   4436     sqlite3_free(pVtab);
  4307   4437     return SQLITE_OK;
  4308   4438   }
  4309   4439   
  4310   4440   /*
  4311   4441   ** Constructor for a new ZipfileCsr object.
  4312   4442   */
................................................................................
  4326   4456   }
  4327   4457   
  4328   4458   /*
  4329   4459   ** Reset a cursor back to the state it was in when first returned
  4330   4460   ** by zipfileOpen().
  4331   4461   */
  4332   4462   static void zipfileResetCursor(ZipfileCsr *pCsr){
  4333         -  sqlite3_free(pCsr->cds.zFile);
  4334         -  pCsr->cds.zFile = 0;
         4463  +  ZipfileEntry *p;
         4464  +  ZipfileEntry *pNext;
         4465  +
  4335   4466     pCsr->bEof = 0;
  4336   4467     if( pCsr->pFile ){
  4337   4468       fclose(pCsr->pFile);
  4338   4469       pCsr->pFile = 0;
         4470  +    zipfileEntryFree(pCsr->pCurrent);
         4471  +    pCsr->pCurrent = 0;
         4472  +  }
         4473  +
         4474  +  for(p=pCsr->pFreeEntry; p; p=pNext){
         4475  +    pNext = p->pNext;
         4476  +    zipfileEntryFree(p);
  4339   4477     }
  4340   4478   }
  4341   4479   
  4342   4480   /*
  4343   4481   ** Destructor for an ZipfileCsr.
  4344   4482   */
  4345   4483   static int zipfileClose(sqlite3_vtab_cursor *cur){
  4346   4484     ZipfileCsr *pCsr = (ZipfileCsr*)cur;
  4347   4485     ZipfileTab *pTab = (ZipfileTab*)(pCsr->base.pVtab);
  4348   4486     ZipfileCsr **pp;
  4349   4487     zipfileResetCursor(pCsr);
  4350   4488   
  4351   4489     /* Remove this cursor from the ZipfileTab.pCsrList list. */
  4352         -  for(pp=&pTab->pCsrList; *pp; pp=&((*pp)->pCsrNext)){
  4353         -    if( *pp==pCsr ){ 
  4354         -      *pp = pCsr->pCsrNext;
  4355         -      break;
  4356         -    }
  4357         -  }
         4490  +  for(pp=&pTab->pCsrList; *pp!=pCsr; pp=&((*pp)->pCsrNext));
         4491  +  *pp = pCsr->pCsrNext;
  4358   4492   
  4359   4493     sqlite3_free(pCsr);
  4360   4494     return SQLITE_OK;
  4361   4495   }
  4362   4496   
  4363   4497   /*
  4364   4498   ** Set the error message for the virtual table associated with cursor
  4365   4499   ** pCsr to the results of vprintf(zFmt, ...).
  4366   4500   */
  4367         -static void zipfileSetErrmsg(ZipfileCsr *pCsr, const char *zFmt, ...){
         4501  +static void zipfileTableErr(ZipfileTab *pTab, const char *zFmt, ...){
         4502  +  va_list ap;
         4503  +  va_start(ap, zFmt);
         4504  +  sqlite3_free(pTab->base.zErrMsg);
         4505  +  pTab->base.zErrMsg = sqlite3_vmprintf(zFmt, ap);
         4506  +  va_end(ap);
         4507  +}
         4508  +static void zipfileCursorErr(ZipfileCsr *pCsr, const char *zFmt, ...){
  4368   4509     va_list ap;
  4369   4510     va_start(ap, zFmt);
         4511  +  sqlite3_free(pCsr->base.pVtab->zErrMsg);
  4370   4512     pCsr->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap);
  4371   4513     va_end(ap);
  4372   4514   }
  4373   4515   
         4516  +/*
         4517  +** Read nRead bytes of data from offset iOff of file pFile into buffer
         4518  +** aRead[]. Return SQLITE_OK if successful, or an SQLite error code
         4519  +** otherwise. 
         4520  +**
         4521  +** If an error does occur, output variable (*pzErrmsg) may be set to point
         4522  +** to an English language error message. It is the responsibility of the
         4523  +** caller to eventually free this buffer using
         4524  +** sqlite3_free().
         4525  +*/
  4374   4526   static int zipfileReadData(
  4375   4527     FILE *pFile,                    /* Read from this file */
  4376   4528     u8 *aRead,                      /* Read into this buffer */
  4377   4529     int nRead,                      /* Number of bytes to read */
  4378   4530     i64 iOff,                       /* Offset to read from */
  4379   4531     char **pzErrmsg                 /* OUT: Error message (from sqlite3_malloc) */
  4380   4532   ){
................................................................................
  4400   4552       pTab->base.zErrMsg = sqlite3_mprintf("error in fwrite()");
  4401   4553       return SQLITE_ERROR;
  4402   4554     }
  4403   4555     pTab->szCurrent += nWrite;
  4404   4556     return SQLITE_OK;
  4405   4557   }
  4406   4558   
         4559  +/*
         4560  +** Read and return a 16-bit little-endian unsigned integer from buffer aBuf.
         4561  +*/
  4407   4562   static u16 zipfileGetU16(const u8 *aBuf){
  4408   4563     return (aBuf[1] << 8) + aBuf[0];
  4409   4564   }
         4565  +
         4566  +/*
         4567  +** Read and return a 32-bit little-endian unsigned integer from buffer aBuf.
         4568  +*/
  4410   4569   static u32 zipfileGetU32(const u8 *aBuf){
  4411   4570     return ((u32)(aBuf[3]) << 24)
  4412   4571          + ((u32)(aBuf[2]) << 16)
  4413   4572          + ((u32)(aBuf[1]) <<  8)
  4414   4573          + ((u32)(aBuf[0]) <<  0);
  4415   4574   }
  4416   4575   
         4576  +/*
         4577  +** Write a 16-bit little endiate integer into buffer aBuf.
         4578  +*/
  4417   4579   static void zipfilePutU16(u8 *aBuf, u16 val){
  4418   4580     aBuf[0] = val & 0xFF;
  4419   4581     aBuf[1] = (val>>8) & 0xFF;
  4420   4582   }
         4583  +
         4584  +/*
         4585  +** Write a 32-bit little endiate integer into buffer aBuf.
         4586  +*/
  4421   4587   static void zipfilePutU32(u8 *aBuf, u32 val){
  4422   4588     aBuf[0] = val & 0xFF;
  4423   4589     aBuf[1] = (val>>8) & 0xFF;
  4424   4590     aBuf[2] = (val>>16) & 0xFF;
  4425   4591     aBuf[3] = (val>>24) & 0xFF;
  4426   4592   }
  4427   4593   
  4428   4594   #define zipfileRead32(aBuf) ( aBuf+=4, zipfileGetU32(aBuf-4) )
  4429   4595   #define zipfileRead16(aBuf) ( aBuf+=2, zipfileGetU16(aBuf-2) )
  4430   4596   
  4431   4597   #define zipfileWrite32(aBuf,val) { zipfilePutU32(aBuf,val); aBuf+=4; }
  4432   4598   #define zipfileWrite16(aBuf,val) { zipfilePutU16(aBuf,val); aBuf+=2; }
  4433   4599   
  4434         -static u8* zipfileCsrBuffer(ZipfileCsr *pCsr){
  4435         -  return ((ZipfileTab*)(pCsr->base.pVtab))->aBuffer;
  4436         -}
  4437         -
  4438   4600   /*
  4439   4601   ** Magic numbers used to read CDS records.
  4440   4602   */
  4441         -#define ZIPFILE_CDS_FIXED_SZ         46
  4442   4603   #define ZIPFILE_CDS_NFILE_OFF        28
         4604  +#define ZIPFILE_CDS_SZCOMPRESSED_OFF 20
  4443   4605   
  4444   4606   /*
  4445   4607   ** Decode the CDS record in buffer aBuf into (*pCDS). Return SQLITE_ERROR
  4446   4608   ** if the record is not well-formed, or SQLITE_OK otherwise.
  4447   4609   */
  4448   4610   static int zipfileReadCDS(u8 *aBuf, ZipfileCDS *pCDS){
  4449   4611     u8 *aRead = aBuf;
................................................................................
  4472   4634       assert( aRead==&aBuf[ZIPFILE_CDS_FIXED_SZ] );
  4473   4635     }
  4474   4636   
  4475   4637     return rc;
  4476   4638   }
  4477   4639   
  4478   4640   /*
  4479         -** Read the CDS record for the current entry from disk into pCsr->cds.
         4641  +** Decode the LFH record in buffer aBuf into (*pLFH). Return SQLITE_ERROR
         4642  +** if the record is not well-formed, or SQLITE_OK otherwise.
  4480   4643   */
  4481         -static int zipfileCsrReadCDS(ZipfileCsr *pCsr){
  4482         -  char **pzErr = &pCsr->base.pVtab->zErrMsg;
  4483         -  u8 *aRead;
  4484         -  int rc = SQLITE_OK;
  4485         -
  4486         -  sqlite3_free(pCsr->cds.zFile);
  4487         -  pCsr->cds.zFile = 0;
  4488         -
  4489         -  if( pCsr->pCurrent==0 ){
  4490         -    aRead = zipfileCsrBuffer(pCsr);
  4491         -    rc = zipfileReadData(
  4492         -        pCsr->pFile, aRead, ZIPFILE_CDS_FIXED_SZ, pCsr->iNextOff, pzErr
  4493         -    );
  4494         -  }else{
  4495         -    aRead = pCsr->pCurrent->aCdsEntry;
  4496         -  }
  4497         -
  4498         -  if( rc==SQLITE_OK ){
  4499         -    rc = zipfileReadCDS(aRead, &pCsr->cds);
  4500         -    if( rc!=SQLITE_OK ){
  4501         -      assert( pCsr->pCurrent==0 );
  4502         -      zipfileSetErrmsg(pCsr,"failed to read CDS at offset %lld",pCsr->iNextOff);
  4503         -    }else{
  4504         -      int nRead;
  4505         -      if( pCsr->pCurrent==0 ){
  4506         -        nRead = pCsr->cds.nFile + pCsr->cds.nExtra;
  4507         -        aRead = zipfileCsrBuffer(pCsr);
  4508         -        pCsr->iNextOff += ZIPFILE_CDS_FIXED_SZ;
  4509         -        rc = zipfileReadData(pCsr->pFile, aRead, nRead, pCsr->iNextOff, pzErr);
  4510         -      }else{
  4511         -        aRead = &aRead[ZIPFILE_CDS_FIXED_SZ];
  4512         -      }
  4513         -
  4514         -      if( rc==SQLITE_OK ){
  4515         -        pCsr->cds.zFile = sqlite3_mprintf("%.*s", (int)pCsr->cds.nFile, aRead);
  4516         -        pCsr->iNextOff += pCsr->cds.nFile;
  4517         -        pCsr->iNextOff += pCsr->cds.nExtra;
  4518         -        pCsr->iNextOff += pCsr->cds.nComment;
  4519         -      }
  4520         -
  4521         -      /* Scan the cds.nExtra bytes of "extra" fields for any that can
  4522         -      ** be interpreted. The general format of an extra field is:
  4523         -      **
  4524         -      **   Header ID    2 bytes
  4525         -      **   Data Size    2 bytes
  4526         -      **   Data         N bytes
  4527         -      **
  4528         -      */
  4529         -      if( rc==SQLITE_OK ){
  4530         -        u8 *p = &aRead[pCsr->cds.nFile];
  4531         -        u8 *pEnd = &p[pCsr->cds.nExtra];
  4532         -
  4533         -        while( p<pEnd ){
  4534         -          u16 id = zipfileRead16(p);
  4535         -          u16 nByte = zipfileRead16(p);
  4536         -
  4537         -          switch( id ){
  4538         -            case ZIPFILE_EXTRA_TIMESTAMP: {
  4539         -              u8 b = p[0];
  4540         -              if( b & 0x01 ){     /* 0x01 -> modtime is present */
  4541         -                pCsr->mTime = zipfileGetU32(&p[1]);
  4542         -                pCsr->flags |= ZIPFILE_MTIME_VALID;
  4543         -              }
  4544         -              break;
  4545         -            }
  4546         -          }
  4547         -
  4548         -          p += nByte;
  4549         -        }
  4550         -      }
  4551         -    }
  4552         -  }
  4553         -
  4554         -  return rc;
  4555         -}
  4556         -
  4557         -static FILE *zipfileGetFd(ZipfileCsr *pCsr){
  4558         -  if( pCsr->pFile ) return pCsr->pFile;
  4559         -  return ((ZipfileTab*)(pCsr->base.pVtab))->pWriteFd;
  4560         -}
  4561         -
  4562   4644   static int zipfileReadLFH(
  4563         -  FILE *pFd, 
  4564         -  i64 iOffset,
  4565         -  u8 *aTmp, 
  4566         -  ZipfileLFH *pLFH, 
  4567         -  char **pzErr
  4568         -){
  4569         -  u8 *aRead = aTmp;
  4570         -  static const int szFix = ZIPFILE_LFH_FIXED_SZ;
  4571         -  int rc;
  4572         -
  4573         -  rc = zipfileReadData(pFd, aRead, szFix, iOffset, pzErr);
  4574         -  if( rc==SQLITE_OK ){
  4575         -    u32 sig = zipfileRead32(aRead);
  4576         -    if( sig!=ZIPFILE_SIGNATURE_LFH ){
  4577         -      *pzErr = sqlite3_mprintf("failed to read LFH at offset %d", (int)iOffset);
  4578         -      rc = SQLITE_ERROR;
  4579         -    }else{
  4580         -      pLFH->iVersionExtract = zipfileRead16(aRead);
  4581         -      pLFH->flags = zipfileRead16(aRead);
  4582         -      pLFH->iCompression = zipfileRead16(aRead);
  4583         -      pLFH->mTime = zipfileRead16(aRead);
  4584         -      pLFH->mDate = zipfileRead16(aRead);
  4585         -      pLFH->crc32 = zipfileRead32(aRead);
  4586         -      pLFH->szCompressed = zipfileRead32(aRead);
  4587         -      pLFH->szUncompressed = zipfileRead32(aRead);
  4588         -      pLFH->nFile = zipfileRead16(aRead);
  4589         -      pLFH->nExtra = zipfileRead16(aRead);
  4590         -      assert( aRead==&aTmp[szFix] );
  4591         -    }
  4592         -  }
  4593         -  return rc;
  4594         -}
  4595         -
  4596         -static int zipfileCsrReadLFH(ZipfileCsr *pCsr){
  4597         -  FILE *pFile = zipfileGetFd(pCsr);
  4598         -  char **pzErr = &pCsr->base.pVtab->zErrMsg;
  4599         -  u8 *aRead = zipfileCsrBuffer(pCsr);
  4600         -  int rc = zipfileReadLFH(pFile, pCsr->cds.iOffset, aRead, &pCsr->lfh, pzErr);
  4601         -  pCsr->iDataOff =  pCsr->cds.iOffset + ZIPFILE_LFH_FIXED_SZ;
  4602         -  pCsr->iDataOff += pCsr->lfh.nFile+pCsr->lfh.nExtra;
  4603         -  return rc;
  4604         -}
  4605         -
  4606         -
  4607         -/*
  4608         -** Advance an ZipfileCsr to its next row of output.
  4609         -*/
  4610         -static int zipfileNext(sqlite3_vtab_cursor *cur){
  4611         -  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
  4612         -  int rc = SQLITE_OK;
  4613         -  pCsr->flags = 0;
  4614         -
  4615         -  if( pCsr->pCurrent==0 ){
  4616         -    i64 iEof = pCsr->eocd.iOffset + pCsr->eocd.nSize;
  4617         -    if( pCsr->iNextOff>=iEof ){
  4618         -      pCsr->bEof = 1;
  4619         -    }
  4620         -  }else{
  4621         -    assert( pCsr->pFile==0 );
  4622         -    do {
  4623         -      pCsr->pCurrent = pCsr->pCurrent->pNext;
  4624         -    }while( pCsr->pCurrent && pCsr->pCurrent->bDeleted );
  4625         -    if( pCsr->pCurrent==0 ){
  4626         -      pCsr->bEof = 1;
  4627         -    }
  4628         -  }
  4629         -
  4630         -  if( pCsr->bEof==0 ){
  4631         -    rc = zipfileCsrReadCDS(pCsr);
  4632         -    if( rc==SQLITE_OK ){
  4633         -      rc = zipfileCsrReadLFH(pCsr);
  4634         -    }
  4635         -  }
  4636         -
  4637         -  return rc;
  4638         -}
  4639         -
  4640         -/*
         4645  +  u8 *aBuffer,
         4646  +  ZipfileLFH *pLFH
         4647  +){
         4648  +  u8 *aRead = aBuffer;
         4649  +  int rc = SQLITE_OK;
         4650  +
         4651  +  u32 sig = zipfileRead32(aRead);
         4652  +  if( sig!=ZIPFILE_SIGNATURE_LFH ){
         4653  +    rc = SQLITE_ERROR;
         4654  +  }else{
         4655  +    pLFH->iVersionExtract = zipfileRead16(aRead);
         4656  +    pLFH->flags = zipfileRead16(aRead);
         4657  +    pLFH->iCompression = zipfileRead16(aRead);
         4658  +    pLFH->mTime = zipfileRead16(aRead);
         4659  +    pLFH->mDate = zipfileRead16(aRead);
         4660  +    pLFH->crc32 = zipfileRead32(aRead);
         4661  +    pLFH->szCompressed = zipfileRead32(aRead);
         4662  +    pLFH->szUncompressed = zipfileRead32(aRead);
         4663  +    pLFH->nFile = zipfileRead16(aRead);
         4664  +    pLFH->nExtra = zipfileRead16(aRead);
         4665  +  }
         4666  +  return rc;
         4667  +}
         4668  +
         4669  +
         4670  +/*
         4671  +** Buffer aExtra (size nExtra bytes) contains zip archive "extra" fields.
         4672  +** Scan through this buffer to find an "extra-timestamp" field. If one
         4673  +** exists, extract the 32-bit modification-timestamp from it and store
         4674  +** the value in output parameter *pmTime.
         4675  +**
         4676  +** Zero is returned if no extra-timestamp record could be found (and so
         4677  +** *pmTime is left unchanged), or non-zero otherwise.
         4678  +**
         4679  +** The general format of an extra field is:
         4680  +**
         4681  +**   Header ID    2 bytes
         4682  +**   Data Size    2 bytes
         4683  +**   Data         N bytes
         4684  +*/
         4685  +static int zipfileScanExtra(u8 *aExtra, int nExtra, u32 *pmTime){
         4686  +  int ret = 0;
         4687  +  u8 *p = aExtra;
         4688  +  u8 *pEnd = &aExtra[nExtra];
         4689  +
         4690  +  while( p<pEnd ){
         4691  +    u16 id = zipfileRead16(p);
         4692  +    u16 nByte = zipfileRead16(p);
         4693  +
         4694  +    switch( id ){
         4695  +      case ZIPFILE_EXTRA_TIMESTAMP: {
         4696  +        u8 b = p[0];
         4697  +        if( b & 0x01 ){     /* 0x01 -> modtime is present */
         4698  +          *pmTime = zipfileGetU32(&p[1]);
         4699  +          ret = 1;
         4700  +        }
         4701  +        break;
         4702  +      }
         4703  +    }
         4704  +
         4705  +    p += nByte;
         4706  +  }
         4707  +  return ret;
         4708  +}
         4709  +
         4710  +/*
         4711  +** Convert the standard MS-DOS timestamp stored in the mTime and mDate
         4712  +** fields of the CDS structure passed as the only argument to a 32-bit
         4713  +** UNIX seconds-since-the-epoch timestamp. Return the result.
         4714  +**
  4641   4715   ** "Standard" MS-DOS time format:
  4642   4716   **
  4643   4717   **   File modification time:
  4644   4718   **     Bits 00-04: seconds divided by 2
  4645   4719   **     Bits 05-10: minute
  4646   4720   **     Bits 11-15: hour
  4647   4721   **   File modification date:
  4648   4722   **     Bits 00-04: day
  4649   4723   **     Bits 05-08: month (1-12)
  4650   4724   **     Bits 09-15: years from 1980 
  4651         -*/
  4652         -static time_t zipfileMtime(ZipfileCsr *pCsr){
  4653         -  struct tm t;
  4654         -  memset(&t, 0, sizeof(t));
  4655         -  t.tm_sec = (pCsr->cds.mTime & 0x1F)*2;
  4656         -  t.tm_min = (pCsr->cds.mTime >> 5) & 0x2F;
  4657         -  t.tm_hour = (pCsr->cds.mTime >> 11) & 0x1F;
  4658         -
  4659         -  t.tm_mday = (pCsr->cds.mDate & 0x1F);
  4660         -  t.tm_mon = ((pCsr->cds.mDate >> 5) & 0x0F) - 1;
  4661         -  t.tm_year = 80 + ((pCsr->cds.mDate >> 9) & 0x7F);
  4662         -
  4663         -  return mktime(&t);
  4664         -}
  4665         -
  4666         -static void zipfileMtimeToDos(ZipfileCDS *pCds, u32 mTime){
  4667         -  time_t t = (time_t)mTime;
  4668         -  struct tm res;
  4669         -
  4670         -#if !defined(_WIN32) && !defined(WIN32)
  4671         -  localtime_r(&t, &res);
  4672         -#else
  4673         -  memcpy(&res, localtime(&t), sizeof(struct tm));
  4674         -#endif
  4675         -
  4676         -  pCds->mTime = (u16)(
  4677         -    (res.tm_sec / 2) + 
  4678         -    (res.tm_min << 5) +
  4679         -    (res.tm_hour << 11));
  4680         -
  4681         -  pCds->mDate = (u16)(
  4682         -    (res.tm_mday-1) +
  4683         -    ((res.tm_mon+1) << 5) +
  4684         -    ((res.tm_year-80) << 9));
  4685         -}
  4686         -
         4725  +**
         4726  +** https://msdn.microsoft.com/en-us/library/9kkf9tah.aspx
         4727  +*/
         4728  +static u32 zipfileMtime(ZipfileCDS *pCDS){
         4729  +  int Y = (1980 + ((pCDS->mDate >> 9) & 0x7F));
         4730  +  int M = ((pCDS->mDate >> 5) & 0x0F);
         4731  +  int D = (pCDS->mDate & 0x1F);
         4732  +  int B = -13;
         4733  +
         4734  +  int sec = (pCDS->mTime & 0x1F)*2;
         4735  +  int min = (pCDS->mTime >> 5) & 0x3F;
         4736  +  int hr = (pCDS->mTime >> 11) & 0x1F;
         4737  +  i64 JD;
         4738  +
         4739  +  /* JD = INT(365.25 * (Y+4716)) + INT(30.6001 * (M+1)) + D + B - 1524.5 */
         4740  +
         4741  +  /* Calculate the JD in seconds for noon on the day in question */
         4742  +  if( M<3 ){
         4743  +    Y = Y-1;
         4744  +    M = M+12;
         4745  +  }
         4746  +  JD = (i64)(24*60*60) * (
         4747  +      (int)(365.25 * (Y + 4716))
         4748  +    + (int)(30.6001 * (M + 1))
         4749  +    + D + B - 1524
         4750  +  );
         4751  +
         4752  +  /* Correct the JD for the time within the day */
         4753  +  JD += (hr-12) * 3600 + min * 60 + sec;
         4754  +
         4755  +  /* Convert JD to unix timestamp (the JD epoch is 2440587.5) */
         4756  +  return (u32)(JD - (i64)(24405875) * 24*60*6);
         4757  +}
         4758  +
         4759  +/*
         4760  +** The opposite of zipfileMtime(). This function populates the mTime and
         4761  +** mDate fields of the CDS structure passed as the first argument according
         4762  +** to the UNIX timestamp value passed as the second.
         4763  +*/
         4764  +static void zipfileMtimeToDos(ZipfileCDS *pCds, u32 mUnixTime){
         4765  +  /* Convert unix timestamp to JD (2440588 is noon on 1/1/1970) */
         4766  +  i64 JD = (i64)2440588 + mUnixTime / (24*60*60);
         4767  +
         4768  +  int A, B, C, D, E;
         4769  +  int yr, mon, day;
         4770  +  int hr, min, sec;
         4771  +
         4772  +  A = (int)((JD - 1867216.25)/36524.25);
         4773  +  A = (int)(JD + 1 + A - (A/4));
         4774  +  B = A + 1524;
         4775  +  C = (int)((B - 122.1)/365.25);
         4776  +  D = (36525*(C&32767))/100;
         4777  +  E = (int)((B-D)/30.6001);
         4778  +
         4779  +  day = B - D - (int)(30.6001*E);
         4780  +  mon = (E<14 ? E-1 : E-13);
         4781  +  yr = mon>2 ? C-4716 : C-4715;
         4782  +
         4783  +  hr = (mUnixTime % (24*60*60)) / (60*60);
         4784  +  min = (mUnixTime % (60*60)) / 60;
         4785  +  sec = (mUnixTime % 60);
         4786  +
         4787  +  if( yr>=1980 ){
         4788  +    pCds->mDate = (u16)(day + (mon << 5) + ((yr-1980) << 9));
         4789  +    pCds->mTime = (u16)(sec/2 + (min<<5) + (hr<<11));
         4790  +  }else{
         4791  +    pCds->mDate = pCds->mTime = 0;
         4792  +  }
         4793  +
         4794  +  assert( mUnixTime<315507600 
         4795  +       || mUnixTime==zipfileMtime(pCds) 
         4796  +       || ((mUnixTime % 2) && mUnixTime-1==zipfileMtime(pCds)) 
         4797  +       /* || (mUnixTime % 2) */
         4798  +  );
         4799  +}
         4800  +
         4801  +/*
         4802  +** If aBlob is not NULL, then it is a pointer to a buffer (nBlob bytes in
         4803  +** size) containing an entire zip archive image. Or, if aBlob is NULL,
         4804  +** then pFile is a file-handle open on a zip file. In either case, this
         4805  +** function creates a ZipfileEntry object based on the zip archive entry
         4806  +** for which the CDS record is at offset iOff.
         4807  +**
         4808  +** If successful, SQLITE_OK is returned and (*ppEntry) set to point to
         4809  +** the new object. Otherwise, an SQLite error code is returned and the
         4810  +** final value of (*ppEntry) undefined.
         4811  +*/
         4812  +static int zipfileGetEntry(
         4813  +  ZipfileTab *pTab,               /* Store any error message here */
         4814  +  const u8 *aBlob,                /* Pointer to in-memory file image */
         4815  +  int nBlob,                      /* Size of aBlob[] in bytes */
         4816  +  FILE *pFile,                    /* If aBlob==0, read from this file */
         4817  +  i64 iOff,                       /* Offset of CDS record */
         4818  +  ZipfileEntry **ppEntry          /* OUT: Pointer to new object */
         4819  +){
         4820  +  u8 *aRead;
         4821  +  char **pzErr = &pTab->base.zErrMsg;
         4822  +  int rc = SQLITE_OK;
         4823  +
         4824  +  if( aBlob==0 ){
         4825  +    aRead = pTab->aBuffer;
         4826  +    rc = zipfileReadData(pFile, aRead, ZIPFILE_CDS_FIXED_SZ, iOff, pzErr);
         4827  +  }else{
         4828  +    aRead = (u8*)&aBlob[iOff];
         4829  +  }
         4830  +
         4831  +  if( rc==SQLITE_OK ){
         4832  +    int nAlloc;
         4833  +    ZipfileEntry *pNew;
         4834  +
         4835  +    int nFile = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF]);
         4836  +    int nExtra = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+2]);
         4837  +    nExtra += zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+4]);
         4838  +
         4839  +    nAlloc = sizeof(ZipfileEntry) + nExtra;
         4840  +    if( aBlob ){
         4841  +      nAlloc += zipfileGetU32(&aRead[ZIPFILE_CDS_SZCOMPRESSED_OFF]);
         4842  +    }
         4843  +
         4844  +    pNew = (ZipfileEntry*)sqlite3_malloc(nAlloc);
         4845  +    if( pNew==0 ){
         4846  +      rc = SQLITE_NOMEM;
         4847  +    }else{
         4848  +      memset(pNew, 0, sizeof(ZipfileEntry));
         4849  +      rc = zipfileReadCDS(aRead, &pNew->cds);
         4850  +      if( rc!=SQLITE_OK ){
         4851  +        *pzErr = sqlite3_mprintf("failed to read CDS at offset %lld", iOff);
         4852  +      }else if( aBlob==0 ){
         4853  +        rc = zipfileReadData(
         4854  +            pFile, aRead, nExtra+nFile, iOff+ZIPFILE_CDS_FIXED_SZ, pzErr
         4855  +        );
         4856  +      }else{
         4857  +        aRead = (u8*)&aBlob[iOff + ZIPFILE_CDS_FIXED_SZ];
         4858  +      }
         4859  +    }
         4860  +
         4861  +    if( rc==SQLITE_OK ){
         4862  +      u32 *pt = &pNew->mUnixTime;
         4863  +      pNew->cds.zFile = sqlite3_mprintf("%.*s", nFile, aRead); 
         4864  +      pNew->aExtra = (u8*)&pNew[1];
         4865  +      memcpy(pNew->aExtra, &aRead[nFile], nExtra);
         4866  +      if( pNew->cds.zFile==0 ){
         4867  +        rc = SQLITE_NOMEM;
         4868  +      }else if( 0==zipfileScanExtra(&aRead[nFile], pNew->cds.nExtra, pt) ){
         4869  +        pNew->mUnixTime = zipfileMtime(&pNew->cds);
         4870  +      }
         4871  +    }
         4872  +
         4873  +    if( rc==SQLITE_OK ){
         4874  +      static const int szFix = ZIPFILE_LFH_FIXED_SZ;
         4875  +      ZipfileLFH lfh;
         4876  +      if( pFile ){
         4877  +        rc = zipfileReadData(pFile, aRead, szFix, pNew->cds.iOffset, pzErr);
         4878  +      }else{
         4879  +        aRead = (u8*)&aBlob[pNew->cds.iOffset];
         4880  +      }
         4881  +
         4882  +      rc = zipfileReadLFH(aRead, &lfh);
         4883  +      if( rc==SQLITE_OK ){
         4884  +        pNew->iDataOff =  pNew->cds.iOffset + ZIPFILE_LFH_FIXED_SZ;
         4885  +        pNew->iDataOff += lfh.nFile + lfh.nExtra;
         4886  +        if( aBlob && pNew->cds.szCompressed ){
         4887  +          pNew->aData = &pNew->aExtra[nExtra];
         4888  +          memcpy(pNew->aData, &aBlob[pNew->iDataOff], pNew->cds.szCompressed);
         4889  +        }
         4890  +      }else{
         4891  +        *pzErr = sqlite3_mprintf("failed to read LFH at offset %d", 
         4892  +            (int)pNew->cds.iOffset
         4893  +        );
         4894  +      }
         4895  +    }
         4896  +
         4897  +    if( rc!=SQLITE_OK ){
         4898  +      zipfileEntryFree(pNew);
         4899  +    }else{
         4900  +      *ppEntry = pNew;
         4901  +    }
         4902  +  }
         4903  +
         4904  +  return rc;
         4905  +}
         4906  +
         4907  +/*
         4908  +** Advance an ZipfileCsr to its next row of output.
         4909  +*/
         4910  +static int zipfileNext(sqlite3_vtab_cursor *cur){
         4911  +  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
         4912  +  int rc = SQLITE_OK;
         4913  +
         4914  +  if( pCsr->pFile ){
         4915  +    i64 iEof = pCsr->eocd.iOffset + pCsr->eocd.nSize;
         4916  +    zipfileEntryFree(pCsr->pCurrent);
         4917  +    pCsr->pCurrent = 0;
         4918  +    if( pCsr->iNextOff>=iEof ){
         4919  +      pCsr->bEof = 1;
         4920  +    }else{
         4921  +      ZipfileEntry *p = 0;
         4922  +      ZipfileTab *pTab = (ZipfileTab*)(cur->pVtab);
         4923  +      rc = zipfileGetEntry(pTab, 0, 0, pCsr->pFile, pCsr->iNextOff, &p);
         4924  +      if( rc==SQLITE_OK ){
         4925  +        pCsr->iNextOff += ZIPFILE_CDS_FIXED_SZ;
         4926  +        pCsr->iNextOff += (int)p->cds.nExtra + p->cds.nFile + p->cds.nComment;
         4927  +      }
         4928  +      pCsr->pCurrent = p;
         4929  +    }
         4930  +  }else{
         4931  +    if( !pCsr->bNoop ){
         4932  +      pCsr->pCurrent = pCsr->pCurrent->pNext;
         4933  +    }
         4934  +    if( pCsr->pCurrent==0 ){
         4935  +      pCsr->bEof = 1;
         4936  +    }
         4937  +  }
         4938  +
         4939  +  pCsr->bNoop = 0;
         4940  +  return rc;
         4941  +}
         4942  +
         4943  +static void zipfileFree(void *p) { 
         4944  +  sqlite3_free(p); 
         4945  +}
         4946  +
         4947  +/*
         4948  +** Buffer aIn (size nIn bytes) contains compressed data. Uncompressed, the
         4949  +** size is nOut bytes. This function uncompresses the data and sets the
         4950  +** return value in context pCtx to the result (a blob).
         4951  +**
         4952  +** If an error occurs, an error code is left in pCtx instead.
         4953  +*/
  4687   4954   static void zipfileInflate(
  4688         -  sqlite3_context *pCtx,          /* Store error here, if any */
         4955  +  sqlite3_context *pCtx,          /* Store result here */
  4689   4956     const u8 *aIn,                  /* Compressed data */
  4690   4957     int nIn,                        /* Size of buffer aIn[] in bytes */
  4691   4958     int nOut                        /* Expected output size */
  4692   4959   ){
  4693   4960     u8 *aRes = sqlite3_malloc(nOut);
  4694   4961     if( aRes==0 ){
  4695   4962       sqlite3_result_error_nomem(pCtx);
................................................................................
  4707   4974       if( err!=Z_OK ){
  4708   4975         zipfileCtxErrorMsg(pCtx, "inflateInit2() failed (%d)", err);
  4709   4976       }else{
  4710   4977         err = inflate(&str, Z_NO_FLUSH);
  4711   4978         if( err!=Z_STREAM_END ){
  4712   4979           zipfileCtxErrorMsg(pCtx, "inflate() failed (%d)", err);
  4713   4980         }else{
  4714         -        sqlite3_result_blob(pCtx, aRes, nOut, SQLITE_TRANSIENT);
         4981  +        sqlite3_result_blob(pCtx, aRes, nOut, zipfileFree);
         4982  +        aRes = 0;
  4715   4983         }
  4716   4984       }
  4717   4985       sqlite3_free(aRes);
  4718   4986       inflateEnd(&str);
  4719   4987     }
  4720   4988   }
  4721   4989   
         4990  +/*
         4991  +** Buffer aIn (size nIn bytes) contains uncompressed data. This function
         4992  +** compresses it and sets (*ppOut) to point to a buffer containing the
         4993  +** compressed data. The caller is responsible for eventually calling
         4994  +** sqlite3_free() to release buffer (*ppOut). Before returning, (*pnOut) 
         4995  +** is set to the size of buffer (*ppOut) in bytes.
         4996  +**
         4997  +** If no error occurs, SQLITE_OK is returned. Otherwise, an SQLite error
         4998  +** code is returned and an error message left in virtual-table handle
         4999  +** pTab. The values of (*ppOut) and (*pnOut) are left unchanged in this
         5000  +** case.
         5001  +*/
  4722   5002   static int zipfileDeflate(
  4723         -  ZipfileTab *pTab,               /* Set error message here */
  4724   5003     const u8 *aIn, int nIn,         /* Input */
  4725         -  u8 **ppOut, int *pnOut          /* Output */
         5004  +  u8 **ppOut, int *pnOut,         /* Output */
         5005  +  char **pzErr                    /* OUT: Error message */
  4726   5006   ){
  4727   5007     int nAlloc = (int)compressBound(nIn);
  4728   5008     u8 *aOut;
  4729   5009     int rc = SQLITE_OK;
  4730   5010   
  4731   5011     aOut = (u8*)sqlite3_malloc(nAlloc);
  4732   5012     if( aOut==0 ){
................................................................................
  4744   5024       res = deflate(&str, Z_FINISH);
  4745   5025   
  4746   5026       if( res==Z_STREAM_END ){
  4747   5027         *ppOut = aOut;
  4748   5028         *pnOut = (int)str.total_out;
  4749   5029       }else{
  4750   5030         sqlite3_free(aOut);
  4751         -      pTab->base.zErrMsg = sqlite3_mprintf("zipfile: deflate() error");
         5031  +      *pzErr = sqlite3_mprintf("zipfile: deflate() error");
  4752   5032         rc = SQLITE_ERROR;
  4753   5033       }
  4754   5034       deflateEnd(&str);
  4755   5035     }
  4756   5036   
  4757   5037     return rc;
  4758   5038   }
................................................................................
  4764   5044   */
  4765   5045   static int zipfileColumn(
  4766   5046     sqlite3_vtab_cursor *cur,   /* The cursor */
  4767   5047     sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
  4768   5048     int i                       /* Which column to return */
  4769   5049   ){
  4770   5050     ZipfileCsr *pCsr = (ZipfileCsr*)cur;
         5051  +  ZipfileCDS *pCDS = &pCsr->pCurrent->cds;
  4771   5052     int rc = SQLITE_OK;
  4772   5053     switch( i ){
  4773   5054       case 0:   /* name */
  4774         -      sqlite3_result_text(ctx, pCsr->cds.zFile, -1, SQLITE_TRANSIENT);
         5055  +      sqlite3_result_text(ctx, pCDS->zFile, -1, SQLITE_TRANSIENT);
  4775   5056         break;
  4776   5057       case 1:   /* mode */
  4777   5058         /* TODO: Whether or not the following is correct surely depends on
  4778   5059         ** the platform on which the archive was created.  */
  4779         -      sqlite3_result_int(ctx, pCsr->cds.iExternalAttr >> 16);
         5060  +      sqlite3_result_int(ctx, pCDS->iExternalAttr >> 16);
  4780   5061         break;
  4781   5062       case 2: { /* mtime */
  4782         -      if( pCsr->flags & ZIPFILE_MTIME_VALID ){
  4783         -        sqlite3_result_int64(ctx, pCsr->mTime);
  4784         -      }else{
  4785         -        sqlite3_result_int64(ctx, zipfileMtime(pCsr));
  4786         -      }
         5063  +      sqlite3_result_int64(ctx, pCsr->pCurrent->mUnixTime);
  4787   5064         break;
  4788   5065       }
  4789   5066       case 3: { /* sz */
  4790   5067         if( sqlite3_vtab_nochange(ctx)==0 ){
  4791         -        sqlite3_result_int64(ctx, pCsr->cds.szUncompressed);
         5068  +        sqlite3_result_int64(ctx, pCDS->szUncompressed);
  4792   5069         }
  4793   5070         break;
  4794   5071       }
  4795   5072       case 4:   /* rawdata */
  4796   5073         if( sqlite3_vtab_nochange(ctx) ) break;
  4797   5074       case 5: { /* data */
  4798         -      if( i==4 || pCsr->cds.iCompression==0 || pCsr->cds.iCompression==8 ){
  4799         -        int sz = pCsr->cds.szCompressed;
  4800         -        int szFinal = pCsr->cds.szUncompressed;
         5075  +      if( i==4 || pCDS->iCompression==0 || pCDS->iCompression==8 ){
         5076  +        int sz = pCDS->szCompressed;
         5077  +        int szFinal = pCDS->szUncompressed;
  4801   5078           if( szFinal>0 ){
  4802         -          u8 *aBuf = sqlite3_malloc(sz);
  4803         -          if( aBuf==0 ){
  4804         -            rc = SQLITE_NOMEM;
         5079  +          u8 *aBuf;
         5080  +          u8 *aFree = 0;
         5081  +          if( pCsr->pCurrent->aData ){
         5082  +            aBuf = pCsr->pCurrent->aData;
  4805   5083             }else{
  4806         -            FILE *pFile = zipfileGetFd(pCsr);
  4807         -            rc = zipfileReadData(pFile, aBuf, sz, pCsr->iDataOff,
  4808         -                &pCsr->base.pVtab->zErrMsg
  4809         -            );
         5084  +            aBuf = aFree = sqlite3_malloc(sz);
         5085  +            if( aBuf==0 ){
         5086  +              rc = SQLITE_NOMEM;
         5087  +            }else{
         5088  +              FILE *pFile = pCsr->pFile;
         5089  +              if( pFile==0 ){
         5090  +                pFile = ((ZipfileTab*)(pCsr->base.pVtab))->pWriteFd;
         5091  +              }
         5092  +              rc = zipfileReadData(pFile, aBuf, sz, pCsr->pCurrent->iDataOff,
         5093  +                  &pCsr->base.pVtab->zErrMsg
         5094  +              );
         5095  +            }
  4810   5096             }
  4811   5097             if( rc==SQLITE_OK ){
  4812         -            if( i==5 && pCsr->cds.iCompression ){
         5098  +            if( i==5 && pCDS->iCompression ){
  4813   5099                 zipfileInflate(ctx, aBuf, sz, szFinal);
  4814   5100               }else{
  4815   5101                 sqlite3_result_blob(ctx, aBuf, sz, SQLITE_TRANSIENT);
  4816   5102               }
  4817         -            sqlite3_free(aBuf);
  4818   5103             }
         5104  +          sqlite3_free(aFree);
  4819   5105           }else{
  4820   5106             /* Figure out if this is a directory or a zero-sized file. Consider
  4821   5107             ** it to be a directory either if the mode suggests so, or if
  4822   5108             ** the final character in the name is '/'.  */
  4823         -          u32 mode = pCsr->cds.iExternalAttr >> 16;
  4824         -          if( !(mode & S_IFDIR) && pCsr->cds.zFile[pCsr->cds.nFile-1]!='/' ){
         5109  +          u32 mode = pCDS->iExternalAttr >> 16;
         5110  +          if( !(mode & S_IFDIR) && pCDS->zFile[pCDS->nFile-1]!='/' ){
  4825   5111               sqlite3_result_blob(ctx, "", 0, SQLITE_STATIC);
  4826   5112             }
  4827   5113           }
  4828   5114         }
  4829   5115         break;
  4830   5116       }
  4831   5117       case 6:   /* method */
  4832         -      sqlite3_result_int(ctx, pCsr->cds.iCompression);
         5118  +      sqlite3_result_int(ctx, pCDS->iCompression);
  4833   5119         break;
  4834         -    case 7:   /* z */
         5120  +    default:  /* z */
         5121  +      assert( i==7 );
  4835   5122         sqlite3_result_int64(ctx, pCsr->iId);
  4836   5123         break;
  4837   5124     }
  4838   5125   
  4839   5126     return rc;
  4840   5127   }
  4841   5128   
  4842   5129   /*
  4843         -** Return the rowid for the current row.
  4844         -*/
  4845         -static int zipfileRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
  4846         -  assert( 0 );
  4847         -  return SQLITE_OK;
  4848         -}
  4849         -
  4850         -/*
  4851         -** Return TRUE if the cursor has been moved off of the last
  4852         -** row of output.
         5130  +** Return TRUE if the cursor is at EOF.
  4853   5131   */
  4854   5132   static int zipfileEof(sqlite3_vtab_cursor *cur){
  4855   5133     ZipfileCsr *pCsr = (ZipfileCsr*)cur;
  4856   5134     return pCsr->bEof;
  4857   5135   }
  4858   5136   
  4859   5137   /*
         5138  +** If aBlob is not NULL, then it points to a buffer nBlob bytes in size
         5139  +** containing an entire zip archive image. Or, if aBlob is NULL, then pFile
         5140  +** is guaranteed to be a file-handle open on a zip file.
         5141  +**
         5142  +** This function attempts to locate the EOCD record within the zip archive
         5143  +** and populate *pEOCD with the results of decoding it. SQLITE_OK is
         5144  +** returned if successful. Otherwise, an SQLite error code is returned and
         5145  +** an English language error message may be left in virtual-table pTab.
  4860   5146   */
  4861   5147   static int zipfileReadEOCD(
  4862   5148     ZipfileTab *pTab,               /* Return errors here */
  4863         -  FILE *pFile,                    /* Read from this file */
         5149  +  const u8 *aBlob,                /* Pointer to in-memory file image */
         5150  +  int nBlob,                      /* Size of aBlob[] in bytes */
         5151  +  FILE *pFile,                    /* Read from this file if aBlob==0 */
  4864   5152     ZipfileEOCD *pEOCD              /* Object to populate */
  4865   5153   ){
  4866   5154     u8 *aRead = pTab->aBuffer;      /* Temporary buffer */
  4867         -  i64 szFile;                     /* Total size of file in bytes */
  4868   5155     int nRead;                      /* Bytes to read from file */
  4869         -  i64 iOff;                       /* Offset to read from */
  4870         -  int rc;
         5156  +  int rc = SQLITE_OK;
  4871   5157   
  4872         -  fseek(pFile, 0, SEEK_END);
  4873         -  szFile = (i64)ftell(pFile);
  4874         -  if( szFile==0 ){
  4875         -    memset(pEOCD, 0, sizeof(ZipfileEOCD));
  4876         -    return SQLITE_OK;
         5158  +  if( aBlob==0 ){
         5159  +    i64 iOff;                     /* Offset to read from */
         5160  +    i64 szFile;                   /* Total size of file in bytes */
         5161  +    fseek(pFile, 0, SEEK_END);
         5162  +    szFile = (i64)ftell(pFile);
         5163  +    if( szFile==0 ){
         5164  +      memset(pEOCD, 0, sizeof(ZipfileEOCD));
         5165  +      return SQLITE_OK;
         5166  +    }
         5167  +    nRead = (int)(MIN(szFile, ZIPFILE_BUFFER_SIZE));
         5168  +    iOff = szFile - nRead;
         5169  +    rc = zipfileReadData(pFile, aRead, nRead, iOff, &pTab->base.zErrMsg);
         5170  +  }else{
         5171  +    nRead = (int)(MIN(nBlob, ZIPFILE_BUFFER_SIZE));
         5172  +    aRead = (u8*)&aBlob[nBlob-nRead];
  4877   5173     }
  4878         -  nRead = (int)(MIN(szFile, ZIPFILE_BUFFER_SIZE));
  4879         -  iOff = szFile - nRead;
  4880   5174   
  4881         -  rc = zipfileReadData(pFile, aRead, nRead, iOff, &pTab->base.zErrMsg);
  4882   5175     if( rc==SQLITE_OK ){
  4883   5176       int i;
  4884   5177   
  4885   5178       /* Scan backwards looking for the signature bytes */
  4886   5179       for(i=nRead-20; i>=0; i--){
  4887   5180         if( aRead[i]==0x50 && aRead[i+1]==0x4b 
  4888   5181          && aRead[i+2]==0x05 && aRead[i+3]==0x06 
................................................................................
  4900   5193       aRead += i+4;
  4901   5194       pEOCD->iDisk = zipfileRead16(aRead);
  4902   5195       pEOCD->iFirstDisk = zipfileRead16(aRead);
  4903   5196       pEOCD->nEntry = zipfileRead16(aRead);
  4904   5197       pEOCD->nEntryTotal = zipfileRead16(aRead);
  4905   5198       pEOCD->nSize = zipfileRead32(aRead);
  4906   5199       pEOCD->iOffset = zipfileRead32(aRead);
  4907         -
  4908         -#if 0
  4909         -    printf("iDisk=%d  iFirstDisk=%d  nEntry=%d  "
  4910         -           "nEntryTotal=%d  nSize=%d  iOffset=%d", 
  4911         -           (int)pEOCD->iDisk, (int)pEOCD->iFirstDisk, (int)pEOCD->nEntry,
  4912         -           (int)pEOCD->nEntryTotal, (int)pEOCD->nSize, (int)pEOCD->iOffset
  4913         -    );
  4914         -#endif
  4915         -  }
  4916         -
  4917         -  return SQLITE_OK;
         5200  +  }
         5201  +
         5202  +  return rc;
         5203  +}
         5204  +
         5205  +/*
         5206  +** Add object pNew to the linked list that begins at ZipfileTab.pFirstEntry 
         5207  +** and ends with pLastEntry. If argument pBefore is NULL, then pNew is added
         5208  +** to the end of the list. Otherwise, it is added to the list immediately
         5209  +** before pBefore (which is guaranteed to be a part of said list).
         5210  +*/
         5211  +static void zipfileAddEntry(
         5212  +  ZipfileTab *pTab, 
         5213  +  ZipfileEntry *pBefore, 
         5214  +  ZipfileEntry *pNew
         5215  +){
         5216  +  assert( (pTab->pFirstEntry==0)==(pTab->pLastEntry==0) );
         5217  +  assert( pNew->pNext==0 );
         5218  +  if( pBefore==0 ){
         5219  +    if( pTab->pFirstEntry==0 ){
         5220  +      pTab->pFirstEntry = pTab->pLastEntry = pNew;
         5221  +    }else{
         5222  +      assert( pTab->pLastEntry->pNext==0 );
         5223  +      pTab->pLastEntry->pNext = pNew;
         5224  +      pTab->pLastEntry = pNew;
         5225  +    }
         5226  +  }else{
         5227  +    ZipfileEntry **pp;
         5228  +    for(pp=&pTab->pFirstEntry; *pp!=pBefore; pp=&((*pp)->pNext));
         5229  +    pNew->pNext = pBefore;
         5230  +    *pp = pNew;
         5231  +  }
         5232  +}
         5233  +
         5234  +static int zipfileLoadDirectory(ZipfileTab *pTab, const u8 *aBlob, int nBlob){
         5235  +  ZipfileEOCD eocd;
         5236  +  int rc;
         5237  +  int i;
         5238  +  i64 iOff;
         5239  +
         5240  +  rc = zipfileReadEOCD(pTab, aBlob, nBlob, pTab->pWriteFd, &eocd);
         5241  +  iOff = eocd.iOffset;
         5242  +  for(i=0; rc==SQLITE_OK && i<eocd.nEntry; i++){
         5243  +    ZipfileEntry *pNew = 0;
         5244  +    rc = zipfileGetEntry(pTab, aBlob, nBlob, pTab->pWriteFd, iOff, &pNew);
         5245  +
         5246  +    if( rc==SQLITE_OK ){
         5247  +      zipfileAddEntry(pTab, 0, pNew);
         5248  +      iOff += ZIPFILE_CDS_FIXED_SZ;
         5249  +      iOff += (int)pNew->cds.nExtra + pNew->cds.nFile + pNew->cds.nComment;
         5250  +    }
         5251  +  }
         5252  +  return rc;
  4918   5253   }
  4919   5254   
  4920   5255   /*
  4921   5256   ** xFilter callback.
  4922   5257   */
  4923   5258   static int zipfileFilter(
  4924   5259     sqlite3_vtab_cursor *cur, 
  4925   5260     int idxNum, const char *idxStr,
  4926   5261     int argc, sqlite3_value **argv
  4927   5262   ){
  4928   5263     ZipfileTab *pTab = (ZipfileTab*)cur->pVtab;
  4929   5264     ZipfileCsr *pCsr = (ZipfileCsr*)cur;
  4930         -  const char *zFile;              /* Zip file to scan */
         5265  +  const char *zFile = 0;          /* Zip file to scan */
  4931   5266     int rc = SQLITE_OK;             /* Return Code */
         5267  +  int bInMemory = 0;              /* True for an in-memory zipfile */
  4932   5268   
  4933   5269     zipfileResetCursor(pCsr);
  4934   5270   
  4935   5271     if( pTab->zFile ){
  4936   5272       zFile = pTab->zFile;
  4937   5273     }else if( idxNum==0 ){
  4938         -    /* Error. This is an eponymous virtual table and the user has not 
  4939         -    ** supplied a file name. */
  4940         -    zipfileSetErrmsg(pCsr, "table function zipfile() requires an argument");
         5274  +    zipfileCursorErr(pCsr, "zipfile() function requires an argument");
  4941   5275       return SQLITE_ERROR;
         5276  +  }else if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){
         5277  +    const u8 *aBlob = (const u8*)sqlite3_value_blob(argv[0]);
         5278  +    int nBlob = sqlite3_value_bytes(argv[0]);
         5279  +    assert( pTab->pFirstEntry==0 );
         5280  +    rc = zipfileLoadDirectory(pTab, aBlob, nBlob);
         5281  +    pCsr->pFreeEntry = pTab->pFirstEntry;
         5282  +    pTab->pFirstEntry = pTab->pLastEntry = 0;
         5283  +    if( rc!=SQLITE_OK ) return rc;
         5284  +    bInMemory = 1;
  4942   5285     }else{
  4943   5286       zFile = (const char*)sqlite3_value_text(argv[0]);
  4944   5287     }
  4945   5288   
  4946         -  if( pTab->pWriteFd==0 ){
         5289  +  if( 0==pTab->pWriteFd && 0==bInMemory ){
  4947   5290       pCsr->pFile = fopen(zFile, "rb");
  4948   5291       if( pCsr->pFile==0 ){
  4949         -      zipfileSetErrmsg(pCsr, "cannot open file: %s", zFile);
         5292  +      zipfileCursorErr(pCsr, "cannot open file: %s", zFile);
  4950   5293         rc = SQLITE_ERROR;
  4951   5294       }else{
  4952         -      rc = zipfileReadEOCD(pTab, pCsr->pFile, &pCsr->eocd);
         5295  +      rc = zipfileReadEOCD(pTab, 0, 0, pCsr->pFile, &pCsr->eocd);
  4953   5296         if( rc==SQLITE_OK ){
  4954   5297           if( pCsr->eocd.nEntry==0 ){
  4955   5298             pCsr->bEof = 1;
  4956   5299           }else{
  4957   5300             pCsr->iNextOff = pCsr->eocd.iOffset;
  4958   5301             rc = zipfileNext(cur);
  4959   5302           }
  4960   5303         }
  4961   5304       }
  4962   5305     }else{
  4963         -    ZipfileEntry e;
  4964         -    memset(&e, 0, sizeof(e));
  4965         -    e.pNext = pTab->pFirstEntry;
  4966         -    pCsr->pCurrent = &e;
         5306  +    pCsr->bNoop = 1;
         5307  +    pCsr->pCurrent = pCsr->pFreeEntry ? pCsr->pFreeEntry : pTab->pFirstEntry;
  4967   5308       rc = zipfileNext(cur);
  4968         -    assert( pCsr->pCurrent!=&e );
  4969   5309     }
  4970   5310   
  4971   5311     return rc;
  4972   5312   }
  4973   5313   
  4974   5314   /*
  4975   5315   ** xBestIndex callback.
................................................................................
  4997   5337       pIdxInfo->estimatedCost = (double)(((sqlite3_int64)1) << 50);
  4998   5338       pIdxInfo->idxNum = 0;
  4999   5339     }
  5000   5340   
  5001   5341     return SQLITE_OK;
  5002   5342   }
  5003   5343   
  5004         -/*
  5005         -** Add object pNew to the end of the linked list that begins at
  5006         -** ZipfileTab.pFirstEntry and ends with pLastEntry.
  5007         -*/
  5008         -static void zipfileAddEntry(
  5009         -  ZipfileTab *pTab, 
  5010         -  ZipfileEntry *pBefore, 
  5011         -  ZipfileEntry *pNew
  5012         -){
  5013         -  assert( (pTab->pFirstEntry==0)==(pTab->pLastEntry==0) );
  5014         -  assert( pNew->pNext==0 );
  5015         -  if( pBefore==0 ){
  5016         -    if( pTab->pFirstEntry==0 ){
  5017         -      pTab->pFirstEntry = pTab->pLastEntry = pNew;
  5018         -    }else{
  5019         -      assert( pTab->pLastEntry->pNext==0 );
  5020         -      pTab->pLastEntry->pNext = pNew;
  5021         -      pTab->pLastEntry = pNew;
  5022         -    }
  5023         -  }else{
  5024         -    ZipfileEntry **pp;
  5025         -    for(pp=&pTab->pFirstEntry; *pp!=pBefore; pp=&((*pp)->pNext));
  5026         -    pNew->pNext = pBefore;
  5027         -    *pp = pNew;
  5028         -  }
  5029         -}
  5030         -
  5031         -static int zipfileLoadDirectory(ZipfileTab *pTab){
  5032         -  ZipfileEOCD eocd;
  5033         -  int rc;
  5034         -
  5035         -  rc = zipfileReadEOCD(pTab, pTab->pWriteFd, &eocd);
  5036         -  if( rc==SQLITE_OK && eocd.nEntry>0 ){
  5037         -    int i;
  5038         -    int iOff = 0;
  5039         -    u8 *aBuf = sqlite3_malloc(eocd.nSize);
  5040         -    if( aBuf==0 ){
  5041         -      rc = SQLITE_NOMEM;
  5042         -    }else{
  5043         -      rc = zipfileReadData(
  5044         -          pTab->pWriteFd, aBuf, eocd.nSize, eocd.iOffset, &pTab->base.zErrMsg
  5045         -      );
  5046         -    }
  5047         -
  5048         -    for(i=0; rc==SQLITE_OK && i<eocd.nEntry; i++){
  5049         -      u16 nFile;
  5050         -      u16 nExtra;
  5051         -      u16 nComment;
  5052         -      ZipfileEntry *pNew;
  5053         -      u8 *aRec = &aBuf[iOff];
  5054         -
  5055         -      nFile = zipfileGetU16(&aRec[ZIPFILE_CDS_NFILE_OFF]);
  5056         -      nExtra = zipfileGetU16(&aRec[ZIPFILE_CDS_NFILE_OFF+2]);
  5057         -      nComment = zipfileGetU16(&aRec[ZIPFILE_CDS_NFILE_OFF+4]);
  5058         -
  5059         -      pNew = sqlite3_malloc(
  5060         -          sizeof(ZipfileEntry) 
  5061         -        + nFile+1 
  5062         -        + ZIPFILE_CDS_FIXED_SZ+nFile+nExtra+nComment
  5063         -      );
  5064         -      if( pNew==0 ){
  5065         -        rc = SQLITE_NOMEM;
  5066         -      }else{
  5067         -        memset(pNew, 0, sizeof(ZipfileEntry));
  5068         -        pNew->zPath = (char*)&pNew[1];
  5069         -        memcpy(pNew->zPath, &aRec[ZIPFILE_CDS_FIXED_SZ], nFile);
  5070         -        pNew->zPath[nFile] = '\0';
  5071         -        pNew->aCdsEntry = (u8*)&pNew->zPath[nFile+1];
  5072         -        pNew->nCdsEntry = ZIPFILE_CDS_FIXED_SZ+nFile+nExtra+nComment;
  5073         -        memcpy(pNew->aCdsEntry, aRec, pNew->nCdsEntry);
  5074         -        zipfileAddEntry(pTab, 0, pNew);
  5075         -      }
  5076         -
  5077         -      iOff += ZIPFILE_CDS_FIXED_SZ+nFile+nExtra+nComment;
  5078         -    }
  5079         -
  5080         -    sqlite3_free(aBuf);
  5081         -  }
  5082         -
  5083         -  return rc;
  5084         -}
  5085         -
  5086         -static ZipfileEntry *zipfileNewEntry(
  5087         -  ZipfileCDS *pCds,               /* Values for fixed size part of CDS */
  5088         -  const char *zPath,              /* Path for new entry */
  5089         -  int nPath,                      /* strlen(zPath) */
  5090         -  u32 mTime                       /* Modification time (or 0) */
  5091         -){
  5092         -  u8 *aWrite;
         5344  +static ZipfileEntry *zipfileNewEntry(const char *zPath){
  5093   5345     ZipfileEntry *pNew;
  5094         -  pCds->nFile = (u16)nPath;
  5095         -  pCds->nExtra = mTime ? 9 : 0;
  5096         -  pNew = (ZipfileEntry*)sqlite3_malloc(
  5097         -    sizeof(ZipfileEntry) + 
  5098         -    nPath+1 + 
  5099         -    ZIPFILE_CDS_FIXED_SZ + nPath + pCds->nExtra
  5100         -  );
  5101         -
         5346  +  pNew = sqlite3_malloc(sizeof(ZipfileEntry));
  5102   5347     if( pNew ){
  5103   5348       memset(pNew, 0, sizeof(ZipfileEntry));
  5104         -    pNew->zPath = (char*)&pNew[1];
  5105         -    pNew->aCdsEntry = (u8*)&pNew->zPath[nPath+1];
  5106         -    pNew->nCdsEntry = ZIPFILE_CDS_FIXED_SZ + nPath + pCds->nExtra;
  5107         -    memcpy(pNew->zPath, zPath, nPath+1);
  5108         -
  5109         -    aWrite = pNew->aCdsEntry;
  5110         -    zipfileWrite32(aWrite, ZIPFILE_SIGNATURE_CDS);
  5111         -    zipfileWrite16(aWrite, pCds->iVersionMadeBy);
  5112         -    zipfileWrite16(aWrite, pCds->iVersionExtract);
  5113         -    zipfileWrite16(aWrite, pCds->flags);
  5114         -    zipfileWrite16(aWrite, pCds->iCompression);
  5115         -    zipfileWrite16(aWrite, pCds->mTime);
  5116         -    zipfileWrite16(aWrite, pCds->mDate);
  5117         -    zipfileWrite32(aWrite, pCds->crc32);
  5118         -    zipfileWrite32(aWrite, pCds->szCompressed);
  5119         -    zipfileWrite32(aWrite, pCds->szUncompressed);
  5120         -    zipfileWrite16(aWrite, pCds->nFile);
  5121         -    zipfileWrite16(aWrite, pCds->nExtra);
  5122         -    zipfileWrite16(aWrite, pCds->nComment);      assert( pCds->nComment==0 );
  5123         -    zipfileWrite16(aWrite, pCds->iDiskStart);
  5124         -    zipfileWrite16(aWrite, pCds->iInternalAttr);
  5125         -    zipfileWrite32(aWrite, pCds->iExternalAttr);
  5126         -    zipfileWrite32(aWrite, pCds->iOffset);
  5127         -    assert( aWrite==&pNew->aCdsEntry[ZIPFILE_CDS_FIXED_SZ] );
  5128         -    memcpy(aWrite, zPath, nPath);
  5129         -    if( pCds->nExtra ){
  5130         -      aWrite += nPath;
  5131         -      zipfileWrite16(aWrite, ZIPFILE_EXTRA_TIMESTAMP);
  5132         -      zipfileWrite16(aWrite, 5);
  5133         -      *aWrite++ = 0x01;
  5134         -      zipfileWrite32(aWrite, mTime);
         5349  +    pNew->cds.zFile = sqlite3_mprintf("%s", zPath);
         5350  +    if( pNew->cds.zFile==0 ){
         5351  +      sqlite3_free(pNew);
         5352  +      pNew = 0;
  5135   5353       }
  5136   5354     }
  5137         -
  5138   5355     return pNew;
  5139   5356   }
         5357  +
         5358  +static int zipfileSerializeLFH(ZipfileEntry *pEntry, u8 *aBuf){
         5359  +  ZipfileCDS *pCds = &pEntry->cds;
         5360  +  u8 *a = aBuf;
         5361  +
         5362  +  pCds->nExtra = 9;
         5363  +
         5364  +  /* Write the LFH itself */
         5365  +  zipfileWrite32(a, ZIPFILE_SIGNATURE_LFH);
         5366  +  zipfileWrite16(a, pCds->iVersionExtract);
         5367  +  zipfileWrite16(a, pCds->flags);
         5368  +  zipfileWrite16(a, pCds->iCompression);
         5369  +  zipfileWrite16(a, pCds->mTime);
         5370  +  zipfileWrite16(a, pCds->mDate);
         5371  +  zipfileWrite32(a, pCds->crc32);
         5372  +  zipfileWrite32(a, pCds->szCompressed);
         5373  +  zipfileWrite32(a, pCds->szUncompressed);
         5374  +  zipfileWrite16(a, (u16)pCds->nFile);
         5375  +  zipfileWrite16(a, pCds->nExtra);
         5376  +  assert( a==&aBuf[ZIPFILE_LFH_FIXED_SZ] );
         5377  +
         5378  +  /* Add the file name */
         5379  +  memcpy(a, pCds->zFile, (int)pCds->nFile);
         5380  +  a += (int)pCds->nFile;
         5381  +
         5382  +  /* The "extra" data */
         5383  +  zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
         5384  +  zipfileWrite16(a, 5);
         5385  +  *a++ = 0x01;
         5386  +  zipfileWrite32(a, pEntry->mUnixTime);
         5387  +
         5388  +  return a-aBuf;
         5389  +}
  5140   5390   
  5141   5391   static int zipfileAppendEntry(
  5142   5392     ZipfileTab *pTab,
  5143         -  ZipfileCDS *pCds,
  5144         -  const char *zPath,              /* Path for new entry */
  5145         -  int nPath,                      /* strlen(zPath) */
         5393  +  ZipfileEntry *pEntry,
  5146   5394     const u8 *pData,
  5147         -  int nData,
  5148         -  u32 mTime
         5395  +  int nData
  5149   5396   ){
  5150   5397     u8 *aBuf = pTab->aBuffer;
         5398  +  int nBuf;
  5151   5399     int rc;
  5152   5400   
  5153         -  zipfileWrite32(aBuf, ZIPFILE_SIGNATURE_LFH);
  5154         -  zipfileWrite16(aBuf, pCds->iVersionExtract);
  5155         -  zipfileWrite16(aBuf, pCds->flags);
  5156         -  zipfileWrite16(aBuf, pCds->iCompression);
  5157         -  zipfileWrite16(aBuf, pCds->mTime);
  5158         -  zipfileWrite16(aBuf, pCds->mDate);
  5159         -  zipfileWrite32(aBuf, pCds->crc32);
  5160         -  zipfileWrite32(aBuf, pCds->szCompressed);
  5161         -  zipfileWrite32(aBuf, pCds->szUncompressed);
  5162         -  zipfileWrite16(aBuf, (u16)nPath);
  5163         -  zipfileWrite16(aBuf, pCds->nExtra);
  5164         -  assert( aBuf==&pTab->aBuffer[ZIPFILE_LFH_FIXED_SZ] );
  5165         -  rc = zipfileAppendData(pTab, pTab->aBuffer, (int)(aBuf - pTab->aBuffer));
         5401  +  nBuf = zipfileSerializeLFH(pEntry, aBuf);
         5402  +  rc = zipfileAppendData(pTab, aBuf, nBuf);
  5166   5403     if( rc==SQLITE_OK ){
  5167         -    rc = zipfileAppendData(pTab, (const u8*)zPath, nPath);
  5168         -  }
  5169         -
  5170         -  if( rc==SQLITE_OK && pCds->nExtra ){
  5171         -    aBuf = pTab->aBuffer;
  5172         -    zipfileWrite16(aBuf, ZIPFILE_EXTRA_TIMESTAMP);
  5173         -    zipfileWrite16(aBuf, 5);
  5174         -    *aBuf++ = 0x01;
  5175         -    zipfileWrite32(aBuf, mTime);
  5176         -    rc = zipfileAppendData(pTab, pTab->aBuffer, 9);
  5177         -  }
  5178         -
  5179         -  if( rc==SQLITE_OK ){
         5404  +    pEntry->iDataOff = pTab->szCurrent;
  5180   5405       rc = zipfileAppendData(pTab, pData, nData);
  5181   5406     }
  5182   5407   
  5183   5408     return rc;
  5184   5409   }
  5185   5410   
  5186   5411   static int zipfileGetMode(
  5187         -  ZipfileTab *pTab, 
  5188   5412     sqlite3_value *pVal, 
  5189         -  u32 defaultMode,                /* Value to use if pVal IS NULL */
  5190         -  u32 *pMode
         5413  +  int bIsDir,                     /* If true, default to directory */
         5414  +  u32 *pMode,                     /* OUT: Mode value */
         5415  +  char **pzErr                    /* OUT: Error message */
  5191   5416   ){
  5192   5417     const char *z = (const char*)sqlite3_value_text(pVal);
  5193   5418     u32 mode = 0;
  5194   5419     if( z==0 ){
  5195         -    mode = defaultMode;
         5420  +    mode = (bIsDir ? (S_IFDIR + 0755) : (S_IFREG + 0644));
  5196   5421     }else if( z[0]>='0' && z[0]<='9' ){
  5197   5422       mode = (unsigned int)sqlite3_value_int(pVal);
  5198   5423     }else{
  5199   5424       const char zTemplate[11] = "-rwxrwxrwx";
  5200   5425       int i;
  5201   5426       if( strlen(z)!=10 ) goto parse_error;
  5202   5427       switch( z[0] ){
  5203   5428         case '-': mode |= S_IFREG; break;
  5204   5429         case 'd': mode |= S_IFDIR; break;
  5205         -#if !defined(_WIN32) && !defined(WIN32)
  5206   5430         case 'l': mode |= S_IFLNK; break;
  5207         -#endif
  5208   5431         default: goto parse_error;
  5209   5432       }
  5210   5433       for(i=1; i<10; i++){
  5211   5434         if( z[i]==zTemplate[i] ) mode |= 1 << (9-i);
  5212   5435         else if( z[i]!='-' ) goto parse_error;
  5213   5436       }
  5214   5437     }
         5438  +  if( ((mode & S_IFDIR)==0)==bIsDir ){
         5439  +    /* The "mode" attribute is a directory, but data has been specified.
         5440  +    ** Or vice-versa - no data but "mode" is a file or symlink.  */
         5441  +    *pzErr = sqlite3_mprintf("zipfile: mode does not match data");
         5442  +    return SQLITE_CONSTRAINT;
         5443  +  }
  5215   5444     *pMode = mode;
  5216   5445     return SQLITE_OK;
  5217   5446   
  5218   5447    parse_error:
  5219         -  pTab->base.zErrMsg = sqlite3_mprintf("zipfile: parse error in mode: %s", z);
         5448  +  *pzErr = sqlite3_mprintf("zipfile: parse error in mode: %s", z);
  5220   5449     return SQLITE_ERROR;
  5221   5450   }
  5222   5451   
  5223   5452   /*
  5224   5453   ** Both (const char*) arguments point to nul-terminated strings. Argument
  5225   5454   ** nB is the value of strlen(zB). This function returns 0 if the strings are
  5226   5455   ** identical, ignoring any trailing '/' character in either path.  */
................................................................................
  5227   5456   static int zipfileComparePath(const char *zA, const char *zB, int nB){
  5228   5457     int nA = (int)strlen(zA);
  5229   5458     if( zA[nA-1]=='/' ) nA--;
  5230   5459     if( zB[nB-1]=='/' ) nB--;
  5231   5460     if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
  5232   5461     return 1;
  5233   5462   }
         5463  +
         5464  +static int zipfileBegin(sqlite3_vtab *pVtab){
         5465  +  ZipfileTab *pTab = (ZipfileTab*)pVtab;
         5466  +  int rc = SQLITE_OK;
         5467  +
         5468  +  assert( pTab->pWriteFd==0 );
         5469  +
         5470  +  /* Open a write fd on the file. Also load the entire central directory
         5471  +  ** structure into memory. During the transaction any new file data is 
         5472  +  ** appended to the archive file, but the central directory is accumulated
         5473  +  ** in main-memory until the transaction is committed.  */
         5474  +  pTab->pWriteFd = fopen(pTab->zFile, "ab+");
         5475  +  if( pTab->pWriteFd==0 ){
         5476  +    pTab->base.zErrMsg = sqlite3_mprintf(
         5477  +        "zipfile: failed to open file %s for writing", pTab->zFile
         5478  +        );
         5479  +    rc = SQLITE_ERROR;
         5480  +  }else{
         5481  +    fseek(pTab->pWriteFd, 0, SEEK_END);
         5482  +    pTab->szCurrent = pTab->szOrig = (i64)ftell(pTab->pWriteFd);
         5483  +    rc = zipfileLoadDirectory(pTab, 0, 0);
         5484  +  }
         5485  +
         5486  +  if( rc!=SQLITE_OK ){
         5487  +    zipfileCleanupTransaction(pTab);
         5488  +  }
         5489  +
         5490  +  return rc;
         5491  +}
         5492  +
         5493  +/*
         5494  +** Return the current time as a 32-bit timestamp in UNIX epoch format (like
         5495  +** time(2)).
         5496  +*/
         5497  +static u32 zipfileTime(void){
         5498  +  sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
         5499  +  u32 ret;
         5500  +  if( pVfs->iVersion>=2 && pVfs->xCurrentTimeInt64 ){
         5501  +    i64 ms;
         5502  +    pVfs->xCurrentTimeInt64(pVfs, &ms);
         5503  +    ret = (u32)((ms/1000) - ((i64)24405875 * 8640));
         5504  +  }else{
         5505  +    double day;
         5506  +    pVfs->xCurrentTime(pVfs, &day);
         5507  +    ret = (u32)((day - 2440587.5) * 86400);
         5508  +  }
         5509  +  return ret;
         5510  +}
         5511  +
         5512  +/*
         5513  +** Return a 32-bit timestamp in UNIX epoch format.
         5514  +**
         5515  +** If the value passed as the only argument is either NULL or an SQL NULL,
         5516  +** return the current time. Otherwise, return the value stored in (*pVal)
         5517  +** cast to a 32-bit unsigned integer.
         5518  +*/
         5519  +static u32 zipfileGetTime(sqlite3_value *pVal){
         5520  +  if( pVal==0 || sqlite3_value_type(pVal)==SQLITE_NULL ){
         5521  +    return zipfileTime();
         5522  +  }
         5523  +  return (u32)sqlite3_value_int64(pVal);
         5524  +}
         5525  +
         5526  +/*
         5527  +** Unless it is NULL, entry pOld is currently part of the pTab->pFirstEntry
         5528  +** linked list.  Remove it from the list and free the object.
         5529  +*/
         5530  +static void zipfileRemoveEntryFromList(ZipfileTab *pTab, ZipfileEntry *pOld){
         5531  +  if( pOld ){
         5532  +    ZipfileEntry **pp;
         5533  +    for(pp=&pTab->pFirstEntry; (*pp)!=pOld; pp=&((*pp)->pNext));
         5534  +    *pp = (*pp)->pNext;
         5535  +    zipfileEntryFree(pOld);
         5536  +  }
         5537  +}
  5234   5538   
  5235   5539   /*
  5236   5540   ** xUpdate method.
  5237   5541   */
  5238   5542   static int zipfileUpdate(
  5239   5543     sqlite3_vtab *pVtab, 
  5240   5544     int nVal, 
................................................................................
  5242   5546     sqlite_int64 *pRowid
  5243   5547   ){
  5244   5548     ZipfileTab *pTab = (ZipfileTab*)pVtab;
  5245   5549     int rc = SQLITE_OK;             /* Return Code */
  5246   5550     ZipfileEntry *pNew = 0;         /* New in-memory CDS entry */
  5247   5551   
  5248   5552     u32 mode = 0;                   /* Mode for new entry */
  5249         -  i64 mTime = 0;                  /* Modification time for new entry */
         5553  +  u32 mTime = 0;                  /* Modification time for new entry */
  5250   5554     i64 sz = 0;                     /* Uncompressed size */
  5251   5555     const char *zPath = 0;          /* Path for new entry */
  5252   5556     int nPath = 0;                  /* strlen(zPath) */
  5253   5557     const u8 *pData = 0;            /* Pointer to buffer containing content */
  5254   5558     int nData = 0;                  /* Size of pData buffer in bytes */
  5255   5559     int iMethod = 0;                /* Compression method for new entry */
  5256   5560     u8 *pFree = 0;                  /* Free this */
  5257   5561     char *zFree = 0;                /* Also free this */
  5258         -  ZipfileCDS cds;                 /* New Central Directory Structure entry */
  5259   5562     ZipfileEntry *pOld = 0;
         5563  +  ZipfileEntry *pOld2 = 0;
         5564  +  int bUpdate = 0;                /* True for an update that modifies "name" */
  5260   5565     int bIsDir = 0;
  5261   5566     u32 iCrc32 = 0;
  5262   5567   
  5263         -  assert( pTab->zFile );
  5264         -  assert( pTab->pWriteFd );
         5568  +  if( pTab->pWriteFd==0 ){
         5569  +    rc = zipfileBegin(pVtab);
         5570  +    if( rc!=SQLITE_OK ) return rc;
         5571  +  }
  5265   5572   
         5573  +  /* If this is a DELETE or UPDATE, find the archive entry to delete. */
  5266   5574     if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
  5267   5575       const char *zDelete = (const char*)sqlite3_value_text(apVal[0]);
  5268   5576       int nDelete = (int)strlen(zDelete);
         5577  +    if( nVal>1 ){
         5578  +      const char *zUpdate = (const char*)sqlite3_value_text(apVal[1]);
         5579  +      if( zUpdate && zipfileComparePath(zUpdate, zDelete, nDelete)!=0 ){
         5580  +        bUpdate = 1;
         5581  +      }
         5582  +    }
  5269   5583       for(pOld=pTab->pFirstEntry; 1; pOld=pOld->pNext){
  5270         -      if( pOld->bDeleted ) continue;
  5271         -      if( zipfileComparePath(pOld->zPath, zDelete, nDelete)==0 ){
  5272         -        pOld->bDeleted = 1;
         5584  +      if( zipfileComparePath(pOld->cds.zFile, zDelete, nDelete)==0 ){
  5273   5585           break;
  5274   5586         }
  5275   5587         assert( pOld->pNext );
  5276   5588       }
  5277         -    if( nVal==1 ) return SQLITE_OK;
  5278         -  }
  5279         -
  5280         -  /* Check that "sz" and "rawdata" are both NULL: */
  5281         -  if( sqlite3_value_type(apVal[5])!=SQLITE_NULL
  5282         -   || sqlite3_value_type(apVal[6])!=SQLITE_NULL
  5283         -  ){
  5284         -    rc = SQLITE_CONSTRAINT;
  5285         -  }
  5286         -
  5287         -  if( rc==SQLITE_OK ){
  5288         -    if( sqlite3_value_type(apVal[7])==SQLITE_NULL ){
  5289         -      /* data=NULL. A directory */
  5290         -      bIsDir = 1;
  5291         -    }else{
  5292         -      /* Value specified for "data", and possibly "method". This must be
  5293         -      ** a regular file or a symlink. */
  5294         -      const u8 *aIn = sqlite3_value_blob(apVal[7]);
  5295         -      int nIn = sqlite3_value_bytes(apVal[7]);
  5296         -      int bAuto = sqlite3_value_type(apVal[8])==SQLITE_NULL;
  5297         -
  5298         -      iMethod = sqlite3_value_int(apVal[8]);
  5299         -      sz = nIn;
  5300         -      pData = aIn;
  5301         -      nData = nIn;
  5302         -      if( iMethod!=0 && iMethod!=8 ){
  5303         -        rc = SQLITE_CONSTRAINT;
  5304         -      }else{
  5305         -        if( bAuto || iMethod ){
  5306         -          int nCmp;
  5307         -          rc = zipfileDeflate(pTab, aIn, nIn, &pFree, &nCmp);
  5308         -          if( rc==SQLITE_OK ){
  5309         -            if( iMethod || nCmp<nIn ){
  5310         -              iMethod = 8;
  5311         -              pData = pFree;
  5312         -              nData = nCmp;
  5313         -            }
  5314         -          }
  5315         -        }
  5316         -        iCrc32 = crc32(0, aIn, nIn);
  5317         -      }
  5318         -    }
  5319         -  }
  5320         -
  5321         -  if( rc==SQLITE_OK ){
  5322         -    rc = zipfileGetMode(pTab, apVal[3], 
  5323         -        (bIsDir ? (S_IFDIR + 0755) : (S_IFREG + 0644)), &mode
  5324         -    );
  5325         -    if( rc==SQLITE_OK && (bIsDir == ((mode & S_IFDIR)==0)) ){
  5326         -      /* The "mode" attribute is a directory, but data has been specified.
  5327         -      ** Or vice-versa - no data but "mode" is a file or symlink.  */
         5589  +  }
         5590  +
         5591  +  if( nVal>1 ){
         5592  +    /* Check that "sz" and "rawdata" are both NULL: */
         5593  +    if( sqlite3_value_type(apVal[5])!=SQLITE_NULL ){
         5594  +      zipfileTableErr(pTab, "sz must be NULL");
         5595  +      rc = SQLITE_CONSTRAINT;
         5596  +    }
         5597  +    if( sqlite3_value_type(apVal[6])!=SQLITE_NULL ){
         5598  +      zipfileTableErr(pTab, "rawdata must be NULL"); 
  5328   5599         rc = SQLITE_CONSTRAINT;
  5329   5600       }
  5330         -  }
  5331         -
  5332         -  if( rc==SQLITE_OK ){
  5333         -    zPath = (const char*)sqlite3_value_text(apVal[2]);
  5334         -    nPath = (int)strlen(zPath);
  5335         -    if( sqlite3_value_type(apVal[4])==SQLITE_NULL ){
  5336         -      mTime = (sqlite3_int64)time(0);
  5337         -    }else{
  5338         -      mTime = sqlite3_value_int64(apVal[4]);
  5339         -    }
  5340         -  }
  5341         -
  5342         -  if( rc==SQLITE_OK && bIsDir ){
  5343         -    /* For a directory, check that the last character in the path is a
  5344         -    ** '/'. This appears to be required for compatibility with info-zip
  5345         -    ** (the unzip command on unix). It does not create directories
  5346         -    ** otherwise.  */
  5347         -    if( zPath[nPath-1]!='/' ){
  5348         -      zFree = sqlite3_mprintf("%s/", zPath);
  5349         -      if( zFree==0 ){ rc = SQLITE_NOMEM; }
  5350         -      zPath = (const char*)zFree;
  5351         -      nPath++;
  5352         -    }
  5353         -  }
  5354         -
  5355         -  /* Check that we're not inserting a duplicate entry */
  5356         -  if( rc==SQLITE_OK ){
  5357         -    ZipfileEntry *p;
  5358         -    for(p=pTab->pFirstEntry; p; p=p->pNext){
  5359         -      if( p->bDeleted ) continue;
  5360         -      if( zipfileComparePath(p->zPath, zPath, nPath)==0 ){
  5361         -        rc = SQLITE_CONSTRAINT;
  5362         -        break;
  5363         -      }
  5364         -    }
  5365         -  }
  5366         -
  5367         -  if( rc==SQLITE_OK ){
  5368         -    /* Create the new CDS record. */
  5369         -    memset(&cds, 0, sizeof(cds));
  5370         -    cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
  5371         -    cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
  5372         -    cds.flags = ZIPFILE_NEWENTRY_FLAGS;
  5373         -    cds.iCompression = (u16)iMethod;
  5374         -    zipfileMtimeToDos(&cds, (u32)mTime);
  5375         -    cds.crc32 = iCrc32;
  5376         -    cds.szCompressed = nData;
  5377         -    cds.szUncompressed = (u32)sz;
  5378         -    cds.iExternalAttr = (mode<<16);
  5379         -    cds.iOffset = (u32)pTab->szCurrent;
  5380         -    pNew = zipfileNewEntry(&cds, zPath, nPath, (u32)mTime);
  5381         -    if( pNew==0 ){
  5382         -      rc = SQLITE_NOMEM;
  5383         -    }else{
  5384         -      zipfileAddEntry(pTab, pOld, pNew);
  5385         -    }
  5386         -  }
  5387         -
  5388         -  /* Append the new header+file to the archive */
  5389         -  if( rc==SQLITE_OK ){
  5390         -    rc = zipfileAppendEntry(pTab, &cds, zPath, nPath, pData, nData, (u32)mTime);
  5391         -  }
  5392         -
  5393         -  if( rc!=SQLITE_OK && pOld ){
  5394         -    pOld->bDeleted = 0;
  5395         -  }
         5601  +
         5602  +    if( rc==SQLITE_OK ){
         5603  +      if( sqlite3_value_type(apVal[7])==SQLITE_NULL ){
         5604  +        /* data=NULL. A directory */
         5605  +        bIsDir = 1;
         5606  +      }else{
         5607  +        /* Value specified for "data", and possibly "method". This must be
         5608  +        ** a regular file or a symlink. */
         5609  +        const u8 *aIn = sqlite3_value_blob(apVal[7]);
         5610  +        int nIn = sqlite3_value_bytes(apVal[7]);
         5611  +        int bAuto = sqlite3_value_type(apVal[8])==SQLITE_NULL;
         5612  +
         5613  +        iMethod = sqlite3_value_int(apVal[8]);
         5614  +        sz = nIn;
         5615  +        pData = aIn;
         5616  +        nData = nIn;
         5617  +        if( iMethod!=0 && iMethod!=8 ){
         5618  +          zipfileTableErr(pTab, "unknown compression method: %d", iMethod);
         5619  +          rc = SQLITE_CONSTRAINT;
         5620  +        }else{
         5621  +          if( bAuto || iMethod ){
         5622  +            int nCmp;
         5623  +            rc = zipfileDeflate(aIn, nIn, &pFree, &nCmp, &pTab->base.zErrMsg);
         5624  +            if( rc==SQLITE_OK ){
         5625  +              if( iMethod || nCmp<nIn ){
         5626  +                iMethod = 8;
         5627  +                pData = pFree;
         5628  +                nData = nCmp;
         5629  +              }
         5630  +            }
         5631  +          }
         5632  +          iCrc32 = crc32(0, aIn, nIn);
         5633  +        }
         5634  +      }
         5635  +    }
         5636  +
         5637  +    if( rc==SQLITE_OK ){
         5638  +      rc = zipfileGetMode(apVal[3], bIsDir, &mode, &pTab->base.zErrMsg);
         5639  +    }
         5640  +
         5641  +    if( rc==SQLITE_OK ){
         5642  +      zPath = (const char*)sqlite3_value_text(apVal[2]);
         5643  +      nPath = (int)strlen(zPath);
         5644  +      mTime = zipfileGetTime(apVal[4]);
         5645  +    }
         5646  +
         5647  +    if( rc==SQLITE_OK && bIsDir ){
         5648  +      /* For a directory, check that the last character in the path is a
         5649  +      ** '/'. This appears to be required for compatibility with info-zip
         5650  +      ** (the unzip command on unix). It does not create directories
         5651  +      ** otherwise.  */
         5652  +      if( zPath[nPath-1]!='/' ){
         5653  +        zFree = sqlite3_mprintf("%s/", zPath);
         5654  +        if( zFree==0 ){ rc = SQLITE_NOMEM; }
         5655  +        zPath = (const char*)zFree;
         5656  +        nPath++;
         5657  +      }
         5658  +    }
         5659  +
         5660  +    /* Check that we're not inserting a duplicate entry -OR- updating an
         5661  +    ** entry with a path, thereby making it into a duplicate. */
         5662  +    if( (pOld==0 || bUpdate) && rc==SQLITE_OK ){
         5663  +      ZipfileEntry *p;
         5664  +      for(p=pTab->pFirstEntry; p; p=p->pNext){
         5665  +        if( zipfileComparePath(p->cds.zFile, zPath, nPath)==0 ){
         5666  +          switch( sqlite3_vtab_on_conflict(pTab->db) ){
         5667  +            case SQLITE_IGNORE: {
         5668  +              goto zipfile_update_done;
         5669  +            }
         5670  +            case SQLITE_REPLACE: {
         5671  +              pOld2 = p;
         5672  +              break;
         5673  +            }
         5674  +            default: {
         5675  +              zipfileTableErr(pTab, "duplicate name: \"%s\"", zPath);
         5676  +              rc = SQLITE_CONSTRAINT;
         5677  +              break;
         5678  +            }
         5679  +          }
         5680  +          break;
         5681  +        }
         5682  +      }
         5683  +    }
         5684  +
         5685  +    if( rc==SQLITE_OK ){
         5686  +      /* Create the new CDS record. */
         5687  +      pNew = zipfileNewEntry(zPath);
         5688  +      if( pNew==0 ){
         5689  +        rc = SQLITE_NOMEM;
         5690  +      }else{
         5691  +        pNew->cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
         5692  +        pNew->cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
         5693  +        pNew->cds.flags = ZIPFILE_NEWENTRY_FLAGS;
         5694  +        pNew->cds.iCompression = (u16)iMethod;
         5695  +        zipfileMtimeToDos(&pNew->cds, mTime);
         5696  +        pNew->cds.crc32 = iCrc32;
         5697  +        pNew->cds.szCompressed = nData;
         5698  +        pNew->cds.szUncompressed = (u32)sz;
         5699  +        pNew->cds.iExternalAttr = (mode<<16);
         5700  +        pNew->cds.iOffset = (u32)pTab->szCurrent;
         5701  +        pNew->cds.nFile = (u16)nPath;
         5702  +        pNew->mUnixTime = (u32)mTime;
         5703  +        rc = zipfileAppendEntry(pTab, pNew, pData, nData);
         5704  +        zipfileAddEntry(pTab, pOld, pNew);
         5705  +      }
         5706  +    }
         5707  +  }
         5708  +
         5709  +  if( rc==SQLITE_OK && (pOld || pOld2) ){
         5710  +    ZipfileCsr *pCsr;
         5711  +    for(pCsr=pTab->pCsrList; pCsr; pCsr=pCsr->pCsrNext){
         5712  +      if( pCsr->pCurrent && (pCsr->pCurrent==pOld || pCsr->pCurrent==pOld2) ){
         5713  +        pCsr->pCurrent = pCsr->pCurrent->pNext;
         5714  +        pCsr->bNoop = 1;
         5715  +      }
         5716  +    }
         5717  +
         5718  +    zipfileRemoveEntryFromList(pTab, pOld);
         5719  +    zipfileRemoveEntryFromList(pTab, pOld2);
         5720  +  }
         5721  +
         5722  +zipfile_update_done:
  5396   5723     sqlite3_free(pFree);
  5397   5724     sqlite3_free(zFree);
  5398   5725     return rc;
  5399   5726   }
         5727  +
         5728  +static int zipfileSerializeEOCD(ZipfileEOCD *p, u8 *aBuf){
         5729  +  u8 *a = aBuf;
         5730  +  zipfileWrite32(a, ZIPFILE_SIGNATURE_EOCD);
         5731  +  zipfileWrite16(a, p->iDisk);
         5732  +  zipfileWrite16(a, p->iFirstDisk);
         5733  +  zipfileWrite16(a, p->nEntry);
         5734  +  zipfileWrite16(a, p->nEntryTotal);
         5735  +  zipfileWrite32(a, p->nSize);
         5736  +  zipfileWrite32(a, p->iOffset);
         5737  +  zipfileWrite16(a, 0);        /* Size of trailing comment in bytes*/
         5738  +
         5739  +  return a-aBuf;
         5740  +}
  5400   5741   
  5401   5742   static int zipfileAppendEOCD(ZipfileTab *pTab, ZipfileEOCD *p){
  5402         -  u8 *aBuf = pTab->aBuffer;
  5403         -
  5404         -  zipfileWrite32(aBuf, ZIPFILE_SIGNATURE_EOCD);
  5405         -  zipfileWrite16(aBuf, p->iDisk);
  5406         -  zipfileWrite16(aBuf, p->iFirstDisk);
  5407         -  zipfileWrite16(aBuf, p->nEntry);
  5408         -  zipfileWrite16(aBuf, p->nEntryTotal);
  5409         -  zipfileWrite32(aBuf, p->nSize);
  5410         -  zipfileWrite32(aBuf, p->iOffset);
  5411         -  zipfileWrite16(aBuf, 0);        /* Size of trailing comment in bytes*/
  5412         -
  5413         -  assert( (aBuf-pTab->aBuffer)==22 );
  5414         -  return zipfileAppendData(pTab, pTab->aBuffer, (int)(aBuf - pTab->aBuffer));
  5415         -}
  5416         -
  5417         -static void zipfileCleanupTransaction(ZipfileTab *pTab){
  5418         -  ZipfileEntry *pEntry;
  5419         -  ZipfileEntry *pNext;
  5420         -
  5421         -  for(pEntry=pTab->pFirstEntry; pEntry; pEntry=pNext){
  5422         -    pNext = pEntry->pNext;
  5423         -    sqlite3_free(pEntry);
  5424         -  }
  5425         -  pTab->pFirstEntry = 0;
  5426         -  pTab->pLastEntry = 0;
  5427         -  fclose(pTab->pWriteFd);
  5428         -  pTab->pWriteFd = 0;
  5429         -  pTab->szCurrent = 0;
  5430         -  pTab->szOrig = 0;
  5431         -}
  5432         -
  5433         -static int zipfileBegin(sqlite3_vtab *pVtab){
  5434         -  ZipfileTab *pTab = (ZipfileTab*)pVtab;
  5435         -  int rc = SQLITE_OK;
  5436         -
  5437         -  assert( pTab->pWriteFd==0 );
  5438         -
  5439         -  /* This table is only writable if a default archive path was specified 
  5440         -  ** as part of the CREATE VIRTUAL TABLE statement. */
  5441         -  if( pTab->zFile==0 ){
  5442         -    pTab->base.zErrMsg = sqlite3_mprintf(
  5443         -        "zipfile: writing requires a default archive"
  5444         -    );
  5445         -    return SQLITE_ERROR;
  5446         -  }
  5447         -
  5448         -  /* Open a write fd on the file. Also load the entire central directory
  5449         -  ** structure into memory. During the transaction any new file data is 
  5450         -  ** appended to the archive file, but the central directory is accumulated
  5451         -  ** in main-memory until the transaction is committed.  */
  5452         -  pTab->pWriteFd = fopen(pTab->zFile, "ab+");
  5453         -  if( pTab->pWriteFd==0 ){
  5454         -    pTab->base.zErrMsg = sqlite3_mprintf(
  5455         -        "zipfile: failed to open file %s for writing", pTab->zFile
  5456         -    );
  5457         -    rc = SQLITE_ERROR;
  5458         -  }else{
  5459         -    fseek(pTab->pWriteFd, 0, SEEK_END);
  5460         -    pTab->szCurrent = pTab->szOrig = (i64)ftell(pTab->pWriteFd);
  5461         -    rc = zipfileLoadDirectory(pTab);
  5462         -  }
  5463         -
  5464         -  if( rc!=SQLITE_OK ){
  5465         -    zipfileCleanupTransaction(pTab);
  5466         -  }
  5467         -
  5468         -  return rc;
         5743  +  int nBuf = zipfileSerializeEOCD(p, pTab->aBuffer);
         5744  +  assert( nBuf==ZIPFILE_EOCD_FIXED_SZ );
         5745  +  return zipfileAppendData(pTab, pTab->aBuffer, nBuf);
         5746  +}
         5747  +
         5748  +/*
         5749  +** Serialize the CDS structure into buffer aBuf[]. Return the number
         5750  +** of bytes written.
         5751  +*/
         5752  +static int zipfileSerializeCDS(ZipfileEntry *pEntry, u8 *aBuf){
         5753  +  u8 *a = aBuf;
         5754  +  ZipfileCDS *pCDS = &pEntry->cds;
         5755  +
         5756  +  if( pEntry->aExtra==0 ){
         5757  +    pCDS->nExtra = 9;
         5758  +  }
         5759  +
         5760  +  zipfileWrite32(a, ZIPFILE_SIGNATURE_CDS);
         5761  +  zipfileWrite16(a, pCDS->iVersionMadeBy);
         5762  +  zipfileWrite16(a, pCDS->iVersionExtract);
         5763  +  zipfileWrite16(a, pCDS->flags);
         5764  +  zipfileWrite16(a, pCDS->iCompression);
         5765  +  zipfileWrite16(a, pCDS->mTime);
         5766  +  zipfileWrite16(a, pCDS->mDate);
         5767  +  zipfileWrite32(a, pCDS->crc32);
         5768  +  zipfileWrite32(a, pCDS->szCompressed);
         5769  +  zipfileWrite32(a, pCDS->szUncompressed);
         5770  +  assert( a==&aBuf[ZIPFILE_CDS_NFILE_OFF] );
         5771  +  zipfileWrite16(a, pCDS->nFile);
         5772  +  zipfileWrite16(a, pCDS->nExtra);
         5773  +  zipfileWrite16(a, pCDS->nComment);
         5774  +  zipfileWrite16(a, pCDS->iDiskStart);
         5775  +  zipfileWrite16(a, pCDS->iInternalAttr);
         5776  +  zipfileWrite32(a, pCDS->iExternalAttr);
         5777  +  zipfileWrite32(a, pCDS->iOffset);
         5778  +
         5779  +  memcpy(a, pCDS->zFile, pCDS->nFile);
         5780  +  a += pCDS->nFile;
         5781  +
         5782  +  if( pEntry->aExtra ){
         5783  +    int n = (int)pCDS->nExtra + (int)pCDS->nComment;
         5784  +    memcpy(a, pEntry->aExtra, n);
         5785  +    a += n;
         5786  +  }else{
         5787  +    assert( pCDS->nExtra==9 );
         5788  +    zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
         5789  +    zipfileWrite16(a, 5);
         5790  +    *a++ = 0x01;
         5791  +    zipfileWrite32(a, pEntry->mUnixTime);
         5792  +  }
         5793  +
         5794  +  return a-aBuf;
  5469   5795   }
  5470   5796   
  5471   5797   static int zipfileCommit(sqlite3_vtab *pVtab){
  5472   5798     ZipfileTab *pTab = (ZipfileTab*)pVtab;
  5473   5799     int rc = SQLITE_OK;
  5474   5800     if( pTab->pWriteFd ){
  5475   5801       i64 iOffset = pTab->szCurrent;
  5476   5802       ZipfileEntry *p;
  5477   5803       ZipfileEOCD eocd;
  5478   5804       int nEntry = 0;
  5479   5805   
  5480         -    /* Write out all undeleted entries */
         5806  +    /* Write out all entries */
  5481   5807       for(p=pTab->pFirstEntry; rc==SQLITE_OK && p; p=p->pNext){
  5482         -      if( p->bDeleted ) continue;
  5483         -      rc = zipfileAppendData(pTab, p->aCdsEntry, p->nCdsEntry);
         5808  +      int n = zipfileSerializeCDS(p, pTab->aBuffer);
         5809  +      rc = zipfileAppendData(pTab, pTab->aBuffer, n);
  5484   5810         nEntry++;
  5485   5811       }
  5486   5812   
  5487   5813       /* Write out the EOCD record */
  5488   5814       eocd.iDisk = 0;
  5489   5815       eocd.iFirstDisk = 0;
  5490   5816       eocd.nEntry = (u16)nEntry;
................................................................................
  5517   5843   ){
  5518   5844     ZipfileCsr *pCsr;
  5519   5845     ZipfileTab *pTab = (ZipfileTab*)sqlite3_user_data(context);
  5520   5846     assert( argc>0 );
  5521   5847   
  5522   5848     pCsr = zipfileFindCursor(pTab, sqlite3_value_int64(argv[0]));
  5523   5849     if( pCsr ){
  5524         -    ZipfileCDS *p = &pCsr->cds;
         5850  +    ZipfileCDS *p = &pCsr->pCurrent->cds;
  5525   5851       char *zRes = sqlite3_mprintf("{"
  5526   5852           "\"version-made-by\" : %u, "
  5527   5853           "\"version-to-extract\" : %u, "
  5528   5854           "\"flags\" : %u, "
  5529   5855           "\"compression\" : %u, "
  5530   5856           "\"time\" : %u, "
  5531   5857           "\"date\" : %u, "
................................................................................
  5554   5880       }else{
  5555   5881         sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT);
  5556   5882         sqlite3_free(zRes);
  5557   5883       }
  5558   5884     }
  5559   5885   }
  5560   5886   
  5561         -
  5562   5887   /*
  5563   5888   ** xFindFunction method.
  5564   5889   */
  5565   5890   static int zipfileFindFunction(
  5566   5891     sqlite3_vtab *pVtab,            /* Virtual table handle */
  5567   5892     int nArg,                       /* Number of SQL function arguments */
  5568   5893     const char *zName,              /* Name of SQL function */
  5569   5894     void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */
  5570   5895     void **ppArg                    /* OUT: User data for *pxFunc */
  5571   5896   ){
  5572         -  if( nArg>0 ){
  5573         -    if( sqlite3_stricmp("zipfile_cds", zName)==0 ){
  5574         -      *pxFunc = zipfileFunctionCds;
  5575         -      *ppArg = (void*)pVtab;
  5576         -      return 1;
  5577         -    }
  5578         -  }
  5579         -
  5580         -  return 0;
  5581         -}
         5897  +  if( sqlite3_stricmp("zipfile_cds", zName)==0 ){
         5898  +    *pxFunc = zipfileFunctionCds;
         5899  +    *ppArg = (void*)pVtab;
         5900  +    return 1;
         5901  +  }
         5902  +  return 0;
         5903  +}
         5904  +
         5905  +typedef struct ZipfileBuffer ZipfileBuffer;
         5906  +struct ZipfileBuffer {
         5907  +  u8 *a;                          /* Pointer to buffer */
         5908  +  int n;                          /* Size of buffer in bytes */
         5909  +  int nAlloc;                     /* Byte allocated at a[] */
         5910  +};
         5911  +
         5912  +typedef struct ZipfileCtx ZipfileCtx;
         5913  +struct ZipfileCtx {
         5914  +  int nEntry;
         5915  +  ZipfileBuffer body;
         5916  +  ZipfileBuffer cds;
         5917  +};
         5918  +
         5919  +static int zipfileBufferGrow(ZipfileBuffer *pBuf, int nByte){
         5920  +  if( pBuf->n+nByte>pBuf->nAlloc ){
         5921  +    u8 *aNew;
         5922  +    int nNew = pBuf->n ? pBuf->n*2 : 512;
         5923  +    int nReq = pBuf->n + nByte;
         5924  +
         5925  +    while( nNew<nReq ) nNew = nNew*2;
         5926  +    aNew = sqlite3_realloc(pBuf->a, nNew);
         5927  +    if( aNew==0 ) return SQLITE_NOMEM;
         5928  +    pBuf->a = aNew;
         5929  +    pBuf->nAlloc = nNew;
         5930  +  }
         5931  +  return SQLITE_OK;
         5932  +}
         5933  +
         5934  +/*
         5935  +** xStep() callback for the zipfile() aggregate. This can be called in
         5936  +** any of the following ways:
         5937  +**
         5938  +**   SELECT zipfile(name,data) ...
         5939  +**   SELECT zipfile(name,mode,mtime,data) ...
         5940  +**   SELECT zipfile(name,mode,mtime,data,method) ...
         5941  +*/
         5942  +void zipfileStep(sqlite3_context *pCtx, int nVal, sqlite3_value **apVal){
         5943  +  ZipfileCtx *p;                  /* Aggregate function context */
         5944  +  ZipfileEntry e;                 /* New entry to add to zip archive */
         5945  +
         5946  +  sqlite3_value *pName = 0;
         5947  +  sqlite3_value *pMode = 0;
         5948  +  sqlite3_value *pMtime = 0;
         5949  +  sqlite3_value *pData = 0;
         5950  +  sqlite3_value *pMethod = 0;
         5951  +
         5952  +  int bIsDir = 0;
         5953  +  u32 mode;
         5954  +  int rc = SQLITE_OK;
         5955  +  char *zErr = 0;
         5956  +
         5957  +  int iMethod = -1;               /* Compression method to use (0 or 8) */
         5958  +
         5959  +  const u8 *aData = 0;            /* Possibly compressed data for new entry */
         5960  +  int nData = 0;                  /* Size of aData[] in bytes */
         5961  +  int szUncompressed = 0;         /* Size of data before compression */
         5962  +  u8 *aFree = 0;                  /* Free this before returning */
         5963  +  u32 iCrc32 = 0;                 /* crc32 of uncompressed data */
         5964  +
         5965  +  char *zName = 0;                /* Path (name) of new entry */
         5966  +  int nName = 0;                  /* Size of zName in bytes */
         5967  +  char *zFree = 0;                /* Free this before returning */
         5968  +  int nByte;
         5969  +
         5970  +  memset(&e, 0, sizeof(e));
         5971  +  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
         5972  +  if( p==0 ) return;
         5973  +
         5974  +  /* Martial the arguments into stack variables */
         5975  +  if( nVal!=2 && nVal!=4 && nVal!=5 ){
         5976  +    zErr = sqlite3_mprintf("wrong number of arguments to function zipfile()");
         5977  +    rc = SQLITE_ERROR;
         5978  +    goto zipfile_step_out;
         5979  +  }
         5980  +  pName = apVal[0];
         5981  +  if( nVal==2 ){
         5982  +    pData = apVal[1];
         5983  +  }else{
         5984  +    pMode = apVal[1];
         5985  +    pMtime = apVal[2];
         5986  +    pData = apVal[3];
         5987  +    if( nVal==5 ){
         5988  +      pMethod = apVal[4];
         5989  +    }
         5990  +  }
         5991  +
         5992  +  /* Check that the 'name' parameter looks ok. */
         5993  +  zName = (char*)sqlite3_value_text(pName);
         5994  +  nName = sqlite3_value_bytes(pName);
         5995  +  if( zName==0 ){
         5996  +    zErr = sqlite3_mprintf("first argument to zipfile() must be non-NULL");
         5997  +    rc = SQLITE_ERROR;
         5998  +    goto zipfile_step_out;
         5999  +  }
         6000  +
         6001  +  /* Inspect the 'method' parameter. This must be either 0 (store), 8 (use
         6002  +  ** deflate compression) or NULL (choose automatically).  */
         6003  +  if( pMethod && SQLITE_NULL!=sqlite3_value_type(pMethod) ){
         6004  +    iMethod = (int)sqlite3_value_int64(pMethod);
         6005  +    if( iMethod!=0 && iMethod!=8 ){
         6006  +      zErr = sqlite3_mprintf("illegal method value: %d", iMethod);
         6007  +      rc = SQLITE_ERROR;
         6008  +      goto zipfile_step_out;
         6009  +    }
         6010  +  }
         6011  +
         6012  +  /* Now inspect the data. If this is NULL, then the new entry must be a
         6013  +  ** directory.  Otherwise, figure out whether or not the data should
         6014  +  ** be deflated or simply stored in the zip archive. */
         6015  +  if( sqlite3_value_type(pData)==SQLITE_NULL ){
         6016  +    bIsDir = 1;
         6017  +    iMethod = 0;
         6018  +  }else{
         6019  +    aData = sqlite3_value_blob(pData);
         6020  +    szUncompressed = nData = sqlite3_value_bytes(pData);
         6021  +    iCrc32 = crc32(0, aData, nData);
         6022  +    if( iMethod<0 || iMethod==8 ){
         6023  +      int nOut = 0;
         6024  +      rc = zipfileDeflate(aData, nData, &aFree, &nOut, &zErr);
         6025  +      if( rc!=SQLITE_OK ){
         6026  +        goto zipfile_step_out;
         6027  +      }
         6028  +      if( iMethod==8 || nOut<nData ){
         6029  +        aData = aFree;
         6030  +        nData = nOut;
         6031  +        iMethod = 8;
         6032  +      }else{
         6033  +        iMethod = 0;
         6034  +      }
         6035  +    }
         6036  +  }
         6037  +
         6038  +  /* Decode the "mode" argument. */
         6039  +  rc = zipfileGetMode(pMode, bIsDir, &mode, &zErr);
         6040  +  if( rc ) goto zipfile_step_out;
         6041  +
         6042  +  /* Decode the "mtime" argument. */
         6043  +  e.mUnixTime = zipfileGetTime(pMtime);
         6044  +
         6045  +  /* If this is a directory entry, ensure that there is exactly one '/'
         6046  +  ** at the end of the path. Or, if this is not a directory and the path
         6047  +  ** ends in '/' it is an error. */
         6048  +  if( bIsDir==0 ){
         6049  +    if( zName[nName-1]=='/' ){
         6050  +      zErr = sqlite3_mprintf("non-directory name must not end with /");
         6051  +      rc = SQLITE_ERROR;
         6052  +      goto zipfile_step_out;
         6053  +    }
         6054  +  }else{
         6055  +    if( zName[nName-1]!='/' ){
         6056  +      zName = zFree = sqlite3_mprintf("%s/", zName);
         6057  +      nName++;
         6058  +      if( zName==0 ){
         6059  +        rc = SQLITE_NOMEM;
         6060  +        goto zipfile_step_out;
         6061  +      }
         6062  +    }else{
         6063  +      while( nName>1 && zName[nName-2]=='/' ) nName--;
         6064  +    }
         6065  +  }
         6066  +
         6067  +  /* Assemble the ZipfileEntry object for the new zip archive entry */
         6068  +  e.cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
         6069  +  e.cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
         6070  +  e.cds.flags = ZIPFILE_NEWENTRY_FLAGS;
         6071  +  e.cds.iCompression = (u16)iMethod;
         6072  +  zipfileMtimeToDos(&e.cds, (u32)e.mUnixTime);
         6073  +  e.cds.crc32 = iCrc32;
         6074  +  e.cds.szCompressed = nData;
         6075  +  e.cds.szUncompressed = szUncompressed;
         6076  +  e.cds.iExternalAttr = (mode<<16);
         6077  +  e.cds.iOffset = p->body.n;
         6078  +  e.cds.nFile = (u16)nName;
         6079  +  e.cds.zFile = zName;
         6080  +
         6081  +  /* Append the LFH to the body of the new archive */
         6082  +  nByte = ZIPFILE_LFH_FIXED_SZ + e.cds.nFile + 9;
         6083  +  if( (rc = zipfileBufferGrow(&p->body, nByte)) ) goto zipfile_step_out;
         6084  +  p->body.n += zipfileSerializeLFH(&e, &p->body.a[p->body.n]);
         6085  +
         6086  +  /* Append the data to the body of the new archive */
         6087  +  if( nData>0 ){
         6088  +    if( (rc = zipfileBufferGrow(&p->body, nData)) ) goto zipfile_step_out;
         6089  +    memcpy(&p->body.a[p->body.n], aData, nData);
         6090  +    p->body.n += nData;
         6091  +  }
         6092  +
         6093  +  /* Append the CDS record to the directory of the new archive */
         6094  +  nByte = ZIPFILE_CDS_FIXED_SZ + e.cds.nFile + 9;
         6095  +  if( (rc = zipfileBufferGrow(&p->cds, nByte)) ) goto zipfile_step_out;
         6096  +  p->cds.n += zipfileSerializeCDS(&e, &p->cds.a[p->cds.n]);
         6097  +
         6098  +  /* Increment the count of entries in the archive */
         6099  +  p->nEntry++;
         6100  +
         6101  + zipfile_step_out:
         6102  +  sqlite3_free(aFree);
         6103  +  sqlite3_free(zFree);
         6104  +  if( rc ){
         6105  +    if( zErr ){
         6106  +      sqlite3_result_error(pCtx, zErr, -1);
         6107  +    }else{
         6108  +      sqlite3_result_error_code(pCtx, rc);
         6109  +    }
         6110  +  }
         6111  +  sqlite3_free(zErr);
         6112  +}
         6113  +
         6114  +/*
         6115  +** xFinalize() callback for zipfile aggregate function.
         6116  +*/
         6117  +void zipfileFinal(sqlite3_context *pCtx){
         6118  +  ZipfileCtx *p;
         6119  +  ZipfileEOCD eocd;
         6120  +  int nZip;
         6121  +  u8 *aZip;
         6122  +
         6123  +  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
         6124  +  if( p==0 ) return;
         6125  +  if( p->nEntry>0 ){
         6126  +    memset(&eocd, 0, sizeof(eocd));
         6127  +    eocd.nEntry = (u16)p->nEntry;
         6128  +    eocd.nEntryTotal = (u16)p->nEntry;
         6129  +    eocd.nSize = p->cds.n;
         6130  +    eocd.iOffset = p->body.n;
         6131  +
         6132  +    nZip = p->body.n + p->cds.n + ZIPFILE_EOCD_FIXED_SZ;
         6133  +    aZip = (u8*)sqlite3_malloc(nZip);
         6134  +    if( aZip==0 ){
         6135  +      sqlite3_result_error_nomem(pCtx);
         6136  +    }else{
         6137  +      memcpy(aZip, p->body.a, p->body.n);
         6138  +      memcpy(&aZip[p->body.n], p->cds.a, p->cds.n);
         6139  +      zipfileSerializeEOCD(&eocd, &aZip[p->body.n + p->cds.n]);
         6140  +      sqlite3_result_blob(pCtx, aZip, nZip, zipfileFree);
         6141  +    }
         6142  +  }
         6143  +
         6144  +  sqlite3_free(p->body.a);
         6145  +  sqlite3_free(p->cds.a);
         6146  +}
         6147  +
  5582   6148   
  5583   6149   /*
  5584   6150   ** Register the "zipfile" virtual table.
  5585   6151   */
  5586   6152   static int zipfileRegister(sqlite3 *db){
  5587   6153     static sqlite3_module zipfileModule = {
  5588   6154       1,                         /* iVersion */
................................................................................
  5593   6159       zipfileDisconnect,         /* xDestroy */
  5594   6160       zipfileOpen,               /* xOpen - open a cursor */
  5595   6161       zipfileClose,              /* xClose - close a cursor */
  5596   6162       zipfileFilter,             /* xFilter - configure scan constraints */
  5597   6163       zipfileNext,               /* xNext - advance a cursor */
  5598   6164       zipfileEof,                /* xEof - check for end of scan */
  5599   6165       zipfileColumn,             /* xColumn - read data */
  5600         -    zipfileRowid,              /* xRowid - read data */
         6166  +    0,                         /* xRowid - read data */
  5601   6167       zipfileUpdate,             /* xUpdate */
  5602   6168       zipfileBegin,              /* xBegin */
  5603   6169       0,                         /* xSync */
  5604   6170       zipfileCommit,             /* xCommit */
  5605   6171       zipfileRollback,           /* xRollback */
  5606   6172       zipfileFindFunction,       /* xFindMethod */
  5607   6173       0,                         /* xRename */
  5608   6174     };
  5609   6175   
  5610   6176     int rc = sqlite3_create_module(db, "zipfile"  , &zipfileModule, 0);
         6177  +  if( rc==SQLITE_OK ) rc = sqlite3_overload_function(db, "zipfile_cds", -1);
  5611   6178     if( rc==SQLITE_OK ){
  5612         -    rc = sqlite3_overload_function(db, "zipfile_cds", -1);
         6179  +    rc = sqlite3_create_function(db, "zipfile", -1, SQLITE_UTF8, 0, 0, 
         6180  +        zipfileStep, zipfileFinal
         6181  +    );
  5613   6182     }
  5614   6183     return rc;
  5615   6184   }
  5616   6185   #else         /* SQLITE_OMIT_VIRTUALTABLE */
  5617   6186   # define zipfileRegister(x) SQLITE_OK
  5618   6187   #endif
  5619   6188   
................................................................................
  7046   7615       IdxHashEntry *pEntry;
  7047   7616       sqlite3_stmt *pExplain = 0;
  7048   7617       idxHashClear(&hIdx);
  7049   7618       rc = idxPrintfPrepareStmt(dbm, &pExplain, pzErr,
  7050   7619           "EXPLAIN QUERY PLAN %s", pStmt->zSql
  7051   7620       );
  7052   7621       while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){
  7053         -      int iSelectid = sqlite3_column_int(pExplain, 0);
  7054         -      int iOrder = sqlite3_column_int(pExplain, 1);
  7055         -      int iFrom = sqlite3_column_int(pExplain, 2);
         7622  +      /* int iId = sqlite3_column_int(pExplain, 0); */
         7623  +      /* int iParent = sqlite3_column_int(pExplain, 1); */
         7624  +      /* int iNotUsed = sqlite3_column_int(pExplain, 2); */
  7056   7625         const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3);
  7057   7626         int nDetail = STRLEN(zDetail);
  7058   7627         int i;
  7059   7628   
  7060   7629         for(i=0; i<nDetail; i++){
  7061   7630           const char *zIdx = 0;
  7062   7631           if( memcmp(&zDetail[i], " USING INDEX ", 13)==0 ){
................................................................................
  7075   7644               idxHashAdd(&rc, &hIdx, zSql, 0);
  7076   7645               if( rc ) goto find_indexes_out;
  7077   7646             }
  7078   7647             break;
  7079   7648           }
  7080   7649         }
  7081   7650   
  7082         -      pStmt->zEQP = idxAppendText(&rc, pStmt->zEQP, "%d|%d|%d|%s\n", 
  7083         -          iSelectid, iOrder, iFrom, zDetail
  7084         -      );
         7651  +      if( zDetail[0]!='-' ){
         7652  +        pStmt->zEQP = idxAppendText(&rc, pStmt->zEQP, "%s\n", zDetail);
         7653  +      }
  7085   7654       }
  7086   7655   
  7087   7656       for(pEntry=hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
  7088   7657         pStmt->zIdx = idxAppendText(&rc, pStmt->zIdx, "%s;\n", pEntry->zKey);
  7089   7658       }
  7090   7659   
  7091   7660       idxFinalize(&rc, pExplain);
................................................................................
  7907   8476   
  7908   8477   typedef struct ExpertInfo ExpertInfo;
  7909   8478   struct ExpertInfo {
  7910   8479     sqlite3expert *pExpert;
  7911   8480     int bVerbose;
  7912   8481   };
  7913   8482   
         8483  +/* A single line in the EQP output */
         8484  +typedef struct EQPGraphRow EQPGraphRow;
         8485  +struct EQPGraphRow {
         8486  +  int iEqpId;           /* ID for this row */
         8487  +  int iParentId;        /* ID of the parent row */
         8488  +  EQPGraphRow *pNext;   /* Next row in sequence */
         8489  +  char zText[1];        /* Text to display for this row */
         8490  +};
         8491  +
         8492  +/* All EQP output is collected into an instance of the following */
         8493  +typedef struct EQPGraph EQPGraph;
         8494  +struct EQPGraph {
         8495  +  EQPGraphRow *pRow;    /* Linked list of all rows of the EQP output */
         8496  +  EQPGraphRow *pLast;   /* Last element of the pRow list */
         8497  +  char zPrefix[100];    /* Graph prefix */
         8498  +};
         8499  +
  7914   8500   /*
  7915   8501   ** State information about the database connection is contained in an
  7916   8502   ** instance of the following structure.
  7917   8503   */
  7918   8504   typedef struct ShellState ShellState;
  7919   8505   struct ShellState {
  7920   8506     sqlite3 *db;           /* The database */
  7921   8507     u8 autoExplain;        /* Automatically turn on .explain mode */
  7922   8508     u8 autoEQP;            /* Run EXPLAIN QUERY PLAN prior to seach SQL stmt */
         8509  +  u8 autoEQPtest;        /* autoEQP is in test mode */
  7923   8510     u8 statsOn;            /* True to display memory stats before each finalize */
  7924   8511     u8 scanstatsOn;        /* True to display scan stats before each finalize */
  7925   8512     u8 openMode;           /* SHELL_OPEN_NORMAL, _APPENDVFS, or _ZIPFILE */
  7926   8513     u8 doXdgOpen;          /* Invoke start/open/xdg-open in output_reset() */
         8514  +  u8 nEqpLevel;          /* Depth of the EQP output graph */
         8515  +  unsigned mEqpLines;    /* Mask of veritical lines in the EQP output graph */
  7927   8516     int outCount;          /* Revert to stdout when reaching zero */
  7928   8517     int cnt;               /* Number of records displayed so far */
  7929   8518     FILE *out;             /* Write results here */
  7930   8519     FILE *traceOut;        /* Output for sqlite3_trace() */
  7931   8520     int nErr;              /* Number of errors seen */
  7932   8521     int mode;              /* An output mode setting */
  7933   8522     int modePrior;         /* Saved mode */
................................................................................
  7953   8542     char *zFreeOnClose;         /* Filename to free when closing */
  7954   8543     const char *zVfs;           /* Name of VFS to use */
  7955   8544     sqlite3_stmt *pStmt;   /* Current statement if any. */
  7956   8545     FILE *pLog;            /* Write log output here */
  7957   8546     int *aiIndent;         /* Array of indents used in MODE_Explain */
  7958   8547     int nIndent;           /* Size of array aiIndent[] */
  7959   8548     int iIndent;           /* Index of current op in aiIndent[] */
         8549  +  EQPGraph sGraph;       /* Information for the graphical EXPLAIN QUERY PLAN */
  7960   8550   #if defined(SQLITE_ENABLE_SESSION)
  7961   8551     int nSession;             /* Number of active sessions */
  7962   8552     OpenSession aSession[4];  /* Array of sessions.  [0] is in focus. */
  7963   8553   #endif
  7964   8554     ExpertInfo expert;        /* Valid if previous command was ".expert OPT..." */
  7965   8555   };
  7966   8556   
  7967   8557   
  7968   8558   /* Allowed values for ShellState.autoEQP
  7969   8559   */
  7970         -#define AUTOEQP_off      0
  7971         -#define AUTOEQP_on       1
  7972         -#define AUTOEQP_trigger  2
  7973         -#define AUTOEQP_full     3
         8560  +#define AUTOEQP_off      0           /* Automatic EXPLAIN QUERY PLAN is off */
         8561  +#define AUTOEQP_on       1           /* Automatic EQP is on */
         8562  +#define AUTOEQP_trigger  2           /* On and also show plans for triggers */
         8563  +#define AUTOEQP_full     3           /* Show full EXPLAIN */
  7974   8564   
  7975   8565   /* Allowed values for ShellState.openMode
  7976   8566   */
  7977   8567   #define SHELL_OPEN_UNSPEC     0      /* No open-mode specified */
  7978   8568   #define SHELL_OPEN_NORMAL     1      /* Normal database file */
  7979   8569   #define SHELL_OPEN_APPENDVFS  2      /* Use appendvfs */
  7980   8570   #define SHELL_OPEN_ZIPFILE    3      /* Use the zipfile virtual table */
         8571  +#define SHELL_OPEN_READONLY   4      /* Open a normal database read-only */
  7981   8572   
  7982   8573   /*
  7983   8574   ** These are the allowed shellFlgs values
  7984   8575   */
  7985   8576   #define SHFLG_Pagecache      0x00000001 /* The --pagecache option is used */
  7986   8577   #define SHFLG_Lookaside      0x00000002 /* Lookaside memory is used */
  7987   8578   #define SHFLG_Backslash      0x00000004 /* The --backslash option is used */
................................................................................
  8008   8599   #define MODE_Insert   5  /* Generate SQL "insert" statements */
  8009   8600   #define MODE_Quote    6  /* Quote values as for SQL */
  8010   8601   #define MODE_Tcl      7  /* Generate ANSI-C or TCL quoted elements */
  8011   8602   #define MODE_Csv      8  /* Quote strings, numbers are plain */
  8012   8603   #define MODE_Explain  9  /* Like MODE_Column, but do not truncate data */
  8013   8604   #define MODE_Ascii   10  /* Use ASCII unit and record separators (0x1F/0x1E) */
  8014   8605   #define MODE_Pretty  11  /* Pretty-print schemas */
         8606  +#define MODE_EQP     12  /* Converts EXPLAIN QUERY PLAN output into a graph */
  8015   8607   
  8016   8608   static const char *modeDescr[] = {
  8017   8609     "line",
  8018   8610     "column",
  8019   8611     "list",
  8020   8612     "semi",
  8021   8613     "html",
................................................................................
  8022   8614     "insert",
  8023   8615     "quote",
  8024   8616     "tcl",
  8025   8617     "csv",
  8026   8618     "explain",
  8027   8619     "ascii",
  8028   8620     "prettyprint",
         8621  +  "eqp"
  8029   8622   };
  8030   8623   
  8031   8624   /*
  8032   8625   ** These are the column/row/line separators used by the various
  8033   8626   ** import/export modes.
  8034   8627   */
  8035   8628   #define SEP_Column    "|"
................................................................................
  8080   8673   **     (4) Delete the temporary file
  8081   8674   **
  8082   8675   ** If the EDITOR argument is omitted, use the value in the VISUAL
  8083   8676   ** environment variable.  If still there is no EDITOR, through an error.
  8084   8677   **
  8085   8678   ** Also throw an error if the EDITOR program returns a non-zero exit code.
  8086   8679   */
         8680  +#ifndef SQLITE_NOHAVE_SYSTEM
  8087   8681   static void editFunc(
  8088   8682     sqlite3_context *context,
  8089   8683     int argc,
  8090   8684     sqlite3_value **argv
  8091   8685   ){
  8092   8686     const char *zEditor;
  8093   8687     char *zTempFile = 0;
................................................................................
  8177   8771     fclose(f);
  8178   8772     f = 0;
  8179   8773     if( x!=sz ){
  8180   8774       sqlite3_result_error(context, "could not read back the whole file", -1);
  8181   8775       goto edit_func_end;
  8182   8776     }
  8183   8777     if( bBin ){
  8184         -    sqlite3_result_blob(context, p, sz, sqlite3_free);
         8778  +    sqlite3_result_blob64(context, p, sz, sqlite3_free);
  8185   8779     }else{
  8186         -    sqlite3_result_text(context, (const char*)p, sz, sqlite3_free);
         8780  +    sqlite3_result_text64(context, (const char*)p, sz,
         8781  +                          sqlite3_free, SQLITE_UTF8);
  8187   8782     }
  8188   8783     p = 0;
  8189   8784   
  8190   8785   edit_func_end:
  8191   8786     if( f ) fclose(f);
  8192   8787     unlink(zTempFile);
  8193   8788     sqlite3_free(zTempFile);
  8194   8789     sqlite3_free(p);
  8195   8790   }
         8791  +#endif /* SQLITE_NOHAVE_SYSTEM */
  8196   8792   
  8197   8793   /*
  8198   8794   ** Save or restore the current output mode
  8199   8795   */
  8200   8796   static void outputModePush(ShellState *p){
  8201   8797     p->modePrior = p->mode;
  8202   8798     memcpy(p->colSepPrior, p->colSeparator, sizeof(p->colSeparator));
................................................................................
  8567   9163       if( z[i]=='\n' ) return 1;
  8568   9164       if( IsSpace(z[i]) ) continue;
  8569   9165       if( z[i]=='-' && z[i+1]=='-' ) return 1;
  8570   9166       return 0;
  8571   9167     }
  8572   9168     return 1;
  8573   9169   }
  8574         -    
         9170  +
         9171  +/*
         9172  +** Add a new entry to the EXPLAIN QUERY PLAN data
         9173  +*/
         9174  +static void eqp_append(ShellState *p, int iEqpId, int p2, const char *zText){
         9175  +  EQPGraphRow *pNew;
         9176  +  int nText = strlen30(zText);
         9177  +  if( p->autoEQPtest ){
         9178  +    utf8_printf(p->out, "%d,%d,%s\n", iEqpId, p2, zText);
         9179  +  }
         9180  +  pNew = sqlite3_malloc64( sizeof(*pNew) + nText );
         9181  +  if( pNew==0 ) shell_out_of_memory();
         9182  +  pNew->iEqpId = iEqpId;
         9183  +  pNew->iParentId = p2;
         9184  +  memcpy(pNew->zText, zText, nText+1);
         9185  +  pNew->pNext = 0;
         9186  +  if( p->sGraph.pLast ){
         9187  +    p->sGraph.pLast->pNext = pNew;
         9188  +  }else{
         9189  +    p->sGraph.pRow = pNew;
         9190  +  }
         9191  +  p->sGraph.pLast = pNew;
         9192  +}
         9193  +
         9194  +/*
         9195  +** Free and reset the EXPLAIN QUERY PLAN data that has been collected
         9196  +** in p->sGraph.
         9197  +*/
         9198  +static void eqp_reset(ShellState *p){
         9199  +  EQPGraphRow *pRow, *pNext;
         9200  +  for(pRow = p->sGraph.pRow; pRow; pRow = pNext){
         9201  +    pNext = pRow->pNext;
         9202  +    sqlite3_free(pRow);
         9203  +  }
         9204  +  memset(&p->sGraph, 0, sizeof(p->sGraph));
         9205  +}
         9206  +
         9207  +/* Return the next EXPLAIN QUERY PLAN line with iEqpId that occurs after
         9208  +** pOld, or return the first such line if pOld is NULL
         9209  +*/
         9210  +static EQPGraphRow *eqp_next_row(ShellState *p, int iEqpId, EQPGraphRow *pOld){
         9211  +  EQPGraphRow *pRow = pOld ? pOld->pNext : p->sGraph.pRow;
         9212  +  while( pRow && pRow->iParentId!=iEqpId ) pRow = pRow->pNext;
         9213  +  return pRow;
         9214  +}
         9215  +
         9216  +/* Render a single level of the graph that has iEqpId as its parent.  Called
         9217  +** recursively to render sublevels.
         9218  +*/
         9219  +static void eqp_render_level(ShellState *p, int iEqpId){
         9220  +  EQPGraphRow *pRow, *pNext;
         9221  +  int n = strlen30(p->sGraph.zPrefix);
         9222  +  char *z;
         9223  +  for(pRow = eqp_next_row(p, iEqpId, 0); pRow; pRow = pNext){
         9224  +    pNext = eqp_next_row(p, iEqpId, pRow);
         9225  +    z = pRow->zText;
         9226  +    utf8_printf(p->out, "%s%s%s\n", p->sGraph.zPrefix, pNext ? "|--" : "`--", z);
         9227  +    if( n<(int)sizeof(p->sGraph.zPrefix)-7 ){
         9228  +      memcpy(&p->sGraph.zPrefix[n], pNext ? "|  " : "   ", 4);
         9229  +      eqp_render_level(p, pRow->iEqpId);
         9230  +      p->sGraph.zPrefix[n] = 0;
         9231  +    }
         9232  +  }
         9233  +}
         9234  +
         9235  +/*
         9236  +** Display and reset the EXPLAIN QUERY PLAN data
         9237  +*/
         9238  +static void eqp_render(ShellState *p){
         9239  +  EQPGraphRow *pRow = p->sGraph.pRow;
         9240  +  if( pRow ){
         9241  +    if( pRow->zText[0]=='-' ){
         9242  +      if( pRow->pNext==0 ){
         9243  +        eqp_reset(p);
         9244  +        return;
         9245  +      }
         9246  +      utf8_printf(p->out, "%s\n", pRow->zText+3);
         9247  +      p->sGraph.pRow = pRow->pNext;
         9248  +      sqlite3_free(pRow);
         9249  +    }else{
         9250  +      utf8_printf(p->out, "QUERY PLAN\n");
         9251  +    }
         9252  +    p->sGraph.zPrefix[0] = 0;
         9253  +    eqp_render_level(p, 0);
         9254  +    eqp_reset(p);
         9255  +  }
         9256  +}
  8575   9257   
  8576   9258   /*
  8577   9259   ** This is the callback routine that the shell
  8578   9260   ** invokes for each row of a query result.
  8579   9261   */
  8580   9262   static int shell_callback(
  8581   9263     void *pArg,
................................................................................
  8918   9600         for(i=0; i<nArg; i++){
  8919   9601           if( i>0 ) utf8_printf(p->out, "%s", p->colSeparator);
  8920   9602           utf8_printf(p->out,"%s",azArg[i] ? azArg[i] : p->nullValue);
  8921   9603         }
  8922   9604         utf8_printf(p->out, "%s", p->rowSeparator);
  8923   9605         break;
  8924   9606       }
         9607  +    case MODE_EQP: {
         9608  +      eqp_append(p, atoi(azArg[0]), atoi(azArg[1]), azArg[3]);
         9609  +      break;
         9610  +    }
  8925   9611     }
  8926   9612     return 0;
  8927   9613   }
  8928   9614   
  8929   9615   /*
  8930   9616   ** This is the callback routine that the SQLite library
  8931   9617   ** invokes for each row of a query result.
................................................................................
  9016   9702       p->zDestTable = 0;
  9017   9703     }
  9018   9704     if( zName==0 ) return;
  9019   9705     cQuote = quoteChar(zName);
  9020   9706     n = strlen30(zName);
  9021   9707     if( cQuote ) n += n+2;
  9022   9708     z = p->zDestTable = malloc( n+1 );
  9023         -  if( z==0 ){
  9024         -    raw_printf(stderr,"Error: out of memory\n");
  9025         -    exit(1);
  9026         -  }
         9709  +  if( z==0 ) shell_out_of_memory();
  9027   9710     n = 0;
  9028   9711     if( cQuote ) z[n++] = cQuote;
  9029   9712     for(i=0; zName[i]; i++){
  9030   9713       z[n++] = zName[i];
  9031   9714       if( zName[i]==cQuote ) z[n++] = cQuote;
  9032   9715     }
  9033   9716     if( cQuote ) z[n++] = cQuote;
................................................................................
  9174   9857   static int display_stats(
  9175   9858     sqlite3 *db,                /* Database to query */
  9176   9859     ShellState *pArg,           /* Pointer to ShellState */
  9177   9860     int bReset                  /* True to reset the stats */
  9178   9861   ){
  9179   9862     int iCur;
  9180   9863     int iHiwtr;
  9181         -
  9182         -  if( pArg && pArg->out ){
  9183         -    displayStatLine(pArg, "Memory Used:",
  9184         -       "%lld (max %lld) bytes", SQLITE_STATUS_MEMORY_USED, bReset);
  9185         -    displayStatLine(pArg, "Number of Outstanding Allocations:",
  9186         -       "%lld (max %lld)", SQLITE_STATUS_MALLOC_COUNT, bReset);
  9187         -    if( pArg->shellFlgs & SHFLG_Pagecache ){
  9188         -      displayStatLine(pArg, "Number of Pcache Pages Used:",
  9189         -         "%lld (max %lld) pages", SQLITE_STATUS_PAGECACHE_USED, bReset);
  9190         -    }
  9191         -    displayStatLine(pArg, "Number of Pcache Overflow Bytes:",
  9192         -       "%lld (max %lld) bytes", SQLITE_STATUS_PAGECACHE_OVERFLOW, bReset);
  9193         -    displayStatLine(pArg, "Largest Allocation:",
  9194         -       "%lld bytes", SQLITE_STATUS_MALLOC_SIZE, bReset);
  9195         -    displayStatLine(pArg, "Largest Pcache Allocation:",
  9196         -       "%lld bytes", SQLITE_STATUS_PAGECACHE_SIZE, bReset);
         9864  +  FILE *out;
         9865  +  if( pArg==0 || pArg->out==0 ) return 0;
         9866  +  out = pArg->out;
         9867  +
         9868  +  if( pArg->pStmt && (pArg->statsOn & 2) ){
         9869  +    int nCol, i, x;
         9870  +    sqlite3_stmt *pStmt = pArg->pStmt;
         9871  +    char z[100];
         9872  +    nCol = sqlite3_column_count(pStmt);
         9873  +    raw_printf(out, "%-36s %d\n", "Number of output columns:", nCol);
         9874  +    for(i=0; i<nCol; i++){
         9875  +      sqlite3_snprintf(sizeof(z),z,"Column %d %nname:", i, &x);
         9876  +      utf8_printf(out, "%-36s %s\n", z, sqlite3_column_name(pStmt,i));
         9877  +#ifndef SQLITE_OMIT_DECLTYPE
         9878  +      sqlite3_snprintf(30, z+x, "declared type:");
         9879  +      utf8_printf(out, "%-36s %s\n", z, sqlite3_column_decltype(pStmt, i));
         9880  +#endif
         9881  +#ifdef SQLITE_ENABLE_COLUMN_METADATA
         9882  +      sqlite3_snprintf(30, z+x, "database name:");
         9883  +      utf8_printf(out, "%-36s %s\n", z, sqlite3_column_database_name(pStmt,i));
         9884  +      sqlite3_snprintf(30, z+x, "table name:");
         9885  +      utf8_printf(out, "%-36s %s\n", z, sqlite3_column_table_name(pStmt,i));
         9886  +      sqlite3_snprintf(30, z+x, "origin name:");
         9887  +      utf8_printf(out, "%-36s %s\n", z, sqlite3_column_origin_name(pStmt,i));
         9888  +#endif
         9889  +    }
         9890  +  }
         9891  +
         9892  +  displayStatLine(pArg, "Memory Used:",
         9893  +     "%lld (max %lld) bytes", SQLITE_STATUS_MEMORY_USED, bReset);
         9894  +  displayStatLine(pArg, "Number of Outstanding Allocations:",
         9895  +     "%lld (max %lld)", SQLITE_STATUS_MALLOC_COUNT, bReset);
         9896  +  if( pArg->shellFlgs & SHFLG_Pagecache ){
         9897  +    displayStatLine(pArg, "Number of Pcache Pages Used:",
         9898  +       "%lld (max %lld) pages", SQLITE_STATUS_PAGECACHE_USED, bReset);
         9899  +  }
         9900  +  displayStatLine(pArg, "Number of Pcache Overflow Bytes:",
         9901  +     "%lld (max %lld) bytes", SQLITE_STATUS_PAGECACHE_OVERFLOW, bReset);
         9902  +  displayStatLine(pArg, "Largest Allocation:",
         9903  +     "%lld bytes", SQLITE_STATUS_MALLOC_SIZE, bReset);
         9904  +  displayStatLine(pArg, "Largest Pcache Allocation:",
         9905  +     "%lld bytes", SQLITE_STATUS_PAGECACHE_SIZE, bReset);
  9197   9906   #ifdef YYTRACKMAXSTACKDEPTH
  9198         -    displayStatLine(pArg, "Deepest Parser Stack:",
  9199         -       "%lld (max %lld)", SQLITE_STATUS_PARSER_STACK, bReset);
         9907  +  displayStatLine(pArg, "Deepest Parser Stack:",
         9908  +     "%lld (max %lld)", SQLITE_STATUS_PARSER_STACK, bReset);
  9200   9909   #endif
  9201         -  }
  9202   9910   
  9203         -  if( pArg && pArg->out && db ){
         9911  +  if( db ){
  9204   9912       if( pArg->shellFlgs & SHFLG_Lookaside ){
  9205   9913         iHiwtr = iCur = -1;
  9206   9914         sqlite3_db_status(db, SQLITE_DBSTATUS_LOOKASIDE_USED,
  9207   9915                           &iCur, &iHiwtr, bReset);
  9208   9916         raw_printf(pArg->out,
  9209   9917                 "Lookaside Slots Used:                %d (max %d)\n",
  9210   9918                 iCur, iHiwtr);
................................................................................
  9231   9939       iHiwtr = iCur = -1;
  9232   9940       sqlite3_db_status(db, SQLITE_DBSTATUS_CACHE_MISS, &iCur, &iHiwtr, 1);
  9233   9941       raw_printf(pArg->out, "Page cache misses:                   %d\n", iCur);
  9234   9942       iHiwtr = iCur = -1;
  9235   9943       sqlite3_db_status(db, SQLITE_DBSTATUS_CACHE_WRITE, &iCur, &iHiwtr, 1);
  9236   9944       raw_printf(pArg->out, "Page cache writes:                   %d\n", iCur);
  9237   9945       iHiwtr = iCur = -1;
         9946  +    sqlite3_db_status(db, SQLITE_DBSTATUS_CACHE_SPILL, &iCur, &iHiwtr, 1);
         9947  +    raw_printf(pArg->out, "Page cache spills:                   %d\n", iCur);
         9948  +    iHiwtr = iCur = -1;
  9238   9949       sqlite3_db_status(db, SQLITE_DBSTATUS_SCHEMA_USED, &iCur, &iHiwtr, bReset);
  9239   9950       raw_printf(pArg->out, "Schema Heap Usage:                   %d bytes\n",
  9240   9951               iCur);
  9241   9952       iHiwtr = iCur = -1;
  9242   9953       sqlite3_db_status(db, SQLITE_DBSTATUS_STMT_USED, &iCur, &iHiwtr, bReset);
  9243   9954       raw_printf(pArg->out, "Statement Heap/Lookaside Usage:      %d bytes\n",
  9244   9955               iCur);
  9245   9956     }
  9246   9957   
  9247         -  if( pArg && pArg->out && db && pArg->pStmt ){
         9958  +  if( pArg->pStmt ){
  9248   9959       iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_FULLSCAN_STEP,
  9249   9960                                  bReset);
  9250   9961       raw_printf(pArg->out, "Fullscan Steps:                      %d\n", iCur);
  9251   9962       iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_SORT, bReset);
  9252   9963       raw_printf(pArg->out, "Sort Operations:                     %d\n", iCur);
  9253   9964       iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_AUTOINDEX,bReset);
  9254   9965       raw_printf(pArg->out, "Autoindex Inserts:                   %d\n", iCur);
  9255   9966       iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_VM_STEP, bReset);
  9256   9967       raw_printf(pArg->out, "Virtual Machine Steps:               %d\n", iCur);
         9968  +    iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_REPREPARE, bReset);
         9969  +    raw_printf(pArg->out, "Reprepare operations:                %d\n", iCur);
         9970  +    iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_RUN, bReset);
         9971  +    raw_printf(pArg->out, "Number of times run:                 %d\n", iCur);
         9972  +    iCur = sqlite3_stmt_status(pArg->pStmt, SQLITE_STMTSTATUS_MEMUSED, bReset);
         9973  +    raw_printf(pArg->out, "Memory used by prepared stmt:        %d\n", iCur);
  9257   9974     }
  9258   9975   
  9259   9976   #ifdef __linux__
  9260   9977     displayLinuxIoStats(pArg->out);
  9261   9978   #endif
  9262   9979   
  9263   9980     /* Do not remove this machine readable comment: extra-stats-output-here */
................................................................................
  9465  10182   }
  9466  10183   
  9467  10184   /*
  9468  10185   ** Run a prepared statement
  9469  10186   */
  9470  10187   static void exec_prepared_stmt(
  9471  10188     ShellState *pArg,                                /* Pointer to ShellState */
  9472         -  sqlite3_stmt *pStmt,                             /* Statment to run */
  9473         -  int (*xCallback)(void*,int,char**,char**,int*)   /* Callback function */
        10189  +  sqlite3_stmt *pStmt                              /* Statment to run */
  9474  10190   ){
  9475  10191     int rc;
  9476  10192   
  9477  10193     /* perform the first step.  this will tell us if we
  9478  10194     ** have a result set or not and how wide it is.
  9479  10195     */
  9480  10196     rc = sqlite3_step(pStmt);
  9481  10197     /* if we have a result set... */
  9482  10198     if( SQLITE_ROW == rc ){
  9483         -    /* if we have a callback... */
  9484         -    if( xCallback ){
  9485         -      /* allocate space for col name ptr, value ptr, and type */
  9486         -      int nCol = sqlite3_column_count(pStmt);
  9487         -      void *pData = sqlite3_malloc64(3*nCol*sizeof(const char*) + 1);
  9488         -      if( !pData ){
  9489         -        rc = SQLITE_NOMEM;
  9490         -      }else{
  9491         -        char **azCols = (char **)pData;      /* Names of result columns */
  9492         -        char **azVals = &azCols[nCol];       /* Results */
  9493         -        int *aiTypes = (int *)&azVals[nCol]; /* Result types */
  9494         -        int i, x;
  9495         -        assert(sizeof(int) <= sizeof(char *));
  9496         -        /* save off ptrs to column names */
        10199  +    /* allocate space for col name ptr, value ptr, and type */
        10200  +    int nCol = sqlite3_column_count(pStmt);
        10201  +    void *pData = sqlite3_malloc64(3*nCol*sizeof(const char*) + 1);
        10202  +    if( !pData ){
        10203  +      rc = SQLITE_NOMEM;
        10204  +    }else{
        10205  +      char **azCols = (char **)pData;      /* Names of result columns */
        10206  +      char **azVals = &azCols[nCol];       /* Results */
        10207  +      int *aiTypes = (int *)&azVals[nCol]; /* Result types */
        10208  +      int i, x;
        10209  +      assert(sizeof(int) <= sizeof(char *));
        10210  +      /* save off ptrs to column names */
        10211  +      for(i=0; i<nCol; i++){
        10212  +        azCols[i] = (char *)sqlite3_column_name(pStmt, i);
        10213  +      }
        10214  +      do{
        10215  +        /* extract the data and data types */
  9497  10216           for(i=0; i<nCol; i++){
  9498         -          azCols[i] = (char *)sqlite3_column_name(pStmt, i);
  9499         -        }
  9500         -        do{
  9501         -          /* extract the data and data types */
  9502         -          for(i=0; i<nCol; i++){
  9503         -            aiTypes[i] = x = sqlite3_column_type(pStmt, i);
  9504         -            if( x==SQLITE_BLOB && pArg && pArg->cMode==MODE_Insert ){
  9505         -              azVals[i] = "";
  9506         -            }else{
  9507         -              azVals[i] = (char*)sqlite3_column_text(pStmt, i);
  9508         -            }
  9509         -            if( !azVals[i] && (aiTypes[i]!=SQLITE_NULL) ){
  9510         -              rc = SQLITE_NOMEM;
  9511         -              break; /* from for */
  9512         -            }
  9513         -          } /* end for */
  9514         -
  9515         -          /* if data and types extracted successfully... */
  9516         -          if( SQLITE_ROW == rc ){
  9517         -            /* call the supplied callback with the result row data */
  9518         -            if( xCallback(pArg, nCol, azVals, azCols, aiTypes) ){
  9519         -              rc = SQLITE_ABORT;
  9520         -            }else{
  9521         -              rc = sqlite3_step(pStmt);
  9522         -            }
  9523         -          }
  9524         -        } while( SQLITE_ROW == rc );
  9525         -        sqlite3_free(pData);
  9526         -      }
  9527         -    }else{
  9528         -      do{
  9529         -        rc = sqlite3_step(pStmt);
  9530         -      } while( rc == SQLITE_ROW );
        10217  +          aiTypes[i] = x = sqlite3_column_type(pStmt, i);
        10218  +          if( x==SQLITE_BLOB && pArg && pArg->cMode==MODE_Insert ){
        10219  +            azVals[i] = "";
        10220  +          }else{
        10221  +            azVals[i] = (char*)sqlite3_column_text(pStmt, i);
        10222  +          }
        10223  +          if( !azVals[i] && (aiTypes[i]!=SQLITE_NULL) ){
        10224  +            rc = SQLITE_NOMEM;
        10225  +            break; /* from for */
        10226  +          }
        10227  +        } /* end for */
        10228  +
        10229  +        /* if data and types extracted successfully... */
        10230  +        if( SQLITE_ROW == rc ){
        10231  +          /* call the supplied callback with the result row data */
        10232  +          if( shell_callback(pArg, nCol, azVals, azCols, aiTypes) ){
        10233  +            rc = SQLITE_ABORT;
        10234  +          }else{
        10235  +            rc = sqlite3_step(pStmt);
        10236  +          }
        10237  +        }
        10238  +      } while( SQLITE_ROW == rc );
        10239  +      sqlite3_free(pData);
  9531  10240       }
  9532  10241     }
  9533  10242   }
  9534  10243   
  9535  10244   #ifndef SQLITE_OMIT_VIRTUALTABLE
  9536  10245   /*
  9537  10246   ** This function is called to process SQL if the previous shell command
................................................................................
  9669  10378   ** set via the supplied callback.
  9670  10379   **
  9671  10380   ** This is very similar to SQLite's built-in sqlite3_exec()
  9672  10381   ** function except it takes a slightly different callback
  9673  10382   ** and callback data argument.
  9674  10383   */
  9675  10384   static int shell_exec(
  9676         -  sqlite3 *db,                              /* An open database */
  9677         -  const char *zSql,                         /* SQL to be evaluated */
  9678         -  int (*xCallback)(void*,int,char**,char**,int*),   /* Callback function */
  9679         -                                            /* (not the same as sqlite3_exec) */
  9680  10385     ShellState *pArg,                         /* Pointer to ShellState */
        10386  +  const char *zSql,                         /* SQL to be evaluated */
  9681  10387     char **pzErrMsg                           /* Error msg written here */
  9682  10388   ){
  9683  10389     sqlite3_stmt *pStmt = NULL;     /* Statement to execute. */
  9684  10390     int rc = SQLITE_OK;             /* Return Code */
  9685  10391     int rc2;
  9686  10392     const char *zLeftover;          /* Tail of unprocessed SQL */
        10393  +  sqlite3 *db = pArg->db;
  9687  10394   
  9688  10395     if( pzErrMsg ){
  9689  10396       *pzErrMsg = NULL;
  9690  10397     }
  9691  10398   
  9692  10399   #ifndef SQLITE_OMIT_VIRTUALTABLE
  9693  10400     if( pArg->expert.pExpert ){
................................................................................
  9735  10442           if( pArg->autoEQP>=AUTOEQP_trigger ){
  9736  10443             sqlite3_db_config(db, SQLITE_DBCONFIG_TRIGGER_EQP, 1, 0);
  9737  10444           }
  9738  10445           zEQP = sqlite3_mprintf("EXPLAIN QUERY PLAN %s", zStmtSql);
  9739  10446           rc = sqlite3_prepare_v2(db, zEQP, -1, &pExplain, 0);
  9740  10447           if( rc==SQLITE_OK ){
  9741  10448             while( sqlite3_step(pExplain)==SQLITE_ROW ){
  9742         -            raw_printf(pArg->out,"--EQP-- %d,",sqlite3_column_int(pExplain, 0));
  9743         -            raw_printf(pArg->out,"%d,", sqlite3_column_int(pExplain, 1));
  9744         -            raw_printf(pArg->out,"%d,", sqlite3_column_int(pExplain, 2));
  9745         -            utf8_printf(pArg->out,"%s\n", sqlite3_column_text(pExplain, 3));
        10449  +            const char *zEQPLine = (const char*)sqlite3_column_text(pExplain,3);
        10450  +            int iEqpId = sqlite3_column_int(pExplain, 0);
        10451  +            int iParentId = sqlite3_column_int(pExplain, 1);
        10452  +            if( zEQPLine[0]=='-' ) eqp_render(pArg);
        10453  +            eqp_append(pArg, iEqpId, iParentId, zEQPLine);
  9746  10454             }
        10455  +          eqp_render(pArg);
  9747  10456           }
  9748  10457           sqlite3_finalize(pExplain);
  9749  10458           sqlite3_free(zEQP);
  9750  10459           if( pArg->autoEQP>=AUTOEQP_full ){
  9751  10460             /* Also do an EXPLAIN for ".eqp full" mode */
  9752  10461             zEQP = sqlite3_mprintf("EXPLAIN %s", zStmtSql);
  9753  10462             rc = sqlite3_prepare_v2(db, zEQP, -1, &pExplain, 0);
  9754  10463             if( rc==SQLITE_OK ){
  9755  10464               pArg->cMode = MODE_Explain;
  9756  10465               explain_data_prepare(pArg, pExplain);
  9757         -            exec_prepared_stmt(pArg, pExplain, xCallback);
        10466  +            exec_prepared_stmt(pArg, pExplain);
  9758  10467               explain_data_delete(pArg);
  9759  10468             }
  9760  10469             sqlite3_finalize(pExplain);
  9761  10470             sqlite3_free(zEQP);
  9762  10471           }
  9763         -        sqlite3_db_config(db, SQLITE_DBCONFIG_TRIGGER_EQP, triggerEQP, 0);
        10472  +        if( pArg->autoEQP>=AUTOEQP_trigger && triggerEQP==0 ){
        10473  +          sqlite3_db_config(db, SQLITE_DBCONFIG_TRIGGER_EQP, 0, 0);
        10474  +          /* Reprepare pStmt before reactiving trace modes */
        10475  +          sqlite3_finalize(pStmt);
        10476  +          sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
        10477  +        }
  9764  10478           restore_debug_trace_modes();
  9765  10479         }
  9766  10480   
  9767  10481         if( pArg ){
  9768  10482           pArg->cMode = pArg->mode;
  9769         -        if( pArg->autoExplain
  9770         -         && sqlite3_column_count(pStmt)==8
  9771         -         && sqlite3_strlike("EXPLAIN%", zStmtSql,0)==0
  9772         -        ){
  9773         -          pArg->cMode = MODE_Explain;
        10483  +        if( pArg->autoExplain ){
        10484  +          if( sqlite3_column_count(pStmt)==8
        10485  +           && sqlite3_strlike("EXPLAIN%", zStmtSql,0)==0
        10486  +          ){
        10487  +            pArg->cMode = MODE_Explain;
        10488  +          }
        10489  +          if( sqlite3_column_count(pStmt)==4
        10490  +           && sqlite3_strlike("EXPLAIN QUERY PLAN%", zStmtSql,0)==0 ){
        10491  +            pArg->cMode = MODE_EQP;
        10492  +          }
  9774  10493           }
  9775  10494   
  9776  10495           /* If the shell is currently in ".explain" mode, gather the extra
  9777  10496           ** data required to add indents to the output.*/
  9778  10497           if( pArg->cMode==MODE_Explain ){
  9779  10498             explain_data_prepare(pArg, pStmt);
  9780  10499           }
  9781  10500         }
  9782  10501   
  9783         -      exec_prepared_stmt(pArg, pStmt, xCallback);
        10502  +      exec_prepared_stmt(pArg, pStmt);
  9784  10503         explain_data_delete(pArg);
        10504  +      eqp_render(pArg);
  9785  10505   
  9786  10506         /* print usage stats if stats on */
  9787  10507         if( pArg && pArg->statsOn ){
  9788  10508           display_stats(db, pArg, 0);
  9789  10509         }
  9790  10510   
  9791  10511         /* print loop-counters if required */
................................................................................
  9855  10575     rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
  9856  10576     sqlite3_free(zSql);
  9857  10577     if( rc ) return 0;
  9858  10578     while( sqlite3_step(pStmt)==SQLITE_ROW ){
  9859  10579       if( nCol>=nAlloc-2 ){
  9860  10580         nAlloc = nAlloc*2 + nCol + 10;
  9861  10581         azCol = sqlite3_realloc(azCol, nAlloc*sizeof(azCol[0]));
  9862         -      if( azCol==0 ){
  9863         -        raw_printf(stderr, "Error: out of memory\n");
  9864         -        exit(1);
  9865         -      }
        10582  +      if( azCol==0 ) shell_out_of_memory();
  9866  10583       }
  9867  10584       azCol[++nCol] = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 1));
  9868  10585       if( sqlite3_column_int(pStmt, 5) ){
  9869  10586         nPK++;
  9870  10587         if( nPK==1
  9871  10588          && sqlite3_stricmp((const char*)sqlite3_column_text(pStmt,2),
  9872  10589                             "INTEGER")==0
................................................................................
 10039  10756       appendText(&sSelect, " FROM ", 0);
 10040  10757       appendText(&sSelect, zTable, quoteChar(zTable));
 10041  10758   
 10042  10759       savedDestTable = p->zDestTable;
 10043  10760       savedMode = p->mode;
 10044  10761       p->zDestTable = sTable.z;
 10045  10762       p->mode = p->cMode = MODE_Insert;
 10046         -    rc = shell_exec(p->db, sSelect.z, shell_callback, p, 0);
        10763  +    rc = shell_exec(p, sSelect.z, 0);
 10047  10764       if( (rc&0xff)==SQLITE_CORRUPT ){
 10048  10765         raw_printf(p->out, "/****** CORRUPTION ERROR *******/\n");
 10049  10766         toggleSelectOrder(p->db);
 10050         -      shell_exec(p->db, sSelect.z, shell_callback, p, 0);
        10767  +      shell_exec(p, sSelect.z, 0);
 10051  10768         toggleSelectOrder(p->db);
 10052  10769       }
 10053  10770       p->zDestTable = savedDestTable;
 10054  10771       p->mode = savedMode;
 10055  10772       freeText(&sTable);
 10056  10773       freeText(&sSelect);
 10057  10774       if( rc ) p->nErr++;
................................................................................
 10104  10821   #if defined(SQLITE_HAVE_ZLIB) && !defined(SQLITE_OMIT_VIRTUALTABLE)
 10105  10822     ".archive ...           Manage SQL archives: \".archive --help\" for details\n"
 10106  10823   #endif
 10107  10824   #ifndef SQLITE_OMIT_AUTHORIZATION
 10108  10825     ".auth ON|OFF           Show authorizer callbacks\n"
 10109  10826   #endif
 10110  10827     ".backup ?DB? FILE      Backup DB (default \"main\") to FILE\n"
        10828  +  "                         Add \"--append\" to open using appendvfs.\n"
 10111  10829     ".bail on|off           Stop after hitting an error.  Default OFF\n"
 10112  10830     ".binary on|off         Turn binary output on or off.  Default OFF\n"
 10113  10831     ".cd DIRECTORY          Change the working directory to DIRECTORY\n"
 10114  10832     ".changes on|off        Show number of rows changed by SQL\n"
 10115  10833     ".check GLOB            Fail if output since .testcase does not match\n"
 10116  10834     ".clone NEWDB           Clone data into NEWDB from the existing database\n"
 10117  10835     ".databases             List names and files of attached databases\n"
        10836  +  ".dbconfig ?op? ?val?   List or change sqlite3_db_config() options\n"
 10118  10837     ".dbinfo ?DB?           Show status information about the database\n"
 10119  10838     ".dump ?TABLE? ...      Dump the database in an SQL text format\n"
 10120  10839     "                         If TABLE specified, only dump tables matching\n"
 10121  10840     "                         LIKE pattern TABLE.\n"
 10122  10841     ".echo on|off           Turn command echo on or off\n"
 10123  10842     ".eqp on|off|full       Enable or disable automatic EXPLAIN QUERY PLAN\n"
 10124  10843     ".excel                 Display the output of next command in a spreadsheet\n"
................................................................................
 10160  10879     "                         tcl      TCL list elements\n"
 10161  10880     ".nullvalue STRING      Use STRING in place of NULL values\n"
 10162  10881     ".once (-e|-x|FILE)     Output for the next SQL command only to FILE\n"
 10163  10882     "                         or invoke system text editor (-e) or spreadsheet (-x)\n"
 10164  10883     "                         on the output.\n"
 10165  10884     ".open ?OPTIONS? ?FILE? Close existing database and reopen FILE\n"
 10166  10885     "                         The --new option starts with an empty file\n"
        10886  +  "                         Other options: --readonly --append --zip\n"
 10167  10887     ".output ?FILE?         Send output to FILE or stdout\n"
 10168  10888     ".print STRING...       Print literal STRING\n"
 10169  10889     ".prompt MAIN CONTINUE  Replace the standard prompts\n"
 10170  10890     ".quit                  Exit this program\n"
 10171  10891     ".read FILENAME         Execute SQL in FILENAME\n"
 10172  10892     ".restore ?DB? FILE     Restore content of DB (default \"main\") from FILE\n"
 10173  10893     ".save FILE             Write in-memory database into FILE\n"
................................................................................
 10177  10897     ".selftest ?--init?     Run tests defined in the SELFTEST table\n"
 10178  10898     ".separator COL ?ROW?   Change the column separator and optionally the row\n"
 10179  10899     "                         separator for both the output mode and .import\n"
 10180  10900   #if defined(SQLITE_ENABLE_SESSION)
 10181  10901     ".session CMD ...       Create or control sessions\n"
 10182  10902   #endif
 10183  10903     ".sha3sum ?OPTIONS...?  Compute a SHA3 hash of database content\n"
        10904  +#ifndef SQLITE_NOHAVE_SYSTEM
 10184  10905     ".shell CMD ARGS...     Run CMD ARGS... in a system shell\n"
        10906  +#endif
 10185  10907     ".show                  Show the current values for various settings\n"
 10186  10908     ".stats ?on|off?        Show stats or turn stats on or off\n"
        10909  +#ifndef SQLITE_NOHAVE_SYSTEM
 10187  10910     ".system CMD ARGS...    Run CMD ARGS... in a system shell\n"
        10911  +#endif
 10188  10912     ".tables ?TABLE?        List names of tables\n"
 10189  10913     "                         If TABLE specified, only list tables matching\n"
 10190  10914     "                         LIKE pattern TABLE.\n"
 10191  10915     ".testcase NAME         Begin redirecting output to 'testcase-out.txt'\n"
 10192  10916     ".timeout MS            Try opening locked tables for MS milliseconds\n"
 10193  10917     ".timer on|off          Turn SQL timer on or off\n"
 10194  10918     ".trace FILE|off        Output each SQL statement as it is run\n"
................................................................................
 10309  11033     return 1;
 10310  11034   }
 10311  11035   #endif
 10312  11036   
 10313  11037   /*
 10314  11038   ** Try to deduce the type of file for zName based on its content.  Return
 10315  11039   ** one of the SHELL_OPEN_* constants.
        11040  +**
        11041  +** If the file does not exist or is empty but its name looks like a ZIP
        11042  +** archive and the dfltZip flag is true, then assume it is a ZIP archive.
        11043  +** Otherwise, assume an ordinary database regardless of the filename if
        11044  +** the type cannot be determined from content.
 10316  11045   */
 10317         -static int deduceDatabaseType(const char *zName){
        11046  +int deduceDatabaseType(const char *zName, int dfltZip){
 10318  11047     FILE *f = fopen(zName, "rb");
 10319  11048     size_t n;
 10320  11049     int rc = SHELL_OPEN_UNSPEC;
 10321  11050     char zBuf[100];
 10322         -  if( f==0 ) return SHELL_OPEN_NORMAL;
        11051  +  if( f==0 ){
        11052  +    if( dfltZip && sqlite3_strlike("%.zip",zName,0)==0 ){
        11053  +       return SHELL_OPEN_ZIPFILE;
        11054  +    }else{
        11055  +       return SHELL_OPEN_NORMAL;
        11056  +    }
        11057  +  }
 10323  11058     fseek(f, -25, SEEK_END);
 10324  11059     n = fread(zBuf, 25, 1, f);
 10325  11060     if( n==1 && memcmp(zBuf, "Start-Of-SQLite3-", 17)==0 ){
 10326  11061       rc = SHELL_OPEN_APPENDVFS;
 10327  11062     }else{
 10328  11063       fseek(f, -22, SEEK_END);
 10329  11064       n = fread(zBuf, 22, 1, f);
 10330  11065       if( n==1 && zBuf[0]==0x50 && zBuf[1]==0x4b && zBuf[2]==0x05
 10331  11066          && zBuf[3]==0x06 ){
 10332  11067         rc = SHELL_OPEN_ZIPFILE;
        11068  +    }else if( n==0 && dfltZip && sqlite3_strlike("%.zip",zName,0)==0 ){
        11069  +      rc = SHELL_OPEN_ZIPFILE;
 10333  11070       }
 10334  11071     }
 10335  11072     fclose(f);
 10336  11073     return rc;  
 10337  11074   }
 10338  11075   
        11076  +/* Flags for open_db().
        11077  +**
        11078  +** The default behavior of open_db() is to exit(1) if the database fails to
        11079  +** open.  The OPEN_DB_KEEPALIVE flag changes that so that it prints an error
        11080  +** but still returns without calling exit.
        11081  +**
        11082  +** The OPEN_DB_ZIPFILE flag causes open_db() to prefer to open files as a
        11083  +** ZIP archive if the file does not exist or is empty and its name matches
        11084  +** the *.zip pattern.
        11085  +*/
        11086  +#define OPEN_DB_KEEPALIVE   0x001   /* Return after error if true */
        11087  +#define OPEN_DB_ZIPFILE     0x002   /* Open as ZIP if name matches *.zip */
        11088  +
 10339  11089   /*
 10340  11090   ** Make sure the database is open.  If it is not, then open it.  If
 10341  11091   ** the database fails to open, print an error message and exit.
 10342  11092   */
 10343         -static void open_db(ShellState *p, int keepAlive){
        11093  +static void open_db(ShellState *p, int openFlags){
 10344  11094     if( p->db==0 ){
 10345         -    sqlite3_initialize();
 10346         -    if( p->openMode==SHELL_OPEN_UNSPEC && access(p->zDbFilename,0)==0 ){
 10347         -      p->openMode = deduceDatabaseType(p->zDbFilename);
        11095  +    if( p->openMode==SHELL_OPEN_UNSPEC ){
        11096  +      if( p->zDbFilename==0 || p->zDbFilename[0]==0 ){
        11097  +        p->openMode = SHELL_OPEN_NORMAL;
        11098  +      }else{
        11099  +        p->openMode = (u8)deduceDatabaseType(p->zDbFilename, 
        11100  +                             (openFlags & OPEN_DB_ZIPFILE)!=0);
        11101  +      }
 10348  11102       }
 10349  11103       switch( p->openMode ){
 10350  11104         case SHELL_OPEN_APPENDVFS: {
 10351  11105           sqlite3_open_v2(p->zDbFilename, &p->db, 
 10352  11106              SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, "apndvfs");
 10353  11107           break;
 10354  11108         }
 10355  11109         case SHELL_OPEN_ZIPFILE: {
 10356  11110           sqlite3_open(":memory:", &p->db);
 10357  11111           break;
        11112  +      }
        11113  +      case SHELL_OPEN_READONLY: {
        11114  +        sqlite3_open_v2(p->zDbFilename, &p->db, SQLITE_OPEN_READONLY, 0);
        11115  +        break;
 10358  11116         }
 10359  11117         case SHELL_OPEN_UNSPEC:
 10360  11118         case SHELL_OPEN_NORMAL: {
 10361  11119           sqlite3_open(p->zDbFilename, &p->db);
 10362  11120           break;
 10363  11121         }
 10364  11122       }
 10365  11123       globalDb = p->db;
 10366  11124       if( p->db==0 || SQLITE_OK!=sqlite3_errcode(p->db) ){
 10367  11125         utf8_printf(stderr,"Error: unable to open database \"%s\": %s\n",
 10368  11126             p->zDbFilename, sqlite3_errmsg(p->db));
 10369         -      if( keepAlive ) return;
        11127  +      if( openFlags & OPEN_DB_KEEPALIVE ) return;
 10370  11128         exit(1);
 10371  11129       }
 10372  11130   #ifndef SQLITE_OMIT_LOAD_EXTENSION
 10373  11131       sqlite3_enable_load_extension(p->db, 1);
 10374  11132   #endif
 10375  11133       sqlite3_fileio_init(p->db, 0, 0);
 10376  11134       sqlite3_shathree_init(p->db, 0, 0);
................................................................................
 10381  11139   #endif
 10382  11140       sqlite3_create_function(p->db, "shell_add_schema", 3, SQLITE_UTF8, 0,
 10383  11141                               shellAddSchemaName, 0, 0);
 10384  11142       sqlite3_create_function(p->db, "shell_module_schema", 1, SQLITE_UTF8, 0,
 10385  11143                               shellModuleSchema, 0, 0);
 10386  11144       sqlite3_create_function(p->db, "shell_putsnl", 1, SQLITE_UTF8, p,
 10387  11145                               shellPutsFunc, 0, 0);
        11146  +#ifndef SQLITE_NOHAVE_SYSTEM
 10388  11147       sqlite3_create_function(p->db, "edit", 1, SQLITE_UTF8, 0,
 10389  11148                               editFunc, 0, 0);
 10390  11149       sqlite3_create_function(p->db, "edit", 2, SQLITE_UTF8, 0,
 10391  11150                               editFunc, 0, 0);
        11151  +#endif
 10392  11152       if( p->openMode==SHELL_OPEN_ZIPFILE ){
 10393  11153         char *zSql = sqlite3_mprintf(
 10394  11154            "CREATE VIRTUAL TABLE zip USING zipfile(%Q);", p->zDbFilename);
 10395  11155         sqlite3_exec(p->db, zSql, 0, 0, 0);
 10396  11156         sqlite3_free(zSql);
 10397  11157       }
 10398  11158     }
 10399  11159   }
        11160  +
        11161  +/*
        11162  +** Attempt to close the databaes connection.  Report errors.
        11163  +*/
        11164  +void close_db(sqlite3 *db){
        11165  +  int rc = sqlite3_close(db);
        11166  +  if( rc ){
        11167  +    utf8_printf(stderr, "Error: sqlite3_close() returns %d: %s\n",
        11168  +        rc, sqlite3_errmsg(db));
        11169  +  } 
        11170  +}
 10400  11171   
 10401  11172   #if HAVE_READLINE || HAVE_EDITLINE
 10402  11173   /*
 10403  11174   ** Readline completion callbacks
 10404  11175   */
 10405  11176   static char *readline_completion_generator(const char *text, int state){
 10406  11177     static sqlite3_stmt *pStmt = 0;
................................................................................
 10435  11206     int nLine = strlen30(zLine);
 10436  11207     int i, iStart;
 10437  11208     sqlite3_stmt *pStmt = 0;
 10438  11209     char *zSql;
 10439  11210     char zBuf[1000];
 10440  11211   
 10441  11212     if( nLine>sizeof(zBuf)-30 ) return;
 10442         -  if( zLine[0]=='.' ) return;
        11213  +  if( zLine[0]=='.' || zLine[0]=='#') return;
 10443  11214     for(i=nLine-1; i>=0 && (isalnum(zLine[i]) || zLine[i]=='_'); i--){}
 10444  11215     if( i==nLine-1 ) return;
 10445  11216     iStart = i+1;
 10446  11217     memcpy(zBuf, zLine, iStart);
 10447  11218     zSql = sqlite3_mprintf("SELECT DISTINCT candidate COLLATE nocase"
 10448  11219                            "  FROM completion(%Q,%Q) ORDER BY 1",
 10449  11220                            &zLine[iStart], zLine);
................................................................................
 10581  11352       if( f==0 ){
 10582  11353         utf8_printf(stderr, "Error: cannot open \"%s\"\n", zFile);
 10583  11354       }
 10584  11355     }
 10585  11356     return f;
 10586  11357   }
 10587  11358   
 10588         -#if !defined(SQLITE_UNTESTABLE)
 10589  11359   #if !defined(SQLITE_OMIT_TRACE) && !defined(SQLITE_OMIT_FLOATING_POINT)
 10590  11360   /*
 10591  11361   ** A routine for handling output from sqlite3_trace().
 10592  11362   */
 10593  11363   static int sql_trace_callback(
 10594  11364     unsigned mType,
 10595  11365     void *pArg,
................................................................................
 10603  11373       const char *z = (const char*)pX;
 10604  11374       int i = strlen30(z);
 10605  11375       while( i>0 && z[i-1]==';' ){ i--; }
 10606  11376       utf8_printf(f, "%.*s;\n", i, z);
 10607  11377     }
 10608  11378     return 0;
 10609  11379   }
 10610         -#endif
 10611  11380   #endif
 10612  11381   
 10613  11382   /*
 10614  11383   ** A no-op routine that runs with the ".breakpoint" doc-command.  This is
 10615  11384   ** a useful spot to set a debugger breakpoint.
 10616  11385   */
 10617  11386   static void test_breakpoint(void){
................................................................................
 10637  11406   };
 10638  11407   
 10639  11408   /* Append a single byte to z[] */
 10640  11409   static void import_append_char(ImportCtx *p, int c){
 10641  11410     if( p->n+1>=p->nAlloc ){
 10642  11411       p->nAlloc += p->nAlloc + 100;
 10643  11412       p->z = sqlite3_realloc64(p->z, p->nAlloc);
 10644         -    if( p->z==0 ){
 10645         -      raw_printf(stderr, "out of memory\n");
 10646         -      exit(1);
 10647         -    }
        11413  +    if( p->z==0 ) shell_out_of_memory();
 10648  11414     }
 10649  11415     p->z[p->n++] = (char)c;
 10650  11416   }
 10651  11417   
 10652  11418   /* Read a single field of CSV text.  Compatible with rfc4180 and extended
 10653  11419   ** with the option of having a separator other than ",".
 10654  11420   **
................................................................................
 10801  11567       utf8_printf(stderr, "Error %d: %s on [%s]\n",
 10802  11568               sqlite3_extended_errcode(p->db), sqlite3_errmsg(p->db),
 10803  11569               zQuery);
 10804  11570       goto end_data_xfer;
 10805  11571     }
 10806  11572     n = sqlite3_column_count(pQuery);
 10807  11573     zInsert = sqlite3_malloc64(200 + nTable + n*3);
 10808         -  if( zInsert==0 ){
 10809         -    raw_printf(stderr, "out of memory\n");
 10810         -    goto end_data_xfer;
 10811         -  }
        11574  +  if( zInsert==0 ) shell_out_of_memory();
 10812  11575     sqlite3_snprintf(200+nTable,zInsert,
 10813  11576                      "INSERT OR IGNORE INTO \"%s\" VALUES(?", zTable);
 10814  11577     i = strlen30(zInsert);
 10815  11578     for(j=1; j<n; j++){
 10816  11579       memcpy(zInsert+i, ",?", 2);
 10817  11580       i += 2;
 10818  11581     }
................................................................................
 10982  11745       sqlite3_exec(p->db, "PRAGMA writable_schema=ON;", 0, 0, 0);
 10983  11746       sqlite3_exec(newDb, "BEGIN EXCLUSIVE;", 0, 0, 0);
 10984  11747       tryToCloneSchema(p, newDb, "type='table'", tryToCloneData);
 10985  11748       tryToCloneSchema(p, newDb, "type!='table'", 0);
 10986  11749       sqlite3_exec(newDb, "COMMIT;", 0, 0, 0);
 10987  11750       sqlite3_exec(p->db, "PRAGMA writable_schema=OFF;", 0, 0, 0);
 10988  11751     }
 10989         -  sqlite3_close(newDb);
        11752  +  close_db(newDb);
 10990  11753   }
 10991  11754   
 10992  11755   /*
 10993  11756   ** Change the output file back to stdout.
 10994  11757   **
 10995  11758   ** If the p->doXdgOpen flag is set, that means the output was being
 10996  11759   ** redirected to a temporary file named by p->zTempFile.  In that case,
................................................................................
 10999  11762   static void output_reset(ShellState *p){
 11000  11763     if( p->outfile[0]=='|' ){
 11001  11764   #ifndef SQLITE_OMIT_POPEN
 11002  11765       pclose(p->out);
 11003  11766   #endif
 11004  11767     }else{
 11005  11768       output_file_close(p->out);
        11769  +#ifndef SQLITE_NOHAVE_SYSTEM
 11006  11770       if( p->doXdgOpen ){
 11007  11771         const char *zXdgOpenCmd =
 11008  11772   #if defined(_WIN32)
 11009  11773         "start";
 11010  11774   #elif defined(__APPLE__)
 11011  11775         "open";
 11012  11776   #else
................................................................................
 11017  11781         if( system(zCmd) ){
 11018  11782           utf8_printf(stderr, "Failed: [%s]\n", zCmd);
 11019  11783         }
 11020  11784         sqlite3_free(zCmd);
 11021  11785         outputModePop(p);
 11022  11786         p->doXdgOpen = 0;
 11023  11787       }
        11788  +#endif /* !defined(SQLITE_NOHAVE_SYSTEM) */
 11024  11789     }
 11025  11790     p->outfile[0] = 0;
 11026  11791     p->out = stdout;
 11027  11792   }
 11028  11793   
 11029  11794   /*
 11030  11795   ** Run an SQL command and return the single integer result.
................................................................................
 11143  11908   */
 11144  11909   static int shellDatabaseError(sqlite3 *db){
 11145  11910     const char *zErr = sqlite3_errmsg(db);
 11146  11911     utf8_printf(stderr, "Error: %s\n", zErr);
 11147  11912     return 1;
 11148  11913   }
 11149  11914   
 11150         -/*
 11151         -** Print an out-of-memory message to stderr and return 1.
 11152         -*/
 11153         -static int shellNomemError(void){
 11154         -  raw_printf(stderr, "Error: out of memory\n");
 11155         -  return 1;
 11156         -}
 11157         -
 11158  11915   /*
 11159  11916   ** Compare the pattern in zGlob[] against the text in z[].  Return TRUE
 11160  11917   ** if they match and FALSE (0) if they do not match.
 11161  11918   **
 11162  11919   ** Globbing rules:
 11163  11920   **
 11164  11921   **      '*'       Matches any sequence of zero or more characters.
................................................................................
 11643  12400   typedef struct ArCommand ArCommand;
 11644  12401   struct ArCommand {
 11645  12402     u8 eCmd;                        /* An AR_CMD_* value */
 11646  12403     u8 bVerbose;                    /* True if --verbose */
 11647  12404     u8 bZip;                        /* True if the archive is a ZIP */
 11648  12405     u8 bDryRun;                     /* True if --dry-run */
 11649  12406     u8 bAppend;                     /* True if --append */
        12407  +  u8 fromCmdLine;                 /* Run from -A instead of .archive */
 11650  12408     int nArg;                       /* Number of command arguments */
 11651  12409     char *zSrcTable;                /* "sqlar", "zipfile($file)" or "zip" */
 11652  12410     const char *zFile;              /* --file argument, or NULL */
 11653  12411     const char *zDir;               /* --directory argument, or NULL */
 11654  12412     char **azArg;                   /* Array of command arguments */
 11655  12413     ShellState *p;                  /* Shell state */
 11656  12414     sqlite3 *db;                    /* Database containing the archive */
................................................................................
 11689  12447     return SQLITE_ERROR;
 11690  12448   }
 11691  12449   
 11692  12450   /*
 11693  12451   ** Print an error message for the .ar command to stderr and return 
 11694  12452   ** SQLITE_ERROR.
 11695  12453   */
 11696         -static int arErrorMsg(const char *zFmt, ...){
        12454  +static int arErrorMsg(ArCommand *pAr, const char *zFmt, ...){
 11697  12455     va_list ap;
 11698  12456     char *z;
 11699  12457     va_start(ap, zFmt);
 11700  12458     z = sqlite3_vmprintf(zFmt, ap);
 11701  12459     va_end(ap);
 11702         -  raw_printf(stderr, "Error: %s (try \".ar --help\")\n", z);
        12460  +  utf8_printf(stderr, "Error: %s\n", z);
        12461  +  if( pAr->fromCmdLine ){
        12462  +    utf8_printf(stderr, "Use \"-A\" for more help\n");
        12463  +  }else{
        12464  +    utf8_printf(stderr, "Use \".archive --help\" for more help\n");
        12465  +  }
 11703  12466     sqlite3_free(z);
 11704  12467     return SQLITE_ERROR;
 11705  12468   }
 11706  12469   
 11707  12470   /*
 11708  12471   ** Values for ArCommand.eCmd.
 11709  12472   */
................................................................................
 11726  12489     switch( eSwitch ){
 11727  12490       case AR_CMD_CREATE:
 11728  12491       case AR_CMD_EXTRACT:
 11729  12492       case AR_CMD_LIST:
 11730  12493       case AR_CMD_UPDATE:
 11731  12494       case AR_CMD_HELP:
 11732  12495         if( pAr->eCmd ){
 11733         -        return arErrorMsg("multiple command options");
        12496  +        return arErrorMsg(pAr, "multiple command options");
 11734  12497         }
 11735  12498         pAr->eCmd = eSwitch;
 11736  12499         break;
 11737  12500   
 11738  12501       case AR_SWITCH_DRYRUN:
 11739  12502         pAr->bDryRun = 1;
 11740  12503         break;
................................................................................
 11786  12549     int nSwitch = sizeof(aSwitch) / sizeof(struct ArSwitch);
 11787  12550     struct ArSwitch *pEnd = &aSwitch[nSwitch];
 11788  12551   
 11789  12552     if( nArg<=1 ){
 11790  12553       return arUsage(stderr);
 11791  12554     }else{
 11792  12555       char *z = azArg[1];
 11793         -    memset(pAr, 0, sizeof(ArCommand));
 11794         -
 11795  12556       if( z[0]!='-' ){
 11796  12557         /* Traditional style [tar] invocation */
 11797  12558         int i;
 11798  12559         int iArg = 2;
 11799  12560         for(i=0; z[i]; i++){
 11800  12561           const char *zArg = 0;
 11801  12562           struct ArSwitch *pOpt;
 11802  12563           for(pOpt=&aSwitch[0]; pOpt<pEnd; pOpt++){
 11803  12564             if( z[i]==pOpt->cShort ) break;
 11804  12565           }
 11805  12566           if( pOpt==pEnd ){
 11806         -          return arErrorMsg("unrecognized option: %c", z[i]);
        12567  +          return arErrorMsg(pAr, "unrecognized option: %c", z[i]);
 11807  12568           }
 11808  12569           if( pOpt->bArg ){
 11809  12570             if( iArg>=nArg ){
 11810         -            return arErrorMsg("option requires an argument: %c",z[i]);
        12571  +            return arErrorMsg(pAr, "option requires an argument: %c",z[i]);
 11811  12572             }
 11812  12573             zArg = azArg[iArg++];
 11813  12574           }
 11814  12575           if( arProcessSwitch(pAr, pOpt->eSwitch, zArg) ) return SQLITE_ERROR;
 11815  12576         }
 11816  12577         pAr->nArg = nArg-iArg;
 11817  12578         if( pAr->nArg>0 ){
................................................................................
 11837  12598             for(i=1; i<n; i++){
 11838  12599               const char *zArg = 0;
 11839  12600               struct ArSwitch *pOpt;
 11840  12601               for(pOpt=&aSwitch[0]; pOpt<pEnd; pOpt++){
 11841  12602                 if( z[i]==pOpt->cShort ) break;
 11842  12603               }
 11843  12604               if( pOpt==pEnd ){
 11844         -              return arErrorMsg("unrecognized option: %c\n", z[i]);
        12605  +              return arErrorMsg(pAr, "unrecognized option: %c", z[i]);
 11845  12606               }
 11846  12607               if( pOpt->bArg ){
 11847  12608                 if( i<(n-1) ){
 11848  12609                   zArg = &z[i+1];
 11849  12610                   i = n;
 11850  12611                 }else{
 11851  12612                   if( iArg>=(nArg-1) ){
 11852         -                  return arErrorMsg("option requires an argument: %c\n",z[i]);
        12613  +                  return arErrorMsg(pAr, "option requires an argument: %c",z[i]);
 11853  12614                   }
 11854  12615                   zArg = azArg[++iArg];
 11855  12616                 }
 11856  12617               }
 11857  12618               if( arProcessSwitch(pAr, pOpt->eSwitch, zArg) ) return SQLITE_ERROR;
 11858  12619             }
 11859  12620           }else if( z[2]=='\0' ){
................................................................................
 11867  12628             const char *zArg = 0;             /* Argument for option, if any */
 11868  12629             struct ArSwitch *pMatch = 0;      /* Matching option */
 11869  12630             struct ArSwitch *pOpt;            /* Iterator */
 11870  12631             for(pOpt=&aSwitch[0]; pOpt<pEnd; pOpt++){
 11871  12632               const char *zLong = pOpt->zLong;
 11872  12633               if( (n-2)<=strlen30(zLong) && 0==memcmp(&z[2], zLong, n-2) ){
 11873  12634                 if( pMatch ){
 11874         -                return arErrorMsg("ambiguous option: %s",z);
        12635  +                return arErrorMsg(pAr, "ambiguous option: %s",z);
 11875  12636                 }else{
 11876  12637                   pMatch = pOpt;
 11877  12638                 }
 11878  12639               }
 11879  12640             }
 11880  12641   
 11881  12642             if( pMatch==0 ){
 11882         -            return arErrorMsg("unrecognized option: %s", z);
        12643  +            return arErrorMsg(pAr, "unrecognized option: %s", z);
 11883  12644             }
 11884  12645             if( pMatch->bArg ){
 11885  12646               if( iArg>=(nArg-1) ){
 11886         -              return arErrorMsg("option requires an argument: %s", z);
        12647  +              return arErrorMsg(pAr, "option requires an argument: %s", z);
 11887  12648               }
 11888  12649               zArg = azArg[++iArg];
 11889  12650             }
 11890  12651             if( arProcessSwitch(pAr, pMatch->eSwitch, zArg) ) return SQLITE_ERROR;
 11891  12652           }
 11892  12653         }
 11893  12654       }
................................................................................
 12008  12769           );
 12009  12770         }else{
 12010  12771           utf8_printf(pAr->p->out, "%s\n", sqlite3_column_text(pSql, 0));
 12011  12772         }
 12012  12773       }
 12013  12774     }
 12014  12775     shellFinalize(&rc, pSql);
        12776  +  sqlite3_free(zWhere);
 12015  12777     return rc;
 12016  12778   }
 12017  12779   
 12018  12780   
 12019  12781   /*
 12020  12782   ** Implementation of .ar "eXtract" command. 
 12021  12783   */
................................................................................
 12127  12889         "  name TEXT PRIMARY KEY,  -- name of the file\n"
 12128  12890         "  mode INT,               -- access permissions\n"
 12129  12891         "  mtime INT,              -- last modification time\n"
 12130  12892         "  sz INT,                 -- original file size\n"
 12131  12893         "  data BLOB               -- compressed content\n"
 12132  12894         ")";
 12133  12895     const char *zDrop = "DROP TABLE IF EXISTS sqlar";
 12134         -  const char *zInsertFmt = 
 12135         -     "REPLACE INTO sqlar(name,mode,mtime,sz,data)\n"
        12896  +  const char *zInsertFmt[2] = {
        12897  +     "REPLACE INTO %s(name,mode,mtime,sz,data)\n"
 12136  12898        "  SELECT\n"
 12137  12899        "    %s,\n"
 12138  12900        "    mode,\n"
 12139  12901        "    mtime,\n"
 12140  12902        "    CASE substr(lsmode(mode),1,1)\n"
 12141  12903        "      WHEN '-' THEN length(data)\n"
 12142  12904        "      WHEN 'd' THEN 0\n"
 12143  12905        "      ELSE -1 END,\n"
 12144         -     "    CASE WHEN lsmode(mode) LIKE 'd%%' THEN NULL else data END\n"
        12906  +     "    sqlar_compress(data)\n"
        12907  +     "  FROM fsdir(%Q,%Q)\n"
        12908  +     "  WHERE lsmode(mode) NOT LIKE '?%%';",
        12909  +     "REPLACE INTO %s(name,mode,mtime,data)\n"
        12910  +     "  SELECT\n"
        12911  +     "    %s,\n"
        12912  +     "    mode,\n"
        12913  +     "    mtime,\n"
        12914  +     "    data\n"
 12145  12915        "  FROM fsdir(%Q,%Q)\n"
 12146         -     "  WHERE lsmode(mode) NOT LIKE '?%%';";
        12916  +     "  WHERE lsmode(mode) NOT LIKE '?%%';"
        12917  +  };
 12147  12918     int i;                          /* For iterating through azFile[] */
 12148  12919     int rc;                         /* Return code */
        12920  +  const char *zTab = 0;           /* SQL table into which to insert */
        12921  +  char *zSql;
        12922  +  char zTemp[50];
 12149  12923   
        12924  +  arExecSql(pAr, "PRAGMA page_size=512");
 12150  12925     rc = arExecSql(pAr, "SAVEPOINT ar;");
 12151  12926     if( rc!=SQLITE_OK ) return rc;
 12152         -  if( bUpdate==0 ){
 12153         -    rc = arExecSql(pAr, zDrop);
 12154         -    if( rc!=SQLITE_OK ) return rc;
        12927  +  zTemp[0] = 0; 
        12928  +  if( pAr->bZip ){
        12929  +    /* Initialize the zipfile virtual table, if necessary */
        12930  +    if( pAr->zFile ){
        12931  +      sqlite3_uint64 r;
        12932  +      sqlite3_randomness(sizeof(r),&r);
        12933  +      sqlite3_snprintf(sizeof(zTemp),zTemp,"zip%016llx",r);
        12934  +      zTab = zTemp;
        12935  +      zSql = sqlite3_mprintf(
        12936  +         "CREATE VIRTUAL TABLE temp.%s USING zipfile(%Q)",
        12937  +         zTab, pAr->zFile
        12938  +      );
        12939  +      rc = arExecSql(pAr, zSql);
        12940  +      sqlite3_free(zSql);
        12941  +    }else{
        12942  +      zTab = "zip";
        12943  +    }
        12944  +  }else{
        12945  +    /* Initialize the table for an SQLAR */
        12946  +    zTab = "sqlar";
        12947  +    if( bUpdate==0 ){
        12948  +      rc = arExecSql(pAr, zDrop);
        12949  +      if( rc!=SQLITE_OK ) goto end_ar_transaction;
        12950  +    }
        12951  +    rc = arExecSql(pAr, zCreate);
 12155  12952     }
 12156         -  rc = arExecSql(pAr, zCreate);
 12157  12953     for(i=0; i<pAr->nArg && rc==SQLITE_OK; i++){
 12158         -    char *zSql = sqlite3_mprintf(zInsertFmt,
        12954  +    char *zSql2 = sqlite3_mprintf(zInsertFmt[pAr->bZip], zTab,
 12159  12955           pAr->bVerbose ? "shell_putsnl(name)" : "name",
 12160  12956           pAr->azArg[i], pAr->zDir);
 12161         -    rc = arExecSql(pAr, zSql);
 12162         -    sqlite3_free(zSql);
        12957  +    rc = arExecSql(pAr, zSql2);
        12958  +    sqlite3_free(zSql2);
 12163  12959     }
        12960  +end_ar_transaction:
 12164  12961     if( rc!=SQLITE_OK ){
 12165  12962       arExecSql(pAr, "ROLLBACK TO ar; RELEASE ar;");
 12166  12963     }else{
 12167  12964       rc = arExecSql(pAr, "RELEASE ar;");
        12965  +    if( pAr->bZip && pAr->zFile ){
        12966  +      zSql = sqlite3_mprintf("DROP TABLE %s", zTemp);
        12967  +      arExecSql(pAr, zSql);
        12968  +      sqlite3_free(zSql);
        12969  +    }
 12168  12970     }
 12169  12971     return rc;
 12170  12972   }
 12171  12973   
 12172  12974   /*
 12173  12975   ** Implementation of ".ar" dot command.
 12174  12976   */
 12175  12977   static int arDotCommand(
 12176  12978     ShellState *pState,             /* Current shell tool state */
        12979  +  int fromCmdLine,                /* True if -A command-line option, not .ar cmd */
 12177  12980     char **azArg,                   /* Array of arguments passed to dot command */
 12178  12981     int nArg                        /* Number of entries in azArg[] */
 12179  12982   ){
 12180  12983     ArCommand cmd;
 12181  12984     int rc;
 12182  12985     memset(&cmd, 0, sizeof(cmd));
        12986  +  cmd.fromCmdLine = fromCmdLine;
 12183  12987     rc = arParseCommand(azArg, nArg, &cmd);
 12184  12988     if( rc==SQLITE_OK ){
 12185  12989       int eDbType = SHELL_OPEN_UNSPEC;
 12186  12990       cmd.p = pState;
 12187  12991       cmd.db = pState->db;
 12188  12992       if( cmd.zFile ){
 12189         -      eDbType = deduceDatabaseType(cmd.zFile);
        12993  +      eDbType = deduceDatabaseType(cmd.zFile, 1);
 12190  12994       }else{
 12191  12995         eDbType = pState->openMode;
 12192  12996       }
 12193  12997       if( eDbType==SHELL_OPEN_ZIPFILE ){
 12194         -      if( cmd.zFile==0 ){
 12195         -        cmd.zSrcTable = sqlite3_mprintf("zip");
 12196         -      }else{
 12197         -        cmd.zSrcTable = sqlite3_mprintf("zipfile(%Q)", cmd.zFile);
 12198         -      }
 12199         -      if( cmd.eCmd==AR_CMD_CREATE || cmd.eCmd==AR_CMD_UPDATE ){
 12200         -        utf8_printf(stderr, "zip archives are read-only\n");
 12201         -        rc = SQLITE_ERROR;
 12202         -        goto end_ar_command;
        12998  +      if( cmd.eCmd==AR_CMD_EXTRACT || cmd.eCmd==AR_CMD_LIST ){
        12999  +        if( cmd.zFile==0 ){
        13000  +          cmd.zSrcTable = sqlite3_mprintf("zip");
        13001  +        }else{
        13002  +          cmd.zSrcTable = sqlite3_mprintf("zipfile(%Q)", cmd.zFile);
        13003  +        }
 12203  13004         }
 12204  13005         cmd.bZip = 1;
 12205  13006       }else if( cmd.zFile ){
 12206  13007         int flags;
 12207  13008         if( cmd.bAppend ) eDbType = SHELL_OPEN_APPENDVFS;
 12208  13009         if( cmd.eCmd==AR_CMD_CREATE || cmd.eCmd==AR_CMD_UPDATE ){
 12209  13010           flags = SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE;
................................................................................
 12220  13021         if( rc!=SQLITE_OK ){
 12221  13022           utf8_printf(stderr, "cannot open file: %s (%s)\n", 
 12222  13023               cmd.zFile, sqlite3_errmsg(cmd.db)
 12223  13024           );
 12224  13025           goto end_ar_command;
 12225  13026         }
 12226  13027         sqlite3_fileio_init(cmd.db, 0, 0);
 12227         -#ifdef SQLITE_HAVE_ZLIB
 12228  13028         sqlite3_sqlar_init(cmd.db, 0, 0);
 12229         -#endif
 12230  13029         sqlite3_create_function(cmd.db, "shell_putsnl", 1, SQLITE_UTF8, cmd.p,
 12231  13030                                 shellPutsFunc, 0, 0);
 12232  13031   
 12233  13032       }
 12234         -    if( cmd.zSrcTable==0 ){
        13033  +    if( cmd.zSrcTable==0 && cmd.bZip==0 && cmd.eCmd!=AR_CMD_HELP ){
 12235  13034         if( cmd.eCmd!=AR_CMD_CREATE
 12236  13035          && sqlite3_table_column_metadata(cmd.db,0,"sqlar","name",0,0,0,0,0)
 12237  13036         ){
 12238  13037           utf8_printf(stderr, "database does not contain an 'sqlar' table\n");
 12239  13038           rc = SQLITE_ERROR;
 12240  13039           goto end_ar_command;
 12241  13040         }
................................................................................
 12263  13062           assert( cmd.eCmd==AR_CMD_UPDATE );
 12264  13063           rc = arCreateOrUpdateCommand(&cmd, 1);
 12265  13064           break;
 12266  13065       }
 12267  13066     }
 12268  13067   end_ar_command:
 12269  13068     if( cmd.db!=pState->db ){
 12270         -    sqlite3_close(cmd.db);
        13069  +    close_db(cmd.db);
 12271  13070     }
 12272  13071     sqlite3_free(cmd.zSrcTable);
 12273  13072   
 12274  13073     return rc;
 12275  13074   }
 12276  13075   /* End of the ".archive" or ".ar" command logic
 12277  13076   **********************************************************************************/
................................................................................
 12343  13142       }
 12344  13143     }else
 12345  13144   #endif
 12346  13145   
 12347  13146   #if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
 12348  13147     if( c=='a' && strncmp(azArg[0], "archive", n)==0 ){
 12349  13148       open_db(p, 0);
 12350         -    rc = arDotCommand(p, azArg, nArg);
        13149  +    rc = arDotCommand(p, 0, azArg, nArg);
 12351  13150     }else
 12352  13151   #endif
 12353  13152   
 12354  13153     if( (c=='b' && n>=3 && strncmp(azArg[0], "backup", n)==0)
 12355  13154      || (c=='s' && n>=3 && strncmp(azArg[0], "save", n)==0)
 12356  13155     ){
 12357  13156       const char *zDestFile = 0;
 12358  13157       const char *zDb = 0;
 12359  13158       sqlite3 *pDest;
 12360  13159       sqlite3_backup *pBackup;
 12361  13160       int j;
        13161  +    const char *zVfs = 0;
 12362  13162       for(j=1; j<nArg; j++){
 12363  13163         const char *z = azArg[j];
 12364  13164         if( z[0]=='-' ){
 12365         -        while( z[0]=='-' ) z++;
 12366         -        /* No options to process at this time */
        13165  +        if( z[1]=='-' ) z++;
        13166  +        if( strcmp(z, "-append")==0 ){
        13167  +          zVfs = "apndvfs";
        13168  +        }else
 12367  13169           {
 12368  13170             utf8_printf(stderr, "unknown option: %s\n", azArg[j]);
 12369  13171             return 1;
 12370  13172           }
 12371  13173         }else if( zDestFile==0 ){
 12372  13174           zDestFile = azArg[j];
 12373  13175         }else if( zDb==0 ){
 12374  13176           zDb = zDestFile;
 12375  13177           zDestFile = azArg[j];
 12376  13178         }else{
 12377         -        raw_printf(stderr, "too many arguments to .backup\n");
        13179  +        raw_printf(stderr, "Usage: .backup ?DB? ?--append? FILENAME\n");
 12378  13180           return 1;
 12379  13181         }
 12380  13182       }
 12381  13183       if( zDestFile==0 ){
 12382  13184         raw_printf(stderr, "missing FILENAME argument on .backup\n");
 12383  13185         return 1;
 12384  13186       }
 12385  13187       if( zDb==0 ) zDb = "main";
 12386         -    rc = sqlite3_open(zDestFile, &pDest);
        13188  +    rc = sqlite3_open_v2(zDestFile, &pDest, 
        13189  +                  SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, zVfs);
 12387  13190       if( rc!=SQLITE_OK ){
 12388  13191         utf8_printf(stderr, "Error: cannot open \"%s\"\n", zDestFile);
 12389         -      sqlite3_close(pDest);
        13192  +      close_db(pDest);
 12390  13193         return 1;
 12391  13194       }
 12392  13195       open_db(p, 0);
 12393  13196       pBackup = sqlite3_backup_init(pDest, "main", p->db, zDb);
 12394  13197       if( pBackup==0 ){
 12395  13198         utf8_printf(stderr, "Error: %s\n", sqlite3_errmsg(pDest));
 12396         -      sqlite3_close(pDest);
        13199  +      close_db(pDest);
 12397  13200         return 1;
 12398  13201       }
 12399  13202       while(  (rc = sqlite3_backup_step(pBackup,100))==SQLITE_OK ){}
 12400  13203       sqlite3_backup_finish(pBackup);
 12401  13204       if( rc==SQLITE_DONE ){
 12402  13205         rc = 0;
 12403  13206       }else{
 12404  13207         utf8_printf(stderr, "Error: %s\n", sqlite3_errmsg(pDest));
 12405  13208         rc = 1;
 12406  13209       }
 12407         -    sqlite3_close(pDest);
        13210  +    close_db(pDest);
 12408  13211     }else
 12409  13212   
 12410  13213     if( c=='b' && n>=3 && strncmp(azArg[0], "bail", n)==0 ){
 12411  13214       if( nArg==2 ){
 12412  13215         bail_on_error = booleanValue(azArg[1]);
 12413  13216       }else{
 12414  13217         raw_printf(stderr, "Usage: .bail on|off\n");
................................................................................
 12512  13315       if( zErrMsg ){
 12513  13316         utf8_printf(stderr,"Error: %s\n", zErrMsg);
 12514  13317         sqlite3_free(zErrMsg);
 12515  13318         rc = 1;
 12516  13319       }
 12517  13320     }else
 12518  13321   
 12519         -  if( c=='d' && strncmp(azArg[0], "dbinfo", n)==0 ){
        13322  +  if( c=='d' && n>=3 && strncmp(azArg[0], "dbconfig", n)==0 ){
        13323  +    static const struct DbConfigChoices {const char *zName; int op;} aDbConfig[] = {
        13324  +        { "enable_fkey",      SQLITE_DBCONFIG_ENABLE_FKEY            },
        13325  +        { "enable_trigger",   SQLITE_DBCONFIG_ENABLE_TRIGGER         },
        13326  +        { "fts3_tokenizer",   SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER  },
        13327  +        { "load_extension",   SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION  },
        13328  +        { "no_ckpt_on_close", SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE       },
        13329  +        { "enable_qpsg",      SQLITE_DBCONFIG_ENABLE_QPSG            },
        13330  +        { "trigger_eqp",      SQLITE_DBCONFIG_TRIGGER_EQP            },
        13331  +        { "reset_database",   SQLITE_DBCONFIG_RESET_DATABASE         },
        13332  +    };
        13333  +    int ii, v;
        13334  +    open_db(p, 0);
        13335  +    for(ii=0; ii<ArraySize(aDbConfig); ii++){
        13336  +      if( nArg>1 && strcmp(azArg[1], aDbConfig[ii].zName)!=0 ) continue;
        13337  +      if( nArg>=3 ){
        13338  +        sqlite3_db_config(p->db, aDbConfig[ii].op, booleanValue(azArg[2]), 0);
        13339  +      }
        13340  +      sqlite3_db_config(p->db, aDbConfig[ii].op, -1, &v);
        13341  +      utf8_printf(p->out, "%18s %s\n", aDbConfig[ii].zName, v ? "on" : "off");
        13342  +      if( nArg>1 ) break;
        13343  +    }
        13344  +    if( nArg>1 && ii==ArraySize(aDbConfig) ){
        13345  +      utf8_printf(stderr, "Error: unknown dbconfig \"%s\"\n", azArg[1]);
        13346  +      utf8_printf(stderr, "Enter \".dbconfig\" with no arguments for a list\n");
        13347  +    }   
        13348  +  }else
        13349  +
        13350  +  if( c=='d' && n>=3 && strncmp(azArg[0], "dbinfo", n)==0 ){
 12520  13351       rc = shell_dbinfo_command(p, nArg, azArg);
 12521  13352     }else
 12522  13353   
 12523  13354     if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){
 12524  13355       const char *zLike = 0;
 12525  13356       int i;
 12526  13357       int savedShowHeader = p->showHeader;
................................................................................
 12615  13446         raw_printf(stderr, "Usage: .echo on|off\n");
 12616  13447         rc = 1;
 12617  13448       }
 12618  13449     }else
 12619  13450   
 12620  13451     if( c=='e' && strncmp(azArg[0], "eqp", n)==0 ){
 12621  13452       if( nArg==2 ){
        13453  +      p->autoEQPtest = 0;
 12622  13454         if( strcmp(azArg[1],"full")==0 ){
 12623  13455           p->autoEQP = AUTOEQP_full;
 12624  13456         }else if( strcmp(azArg[1],"trigger")==0 ){
 12625  13457           p->autoEQP = AUTOEQP_trigger;
        13458  +      }else if( strcmp(azArg[1],"test")==0 ){
        13459  +        p->autoEQP = AUTOEQP_on;
        13460  +        p->autoEQPtest = 1;
 12626  13461         }else{
 12627         -        p->autoEQP = booleanValue(azArg[1]);
        13462  +        p->autoEQP = (u8)booleanValue(azArg[1]);
 12628  13463         }
 12629  13464       }else{
 12630  13465         raw_printf(stderr, "Usage: .eqp off|on|trigger|full\n");
 12631  13466         rc = 1;
 12632  13467       }
 12633  13468     }else
 12634  13469   
................................................................................
 12707  13542         raw_printf(p->out, "/* No STAT tables available */\n");
 12708  13543       }else{
 12709  13544         raw_printf(p->out, "ANALYZE sqlite_master;\n");
 12710  13545         sqlite3_exec(p->db, "SELECT 'ANALYZE sqlite_master'",
 12711  13546                      callback, &data, &zErrMsg);
 12712  13547         data.cMode = data.mode = MODE_Insert;
 12713  13548         data.zDestTable = "sqlite_stat1";
 12714         -      shell_exec(p->db, "SELECT * FROM sqlite_stat1",
 12715         -                 shell_callback, &data,&zErrMsg);
        13549  +      shell_exec(&data, "SELECT * FROM sqlite_stat1", &zErrMsg);
 12716  13550         data.zDestTable = "sqlite_stat3";
 12717         -      shell_exec(p->db, "SELECT * FROM sqlite_stat3",
 12718         -                 shell_callback, &data,&zErrMsg);
        13551  +      shell_exec(&data, "SELECT * FROM sqlite_stat3", &zErrMsg);
 12719  13552         data.zDestTable = "sqlite_stat4";
 12720         -      shell_exec(p->db, "SELECT * FROM sqlite_stat4",
 12721         -                 shell_callback, &data, &zErrMsg);
        13553  +      shell_exec(&data, "SELECT * FROM sqlite_stat4", &zErrMsg);
 12722  13554         raw_printf(p->out, "ANALYZE sqlite_master;\n");
 12723  13555       }
 12724  13556     }else
 12725  13557   
 12726  13558     if( c=='h' && strncmp(azArg[0], "headers", n)==0 ){
 12727  13559       if( nArg==2 ){
 12728  13560         p->showHeader = booleanValue(azArg[1]);
................................................................................
 12812  13644         utf8_printf(stderr, "Error: cannot open \"%s\"\n", zFile);
 12813  13645         return 1;
 12814  13646       }
 12815  13647       sCtx.cColSep = p->colSeparator[0];
 12816  13648       sCtx.cRowSep = p->rowSeparator[0];
 12817  13649       zSql = sqlite3_mprintf("SELECT * FROM %s", zTable);
 12818  13650       if( zSql==0 ){
 12819         -      raw_printf(stderr, "Error: out of memory\n");
 12820  13651         xCloser(sCtx.in);
 12821         -      return 1;
        13652  +      shell_out_of_memory();
 12822  13653       }
 12823  13654       nByte = strlen30(zSql);
 12824  13655       rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
 12825  13656       import_append_char(&sCtx, 0);    /* To ensure sCtx.z is allocated */
 12826  13657       if( rc && sqlite3_strglob("no such table: *", sqlite3_errmsg(p->db))==0 ){
 12827  13658         char *zCreate = sqlite3_mprintf("CREATE TABLE %s", zTable);
 12828  13659         char cSep = '(';
................................................................................
 12859  13690       }
 12860  13691       nCol = sqlite3_column_count(pStmt);
 12861  13692       sqlite3_finalize(pStmt);
 12862  13693       pStmt = 0;
 12863  13694       if( nCol==0 ) return 0; /* no columns, no error */
 12864  13695       zSql = sqlite3_malloc64( nByte*2 + 20 + nCol*2 );
 12865  13696       if( zSql==0 ){
 12866         -      raw_printf(stderr, "Error: out of memory\n");
 12867  13697         xCloser(sCtx.in);
 12868         -      return 1;
        13698  +      shell_out_of_memory();
 12869  13699       }
 12870  13700       sqlite3_snprintf(nByte+20, zSql, "INSERT INTO \"%w\" VALUES(?", zTable);
 12871  13701       j = strlen30(zSql);
 12872  13702       for(i=1; i<nCol; i++){
 12873  13703         zSql[j++] = ',';
 12874  13704         zSql[j++] = '?';
 12875  13705       }
................................................................................
 12937  13767   #ifndef SQLITE_UNTESTABLE
 12938  13768     if( c=='i' && strncmp(azArg[0], "imposter", n)==0 ){
 12939  13769       char *zSql;
 12940  13770       char *zCollist = 0;
 12941  13771       sqlite3_stmt *pStmt;
 12942  13772       int tnum = 0;
 12943  13773       int i;
 12944         -    if( nArg!=3 ){
 12945         -      utf8_printf(stderr, "Usage: .imposter INDEX IMPOSTER\n");
        13774  +    if( !(nArg==3 || (nArg==2 && sqlite3_stricmp(azArg[1],"off")==0)) ){
        13775  +      utf8_printf(stderr, "Usage: .imposter INDEX IMPOSTER\n"
        13776  +                          "       .imposter off\n");
 12946  13777         rc = 1;
 12947  13778         goto meta_command_exit;
 12948  13779       }
 12949  13780       open_db(p, 0);
        13781  +    if( nArg==2 ){
        13782  +      sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, p->db, "main", 0, 1);
        13783  +      goto meta_command_exit;
        13784  +    }
 12950  13785       zSql = sqlite3_mprintf("SELECT rootpage FROM sqlite_master"
 12951  13786                              " WHERE name='%q' AND type='index'", azArg[1]);
 12952  13787       sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
 12953  13788       sqlite3_free(zSql);
 12954  13789       if( sqlite3_step(pStmt)==SQLITE_ROW ){
 12955  13790         tnum = sqlite3_column_int(pStmt, 0);
 12956  13791       }
................................................................................
 13184  14019   
 13185  14020     if( c=='o' && strncmp(azArg[0], "open", n)==0 && n>=2 ){
 13186  14021       char *zNewFilename;  /* Name of the database file to open */
 13187  14022       int iName = 1;       /* Index in azArg[] of the filename */
 13188  14023       int newFlag = 0;     /* True to delete file before opening */
 13189  14024       /* Close the existing database */
 13190  14025       session_close_all(p);
 13191         -    sqlite3_close(p->db);
        14026  +    close_db(p->db);
 13192  14027       p->db = 0;
 13193  14028       p->zDbFilename = 0;
 13194  14029       sqlite3_free(p->zFreeOnClose);
 13195  14030       p->zFreeOnClose = 0;
 13196  14031       p->openMode = SHELL_OPEN_UNSPEC;
 13197  14032       /* Check for command-line arguments */
 13198  14033       for(iName=1; iName<nArg && azArg[iName][0]=='-'; iName++){
 13199  14034         const char *z = azArg[iName];
 13200  14035         if( optionMatch(z,"new") ){
 13201  14036           newFlag = 1;
 13202         -#ifdef SQLITE_HAVE_ZIP
        14037  +#ifdef SQLITE_HAVE_ZLIB
 13203  14038         }else if( optionMatch(z, "zip") ){
 13204  14039           p->openMode = SHELL_OPEN_ZIPFILE;
 13205  14040   #endif
 13206  14041         }else if( optionMatch(z, "append") ){
 13207  14042           p->openMode = SHELL_OPEN_APPENDVFS;
        14043  +      }else if( optionMatch(z, "readonly") ){
        14044  +        p->openMode = SHELL_OPEN_READONLY;
 13208  14045         }else if( z[0]=='-' ){
 13209  14046           utf8_printf(stderr, "unknown option: %s\n", z);
 13210  14047           rc = 1;
 13211  14048           goto meta_command_exit;
 13212  14049         }
 13213  14050       }
 13214  14051       /* If a filename is specified, try to open it first */
 13215  14052       zNewFilename = nArg>iName ? sqlite3_mprintf("%s", azArg[iName]) : 0;
 13216  14053       if( zNewFilename ){
 13217  14054         if( newFlag ) shellDeleteFile(zNewFilename);
 13218  14055         p->zDbFilename = zNewFilename;
 13219         -      open_db(p, 1);
        14056  +      open_db(p, OPEN_DB_KEEPALIVE);
 13220  14057         if( p->db==0 ){
 13221  14058           utf8_printf(stderr, "Error: cannot open '%s'\n", zNewFilename);
 13222  14059           sqlite3_free(zNewFilename);
 13223  14060         }else{
 13224  14061           p->zFreeOnClose = zNewFilename;
 13225  14062         }
 13226  14063       }
................................................................................
 13257  14094         }
 13258  14095         p->outCount = 2;
 13259  14096       }else{
 13260  14097         p->outCount = 0;
 13261  14098       }
 13262  14099       output_reset(p);
 13263  14100       if( zFile[0]=='-' && zFile[1]=='-' ) zFile++;
        14101  +#ifndef SQLITE_NOHAVE_SYSTEM
 13264  14102       if( strcmp(zFile, "-e")==0 || strcmp(zFile, "-x")==0 ){
 13265  14103         p->doXdgOpen = 1;
 13266  14104         outputModePush(p);
 13267  14105         if( zFile[1]=='x' ){
 13268  14106           newTempFile(p, "csv");
 13269  14107           p->mode = MODE_Csv;
 13270  14108           sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Comma);
................................................................................
 13271  14109           sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_CrLf);
 13272  14110         }else{
 13273  14111           newTempFile(p, "txt");
 13274  14112           bTxtMode = 1;
 13275  14113         }
 13276  14114         zFile = p->zTempFile;
 13277  14115       }
        14116  +#endif /* SQLITE_NOHAVE_SYSTEM */
 13278  14117       if( zFile[0]=='|' ){
 13279  14118   #ifdef SQLITE_OMIT_POPEN
 13280  14119         raw_printf(stderr, "Error: pipes are not supported in this OS\n");
 13281  14120         rc = 1;
 13282  14121         p->out = stdout;
 13283  14122   #else
 13284  14123         p->out = popen(zFile + 1, "w");
................................................................................
 13360  14199         raw_printf(stderr, "Usage: .restore ?DB? FILE\n");
 13361  14200         rc = 1;
 13362  14201         goto meta_command_exit;
 13363  14202       }
 13364  14203       rc = sqlite3_open(zSrcFile, &pSrc);
 13365  14204       if( rc!=SQLITE_OK ){
 13366  14205         utf8_printf(stderr, "Error: cannot open \"%s\"\n", zSrcFile);
 13367         -      sqlite3_close(pSrc);
        14206  +      close_db(pSrc);
 13368  14207         return 1;
 13369  14208       }
 13370  14209       open_db(p, 0);
 13371  14210       pBackup = sqlite3_backup_init(p->db, zDb, pSrc, "main");
 13372  14211       if( pBackup==0 ){
 13373  14212         utf8_printf(stderr, "Error: %s\n", sqlite3_errmsg(p->db));
 13374         -      sqlite3_close(pSrc);
        14213  +      close_db(pSrc);
 13375  14214         return 1;
 13376  14215       }
 13377  14216       while( (rc = sqlite3_backup_step(pBackup,100))==SQLITE_OK
 13378  14217             || rc==SQLITE_BUSY  ){
 13379  14218         if( rc==SQLITE_BUSY ){
 13380  14219           if( nTimeout++ >= 3 ) break;
 13381  14220           sqlite3_sleep(100);
................................................................................
 13387  14226       }else if( rc==SQLITE_BUSY || rc==SQLITE_LOCKED ){
 13388  14227         raw_printf(stderr, "Error: source database is busy\n");
 13389  14228         rc = 1;
 13390  14229       }else{
 13391  14230         utf8_printf(stderr, "Error: %s\n", sqlite3_errmsg(p->db));
 13392  14231         rc = 1;
 13393  14232       }
 13394         -    sqlite3_close(pSrc);
        14233  +    close_db(pSrc);
 13395  14234     }else
 13396         -
 13397  14235   
 13398  14236     if( c=='s' && strncmp(azArg[0], "scanstats", n)==0 ){
 13399  14237       if( nArg==2 ){
 13400         -      p->scanstatsOn = booleanValue(azArg[1]);
        14238  +      p->scanstatsOn = (u8)booleanValue(azArg[1]);
 13401  14239   #ifndef SQLITE_ENABLE_STMT_SCANSTATUS
 13402  14240         raw_printf(stderr, "Warning: .scanstats not available in this build.\n");
 13403  14241   #endif
 13404  14242       }else{
 13405  14243         raw_printf(stderr, "Usage: .scanstats on|off\n");
 13406  14244         rc = 1;
 13407  14245       }
................................................................................
 13432  14270         }else{
 13433  14271           raw_printf(stderr, "Usage: .schema ?--indent? ?LIKE-PATTERN?\n");
 13434  14272           rc = 1;
 13435  14273           goto meta_command_exit;
 13436  14274         }
 13437  14275       }
 13438  14276       if( zName!=0 ){
 13439         -      int isMaster = sqlite3_strlike(zName, "sqlite_master", 0)==0;
 13440         -      if( isMaster || sqlite3_strlike(zName,"sqlite_temp_master",0)==0 ){
        14277  +      int isMaster = sqlite3_strlike(zName, "sqlite_master", '\\')==0;
        14278  +      if( isMaster || sqlite3_strlike(zName,"sqlite_temp_master", '\\')==0 ){
 13441  14279           char *new_argv[2], *new_colv[2];
 13442  14280           new_argv[0] = sqlite3_mprintf(
 13443  14281                         "CREATE TABLE %s (\n"
 13444  14282                         "  type text,\n"
 13445  14283                         "  name text,\n"
 13446  14284                         "  tbl_name text,\n"
 13447  14285                         "  rootpage integer,\n"
................................................................................
 13493  14331              " UNION ALL SELECT shell_module_schema(name),"
 13494  14332              " 'table', name, name, name, 9e+99, 'main' FROM pragma_module_list", 0);
 13495  14333         }
 13496  14334   #endif
 13497  14335         appendText(&sSelect, ") WHERE ", 0);
 13498  14336         if( zName ){
 13499  14337           char *zQarg = sqlite3_mprintf("%Q", zName);
        14338  +        int bGlob = strchr(zName, '*') != 0 || strchr(zName, '?') != 0 ||
        14339  +                    strchr(zName, '[') != 0;
 13500  14340           if( strchr(zName, '.') ){
 13501  14341             appendText(&sSelect, "lower(printf('%s.%s',sname,tbl_name))", 0);
 13502  14342           }else{
 13503  14343             appendText(&sSelect, "lower(tbl_name)", 0);
 13504  14344           }
 13505         -        appendText(&sSelect, strchr(zName, '*') ? " GLOB " : " LIKE ", 0);
        14345  +        appendText(&sSelect, bGlob ? " GLOB " : " LIKE ", 0);
 13506  14346           appendText(&sSelect, zQarg, 0);
        14347  +        if( !bGlob ){
        14348  +          appendText(&sSelect, " ESCAPE '\\' ", 0);
        14349  +        }
 13507  14350           appendText(&sSelect, " AND ", 0);
 13508  14351           sqlite3_free(zQarg);
 13509  14352         }
 13510  14353         appendText(&sSelect, "type!='meta' AND sql IS NOT NULL"
 13511  14354                              " ORDER BY snum, rowid", 0);
 13512  14355         if( bDebug ){
 13513  14356           utf8_printf(p->out, "SQL: %s;\n", sSelect.z);
................................................................................
 13903  14746           if( strcmp(z,"debug")==0 ){
 13904  14747             bDebug = 1;
 13905  14748           }else
 13906  14749           {
 13907  14750             utf8_printf(stderr, "Unknown option \"%s\" on \"%s\"\n",
 13908  14751                         azArg[i], azArg[0]);
 13909  14752             raw_printf(stderr, "Should be one of: --schema"
 13910         -                             " --sha3-224 --sha3-255 --sha3-384 --sha3-512\n");
        14753  +                             " --sha3-224 --sha3-256 --sha3-384 --sha3-512\n");
 13911  14754             rc = 1;
 13912  14755             goto meta_command_exit;
 13913  14756           }
 13914  14757         }else if( zLike ){
 13915  14758           raw_printf(stderr, "Usage: .sha3sum ?OPTIONS? ?LIKE-PATTERN?\n");
 13916  14759           rc = 1;
 13917  14760           goto meta_command_exit;
 13918  14761         }else{
 13919  14762           zLike = z;
 13920  14763           bSeparate = 1;
 13921         -        if( sqlite3_strlike("sqlite_%", zLike, 0)==0 ) bSchema = 1;
        14764  +        if( sqlite3_strlike("sqlite\\_%", zLike, '\\')==0 ) bSchema = 1;
 13922  14765         }
 13923  14766       }
 13924  14767       if( bSchema ){
 13925  14768         zSql = "SELECT lower(name) FROM sqlite_master"
 13926  14769                " WHERE type='table' AND coalesce(rootpage,0)>1"
 13927  14770                " UNION ALL SELECT 'sqlite_master'"
 13928  14771                " ORDER BY 1 collate nocase";
................................................................................
 13981  14824             sSql.z, iSize);
 13982  14825       }
 13983  14826       freeText(&sQuery);
 13984  14827       freeText(&sSql);
 13985  14828       if( bDebug ){
 13986  14829         utf8_printf(p->out, "%s\n", zSql);
 13987  14830       }else{
 13988         -      shell_exec(p->db, zSql, shell_callback, p, 0);
        14831  +      shell_exec(p, zSql, 0);
 13989  14832       }
 13990  14833       sqlite3_free(zSql);
 13991  14834     }else
 13992  14835   
        14836  +#ifndef SQLITE_NOHAVE_SYSTEM
 13993  14837     if( c=='s'
 13994  14838      && (strncmp(azArg[0], "shell", n)==0 || strncmp(azArg[0],"system",n)==0)
 13995  14839     ){
 13996  14840       char *zCmd;
 13997  14841       int i, x;
 13998  14842       if( nArg<2 ){
 13999  14843         raw_printf(stderr, "Usage: .system COMMAND\n");
................................................................................
 14005  14849         zCmd = sqlite3_mprintf(strchr(azArg[i],' ')==0?"%z %s":"%z \"%s\"",
 14006  14850                                zCmd, azArg[i]);
 14007  14851       }
 14008  14852       x = system(zCmd);
 14009  14853       sqlite3_free(zCmd);
 14010  14854       if( x ) raw_printf(stderr, "System command returns %d\n", x);
 14011  14855     }else
        14856  +#endif /* !defined(SQLITE_NOHAVE_SYSTEM) */
 14012  14857   
 14013  14858     if( c=='s' && strncmp(azArg[0], "show", n)==0 ){
 14014  14859       static const char *azBool[] = { "off", "on", "trigger", "full"};
 14015  14860       int i;
 14016  14861       if( nArg!=1 ){
 14017  14862         raw_printf(stderr, "Usage: .show\n");
 14018  14863         rc = 1;
................................................................................
 14044  14889       raw_printf(p->out, "\n");
 14045  14890       utf8_printf(p->out, "%12.12s: %s\n", "filename",
 14046  14891                   p->zDbFilename ? p->zDbFilename : "");
 14047  14892     }else
 14048  14893   
 14049  14894     if( c=='s' && strncmp(azArg[0], "stats", n)==0 ){
 14050  14895       if( nArg==2 ){
 14051         -      p->statsOn = booleanValue(azArg[1]);
        14896  +      p->statsOn = (u8)booleanValue(azArg[1]);
 14052  14897       }else if( nArg==1 ){
 14053  14898         display_stats(p->db, p, 0);
 14054  14899       }else{
 14055  14900         raw_printf(stderr, "Usage: .stats ?on|off?\n");
 14056  14901         rc = 1;
 14057  14902       }
 14058  14903     }else
................................................................................
 14065  14910       char **azResult;
 14066  14911       int nRow, nAlloc;
 14067  14912       int ii;
 14068  14913       ShellText s;
 14069  14914       initText(&s);
 14070  14915       open_db(p, 0);
 14071  14916       rc = sqlite3_prepare_v2(p->db, "PRAGMA database_list", -1, &pStmt, 0);
 14072         -    if( rc ) return shellDatabaseError(p->db);
        14917  +    if( rc ){
        14918  +      sqlite3_finalize(pStmt);
        14919  +      return shellDatabaseError(p->db);
        14920  +    }
 14073  14921   
 14074  14922       if( nArg>2 && c=='i' ){
 14075  14923         /* It is an historical accident that the .indexes command shows an error
 14076  14924         ** when called with the wrong number of arguments whereas the .tables
 14077  14925         ** command does not. */
 14078  14926         raw_printf(stderr, "Usage: .indexes ?LIKE-PATTERN?\n");
 14079  14927         rc = 1;
        14928  +      sqlite3_finalize(pStmt);
 14080  14929         goto meta_command_exit;
 14081  14930       }
 14082  14931       for(ii=0; sqlite3_step(pStmt)==SQLITE_ROW; ii++){
 14083  14932         const char *zDbName = (const char*)sqlite3_column_text(pStmt, 1);
 14084  14933         if( zDbName==0 ) continue;
 14085  14934         if( s.z && s.z[0] ) appendText(&s, " UNION ALL ", 0);
 14086  14935         if( sqlite3_stricmp(zDbName, "main")==0 ){
................................................................................
 14117  14966         sqlite3_bind_text(pStmt, 1, "%", -1, SQLITE_STATIC);
 14118  14967       }
 14119  14968       while( sqlite3_step(pStmt)==SQLITE_ROW ){
 14120  14969         if( nRow>=nAlloc ){
 14121  14970           char **azNew;
 14122  14971           int n2 = nAlloc*2 + 10;
 14123  14972           azNew = sqlite3_realloc64(azResult, sizeof(azResult[0])*n2);
 14124         -        if( azNew==0 ){
 14125         -          rc = shellNomemError();
 14126         -          break;
 14127         -        }
        14973  +        if( azNew==0 ) shell_out_of_memory();
 14128  14974           nAlloc = n2;
 14129  14975           azResult = azNew;
 14130  14976         }
 14131  14977         azResult[nRow] = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 0));
 14132         -      if( 0==azResult[nRow] ){
 14133         -        rc = shellNomemError();
 14134         -        break;
 14135         -      }
        14978  +      if( 0==azResult[nRow] ) shell_out_of_memory();
 14136  14979         nRow++;
 14137  14980       }
 14138  14981       if( sqlite3_finalize(pStmt)!=SQLITE_OK ){
 14139  14982         rc = shellDatabaseError(p->db);
 14140  14983       }
 14141  14984   
 14142  14985       /* Pretty-print the contents of array azResult[] to the output */
................................................................................
 14189  15032         { "always",             SQLITE_TESTCTRL_ALWAYS,        "BOOLEAN"            },
 14190  15033         { "assert",             SQLITE_TESTCTRL_ASSERT,        "BOOLEAN"            },
 14191  15034       /*{ "benign_malloc_hooks",SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS, ""          },*/
 14192  15035       /*{ "bitvec_test",        SQLITE_TESTCTRL_BITVEC_TEST,   ""                },*/
 14193  15036         { "byteorder",          SQLITE_TESTCTRL_BYTEORDER,     ""                   },
 14194  15037       /*{ "fault_install",      SQLITE_TESTCTRL_FAULT_INSTALL, ""                }, */
 14195  15038         { "imposter",           SQLITE_TESTCTRL_IMPOSTER,   "SCHEMA ON/OFF ROOTPAGE"},
 14196         -#ifdef SQLITE_N_KEYWORD
 14197         -      { "iskeyword",          SQLITE_TESTCTRL_ISKEYWORD,     "IDENTIFIER"         },
 14198         -#endif
 14199  15039         { "localtime_fault",    SQLITE_TESTCTRL_LOCALTIME_FAULT,"BOOLEAN"           },
 14200  15040         { "never_corrupt",      SQLITE_TESTCTRL_NEVER_CORRUPT, "BOOLEAN"            },
 14201  15041         { "optimizations",      SQLITE_TESTCTRL_OPTIMIZATIONS, "DISABLE-MASK"       },
 14202  15042   #ifdef YYCOVERAGE
 14203  15043         { "parser_coverage",    SQLITE_TESTCTRL_PARSER_COVERAGE, ""                 },
 14204  15044   #endif
 14205  15045         { "pending_byte",       SQLITE_TESTCTRL_PENDING_BYTE,  "OFFSET  "           },
................................................................................
 14303  15143             if( nArg==3 ){
 14304  15144               int opt = booleanValue(azArg[2]);
 14305  15145               rc2 = sqlite3_test_control(testctrl, opt);
 14306  15146               isOk = 3;
 14307  15147             }
 14308  15148             break;
 14309  15149   
 14310         -        /* sqlite3_test_control(int, char *) */
 14311         -#ifdef SQLITE_N_KEYWORD
 14312         -        case SQLITE_TESTCTRL_ISKEYWORD:
 14313         -          if( nArg==3 ){
 14314         -            const char *opt = azArg[2];
 14315         -            rc2 = sqlite3_test_control(testctrl, opt);
 14316         -            isOk = 1;
 14317         -          }
 14318         -          break;
 14319         -#endif
 14320         -
 14321  15150           case SQLITE_TESTCTRL_IMPOSTER:
 14322  15151             if( nArg==5 ){
 14323  15152               rc2 = sqlite3_test_control(testctrl, p->db,
 14324  15153                             azArg[2],
 14325  15154                             integerValue(azArg[3]),
 14326  15155                             integerValue(azArg[4]));
 14327  15156               isOk = 3;
................................................................................
 14584  15413     if( ToLower(zLine[0])=='g' && ToLower(zLine[1])=='o'
 14585  15414            && _all_whitespace(&zLine[2]) ){
 14586  15415       return 1;  /* SQL Server */
 14587  15416     }
 14588  15417     return 0;
 14589  15418   }
 14590  15419   
        15420  +/*
        15421  +** We need a default sqlite3_complete() implementation to use in case
        15422  +** the shell is compiled with SQLITE_OMIT_COMPLETE.  The default assumes
        15423  +** any arbitrary text is a complete SQL statement.  This is not very
        15424  +** user-friendly, but it does seem to work.
        15425  +*/
        15426  +#ifdef SQLITE_OMIT_COMPLETE
        15427  +int sqlite3_complete(const char *zSql){ return 1; }
        15428  +#endif
        15429  +
 14591  15430   /*
 14592  15431   ** Return true if zSql is a complete SQL statement.  Return false if it
 14593  15432   ** ends in the middle of a string literal or C-style comment.
 14594  15433   */
 14595  15434   static int line_is_complete(char *zSql, int nSql){
 14596  15435     int rc;
 14597  15436     if( zSql==0 ) return 1;
................................................................................
 14599  15438     zSql[nSql+1] = 0;
 14600  15439     rc = sqlite3_complete(zSql);
 14601  15440     zSql[nSql] = 0;
 14602  15441     return rc;
 14603  15442   }
 14604  15443   
 14605  15444   /*
 14606         -** Run a single line of SQL
        15445  +** Run a single line of SQL.  Return the number of errors.
 14607  15446   */
 14608  15447   static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
 14609  15448     int rc;
 14610  15449     char *zErrMsg = 0;
 14611  15450   
 14612  15451     open_db(p, 0);
 14613  15452     if( ShellHasFlag(p,SHFLG_Backslash) ) resolve_backslashes(zSql);
 14614  15453     BEGIN_TIMER;
 14615         -  rc = shell_exec(p->db, zSql, shell_callback, p, &zErrMsg);
        15454  +  rc = shell_exec(p, zSql, &zErrMsg);
 14616  15455     END_TIMER;
 14617  15456     if( rc || zErrMsg ){
 14618  15457       char zPrefix[100];
 14619  15458       if( in!=0 || !stdin_is_interactive ){
 14620  15459         sqlite3_snprintf(sizeof(zPrefix), zPrefix,
 14621  15460                          "Error: near line %d:", startline);
 14622  15461       }else{
................................................................................
 14672  15511         seenInterrupt = 0;
 14673  15512       }
 14674  15513       lineno++;
 14675  15514       if( nSql==0 && _all_whitespace(zLine) ){
 14676  15515         if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zLine);
 14677  15516         continue;
 14678  15517       }
 14679         -    if( zLine && zLine[0]=='.' && nSql==0 ){
        15518  +    if( zLine && (zLine[0]=='.' || zLine[0]=='#') && nSql==0 ){
 14680  15519         if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zLine);
 14681         -      rc = do_meta_command(zLine, p);
 14682         -      if( rc==2 ){ /* exit requested */
 14683         -        break;
 14684         -      }else if( rc ){
 14685         -        errCnt++;
        15520  +      if( zLine[0]=='.' ){
        15521  +        rc = do_meta_command(zLine, p);
        15522  +        if( rc==2 ){ /* exit requested */
        15523  +          break;
        15524  +        }else if( rc ){
        15525  +          errCnt++;
        15526  +        }
 14686  15527         }
 14687  15528         continue;
 14688  15529       }
 14689  15530       if( line_is_command_terminator(zLine) && line_is_complete(zSql, nSql) ){
 14690  15531         memcpy(zLine,";",2);
 14691  15532       }
 14692  15533       nLine = strlen30(zLine);
 14693  15534       if( nSql+nLine+2>=nAlloc ){
 14694  15535         nAlloc = nSql+nLine+100;
 14695  15536         zSql = realloc(zSql, nAlloc);
 14696         -      if( zSql==0 ){
 14697         -        raw_printf(stderr, "Error: out of memory\n");
 14698         -        exit(1);
 14699         -      }
        15537  +      if( zSql==0 ) shell_out_of_memory();
 14700  15538       }
 14701  15539       nSqlPrior = nSql;
 14702  15540       if( nSql==0 ){
 14703  15541         int i;
 14704  15542         for(i=0; zLine[i] && IsSpace(zLine[i]); i++){}
 14705  15543         assert( nAlloc>0 && zSql!=0 );
 14706  15544         memcpy(zSql, zLine+i, nLine+1-i);
................................................................................
 14723  15561         }
 14724  15562       }else if( nSql && _all_whitespace(zSql) ){
 14725  15563         if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zSql);
 14726  15564         nSql = 0;
 14727  15565       }
 14728  15566     }
 14729  15567     if( nSql && !_all_whitespace(zSql) ){
 14730         -    runOneSqlLine(p, zSql, in, startline);
        15568  +    errCnt += runOneSqlLine(p, zSql, in, startline);
 14731  15569     }
 14732  15570     free(zSql);
 14733  15571     free(zLine);
 14734  15572     return errCnt>0;
 14735  15573   }
 14736  15574   
 14737  15575   /*
................................................................................
 14821  15659     if (sqliterc == NULL) {
 14822  15660       home_dir = find_home_dir(0);
 14823  15661       if( home_dir==0 ){
 14824  15662         raw_printf(stderr, "-- warning: cannot find home directory;"
 14825  15663                         " cannot read ~/.sqliterc\n");
 14826  15664         return;
 14827  15665       }
 14828         -    sqlite3_initialize();
 14829  15666       zBuf = sqlite3_mprintf("%s/.sqliterc",home_dir);
 14830  15667       sqliterc = zBuf;
 14831  15668     }
 14832  15669     in = fopen(sqliterc,"rb");
 14833  15670     if( in ){
 14834  15671       if( stdin_is_interactive ){
 14835  15672         utf8_printf(stderr,"-- Loading resources from %s\n",sqliterc);
................................................................................
 14840  15677     sqlite3_free(zBuf);
 14841  15678   }
 14842  15679   
 14843  15680   /*
 14844  15681   ** Show available command line options
 14845  15682   */
 14846  15683   static const char zOptions[] =
        15684  +#if defined(SQLITE_HAVE_ZLIB) && !defined(SQLITE_OMIT_VIRTUALTABLE)
        15685  +  "   -A ARGS...           run \".archive ARGS\" and exit\n"
        15686  +#endif
        15687  +  "   -append              append the database to the end of the file\n"
 14847  15688     "   -ascii               set output mode to 'ascii'\n"
 14848  15689     "   -bail                stop after hitting an error\n"
 14849  15690     "   -batch               force batch I/O\n"
 14850  15691     "   -column              set output mode to 'column'\n"
 14851  15692     "   -cmd COMMAND         run \"COMMAND\" before reading stdin\n"
 14852  15693     "   -csv                 set output mode to 'csv'\n"
 14853  15694     "   -echo                print commands before execution\n"
................................................................................
 14866  15707   #ifdef SQLITE_ENABLE_MULTIPLEX
 14867  15708     "   -multiplex           enable the multiplexor VFS\n"
 14868  15709   #endif
 14869  15710     "   -newline SEP         set output row separator. Default: '\\n'\n"
 14870  15711     "   -nullvalue TEXT      set text string for NULL values. Default ''\n"
 14871  15712     "   -pagecache SIZE N    use N slots of SZ bytes each for page cache memory\n"
 14872  15713     "   -quote               set output mode to 'quote'\n"
        15714  +  "   -readonly            open the database read-only\n"
 14873  15715     "   -separator SEP       set output column separator. Default: '|'\n"
        15716  +#ifdef SQLITE_ENABLE_SORTER_REFERENCES
        15717  +  "   -sorterref SIZE      sorter references threshold size\n"
        15718  +#endif
 14874  15719     "   -stats               print memory stats before each finalize\n"
 14875  15720     "   -version             show SQLite version\n"
 14876  15721     "   -vfs NAME            use NAME as the default VFS\n"
 14877  15722   #ifdef SQLITE_ENABLE_VFSTRACE
 14878  15723     "   -vfstrace            enable tracing of all VFS calls\n"
 14879  15724   #endif
        15725  +#ifdef SQLITE_HAVE_ZLIB
        15726  +  "   -zip                 open the file as a ZIP Archive\n"
        15727  +#endif
 14880  15728   ;
 14881  15729   static void usage(int showDetail){
 14882  15730     utf8_printf(stderr,
 14883  15731         "Usage: %s [OPTIONS] FILENAME [SQL]\n"
 14884  15732         "FILENAME is the name of an SQLite database. A new database is created\n"
 14885  15733         "if the file does not previously exist.\n", Argv0);
 14886  15734     if( showDetail ){
 14887  15735       utf8_printf(stderr, "OPTIONS include:\n%s", zOptions);
 14888  15736     }else{
 14889  15737       raw_printf(stderr, "Use the -help option for additional information\n");
 14890  15738     }
 14891  15739     exit(1);
 14892  15740   }
        15741  +
        15742  +/*
        15743  +** Internal check:  Verify that the SQLite is uninitialized.  Print a
        15744  +** error message if it is initialized.
        15745  +*/
        15746  +static void verify_uninitialized(void){
        15747  +  if( sqlite3_config(-1)==SQLITE_MISUSE ){
        15748  +    utf8_printf(stdout, "WARNING: attempt to configure SQLite after"
        15749  +                        " initialization.\n");
        15750  +  }
        15751  +}
 14893  15752   
 14894  15753   /*
 14895  15754   ** Initialize the state information in data
 14896  15755   */
 14897  15756   static void main_init(ShellState *data) {
 14898  15757     memset(data, 0, sizeof(*data));
 14899  15758     data->normalMode = data->cMode = data->mode = MODE_List;
 14900  15759     data->autoExplain = 1;
 14901  15760     memcpy(data->colSeparator,SEP_Column, 2);
 14902  15761     memcpy(data->rowSeparator,SEP_Row, 2);
 14903  15762     data->showHeader = 0;
 14904  15763     data->shellFlgs = SHFLG_Lookaside;
        15764  +  verify_uninitialized();
 14905  15765     sqlite3_config(SQLITE_CONFIG_URI, 1);
 14906  15766     sqlite3_config(SQLITE_CONFIG_LOG, shellLog, data);
 14907  15767     sqlite3_config(SQLITE_CONFIG_MULTITHREAD);
 14908  15768     sqlite3_snprintf(sizeof(mainPrompt), mainPrompt,"sqlite> ");
 14909  15769     sqlite3_snprintf(sizeof(continuePrompt), continuePrompt,"   ...> ");
 14910  15770   }
 14911  15771   
................................................................................
 14961  15821     const char *zInitFile = 0;
 14962  15822     int i;
 14963  15823     int rc = 0;
 14964  15824     int warnInmemoryDb = 0;
 14965  15825     int readStdin = 1;
 14966  15826     int nCmd = 0;
 14967  15827     char **azCmd = 0;
        15828  +  const char *zVfs = 0;           /* Value of -vfs command-line option */
        15829  +#if !SQLITE_SHELL_IS_UTF8
        15830  +  char **argvToFree = 0;
        15831  +  int argcToFree = 0;
        15832  +#endif
 14968  15833   
 14969  15834     setBinaryMode(stdin, 0);
 14970  15835     setvbuf(stderr, 0, _IONBF, 0); /* Make sure stderr is unbuffered */
 14971  15836     stdin_is_interactive = isatty(0);
 14972  15837     stdout_is_console = isatty(1);
 14973  15838   
 14974  15839   #if USE_SYSTEM_SQLITE+0!=1
................................................................................
 14975  15840     if( strncmp(sqlite3_sourceid(),SQLITE_SOURCE_ID,60)!=0 ){
 14976  15841       utf8_printf(stderr, "SQLite header and source version mismatch\n%s\n%s\n",
 14977  15842               sqlite3_sourceid(), SQLITE_SOURCE_ID);
 14978  15843       exit(1);
 14979  15844     }
 14980  15845   #endif
 14981  15846     main_init(&data);
        15847  +
        15848  +  /* On Windows, we must translate command-line arguments into UTF-8.
        15849  +  ** The SQLite memory allocator subsystem has to be enabled in order to
        15850  +  ** do this.  But we want to run an sqlite3_shutdown() afterwards so that
        15851  +  ** subsequent sqlite3_config() calls will work.  So copy all results into
        15852  +  ** memory that does not come from the SQLite memory allocator.
        15853  +  */
 14982  15854   #if !SQLITE_SHELL_IS_UTF8
 14983  15855     sqlite3_initialize();
 14984         -  argv = sqlite3_malloc64(sizeof(argv[0])*argc);
 14985         -  if( argv==0 ){
 14986         -    raw_printf(stderr, "out of memory\n");
 14987         -    exit(1);
 14988         -  }
        15856  +  argvToFree = malloc(sizeof(argv[0])*argc*2);
        15857  +  argcToFree = argc;
        15858  +  argv = argvToFree + argc;
        15859  +  if( argv==0 ) shell_out_of_memory();
 14989  15860     for(i=0; i<argc; i++){
 14990         -    argv[i] = sqlite3_win32_unicode_to_utf8(wargv[i]);
 14991         -    if( argv[i]==0 ){
 14992         -      raw_printf(stderr, "out of memory\n");
 14993         -      exit(1);
 14994         -    }
        15861  +    char *z = sqlite3_win32_unicode_to_utf8(wargv[i]);
        15862  +    int n;
        15863  +    if( z==0 ) shell_out_of_memory();
        15864  +    n = (int)strlen(z);
        15865  +    argv[i] = malloc( n+1 );
        15866  +    if( argv[i]==0 ) shell_out_of_memory();
        15867  +    memcpy(argv[i], z, n+1);
        15868  +    argvToFree[i] = argv[i];
        15869  +    sqlite3_free(z);
 14995  15870     }
        15871  +  sqlite3_shutdown();
 14996  15872   #endif
        15873  +
 14997  15874     assert( argc>=1 && argv && argv[0] );
 14998  15875     Argv0 = argv[0];
 14999  15876   
 15000  15877     /* Make sure we have a valid signal handler early, before anything
 15001  15878     ** else is done.
 15002  15879     */
 15003  15880   #ifdef SIGINT
................................................................................
 15019  15896   #endif
 15020  15897   
 15021  15898     /* Do an initial pass through the command-line argument to locate
 15022  15899     ** the name of the database file, the name of the initialization file,
 15023  15900     ** the size of the alternative malloc heap,
 15024  15901     ** and the first command to execute.
 15025  15902     */
        15903  +  verify_uninitialized();
 15026  15904     for(i=1; i<argc; i++){
 15027  15905       char *z;
 15028  15906       z = argv[i];
 15029  15907       if( z[0]!='-' ){
 15030  15908         if( data.zDbFilename==0 ){
 15031  15909           data.zDbFilename = z;
 15032  15910         }else{
 15033  15911           /* Excesss arguments are interpreted as SQL (or dot-commands) and
 15034  15912           ** mean that nothing is read from stdin */
 15035  15913           readStdin = 0;
 15036  15914           nCmd++;
 15037  15915           azCmd = realloc(azCmd, sizeof(azCmd[0])*nCmd);
 15038         -        if( azCmd==0 ){
 15039         -          raw_printf(stderr, "out of memory\n");
 15040         -          exit(1);
 15041         -        }
        15916  +        if( azCmd==0 ) shell_out_of_memory();
 15042  15917           azCmd[nCmd-1] = z;
 15043  15918         }
 15044  15919       }
 15045  15920       if( z[1]=='-' ) z++;
 15046  15921       if( strcmp(z,"-separator")==0
 15047  15922        || strcmp(z,"-nullvalue")==0
 15048  15923        || strcmp(z,"-newline")==0
................................................................................
 15101  15976       }else if( strcmp(z,"-multiplex")==0 ){
 15102  15977         extern int sqlite3_multiple_initialize(const char*,int);
 15103  15978         sqlite3_multiplex_initialize(0, 1);
 15104  15979   #endif
 15105  15980       }else if( strcmp(z,"-mmap")==0 ){
 15106  15981         sqlite3_int64 sz = integerValue(cmdline_option_value(argc,argv,++i));
 15107  15982         sqlite3_config(SQLITE_CONFIG_MMAP_SIZE, sz, sz);
        15983  +#ifdef SQLITE_ENABLE_SORTER_REFERENCES
        15984  +    }else if( strcmp(z,"-sorterref")==0 ){
        15985  +      sqlite3_int64 sz = integerValue(cmdline_option_value(argc,argv,++i));
        15986  +      sqlite3_config(SQLITE_CONFIG_SORTERREF_SIZE, (int)sz);
        15987  +#endif
 15108  15988       }else if( strcmp(z,"-vfs")==0 ){
 15109         -      sqlite3_vfs *pVfs = sqlite3_vfs_find(cmdline_option_value(argc,argv,++i));
 15110         -      if( pVfs ){
 15111         -        sqlite3_vfs_register(pVfs, 1);
 15112         -      }else{
 15113         -        utf8_printf(stderr, "no such VFS: \"%s\"\n", argv[i]);
 15114         -        exit(1);
 15115         -      }
 15116         -#ifdef SQLITE_HAVE_ZIP
        15989  +      zVfs = cmdline_option_value(argc, argv, ++i);
        15990  +#ifdef SQLITE_HAVE_ZLIB
 15117  15991       }else if( strcmp(z,"-zip")==0 ){
 15118  15992         data.openMode = SHELL_OPEN_ZIPFILE;
 15119  15993   #endif
 15120  15994       }else if( strcmp(z,"-append")==0 ){
 15121  15995         data.openMode = SHELL_OPEN_APPENDVFS;
        15996  +    }else if( strcmp(z,"-readonly")==0 ){
        15997  +      data.openMode = SHELL_OPEN_READONLY;
        15998  +#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
        15999  +    }else if( strncmp(z, "-A",2)==0 ){
        16000  +      /* All remaining command-line arguments are passed to the ".archive"
        16001  +      ** command, so ignore them */
        16002  +      break;
        16003  +#endif
        16004  +    }
        16005  +  }
        16006  +  verify_uninitialized();
        16007  +
        16008  +
        16009  +#ifdef SQLITE_SHELL_INIT_PROC
        16010  +  {
        16011  +    /* If the SQLITE_SHELL_INIT_PROC macro is defined, then it is the name
        16012  +    ** of a C-function that will perform initialization actions on SQLite that
        16013  +    ** occur just before or after sqlite3_initialize(). Use this compile-time
        16014  +    ** option to embed this shell program in larger applications. */
        16015  +    extern void SQLITE_SHELL_INIT_PROC(void);
        16016  +    SQLITE_SHELL_INIT_PROC();
        16017  +  }
        16018  +#else
        16019  +  /* All the sqlite3_config() calls have now been made. So it is safe
        16020  +  ** to call sqlite3_initialize() and process any command line -vfs option. */
        16021  +  sqlite3_initialize();
        16022  +#endif
        16023  +
        16024  +  if( zVfs ){
        16025  +    sqlite3_vfs *pVfs = sqlite3_vfs_find(zVfs);
        16026  +    if( pVfs ){
        16027  +      sqlite3_vfs_register(pVfs, 1);
        16028  +    }else{
        16029  +      utf8_printf(stderr, "no such VFS: \"%s\"\n", argv[i]);
        16030  +      exit(1);
 15122  16031       }
 15123  16032     }
        16033  +
 15124  16034     if( data.zDbFilename==0 ){
 15125  16035   #ifndef SQLITE_OMIT_MEMORYDB
 15126  16036       data.zDbFilename = ":memory:";
 15127  16037       warnInmemoryDb = argc==1;
 15128  16038   #else
 15129  16039       utf8_printf(stderr,"%s: Error: no database filename specified\n", Argv0);
 15130  16040       return 1;
................................................................................
 15168  16078       }else if( strcmp(z,"-line")==0 ){
 15169  16079         data.mode = MODE_Line;
 15170  16080       }else if( strcmp(z,"-column")==0 ){
 15171  16081         data.mode = MODE_Column;
 15172  16082       }else if( strcmp(z,"-csv")==0 ){
 15173  16083         data.mode = MODE_Csv;
 15174  16084         memcpy(data.colSeparator,",",2);
 15175         -#ifdef SQLITE_HAVE_ZIP
        16085  +#ifdef SQLITE_HAVE_ZLIB
 15176  16086       }else if( strcmp(z,"-zip")==0 ){
 15177  16087         data.openMode = SHELL_OPEN_ZIPFILE;
 15178  16088   #endif
 15179  16089       }else if( strcmp(z,"-append")==0 ){
 15180  16090         data.openMode = SHELL_OPEN_APPENDVFS;
        16091  +    }else if( strcmp(z,"-readonly")==0 ){
        16092  +      data.openMode = SHELL_OPEN_READONLY;
 15181  16093       }else if( strcmp(z,"-ascii")==0 ){
 15182  16094         data.mode = MODE_Ascii;
 15183  16095         sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator,
 15184  16096                          SEP_Unit);
 15185  16097         sqlite3_snprintf(sizeof(data.rowSeparator), data.rowSeparator,
 15186  16098                          SEP_Record);
 15187  16099       }else if( strcmp(z,"-separator")==0 ){
................................................................................
 15227  16139         i++;
 15228  16140       }else if( strcmp(z,"-pagecache")==0 ){
 15229  16141         i+=2;
 15230  16142       }else if( strcmp(z,"-lookaside")==0 ){
 15231  16143         i+=2;
 15232  16144       }else if( strcmp(z,"-mmap")==0 ){
 15233  16145         i++;
        16146  +#ifdef SQLITE_ENABLE_SORTER_REFERENCES
        16147  +    }else if( strcmp(z,"-sorterref")==0 ){
        16148  +      i++;
        16149  +#endif
 15234  16150       }else if( strcmp(z,"-vfs")==0 ){
 15235  16151         i++;
 15236  16152   #ifdef SQLITE_ENABLE_VFSTRACE
 15237  16153       }else if( strcmp(z,"-vfstrace")==0 ){
 15238  16154         i++;
 15239  16155   #endif
 15240  16156   #ifdef SQLITE_ENABLE_MULTIPLEX
................................................................................
 15251  16167         if( i==argc-1 ) break;
 15252  16168         z = cmdline_option_value(argc,argv,++i);
 15253  16169         if( z[0]=='.' ){
 15254  16170           rc = do_meta_command(z, &data);
 15255  16171           if( rc && bail_on_error ) return rc==2 ? 0 : rc;
 15256  16172         }else{
 15257  16173           open_db(&data, 0);
 15258         -        rc = shell_exec(data.db, z, shell_callback, &data, &zErrMsg);
        16174  +        rc = shell_exec(&data, z, &zErrMsg);
 15259  16175           if( zErrMsg!=0 ){
 15260  16176             utf8_printf(stderr,"Error: %s\n", zErrMsg);
 15261  16177             if( bail_on_error ) return rc!=0 ? rc : 1;
 15262  16178           }else if( rc!=0 ){
 15263  16179             utf8_printf(stderr,"Error: unable to process SQL \"%s\"\n", z);
 15264  16180             if( bail_on_error ) return rc;
 15265  16181           }
 15266  16182         }
        16183  +#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
        16184  +    }else if( strncmp(z, "-A", 2)==0 ){
        16185  +      if( nCmd>0 ){
        16186  +        utf8_printf(stderr, "Error: cannot mix regular SQL or dot-commands"
        16187  +                            " with \"%s\"\n", z);
        16188  +        return 1;
        16189  +      }
        16190  +      open_db(&data, OPEN_DB_ZIPFILE);
        16191  +      if( z[2] ){
        16192  +        argv[i] = &z[2];
        16193  +        arDotCommand(&data, 1, argv+(i-1), argc-(i-1));
        16194  +      }else{
        16195  +        arDotCommand(&data, 1, argv+i, argc-i);
        16196  +      }
        16197  +      readStdin = 0;
        16198  +      break;
        16199  +#endif
 15267  16200       }else{
 15268  16201         utf8_printf(stderr,"%s: Error: unknown option: %s\n", Argv0, z);
 15269  16202         raw_printf(stderr,"Use -help for a list of options.\n");
 15270  16203         return 1;
 15271  16204       }
 15272  16205       data.cMode = data.mode;
 15273  16206     }
................................................................................
 15279  16212       */
 15280  16213       for(i=0; i<nCmd; i++){
 15281  16214         if( azCmd[i][0]=='.' ){
 15282  16215           rc = do_meta_command(azCmd[i], &data);
 15283  16216           if( rc ) return rc==2 ? 0 : rc;
 15284  16217         }else{
 15285  16218           open_db(&data, 0);
 15286         -        rc = shell_exec(data.db, azCmd[i], shell_callback, &data, &zErrMsg);
        16219  +        rc = shell_exec(&data, azCmd[i], &zErrMsg);
 15287  16220           if( zErrMsg!=0 ){
 15288  16221             utf8_printf(stderr,"Error: %s\n", zErrMsg);
 15289  16222             return rc!=0 ? rc : 1;
 15290  16223           }else if( rc!=0 ){
 15291  16224             utf8_printf(stderr,"Error: unable to process SQL: %s\n", azCmd[i]);
 15292  16225             return rc;
 15293  16226           }
................................................................................
 15334  16267       }else{
 15335  16268         rc = process_input(&data, stdin);
 15336  16269       }
 15337  16270     }
 15338  16271     set_table_name(&data, 0);
 15339  16272     if( data.db ){
 15340  16273       session_close_all(&data);
 15341         -    sqlite3_close(data.db);
        16274  +    close_db(data.db);
 15342  16275     }
 15343  16276     sqlite3_free(data.zFreeOnClose);
 15344  16277     find_home_dir(1);
 15345  16278     output_reset(&data);
 15346  16279     data.doXdgOpen = 0;
 15347  16280     clearTempFile(&data);
 15348  16281   #if !SQLITE_SHELL_IS_UTF8
 15349         -  for(i=0; i<argc; i++) sqlite3_free(argv[i]);
 15350         -  sqlite3_free(argv);
        16282  +  for(i=0; i<argcToFree; i++) free(argvToFree[i]);
        16283  +  free(argvToFree);
 15351  16284   #endif
        16285  +  /* Clear the global data structure so that valgrind will detect memory
        16286  +  ** leaks */
        16287  +  memset(&data, 0, sizeof(data));
 15352  16288     return rc;
 15353  16289   }

Changes to src/skins.c.

   465    465     }
   466    466     db_begin_transaction();
   467    467     zCurrent = getSkin(0);
   468    468     for(i=0; i<count(aBuiltinSkin); i++){
   469    469       aBuiltinSkin[i].zSQL = getSkin(aBuiltinSkin[i].zLabel);
   470    470     }
   471    471   
   472         -  /* Process requests to delete a user-defined skin */
   473         -  if( P("del1") && (zName = skinVarName(P("sn"), 1))!=0 ){
   474         -    style_header("Confirm Custom Skin Delete");
   475         -    @ <form action="%s(g.zTop)/setup_skin_admin" method="post"><div>
   476         -    @ <p>Deletion of a custom skin is a permanent action that cannot
   477         -    @ be undone.  Please confirm that this is what you want to do:</p>
   478         -    @ <input type="hidden" name="sn" value="%h(P("sn"))" />
   479         -    @ <input type="submit" name="del2" value="Confirm - Delete The Skin" />
   480         -    @ <input type="submit" name="cancel" value="Cancel - Do Not Delete" />
   481         -    login_insert_csrf_secret();
   482         -    @ </div></form>
   483         -    style_footer();
   484         -    return;
   485         -  }
   486         -  if( P("del2")!=0 && (zName = skinVarName(P("sn"), 1))!=0 ){
   487         -    db_multi_exec("DELETE FROM config WHERE name=%Q", zName);
   488         -  }
   489         -  if( P("draftdel")!=0 ){
   490         -    const char *zDraft = P("name");
   491         -    if( sqlite3_strglob("draft[1-9]",zDraft)==0 ){
   492         -      db_multi_exec("DELETE FROM config WHERE name GLOB '%q-*'", zDraft);
   493         -    }
   494         -  }
   495         -  if( skinRename() ) return;
   496         -  if( skinSave(zCurrent) ) return;
   497         -
   498         -  /* The user pressed one of the "Install" buttons. */
   499         -  if( P("load") && (z = P("sn"))!=0 && z[0] ){
   500         -    int seen = 0;
   501         -
   502         -    /* Check to see if the current skin is already saved.  If it is, there
   503         -    ** is no need to create a backup */
   504         -    zCurrent = getSkin(0);
   505         -    for(i=0; i<count(aBuiltinSkin); i++){
   506         -      if( fossil_strcmp(aBuiltinSkin[i].zSQL, zCurrent)==0 ){
   507         -        seen = 1;
   508         -        break;
          472  +  if( cgi_csrf_safe(1) ){
          473  +    /* Process requests to delete a user-defined skin */
          474  +    if( P("del1") && (zName = skinVarName(P("sn"), 1))!=0 ){
          475  +      style_header("Confirm Custom Skin Delete");
          476  +      @ <form action="%s(g.zTop)/setup_skin_admin" method="post"><div>
          477  +      @ <p>Deletion of a custom skin is a permanent action that cannot
          478  +      @ be undone.  Please confirm that this is what you want to do:</p>
          479  +      @ <input type="hidden" name="sn" value="%h(P("sn"))" />
          480  +      @ <input type="submit" name="del2" value="Confirm - Delete The Skin" />
          481  +      @ <input type="submit" name="cancel" value="Cancel - Do Not Delete" />
          482  +      login_insert_csrf_secret();
          483  +      @ </div></form>
          484  +      style_footer();
          485  +      return;
          486  +    }
          487  +    if( P("del2")!=0 && (zName = skinVarName(P("sn"), 1))!=0 ){
          488  +      db_multi_exec("DELETE FROM config WHERE name=%Q", zName);
          489  +    }
          490  +    if( P("draftdel")!=0 ){
          491  +      const char *zDraft = P("name");
          492  +      if( sqlite3_strglob("draft[1-9]",zDraft)==0 ){
          493  +        db_multi_exec("DELETE FROM config WHERE name GLOB '%q-*'", zDraft);
          494  +      }
          495  +    }
          496  +    if( skinRename() ) return;
          497  +    if( skinSave(zCurrent) ) return;
          498  +  
          499  +    /* The user pressed one of the "Install" buttons. */
          500  +    if( P("load") && (z = P("sn"))!=0 && z[0] ){
          501  +      int seen = 0;
          502  +  
          503  +      /* Check to see if the current skin is already saved.  If it is, there
          504  +      ** is no need to create a backup */
          505  +      zCurrent = getSkin(0);
          506  +      for(i=0; i<count(aBuiltinSkin); i++){
          507  +        if( fossil_strcmp(aBuiltinSkin[i].zSQL, zCurrent)==0 ){
          508  +          seen = 1;
          509  +          break;
          510  +        }
          511  +      }
          512  +      if( !seen ){
          513  +        seen = db_exists("SELECT 1 FROM config WHERE name GLOB 'skin:*'"
          514  +                         " AND value=%Q", zCurrent);
          515  +        if( !seen ){
          516  +          db_multi_exec(
          517  +            "INSERT INTO config(name,value,mtime) VALUES("
          518  +            "  strftime('skin:Backup On %%Y-%%m-%%d %%H:%%M:%%S'),"
          519  +            "  %Q,now())", zCurrent
          520  +          );
          521  +        }
          522  +      }
          523  +      seen = 0;
          524  +      for(i=0; i<count(aBuiltinSkin); i++){
          525  +        if( fossil_strcmp(aBuiltinSkin[i].zDesc, z)==0 ){
          526  +          seen = 1;
          527  +          zCurrent = aBuiltinSkin[i].zSQL;
          528  +          db_multi_exec("%s", zCurrent/*safe-for-%s*/);
          529  +          break;
          530  +        }
          531  +      }
          532  +      if( !seen ){
          533  +        zName = skinVarName(z,0);
          534  +        zCurrent = db_get(zName, 0);
          535  +        db_multi_exec("%s", zCurrent/*safe-for-%s*/);
   509    536         }
   510    537       }
   511         -    if( !seen ){
   512         -      seen = db_exists("SELECT 1 FROM config WHERE name GLOB 'skin:*'"
   513         -                       " AND value=%Q", zCurrent);
   514         -      if( !seen ){
   515         -        db_multi_exec(
   516         -          "INSERT INTO config(name,value,mtime) VALUES("
   517         -          "  strftime('skin:Backup On %%Y-%%m-%%d %%H:%%M:%%S'),"
   518         -          "  %Q,now())", zCurrent
   519         -        );
   520         -      }
   521         -    }
   522         -    seen = 0;
   523         -    for(i=0; i<count(aBuiltinSkin); i++){
   524         -      if( fossil_strcmp(aBuiltinSkin[i].zDesc, z)==0 ){
   525         -        seen = 1;
   526         -        zCurrent = aBuiltinSkin[i].zSQL;
   527         -        db_multi_exec("%s", zCurrent/*safe-for-%s*/);
   528         -        break;
   529         -      }
   530         -    }
   531         -    if( !seen ){
   532         -      zName = skinVarName(z,0);
   533         -      zCurrent = db_get(zName, 0);
   534         -      db_multi_exec("%s", zCurrent/*safe-for-%s*/);
   535         -    }
   536    538     }
   537         -
          539  +  
   538    540     style_header("Skins");
   539    541     if( zErr ){
   540    542       @ <p style="color:red">%h(zErr)</p>
   541    543     }
   542    544     @ <table border="0">
   543    545     @ <tr><td colspan=4><h2>Built-in Skins:</h2></td></th>
   544    546     for(i=0; i<count(aBuiltinSkin); i++){
................................................................................
  1014   1016     fossil_free(zBase);
  1015   1017     @ </ul>
  1016   1018     @
  1017   1019     @ <p>You will probably need to press Reload on your browser before any
  1018   1020     @ CSS changes will take effect.</p>
  1019   1021     @
  1020   1022     @ <a hame='step6'></a>
  1021         -  @ <h1>Step 6: Interate</h1>
         1023  +  @ <h1>Step 6: Iterate</h1>
  1022   1024     @
  1023   1025     @ <p>Repeat <a href='#step4'>step 4</a> and
  1024   1026     @ <a href='#step5'>step 5</a> as many times as necessary to create
  1025   1027     @ a production-ready skin.
  1026   1028     @
  1027   1029     @ <a name='step7'></a>
  1028   1030     @ <h1>Step 7: Publish</h1>

Changes to src/sqlcmd.c.

   173    173   ** atexit() handler that cleans up global state modified by this module.
   174    174   */
   175    175   static void sqlcmd_atexit(void) {
   176    176     g.zConfigDbName = 0; /* prevent panic */
   177    177   }
   178    178   
   179    179   /*
   180         -** This routine is called by the patched sqlite3 command-line shell in order
   181         -** to load the name and database connection for the open Fossil database.
          180  +** This routine is called by the sqlite3 command-line shell to
          181  +** to load the name the Fossil repository database.
   182    182   */
   183         -void fossil_open(const char **pzRepoName){
   184         -  sqlite3_auto_extension((void(*)(void))sqlcmd_autoinit);
          183  +void sqlcmd_get_dbname(const char **pzRepoName){
   185    184     *pzRepoName = g.zRepositoryName;
   186    185   }
          186  +
          187  +/*
          188  +** This routine is called by the sqlite3 command-line shell to do
          189  +** extra initialization prior to starting up the shell.
          190  +*/
          191  +void sqlcmd_init_proc(void){
          192  +  sqlite3_initialize();
          193  +  sqlite3_auto_extension((void(*)(void))sqlcmd_autoinit);
          194  +}
   187    195   
   188    196   #if USE_SEE
   189    197   /*
   190    198   ** This routine is called by the patched sqlite3 command-line shell in order
   191    199   ** to load the encryption key for the open Fossil database.  The memory that
   192    200   ** is pointed to by the value placed in pzKey must be obtained from SQLite.
   193    201   */

Changes to src/sqlite3.c.

more than 10,000 changes

Changes to src/sqlite3.h.

   119    119   ** been edited in any way since it was last checked in, then the last
   120    120   ** four hexadecimal digits of the hash may be modified.
   121    121   **
   122    122   ** See also: [sqlite3_libversion()],
   123    123   ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
   124    124   ** [sqlite_version()] and [sqlite_source_id()].
   125    125   */
   126         -#define SQLITE_VERSION        "3.22.0"
   127         -#define SQLITE_VERSION_NUMBER 3022000
   128         -#define SQLITE_SOURCE_ID      "2018-01-22 18:45:57 0c55d179733b46d8d0ba4d88e01a25e10677046ee3da1d5b1581e86726f2171d"
          126  +#define SQLITE_VERSION        "3.24.0"
          127  +#define SQLITE_VERSION_NUMBER 3024000
          128  +#define SQLITE_SOURCE_ID      "2018-06-04 19:24:41 c7ee0833225bfd8c5ec2f9bf62b97c4e04d03bd9566366d5221ac8fb199a87ca"
   129    129   
   130    130   /*
   131    131   ** CAPI3REF: Run-Time Library Version Numbers
   132    132   ** KEYWORDS: sqlite3_version sqlite3_sourceid
   133    133   **
   134    134   ** These interfaces provide the same information as the [SQLITE_VERSION],
   135    135   ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
   500    500   #define SQLITE_IOERR_CONVPATH          (SQLITE_IOERR | (26<<8))
   501    501   #define SQLITE_IOERR_VNODE             (SQLITE_IOERR | (27<<8))
   502    502   #define SQLITE_IOERR_AUTH              (SQLITE_IOERR | (28<<8))
   503    503   #define SQLITE_IOERR_BEGIN_ATOMIC      (SQLITE_IOERR | (29<<8))
   504    504   #define SQLITE_IOERR_COMMIT_ATOMIC     (SQLITE_IOERR | (30<<8))
   505    505   #define SQLITE_IOERR_ROLLBACK_ATOMIC   (SQLITE_IOERR | (31<<8))
   506    506   #define SQLITE_LOCKED_SHAREDCACHE      (SQLITE_LOCKED |  (1<<8))
          507  +#define SQLITE_LOCKED_VTAB             (SQLITE_LOCKED |  (2<<8))
   507    508   #define SQLITE_BUSY_RECOVERY           (SQLITE_BUSY   |  (1<<8))
   508    509   #define SQLITE_BUSY_SNAPSHOT           (SQLITE_BUSY   |  (2<<8))
   509    510   #define SQLITE_CANTOPEN_NOTEMPDIR      (SQLITE_CANTOPEN | (1<<8))
   510    511   #define SQLITE_CANTOPEN_ISDIR          (SQLITE_CANTOPEN | (2<<8))
   511    512   #define SQLITE_CANTOPEN_FULLPATH       (SQLITE_CANTOPEN | (3<<8))
   512    513   #define SQLITE_CANTOPEN_CONVPATH       (SQLITE_CANTOPEN | (4<<8))
   513    514   #define SQLITE_CORRUPT_VTAB            (SQLITE_CORRUPT | (1<<8))
          515  +#define SQLITE_CORRUPT_SEQUENCE        (SQLITE_CORRUPT | (2<<8))
   514    516   #define SQLITE_READONLY_RECOVERY       (SQLITE_READONLY | (1<<8))
   515    517   #define SQLITE_READONLY_CANTLOCK       (SQLITE_READONLY | (2<<8))
   516    518   #define SQLITE_READONLY_ROLLBACK       (SQLITE_READONLY | (3<<8))
   517    519   #define SQLITE_READONLY_DBMOVED        (SQLITE_READONLY | (4<<8))
   518    520   #define SQLITE_READONLY_CANTINIT       (SQLITE_READONLY | (5<<8))
   519    521   #define SQLITE_READONLY_DIRECTORY      (SQLITE_READONLY | (6<<8))
   520    522   #define SQLITE_ABORT_ROLLBACK          (SQLITE_ABORT | (2<<8))
................................................................................
  1060   1062   ** The [SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE] opcode causes all write
  1061   1063   ** operations since the previous successful call to 
  1062   1064   ** [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] to be rolled back.
  1063   1065   ** ^This file control takes the file descriptor out of batch write mode
  1064   1066   ** so that all subsequent write operations are independent.
  1065   1067   ** ^SQLite will never invoke SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE without
  1066   1068   ** a prior successful call to [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE].
         1069  +**
         1070  +** <li>[[SQLITE_FCNTL_LOCK_TIMEOUT]]
         1071  +** The [SQLITE_FCNTL_LOCK_TIMEOUT] opcode causes attempts to obtain
         1072  +** a file lock using the xLock or xShmLock methods of the VFS to wait
         1073  +** for up to M milliseconds before failing, where M is the single 
         1074  +** unsigned integer parameter.
  1067   1075   ** </ul>
  1068   1076   */
  1069   1077   #define SQLITE_FCNTL_LOCKSTATE               1
  1070   1078   #define SQLITE_FCNTL_GET_LOCKPROXYFILE       2
  1071   1079   #define SQLITE_FCNTL_SET_LOCKPROXYFILE       3
  1072   1080   #define SQLITE_FCNTL_LAST_ERRNO              4
  1073   1081   #define SQLITE_FCNTL_SIZE_HINT               5
................................................................................
  1094   1102   #define SQLITE_FCNTL_VFS_POINTER            27
  1095   1103   #define SQLITE_FCNTL_JOURNAL_POINTER        28
  1096   1104   #define SQLITE_FCNTL_WIN32_GET_HANDLE       29
  1097   1105   #define SQLITE_FCNTL_PDB                    30
  1098   1106   #define SQLITE_FCNTL_BEGIN_ATOMIC_WRITE     31
  1099   1107   #define SQLITE_FCNTL_COMMIT_ATOMIC_WRITE    32
  1100   1108   #define SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE  33
         1109  +#define SQLITE_FCNTL_LOCK_TIMEOUT           34
  1101   1110   
  1102   1111   /* deprecated names */
  1103   1112   #define SQLITE_GET_LOCKPROXYFILE      SQLITE_FCNTL_GET_LOCKPROXYFILE
  1104   1113   #define SQLITE_SET_LOCKPROXYFILE      SQLITE_FCNTL_SET_LOCKPROXYFILE
  1105   1114   #define SQLITE_LAST_ERRNO             SQLITE_FCNTL_LAST_ERRNO
  1106   1115   
  1107   1116   
................................................................................
  1919   1928   ** Or if the threshold is -1, statement journals are always held
  1920   1929   ** exclusively in memory.
  1921   1930   ** Since many statement journals never become large, setting the spill
  1922   1931   ** threshold to a value such as 64KiB can greatly reduce the amount of
  1923   1932   ** I/O required to support statement rollback.
  1924   1933   ** The default value for this setting is controlled by the
  1925   1934   ** [SQLITE_STMTJRNL_SPILL] compile-time option.
         1935  +**
         1936  +** [[SQLITE_CONFIG_SORTERREF_SIZE]]
         1937  +** <dt>SQLITE_CONFIG_SORTERREF_SIZE
         1938  +** <dd>The SQLITE_CONFIG_SORTERREF_SIZE option accepts a single parameter
         1939  +** of type (int) - the new value of the sorter-reference size threshold.
         1940  +** Usually, when SQLite uses an external sort to order records according
         1941  +** to an ORDER BY clause, all fields required by the caller are present in the
         1942  +** sorted records. However, if SQLite determines based on the declared type
         1943  +** of a table column that its values are likely to be very large - larger
         1944  +** than the configured sorter-reference size threshold - then a reference
         1945  +** is stored in each sorted record and the required column values loaded
         1946  +** from the database as records are returned in sorted order. The default
         1947  +** value for this option is to never use this optimization. Specifying a 
         1948  +** negative value for this option restores the default behaviour.
         1949  +** This option is only available if SQLite is compiled with the
         1950  +** [SQLITE_ENABLE_SORTER_REFERENCES] compile-time option.
  1926   1951   ** </dl>
  1927   1952   */
  1928   1953   #define SQLITE_CONFIG_SINGLETHREAD  1  /* nil */
  1929   1954   #define SQLITE_CONFIG_MULTITHREAD   2  /* nil */
  1930   1955   #define SQLITE_CONFIG_SERIALIZED    3  /* nil */
  1931   1956   #define SQLITE_CONFIG_MALLOC        4  /* sqlite3_mem_methods* */
  1932   1957   #define SQLITE_CONFIG_GETMALLOC     5  /* sqlite3_mem_methods* */
................................................................................
  1948   1973   #define SQLITE_CONFIG_SQLLOG       21  /* xSqllog, void* */
  1949   1974   #define SQLITE_CONFIG_MMAP_SIZE    22  /* sqlite3_int64, sqlite3_int64 */
  1950   1975   #define SQLITE_CONFIG_WIN32_HEAPSIZE      23  /* int nByte */
  1951   1976   #define SQLITE_CONFIG_PCACHE_HDRSZ        24  /* int *psz */
  1952   1977   #define SQLITE_CONFIG_PMASZ               25  /* unsigned int szPma */
  1953   1978   #define SQLITE_CONFIG_STMTJRNL_SPILL      26  /* int nByte */
  1954   1979   #define SQLITE_CONFIG_SMALL_MALLOC        27  /* boolean */
         1980  +#define SQLITE_CONFIG_SORTERREF_SIZE      28  /* int nByte */
  1955   1981   
  1956   1982   /*
  1957   1983   ** CAPI3REF: Database Connection Configuration Options
  1958   1984   **
  1959   1985   ** These constants are the available integer configuration options that
  1960   1986   ** can be passed as the second argument to the [sqlite3_db_config()] interface.
  1961   1987   **
................................................................................
  2050   2076   **
  2051   2077   ** <dt>SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE</dt>
  2052   2078   ** <dd> Usually, when a database in wal mode is closed or detached from a 
  2053   2079   ** database handle, SQLite checks if this will mean that there are now no 
  2054   2080   ** connections at all to the database. If so, it performs a checkpoint 
  2055   2081   ** operation before closing the connection. This option may be used to
  2056   2082   ** override this behaviour. The first parameter passed to this operation
  2057         -** is an integer - non-zero to disable checkpoints-on-close, or zero (the
  2058         -** default) to enable them. The second parameter is a pointer to an integer
         2083  +** is an integer - positive to disable checkpoints-on-close, or zero (the
         2084  +** default) to enable them, and negative to leave the setting unchanged.
         2085  +** The second parameter is a pointer to an integer
  2059   2086   ** into which is written 0 or 1 to indicate whether checkpoints-on-close
  2060   2087   ** have been disabled - 0 if they are not disabled, 1 if they are.
  2061   2088   ** </dd>
         2089  +**
  2062   2090   ** <dt>SQLITE_DBCONFIG_ENABLE_QPSG</dt>
  2063   2091   ** <dd>^(The SQLITE_DBCONFIG_ENABLE_QPSG option activates or deactivates
  2064   2092   ** the [query planner stability guarantee] (QPSG).  When the QPSG is active,
  2065   2093   ** a single SQL query statement will always use the same algorithm regardless
  2066   2094   ** of values of [bound parameters].)^ The QPSG disables some query optimizations
  2067   2095   ** that look at the values of bound parameters, which can make some queries
  2068   2096   ** slower.  But the QPSG has the advantage of more predictable behavior.  With
  2069   2097   ** the QPSG active, SQLite will always use the same query plan in the field as
  2070   2098   ** was used during testing in the lab.
         2099  +** The first argument to this setting is an integer which is 0 to disable 
         2100  +** the QPSG, positive to enable QPSG, or negative to leave the setting
         2101  +** unchanged. The second parameter is a pointer to an integer into which
         2102  +** is written 0 or 1 to indicate whether the QPSG is disabled or enabled
         2103  +** following this call.
  2071   2104   ** </dd>
         2105  +**
  2072   2106   ** <dt>SQLITE_DBCONFIG_TRIGGER_EQP</dt>
  2073   2107   ** <dd> By default, the output of EXPLAIN QUERY PLAN commands does not 
  2074   2108   ** include output for any operations performed by trigger programs. This
  2075   2109   ** option is used to set or clear (the default) a flag that governs this
  2076   2110   ** behavior. The first parameter passed to this operation is an integer -
  2077         -** non-zero to enable output for trigger programs, or zero to disable it.
         2111  +** positive to enable output for trigger programs, or zero to disable it,
         2112  +** or negative to leave the setting unchanged.
  2078   2113   ** The second parameter is a pointer to an integer into which is written 
  2079   2114   ** 0 or 1 to indicate whether output-for-triggers has been disabled - 0 if 
  2080   2115   ** it is not disabled, 1 if it is.  
  2081   2116   ** </dd>
         2117  +**
         2118  +** <dt>SQLITE_DBCONFIG_RESET_DATABASE</dt>
         2119  +** <dd> Set the SQLITE_DBCONFIG_RESET_DATABASE flag and then run
         2120  +** [VACUUM] in order to reset a database back to an empty database
         2121  +** with no schema and no content. The following process works even for
         2122  +** a badly corrupted database file:
         2123  +** <ol>
         2124  +** <li> sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 1, 0);
         2125  +** <li> [sqlite3_exec](db, "[VACUUM]", 0, 0, 0);
         2126  +** <li> sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 0, 0);
         2127  +** </ol>
         2128  +** Because resetting a database is destructive and irreversible, the
         2129  +** process requires the use of this obscure API and multiple steps to help
         2130  +** ensure that it does not happen by accident.
         2131  +** </dd>
  2082   2132   ** </dl>
  2083   2133   */
  2084   2134   #define SQLITE_DBCONFIG_MAINDBNAME            1000 /* const char* */
  2085   2135   #define SQLITE_DBCONFIG_LOOKASIDE             1001 /* void* int int */
  2086   2136   #define SQLITE_DBCONFIG_ENABLE_FKEY           1002 /* int int* */
  2087   2137   #define SQLITE_DBCONFIG_ENABLE_TRIGGER        1003 /* int int* */
  2088   2138   #define SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1004 /* int int* */
  2089   2139   #define SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION 1005 /* int int* */
  2090   2140   #define SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE      1006 /* int int* */
  2091   2141   #define SQLITE_DBCONFIG_ENABLE_QPSG           1007 /* int int* */
  2092   2142   #define SQLITE_DBCONFIG_TRIGGER_EQP           1008 /* int int* */
  2093         -#define SQLITE_DBCONFIG_MAX                   1008 /* Largest DBCONFIG */
         2143  +#define SQLITE_DBCONFIG_RESET_DATABASE        1009 /* int int* */
         2144  +#define SQLITE_DBCONFIG_MAX                   1009 /* Largest DBCONFIG */
  2094   2145   
  2095   2146   /*
  2096   2147   ** CAPI3REF: Enable Or Disable Extended Result Codes
  2097   2148   ** METHOD: sqlite3
  2098   2149   **
  2099   2150   ** ^The sqlite3_extended_result_codes() routine enables or disables the
  2100   2151   ** [extended result codes] feature of SQLite. ^The extended result
................................................................................
  2492   2543   SQLITE_API void sqlite3_free_table(char **result);
  2493   2544   
  2494   2545   /*
  2495   2546   ** CAPI3REF: Formatted String Printing Functions
  2496   2547   **
  2497   2548   ** These routines are work-alikes of the "printf()" family of functions
  2498   2549   ** from the standard C library.
  2499         -** These routines understand most of the common K&R formatting options,
  2500         -** plus some additional non-standard formats, detailed below.
  2501         -** Note that some of the more obscure formatting options from recent
  2502         -** C-library standards are omitted from this implementation.
         2550  +** These routines understand most of the common formatting options from
         2551  +** the standard library printf() 
         2552  +** plus some additional non-standard formats ([%q], [%Q], [%w], and [%z]).
         2553  +** See the [built-in printf()] documentation for details.
  2503   2554   **
  2504   2555   ** ^The sqlite3_mprintf() and sqlite3_vmprintf() routines write their
  2505         -** results into memory obtained from [sqlite3_malloc()].
         2556  +** results into memory obtained from [sqlite3_malloc64()].
  2506   2557   ** The strings returned by these two routines should be
  2507   2558   ** released by [sqlite3_free()].  ^Both routines return a
  2508         -** NULL pointer if [sqlite3_malloc()] is unable to allocate enough
         2559  +** NULL pointer if [sqlite3_malloc64()] is unable to allocate enough
  2509   2560   ** memory to hold the resulting string.
  2510   2561   **
  2511   2562   ** ^(The sqlite3_snprintf() routine is similar to "snprintf()" from
  2512   2563   ** the standard C library.  The result is written into the
  2513   2564   ** buffer supplied as the second parameter whose size is given by
  2514   2565   ** the first parameter. Note that the order of the
  2515   2566   ** first two parameters is reversed from snprintf().)^  This is an
................................................................................
  2525   2576   ** guarantees that the buffer is always zero-terminated.  ^The first
  2526   2577   ** parameter "n" is the total size of the buffer, including space for
  2527   2578   ** the zero terminator.  So the longest string that can be completely
  2528   2579   ** written will be n-1 characters.
  2529   2580   **
  2530   2581   ** ^The sqlite3_vsnprintf() routine is a varargs version of sqlite3_snprintf().
  2531   2582   **
  2532         -** These routines all implement some additional formatting
  2533         -** options that are useful for constructing SQL statements.
  2534         -** All of the usual printf() formatting options apply.  In addition, there
  2535         -** is are "%q", "%Q", "%w" and "%z" options.
  2536         -**
  2537         -** ^(The %q option works like %s in that it substitutes a nul-terminated
  2538         -** string from the argument list.  But %q also doubles every '\'' character.
  2539         -** %q is designed for use inside a string literal.)^  By doubling each '\''
  2540         -** character it escapes that character and allows it to be inserted into
  2541         -** the string.
  2542         -**
  2543         -** For example, assume the string variable zText contains text as follows:
  2544         -**
  2545         -** <blockquote><pre>
  2546         -**  char *zText = "It's a happy day!";
  2547         -** </pre></blockquote>
  2548         -**
  2549         -** One can use this text in an SQL statement as follows:
  2550         -**
  2551         -** <blockquote><pre>
  2552         -**  char *zSQL = sqlite3_mprintf("INSERT INTO table VALUES('%q')", zText);
  2553         -**  sqlite3_exec(db, zSQL, 0, 0, 0);
  2554         -**  sqlite3_free(zSQL);
  2555         -** </pre></blockquote>
  2556         -**
  2557         -** Because the %q format string is used, the '\'' character in zText
  2558         -** is escaped and the SQL generated is as follows:
  2559         -**
  2560         -** <blockquote><pre>
  2561         -**  INSERT INTO table1 VALUES('It''s a happy day!')
  2562         -** </pre></blockquote>
  2563         -**
  2564         -** This is correct.  Had we used %s instead of %q, the generated SQL
  2565         -** would have looked like this:
  2566         -**
  2567         -** <blockquote><pre>
  2568         -**  INSERT INTO table1 VALUES('It's a happy day!');
  2569         -** </pre></blockquote>
  2570         -**
  2571         -** This second example is an SQL syntax error.  As a general rule you should
  2572         -** always use %q instead of %s when inserting text into a string literal.
  2573         -**
  2574         -** ^(The %Q option works like %q except it also adds single quotes around
  2575         -** the outside of the total string.  Additionally, if the parameter in the
  2576         -** argument list is a NULL pointer, %Q substitutes the text "NULL" (without
  2577         -** single quotes).)^  So, for example, one could say:
  2578         -**
  2579         -** <blockquote><pre>
  2580         -**  char *zSQL = sqlite3_mprintf("INSERT INTO table VALUES(%Q)", zText);
  2581         -**  sqlite3_exec(db, zSQL, 0, 0, 0);
  2582         -**  sqlite3_free(zSQL);
  2583         -** </pre></blockquote>
  2584         -**
  2585         -** The code above will render a correct SQL statement in the zSQL
  2586         -** variable even if the zText variable is a NULL pointer.
  2587         -**
  2588         -** ^(The "%w" formatting option is like "%q" except that it expects to
  2589         -** be contained within double-quotes instead of single quotes, and it
  2590         -** escapes the double-quote character instead of the single-quote
  2591         -** character.)^  The "%w" formatting option is intended for safely inserting
  2592         -** table and column names into a constructed SQL statement.
  2593         -**
  2594         -** ^(The "%z" formatting option works like "%s" but with the
  2595         -** addition that after the string has been read and copied into
  2596         -** the result, [sqlite3_free()] is called on the input string.)^
         2583  +** See also:  [built-in printf()], [printf() SQL function]
  2597   2584   */
  2598   2585   SQLITE_API char *sqlite3_mprintf(const char*,...);
  2599   2586   SQLITE_API char *sqlite3_vmprintf(const char*, va_list);
  2600   2587   SQLITE_API char *sqlite3_snprintf(int,char*,const char*, ...);
  2601   2588   SQLITE_API char *sqlite3_vsnprintf(int,char*,const char*, va_list);
  2602   2589   
  2603   2590   /*
................................................................................
  3655   3642   ** a schema change, on the first  [sqlite3_step()] call following any change
  3656   3643   ** to the [sqlite3_bind_text | bindings] of that [parameter]. 
  3657   3644   ** ^The specific value of WHERE-clause [parameter] might influence the 
  3658   3645   ** choice of query plan if the parameter is the left-hand side of a [LIKE]
  3659   3646   ** or [GLOB] operator or if the parameter is compared to an indexed column
  3660   3647   ** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled.
  3661   3648   ** </li>
         3649  +** </ol>
  3662   3650   **
  3663   3651   ** <p>^sqlite3_prepare_v3() differs from sqlite3_prepare_v2() only in having
  3664   3652   ** the extra prepFlags parameter, which is a bit array consisting of zero or
  3665   3653   ** more of the [SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_*] flags.  ^The
  3666   3654   ** sqlite3_prepare_v2() interface works exactly the same as
  3667   3655   ** sqlite3_prepare_v3() with a zero prepFlags parameter.
  3668         -** </ol>
  3669   3656   */
  3670   3657   SQLITE_API int sqlite3_prepare(
  3671   3658     sqlite3 *db,            /* Database handle */
  3672   3659     const char *zSql,       /* SQL statement, UTF-8 encoded */
  3673   3660     int nByte,              /* Maximum length of zSql in bytes. */
  3674   3661     sqlite3_stmt **ppStmt,  /* OUT: Statement handle */
  3675   3662     const char **pzTail     /* OUT: Pointer to unused portion of zSql */
................................................................................
  5536   5523   ** using [sqlite3_free].
  5537   5524   ** Hence, if this variable is modified directly, either it should be
  5538   5525   ** made NULL or made to point to memory obtained from [sqlite3_malloc]
  5539   5526   ** or else the use of the [data_store_directory pragma] should be avoided.
  5540   5527   */
  5541   5528   SQLITE_API SQLITE_EXTERN char *sqlite3_data_directory;
  5542   5529   
         5530  +/*
         5531  +** CAPI3REF: Win32 Specific Interface
         5532  +**
         5533  +** These interfaces are available only on Windows.  The
         5534  +** [sqlite3_win32_set_directory] interface is used to set the value associated
         5535  +** with the [sqlite3_temp_directory] or [sqlite3_data_directory] variable, to
         5536  +** zValue, depending on the value of the type parameter.  The zValue parameter
         5537  +** should be NULL to cause the previous value to be freed via [sqlite3_free];
         5538  +** a non-NULL value will be copied into memory obtained from [sqlite3_malloc]
         5539  +** prior to being used.  The [sqlite3_win32_set_directory] interface returns
         5540  +** [SQLITE_OK] to indicate success, [SQLITE_ERROR] if the type is unsupported,
         5541  +** or [SQLITE_NOMEM] if memory could not be allocated.  The value of the
         5542  +** [sqlite3_data_directory] variable is intended to act as a replacement for
         5543  +** the current directory on the sub-platforms of Win32 where that concept is
         5544  +** not present, e.g. WinRT and UWP.  The [sqlite3_win32_set_directory8] and
         5545  +** [sqlite3_win32_set_directory16] interfaces behave exactly the same as the
         5546  +** sqlite3_win32_set_directory interface except the string parameter must be
         5547  +** UTF-8 or UTF-16, respectively.
         5548  +*/
         5549  +SQLITE_API int sqlite3_win32_set_directory(
         5550  +  unsigned long type, /* Identifier for directory being set or reset */
         5551  +  void *zValue        /* New value for directory being set or reset */
         5552  +);
         5553  +SQLITE_API int sqlite3_win32_set_directory8(unsigned long type, const char *zValue);
         5554  +SQLITE_API int sqlite3_win32_set_directory16(unsigned long type, const void *zValue);
         5555  +
         5556  +/*
         5557  +** CAPI3REF: Win32 Directory Types
         5558  +**
         5559  +** These macros are only available on Windows.  They define the allowed values
         5560  +** for the type argument to the [sqlite3_win32_set_directory] interface.
         5561  +*/
         5562  +#define SQLITE_WIN32_DATA_DIRECTORY_TYPE  1
         5563  +#define SQLITE_WIN32_TEMP_DIRECTORY_TYPE  2
         5564  +
  5543   5565   /*
  5544   5566   ** CAPI3REF: Test For Auto-Commit Mode
  5545   5567   ** KEYWORDS: {autocommit mode}
  5546   5568   ** METHOD: sqlite3
  5547   5569   **
  5548   5570   ** ^The sqlite3_get_autocommit() interface returns non-zero or
  5549   5571   ** zero if the given database connection is or is not in autocommit mode,
................................................................................
  6268   6290     int idxFlags;              /* Mask of SQLITE_INDEX_SCAN_* flags */
  6269   6291     /* Fields below are only available in SQLite 3.10.0 and later */
  6270   6292     sqlite3_uint64 colUsed;    /* Input: Mask of columns used by statement */
  6271   6293   };
  6272   6294   
  6273   6295   /*
  6274   6296   ** CAPI3REF: Virtual Table Scan Flags
         6297  +**
         6298  +** Virtual table implementations are allowed to set the 
         6299  +** [sqlite3_index_info].idxFlags field to some combination of
         6300  +** these bits.
  6275   6301   */
  6276   6302   #define SQLITE_INDEX_SCAN_UNIQUE      1     /* Scan visits at most 1 row */
  6277   6303   
  6278   6304   /*
  6279   6305   ** CAPI3REF: Virtual Table Constraint Operator Codes
  6280   6306   **
  6281   6307   ** These macros defined the allowed values for the
................................................................................
  7043   7069   #define SQLITE_TESTCTRL_FAULT_INSTALL            9
  7044   7070   #define SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS     10
  7045   7071   #define SQLITE_TESTCTRL_PENDING_BYTE            11
  7046   7072   #define SQLITE_TESTCTRL_ASSERT                  12
  7047   7073   #define SQLITE_TESTCTRL_ALWAYS                  13
  7048   7074   #define SQLITE_TESTCTRL_RESERVE                 14
  7049   7075   #define SQLITE_TESTCTRL_OPTIMIZATIONS           15
  7050         -#define SQLITE_TESTCTRL_ISKEYWORD               16
         7076  +#define SQLITE_TESTCTRL_ISKEYWORD               16  /* NOT USED */
  7051   7077   #define SQLITE_TESTCTRL_SCRATCHMALLOC           17  /* NOT USED */
  7052   7078   #define SQLITE_TESTCTRL_LOCALTIME_FAULT         18
  7053   7079   #define SQLITE_TESTCTRL_EXPLAIN_STMT            19  /* NOT USED */
  7054   7080   #define SQLITE_TESTCTRL_ONCE_RESET_THRESHOLD    19
  7055   7081   #define SQLITE_TESTCTRL_NEVER_CORRUPT           20
  7056   7082   #define SQLITE_TESTCTRL_VDBE_COVERAGE           21
  7057   7083   #define SQLITE_TESTCTRL_BYTEORDER               22
  7058   7084   #define SQLITE_TESTCTRL_ISINIT                  23
  7059   7085   #define SQLITE_TESTCTRL_SORTER_MMAP             24
  7060   7086   #define SQLITE_TESTCTRL_IMPOSTER                25
  7061   7087   #define SQLITE_TESTCTRL_PARSER_COVERAGE         26
  7062   7088   #define SQLITE_TESTCTRL_LAST                    26  /* Largest TESTCTRL */
  7063   7089   
         7090  +/*
         7091  +** CAPI3REF: SQL Keyword Checking
         7092  +**
         7093  +** These routines provide access to the set of SQL language keywords 
         7094  +** recognized by SQLite.  Applications can uses these routines to determine
         7095  +** whether or not a specific identifier needs to be escaped (for example,
         7096  +** by enclosing in double-quotes) so as not to confuse the parser.
         7097  +**
         7098  +** The sqlite3_keyword_count() interface returns the number of distinct
         7099  +** keywords understood by SQLite.
         7100  +**
         7101  +** The sqlite3_keyword_name(N,Z,L) interface finds the N-th keyword and
         7102  +** makes *Z point to that keyword expressed as UTF8 and writes the number
         7103  +** of bytes in the keyword into *L.  The string that *Z points to is not
         7104  +** zero-terminated.  The sqlite3_keyword_name(N,Z,L) routine returns
         7105  +** SQLITE_OK if N is within bounds and SQLITE_ERROR if not. If either Z
         7106  +** or L are NULL or invalid pointers then calls to
         7107  +** sqlite3_keyword_name(N,Z,L) result in undefined behavior.
         7108  +**
         7109  +** The sqlite3_keyword_check(Z,L) interface checks to see whether or not
         7110  +** the L-byte UTF8 identifier that Z points to is a keyword, returning non-zero
         7111  +** if it is and zero if not.
         7112  +**
         7113  +** The parser used by SQLite is forgiving.  It is often possible to use
         7114  +** a keyword as an identifier as long as such use does not result in a
         7115  +** parsing ambiguity.  For example, the statement
         7116  +** "CREATE TABLE BEGIN(REPLACE,PRAGMA,END);" is accepted by SQLite, and
         7117  +** creates a new table named "BEGIN" with three columns named
         7118  +** "REPLACE", "PRAGMA", and "END".  Nevertheless, best practice is to avoid
         7119  +** using keywords as identifiers.  Common techniques used to avoid keyword
         7120  +** name collisions include:
         7121  +** <ul>
         7122  +** <li> Put all identifier names inside double-quotes.  This is the official
         7123  +**      SQL way to escape identifier names.
         7124  +** <li> Put identifier names inside &#91;...&#93;.  This is not standard SQL,
         7125  +**      but it is what SQL Server does and so lots of programmers use this
         7126  +**      technique.
         7127  +** <li> Begin every identifier with the letter "Z" as no SQL keywords start
         7128  +**      with "Z".
         7129  +** <li> Include a digit somewhere in every identifier name.
         7130  +** </ul>
         7131  +**
         7132  +** Note that the number of keywords understood by SQLite can depend on
         7133  +** compile-time options.  For example, "VACUUM" is not a keyword if
         7134  +** SQLite is compiled with the [-DSQLITE_OMIT_VACUUM] option.  Also,
         7135  +** new keywords may be added to future releases of SQLite.
         7136  +*/
         7137  +SQLITE_API int sqlite3_keyword_count(void);
         7138  +SQLITE_API int sqlite3_keyword_name(int,const char**,int*);
         7139  +SQLITE_API int sqlite3_keyword_check(const char*,int);
         7140  +
         7141  +/*
         7142  +** CAPI3REF: Dynamic String Object
         7143  +** KEYWORDS: {dynamic string}
         7144  +**
         7145  +** An instance of the sqlite3_str object contains a dynamically-sized
         7146  +** string under construction.
         7147  +**
         7148  +** The lifecycle of an sqlite3_str object is as follows:
         7149  +** <ol>
         7150  +** <li> ^The sqlite3_str object is created using [sqlite3_str_new()].
         7151  +** <li> ^Text is appended to the sqlite3_str object using various
         7152  +** methods, such as [sqlite3_str_appendf()].
         7153  +** <li> ^The sqlite3_str object is destroyed and the string it created
         7154  +** is returned using the [sqlite3_str_finish()] interface.
         7155  +** </ol>
         7156  +*/
         7157  +typedef struct sqlite3_str sqlite3_str;
         7158  +
         7159  +/*
         7160  +** CAPI3REF: Create A New Dynamic String Object
         7161  +** CONSTRUCTOR: sqlite3_str
         7162  +**
         7163  +** ^The [sqlite3_str_new(D)] interface allocates and initializes
         7164  +** a new [sqlite3_str] object.  To avoid memory leaks, the object returned by
         7165  +** [sqlite3_str_new()] must be freed by a subsequent call to 
         7166  +** [sqlite3_str_finish(X)].
         7167  +**
         7168  +** ^The [sqlite3_str_new(D)] interface always returns a pointer to a
         7169  +** valid [sqlite3_str] object, though in the event of an out-of-memory
         7170  +** error the returned object might be a special singleton that will
         7171  +** silently reject new text, always return SQLITE_NOMEM from 
         7172  +** [sqlite3_str_errcode()], always return 0 for 
         7173  +** [sqlite3_str_length()], and always return NULL from
         7174  +** [sqlite3_str_finish(X)].  It is always safe to use the value
         7175  +** returned by [sqlite3_str_new(D)] as the sqlite3_str parameter
         7176  +** to any of the other [sqlite3_str] methods.
         7177  +**
         7178  +** The D parameter to [sqlite3_str_new(D)] may be NULL.  If the
         7179  +** D parameter in [sqlite3_str_new(D)] is not NULL, then the maximum
         7180  +** length of the string contained in the [sqlite3_str] object will be
         7181  +** the value set for [sqlite3_limit](D,[SQLITE_LIMIT_LENGTH]) instead
         7182  +** of [SQLITE_MAX_LENGTH].
         7183  +*/
         7184  +SQLITE_API sqlite3_str *sqlite3_str_new(sqlite3*);
         7185  +
         7186  +/*
         7187  +** CAPI3REF: Finalize A Dynamic String
         7188  +** DESTRUCTOR: sqlite3_str
         7189  +**
         7190  +** ^The [sqlite3_str_finish(X)] interface destroys the sqlite3_str object X
         7191  +** and returns a pointer to a memory buffer obtained from [sqlite3_malloc64()]
         7192  +** that contains the constructed string.  The calling application should
         7193  +** pass the returned value to [sqlite3_free()] to avoid a memory leak.
         7194  +** ^The [sqlite3_str_finish(X)] interface may return a NULL pointer if any
         7195  +** errors were encountered during construction of the string.  ^The
         7196  +** [sqlite3_str_finish(X)] interface will also return a NULL pointer if the
         7197  +** string in [sqlite3_str] object X is zero bytes long.
         7198  +*/
         7199  +SQLITE_API char *sqlite3_str_finish(sqlite3_str*);
         7200  +
         7201  +/*
         7202  +** CAPI3REF: Add Content To A Dynamic String
         7203  +** METHOD: sqlite3_str
         7204  +**
         7205  +** These interfaces add content to an sqlite3_str object previously obtained
         7206  +** from [sqlite3_str_new()].
         7207  +**
         7208  +** ^The [sqlite3_str_appendf(X,F,...)] and 
         7209  +** [sqlite3_str_vappendf(X,F,V)] interfaces uses the [built-in printf]
         7210  +** functionality of SQLite to append formatted text onto the end of 
         7211  +** [sqlite3_str] object X.
         7212  +**
         7213  +** ^The [sqlite3_str_append(X,S,N)] method appends exactly N bytes from string S
         7214  +** onto the end of the [sqlite3_str] object X.  N must be non-negative.
         7215  +** S must contain at least N non-zero bytes of content.  To append a
         7216  +** zero-terminated string in its entirety, use the [sqlite3_str_appendall()]
         7217  +** method instead.
         7218  +**
         7219  +** ^The [sqlite3_str_appendall(X,S)] method appends the complete content of
         7220  +** zero-terminated string S onto the end of [sqlite3_str] object X.
         7221  +**
         7222  +** ^The [sqlite3_str_appendchar(X,N,C)] method appends N copies of the
         7223  +** single-byte character C onto the end of [sqlite3_str] object X.
         7224  +** ^This method can be used, for example, to add whitespace indentation.
         7225  +**
         7226  +** ^The [sqlite3_str_reset(X)] method resets the string under construction
         7227  +** inside [sqlite3_str] object X back to zero bytes in length.  
         7228  +**
         7229  +** These methods do not return a result code.  ^If an error occurs, that fact
         7230  +** is recorded in the [sqlite3_str] object and can be recovered by a
         7231  +** subsequent call to [sqlite3_str_errcode(X)].
         7232  +*/
         7233  +SQLITE_API void sqlite3_str_appendf(sqlite3_str*, const char *zFormat, ...);
         7234  +SQLITE_API void sqlite3_str_vappendf(sqlite3_str*, const char *zFormat, va_list);
         7235  +SQLITE_API void sqlite3_str_append(sqlite3_str*, const char *zIn, int N);
         7236  +SQLITE_API void sqlite3_str_appendall(sqlite3_str*, const char *zIn);
         7237  +SQLITE_API void sqlite3_str_appendchar(sqlite3_str*, int N, char C);
         7238  +SQLITE_API void sqlite3_str_reset(sqlite3_str*);
         7239  +
         7240  +/*
         7241  +** CAPI3REF: Status Of A Dynamic String
         7242  +** METHOD: sqlite3_str
         7243  +**
         7244  +** These interfaces return the current status of an [sqlite3_str] object.
         7245  +**
         7246  +** ^If any prior errors have occurred while constructing the dynamic string
         7247  +** in sqlite3_str X, then the [sqlite3_str_errcode(X)] method will return
         7248  +** an appropriate error code.  ^The [sqlite3_str_errcode(X)] method returns
         7249  +** [SQLITE_NOMEM] following any out-of-memory error, or
         7250  +** [SQLITE_TOOBIG] if the size of the dynamic string exceeds
         7251  +** [SQLITE_MAX_LENGTH], or [SQLITE_OK] if there have been no errors.
         7252  +**
         7253  +** ^The [sqlite3_str_length(X)] method returns the current length, in bytes,
         7254  +** of the dynamic string under construction in [sqlite3_str] object X.
         7255  +** ^The length returned by [sqlite3_str_length(X)] does not include the
         7256  +** zero-termination byte.
         7257  +**
         7258  +** ^The [sqlite3_str_value(X)] method returns a pointer to the current
         7259  +** content of the dynamic string under construction in X.  The value
         7260  +** returned by [sqlite3_str_value(X)] is managed by the sqlite3_str object X
         7261  +** and might be freed or altered by any subsequent method on the same
         7262  +** [sqlite3_str] object.  Applications must not used the pointer returned
         7263  +** [sqlite3_str_value(X)] after any subsequent method call on the same
         7264  +** object.  ^Applications may change the content of the string returned
         7265  +** by [sqlite3_str_value(X)] as long as they do not write into any bytes
         7266  +** outside the range of 0 to [sqlite3_str_length(X)] and do not read or
         7267  +** write any byte after any subsequent sqlite3_str method call.
         7268  +*/
         7269  +SQLITE_API int sqlite3_str_errcode(sqlite3_str*);
         7270  +SQLITE_API int sqlite3_str_length(sqlite3_str*);
         7271  +SQLITE_API char *sqlite3_str_value(sqlite3_str*);
         7272  +
  7064   7273   /*
  7065   7274   ** CAPI3REF: SQLite Runtime Status
  7066   7275   **
  7067   7276   ** ^These interfaces are used to retrieve runtime status information
  7068   7277   ** about the performance of SQLite, and optionally to reset various
  7069   7278   ** highwater marks.  ^The first argument is an integer code for
  7070   7279   ** the specific parameter to measure.  ^(Recognized integer codes
................................................................................
  7289   7498   ** wal file in wal mode databases, or the number of pages written to the
  7290   7499   ** database file in rollback mode databases. Any pages written as part of
  7291   7500   ** transaction rollback or database recovery operations are not included.
  7292   7501   ** If an IO or other error occurs while writing a page to disk, the effect
  7293   7502   ** on subsequent SQLITE_DBSTATUS_CACHE_WRITE requests is undefined.)^ ^The
  7294   7503   ** highwater mark associated with SQLITE_DBSTATUS_CACHE_WRITE is always 0.
  7295   7504   ** </dd>
         7505  +**
         7506  +** [[SQLITE_DBSTATUS_CACHE_SPILL]] ^(<dt>SQLITE_DBSTATUS_CACHE_SPILL</dt>
         7507  +** <dd>This parameter returns the number of dirty cache entries that have
         7508  +** been written to disk in the middle of a transaction due to the page
         7509  +** cache overflowing. Transactions are more efficient if they are written
         7510  +** to disk all at once. When pages spill mid-transaction, that introduces
         7511  +** additional overhead. This parameter can be used help identify
         7512  +** inefficiencies that can be resolve by increasing the cache size.
         7513  +** </dd>
  7296   7514   **
  7297   7515   ** [[SQLITE_DBSTATUS_DEFERRED_FKS]] ^(<dt>SQLITE_DBSTATUS_DEFERRED_FKS</dt>
  7298   7516   ** <dd>This parameter returns zero for the current value if and only if
  7299   7517   ** all foreign key constraints (deferred or immediate) have been
  7300   7518   ** resolved.)^  ^The highwater mark is always 0.
  7301   7519   ** </dd>
  7302   7520   ** </dl>
................................................................................
  7309   7527   #define SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE  5
  7310   7528   #define SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL  6
  7311   7529   #define SQLITE_DBSTATUS_CACHE_HIT            7
  7312   7530   #define SQLITE_DBSTATUS_CACHE_MISS           8
  7313   7531   #define SQLITE_DBSTATUS_CACHE_WRITE          9
  7314   7532   #define SQLITE_DBSTATUS_DEFERRED_FKS        10
  7315   7533   #define SQLITE_DBSTATUS_CACHE_USED_SHARED   11
  7316         -#define SQLITE_DBSTATUS_MAX                 11   /* Largest defined DBSTATUS */
         7534  +#define SQLITE_DBSTATUS_CACHE_SPILL         12
         7535  +#define SQLITE_DBSTATUS_MAX                 12   /* Largest defined DBSTATUS */
  7317   7536   
  7318   7537   
  7319   7538   /*
  7320   7539   ** CAPI3REF: Prepared Statement Status
  7321   7540   ** METHOD: sqlite3_stmt
  7322   7541   **
  7323   7542   ** ^(Each prepared statement maintains various
................................................................................
  8316   8535   /*
  8317   8536   ** CAPI3REF: Determine If Virtual Table Column Access Is For UPDATE
  8318   8537   **
  8319   8538   ** If the sqlite3_vtab_nochange(X) routine is called within the [xColumn]
  8320   8539   ** method of a [virtual table], then it returns true if and only if the
  8321   8540   ** column is being fetched as part of an UPDATE operation during which the
  8322   8541   ** column value will not change.  Applications might use this to substitute
  8323         -** a lighter-weight value to return that the corresponding [xUpdate] method
  8324         -** understands as a "no-change" value.
         8542  +** a return value that is less expensive to compute and that the corresponding
         8543  +** [xUpdate] method understands as a "no-change" value.
  8325   8544   **
  8326   8545   ** If the [xColumn] method calls sqlite3_vtab_nochange() and finds that
  8327         -** the column is not changed by the UPDATE statement, they the xColumn
         8546  +** the column is not changed by the UPDATE statement, then the xColumn
  8328   8547   ** method can optionally return without setting a result, without calling
  8329   8548   ** any of the [sqlite3_result_int|sqlite3_result_xxxxx() interfaces].
  8330   8549   ** In that case, [sqlite3_value_nochange(X)] will return true for the
  8331   8550   ** same column in the [xUpdate] method.
  8332   8551   */
  8333   8552   SQLITE_API int sqlite3_vtab_nochange(sqlite3_context*);
  8334   8553   
................................................................................
  8789   9008   ** transaction open on the database, or if the database is not a wal mode
  8790   9009   ** database.
  8791   9010   **
  8792   9011   ** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
  8793   9012   */
  8794   9013   SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_snapshot_recover(sqlite3 *db, const char *zDb);
  8795   9014   
         9015  +/*
         9016  +** CAPI3REF: Serialize a database
         9017  +**
         9018  +** The sqlite3_serialize(D,S,P,F) interface returns a pointer to memory
         9019  +** that is a serialization of the S database on [database connection] D.
         9020  +** If P is not a NULL pointer, then the size of the database in bytes
         9021  +** is written into *P.
         9022  +**
         9023  +** For an ordinary on-disk database file, the serialization is just a
         9024  +** copy of the disk file.  For an in-memory database or a "TEMP" database,
         9025  +** the serialization is the same sequence of bytes which would be written
         9026  +** to disk if that database where backed up to disk.
         9027  +**
         9028  +** The usual case is that sqlite3_serialize() copies the serialization of
         9029  +** the database into memory obtained from [sqlite3_malloc64()] and returns
         9030  +** a pointer to that memory.  The caller is responsible for freeing the
         9031  +** returned value to avoid a memory leak.  However, if the F argument
         9032  +** contains the SQLITE_SERIALIZE_NOCOPY bit, then no memory allocations
         9033  +** are made, and the sqlite3_serialize() function will return a pointer
         9034  +** to the contiguous memory representation of the database that SQLite
         9035  +** is currently using for that database, or NULL if the no such contiguous
         9036  +** memory representation of the database exists.  A contiguous memory
         9037  +** representation of the database will usually only exist if there has
         9038  +** been a prior call to [sqlite3_deserialize(D,S,...)] with the same
         9039  +** values of D and S.
         9040  +** The size of the database is written into *P even if the 
         9041  +** SQLITE_SERIALIZE_NOCOPY bit is set but no contiguous copy
         9042  +** of the database exists.
         9043  +**
         9044  +** A call to sqlite3_serialize(D,S,P,F) might return NULL even if the
         9045  +** SQLITE_SERIALIZE_NOCOPY bit is omitted from argument F if a memory
         9046  +** allocation error occurs.
         9047  +**
         9048  +** This interface is only available if SQLite is compiled with the
         9049  +** [SQLITE_ENABLE_DESERIALIZE] option.
         9050  +*/
         9051  +SQLITE_API unsigned char *sqlite3_serialize(
         9052  +  sqlite3 *db,           /* The database connection */
         9053  +  const char *zSchema,   /* Which DB to serialize. ex: "main", "temp", ... */
         9054  +  sqlite3_int64 *piSize, /* Write size of the DB here, if not NULL */
         9055  +  unsigned int mFlags    /* Zero or more SQLITE_SERIALIZE_* flags */
         9056  +);
         9057  +
         9058  +/*
         9059  +** CAPI3REF: Flags for sqlite3_serialize
         9060  +**
         9061  +** Zero or more of the following constants can be OR-ed together for
         9062  +** the F argument to [sqlite3_serialize(D,S,P,F)].
         9063  +**
         9064  +** SQLITE_SERIALIZE_NOCOPY means that [sqlite3_serialize()] will return
         9065  +** a pointer to contiguous in-memory database that it is currently using,
         9066  +** without making a copy of the database.  If SQLite is not currently using
         9067  +** a contiguous in-memory database, then this option causes
         9068  +** [sqlite3_serialize()] to return a NULL pointer.  SQLite will only be
         9069  +** using a contiguous in-memory database if it has been initialized by a
         9070  +** prior call to [sqlite3_deserialize()].
         9071  +*/
         9072  +#define SQLITE_SERIALIZE_NOCOPY 0x001   /* Do no memory allocations */
         9073  +
         9074  +/*
         9075  +** CAPI3REF: Deserialize a database
         9076  +**
         9077  +** The sqlite3_deserialize(D,S,P,N,M,F) interface causes the 
         9078  +** [database connection] D to disconnect from database S and then
         9079  +** reopen S as an in-memory database based on the serialization contained
         9080  +** in P.  The serialized database P is N bytes in size.  M is the size of
         9081  +** the buffer P, which might be larger than N.  If M is larger than N, and
         9082  +** the SQLITE_DESERIALIZE_READONLY bit is not set in F, then SQLite is
         9083  +** permitted to add content to the in-memory database as long as the total
         9084  +** size does not exceed M bytes.
         9085  +**
         9086  +** If the SQLITE_DESERIALIZE_FREEONCLOSE bit is set in F, then SQLite will
         9087  +** invoke sqlite3_free() on the serialization buffer when the database
         9088  +** connection closes.  If the SQLITE_DESERIALIZE_RESIZEABLE bit is set, then
         9089  +** SQLite will try to increase the buffer size using sqlite3_realloc64()
         9090  +** if writes on the database cause it to grow larger than M bytes.
         9091  +**
         9092  +** The sqlite3_deserialize() interface will fail with SQLITE_BUSY if the
         9093  +** database is currently in a read transaction or is involved in a backup
         9094  +** operation.
         9095  +**
         9096  +** If sqlite3_deserialize(D,S,P,N,M,F) fails for any reason and if the 
         9097  +** SQLITE_DESERIALIZE_FREEONCLOSE bit is set in argument F, then
         9098  +** [sqlite3_free()] is invoked on argument P prior to returning.
         9099  +**
         9100  +** This interface is only available if SQLite is compiled with the
         9101  +** [SQLITE_ENABLE_DESERIALIZE] option.
         9102  +*/
         9103  +SQLITE_API int sqlite3_deserialize(
         9104  +  sqlite3 *db,            /* The database connection */
         9105  +  const char *zSchema,    /* Which DB to reopen with the deserialization */
         9106  +  unsigned char *pData,   /* The serialized database content */
         9107  +  sqlite3_int64 szDb,     /* Number bytes in the deserialization */
         9108  +  sqlite3_int64 szBuf,    /* Total size of buffer pData[] */
         9109  +  unsigned mFlags         /* Zero or more SQLITE_DESERIALIZE_* flags */
         9110  +);
         9111  +
         9112  +/*
         9113  +** CAPI3REF: Flags for sqlite3_deserialize()
         9114  +**
         9115  +** The following are allowed values for 6th argument (the F argument) to
         9116  +** the [sqlite3_deserialize(D,S,P,N,M,F)] interface.
         9117  +**
         9118  +** The SQLITE_DESERIALIZE_FREEONCLOSE means that the database serialization
         9119  +** in the P argument is held in memory obtained from [sqlite3_malloc64()]
         9120  +** and that SQLite should take ownership of this memory and automatically
         9121  +** free it when it has finished using it.  Without this flag, the caller
         9122  +** is resposible for freeing any dynamically allocated memory.
         9123  +**
         9124  +** The SQLITE_DESERIALIZE_RESIZEABLE flag means that SQLite is allowed to
         9125  +** grow the size of the database using calls to [sqlite3_realloc64()].  This
         9126  +** flag should only be used if SQLITE_DESERIALIZE_FREEONCLOSE is also used.
         9127  +** Without this flag, the deserialized database cannot increase in size beyond
         9128  +** the number of bytes specified by the M parameter.
         9129  +**
         9130  +** The SQLITE_DESERIALIZE_READONLY flag means that the deserialized database
         9131  +** should be treated as read-only.
         9132  +*/
         9133  +#define SQLITE_DESERIALIZE_FREEONCLOSE 1 /* Call sqlite3_free() on close */
         9134  +#define SQLITE_DESERIALIZE_RESIZEABLE  2 /* Resize using sqlite3_realloc64() */
         9135  +#define SQLITE_DESERIALIZE_READONLY    4 /* Database is read-only */
         9136  +
  8796   9137   /*
  8797   9138   ** Undo the hack that converts floating point types to integer for
  8798   9139   ** builds on processors without floating point support.
  8799   9140   */
  8800   9141   #ifdef SQLITE_OMIT_FLOATING_POINT
  8801   9142   # undef double
  8802   9143   #endif
................................................................................
  8936   9277   #ifdef __cplusplus
  8937   9278   extern "C" {
  8938   9279   #endif
  8939   9280   
  8940   9281   
  8941   9282   /*
  8942   9283   ** CAPI3REF: Session Object Handle
         9284  +**
         9285  +** An instance of this object is a [session] that can be used to
         9286  +** record changes to a database.
  8943   9287   */
  8944   9288   typedef struct sqlite3_session sqlite3_session;
  8945   9289   
  8946   9290   /*
  8947   9291   ** CAPI3REF: Changeset Iterator Handle
         9292  +**
         9293  +** An instance of this object acts as a cursor for iterating
         9294  +** over the elements of a [changeset] or [patchset].
  8948   9295   */
  8949   9296   typedef struct sqlite3_changeset_iter sqlite3_changeset_iter;
  8950   9297   
  8951   9298   /*
  8952   9299   ** CAPI3REF: Create A New Session Object
         9300  +** CONSTRUCTOR: sqlite3_session
  8953   9301   **
  8954   9302   ** Create a new session object attached to database handle db. If successful,
  8955   9303   ** a pointer to the new object is written to *ppSession and SQLITE_OK is
  8956   9304   ** returned. If an error occurs, *ppSession is set to NULL and an SQLite
  8957   9305   ** error code (e.g. SQLITE_NOMEM) is returned.
  8958   9306   **
  8959   9307   ** It is possible to create multiple session objects attached to a single
................................................................................
  8982   9330     sqlite3 *db,                    /* Database handle */
  8983   9331     const char *zDb,                /* Name of db (e.g. "main") */
  8984   9332     sqlite3_session **ppSession     /* OUT: New session object */
  8985   9333   );
  8986   9334   
  8987   9335   /*
  8988   9336   ** CAPI3REF: Delete A Session Object
         9337  +** DESTRUCTOR: sqlite3_session
  8989   9338   **
  8990   9339   ** Delete a session object previously allocated using 
  8991   9340   ** [sqlite3session_create()]. Once a session object has been deleted, the
  8992   9341   ** results of attempting to use pSession with any other session module
  8993   9342   ** function are undefined.
  8994   9343   **
  8995   9344   ** Session objects must be deleted before the database handle to which they
................................................................................
  8997   9346   ** [sqlite3session_create()] for details.
  8998   9347   */
  8999   9348   SQLITE_API void sqlite3session_delete(sqlite3_session *pSession);
  9000   9349   
  9001   9350   
  9002   9351   /*
  9003   9352   ** CAPI3REF: Enable Or Disable A Session Object
         9353  +** METHOD: sqlite3_session
  9004   9354   **
  9005   9355   ** Enable or disable the recording of changes by a session object. When
  9006   9356   ** enabled, a session object records changes made to the database. When
  9007   9357   ** disabled - it does not. A newly created session object is enabled.
  9008   9358   ** Refer to the documentation for [sqlite3session_changeset()] for further
  9009   9359   ** details regarding how enabling and disabling a session object affects
  9010   9360   ** the eventual changesets.
................................................................................
  9016   9366   ** The return value indicates the final state of the session object: 0 if 
  9017   9367   ** the session is disabled, or 1 if it is enabled.
  9018   9368   */
  9019   9369   SQLITE_API int sqlite3session_enable(sqlite3_session *pSession, int bEnable);
  9020   9370   
  9021   9371   /*
  9022   9372   ** CAPI3REF: Set Or Clear the Indirect Change Flag
         9373  +** METHOD: sqlite3_session
  9023   9374   **
  9024   9375   ** Each change recorded by a session object is marked as either direct or
  9025   9376   ** indirect. A change is marked as indirect if either:
  9026   9377   **
  9027   9378   ** <ul>
  9028   9379   **   <li> The session object "indirect" flag is set when the change is
  9029   9380   **        made, or
................................................................................
  9045   9396   ** The return value indicates the final state of the indirect flag: 0 if 
  9046   9397   ** it is clear, or 1 if it is set.
  9047   9398   */
  9048   9399   SQLITE_API int sqlite3session_indirect(sqlite3_session *pSession, int bIndirect);
  9049   9400   
  9050   9401   /*
  9051   9402   ** CAPI3REF: Attach A Table To A Session Object
         9403  +** METHOD: sqlite3_session
  9052   9404   **
  9053   9405   ** If argument zTab is not NULL, then it is the name of a table to attach
  9054   9406   ** to the session object passed as the first argument. All subsequent changes 
  9055   9407   ** made to the table while the session object is enabled will be recorded. See 
  9056   9408   ** documentation for [sqlite3session_changeset()] for further details.
  9057   9409   **
  9058   9410   ** Or, if argument zTab is NULL, then changes are recorded for all tables
................................................................................
  9107   9459   SQLITE_API int sqlite3session_attach(
  9108   9460     sqlite3_session *pSession,      /* Session object */
  9109   9461     const char *zTab                /* Table name */
  9110   9462   );
  9111   9463   
  9112   9464   /*
  9113   9465   ** CAPI3REF: Set a table filter on a Session Object.
         9466  +** METHOD: sqlite3_session
  9114   9467   **
  9115   9468   ** The second argument (xFilter) is the "filter callback". For changes to rows 
  9116   9469   ** in tables that are not attached to the Session object, the filter is called
  9117   9470   ** to determine whether changes to the table's rows should be tracked or not. 
  9118   9471   ** If xFilter returns 0, changes is not tracked. Note that once a table is 
  9119   9472   ** attached, xFilter will not be called again.
  9120   9473   */
................................................................................
  9125   9478       const char *zTab              /* Table name */
  9126   9479     ),
  9127   9480     void *pCtx                      /* First argument passed to xFilter */
  9128   9481   );
  9129   9482   
  9130   9483   /*
  9131   9484   ** CAPI3REF: Generate A Changeset From A Session Object
         9485  +** METHOD: sqlite3_session
  9132   9486   **
  9133   9487   ** Obtain a changeset containing changes to the tables attached to the 
  9134   9488   ** session object passed as the first argument. If successful, 
  9135   9489   ** set *ppChangeset to point to a buffer containing the changeset 
  9136   9490   ** and *pnChangeset to the size of the changeset in bytes before returning
  9137   9491   ** SQLITE_OK. If an error occurs, set both *ppChangeset and *pnChangeset to
  9138   9492   ** zero and return an SQLite error code.
................................................................................
  9234   9588   SQLITE_API int sqlite3session_changeset(
  9235   9589     sqlite3_session *pSession,      /* Session object */
  9236   9590     int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  9237   9591     void **ppChangeset              /* OUT: Buffer containing changeset */
  9238   9592   );
  9239   9593   
  9240   9594   /*
  9241         -** CAPI3REF: Load The Difference Between Tables Into A Session 
         9595  +** CAPI3REF: Load The Difference Between Tables Into A Session
         9596  +** METHOD: sqlite3_session
  9242   9597   **
  9243   9598   ** If it is not already attached to the session object passed as the first
  9244   9599   ** argument, this function attaches table zTbl in the same manner as the
  9245   9600   ** [sqlite3session_attach()] function. If zTbl does not exist, or if it
  9246   9601   ** does not have a primary key, this function is a no-op (but does not return
  9247   9602   ** an error).
  9248   9603   **
................................................................................
  9299   9654     const char *zTbl,
  9300   9655     char **pzErrMsg
  9301   9656   );
  9302   9657   
  9303   9658   
  9304   9659   /*
  9305   9660   ** CAPI3REF: Generate A Patchset From A Session Object
         9661  +** METHOD: sqlite3_session
  9306   9662   **
  9307   9663   ** The differences between a patchset and a changeset are that:
  9308   9664   **
  9309   9665   ** <ul>
  9310   9666   **   <li> DELETE records consist of the primary key fields only. The 
  9311   9667   **        original values of other fields are omitted.
  9312   9668   **   <li> The original values of any modified fields are omitted from 
................................................................................
  9350   9706   ** guaranteed that a call to sqlite3session_changeset() will return a 
  9351   9707   ** changeset containing zero changes.
  9352   9708   */
  9353   9709   SQLITE_API int sqlite3session_isempty(sqlite3_session *pSession);
  9354   9710   
  9355   9711   /*
  9356   9712   ** CAPI3REF: Create An Iterator To Traverse A Changeset 
         9713  +** CONSTRUCTOR: sqlite3_changeset_iter
  9357   9714   **
  9358   9715   ** Create an iterator used to iterate through the contents of a changeset.
  9359   9716   ** If successful, *pp is set to point to the iterator handle and SQLITE_OK
  9360   9717   ** is returned. Otherwise, if an error occurs, *pp is set to zero and an
  9361   9718   ** SQLite error code is returned.
  9362   9719   **
  9363   9720   ** The following functions can be used to advance and query a changeset 
................................................................................
  9390   9747     int nChangeset,                 /* Size of changeset blob in bytes */
  9391   9748     void *pChangeset                /* Pointer to blob containing changeset */
  9392   9749   );
  9393   9750   
  9394   9751   
  9395   9752   /*
  9396   9753   ** CAPI3REF: Advance A Changeset Iterator
         9754  +** METHOD: sqlite3_changeset_iter
  9397   9755   **
  9398   9756   ** This function may only be used with iterators created by function
  9399   9757   ** [sqlite3changeset_start()]. If it is called on an iterator passed to
  9400   9758   ** a conflict-handler callback by [sqlite3changeset_apply()], SQLITE_MISUSE
  9401   9759   ** is returned and the call has no effect.
  9402   9760   **
  9403   9761   ** Immediately after an iterator is created by sqlite3changeset_start(), it
................................................................................
  9414   9772   ** codes include SQLITE_CORRUPT (if the changeset buffer is corrupt) or 
  9415   9773   ** SQLITE_NOMEM.
  9416   9774   */
  9417   9775   SQLITE_API int sqlite3changeset_next(sqlite3_changeset_iter *pIter);
  9418   9776   
  9419   9777   /*
  9420   9778   ** CAPI3REF: Obtain The Current Operation From A Changeset Iterator
         9779  +** METHOD: sqlite3_changeset_iter
  9421   9780   **
  9422   9781   ** The pIter argument passed to this function may either be an iterator
  9423   9782   ** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator
  9424   9783   ** created by [sqlite3changeset_start()]. In the latter case, the most recent
  9425   9784   ** call to [sqlite3changeset_next()] must have returned [SQLITE_ROW]. If this
  9426   9785   ** is not the case, this function returns [SQLITE_MISUSE].
  9427   9786   **
................................................................................
  9448   9807     int *pnCol,                     /* OUT: Number of columns in table */
  9449   9808     int *pOp,                       /* OUT: SQLITE_INSERT, DELETE or UPDATE */
  9450   9809     int *pbIndirect                 /* OUT: True for an 'indirect' change */
  9451   9810   );
  9452   9811   
  9453   9812   /*
  9454   9813   ** CAPI3REF: Obtain The Primary Key Definition Of A Table
         9814  +** METHOD: sqlite3_changeset_iter
  9455   9815   **
  9456   9816   ** For each modified table, a changeset includes the following:
  9457   9817   **
  9458   9818   ** <ul>
  9459   9819   **   <li> The number of columns in the table, and
  9460   9820   **   <li> Which of those columns make up the tables PRIMARY KEY.
  9461   9821   ** </ul>
................................................................................
  9479   9839     sqlite3_changeset_iter *pIter,  /* Iterator object */
  9480   9840     unsigned char **pabPK,          /* OUT: Array of boolean - true for PK cols */
  9481   9841     int *pnCol                      /* OUT: Number of entries in output array */
  9482   9842   );
  9483   9843   
  9484   9844   /*
  9485   9845   ** CAPI3REF: Obtain old.* Values From A Changeset Iterator
         9846  +** METHOD: sqlite3_changeset_iter
  9486   9847   **
  9487   9848   ** The pIter argument passed to this function may either be an iterator
  9488   9849   ** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator
  9489   9850   ** created by [sqlite3changeset_start()]. In the latter case, the most recent
  9490   9851   ** call to [sqlite3changeset_next()] must have returned SQLITE_ROW. 
  9491   9852   ** Furthermore, it may only be called if the type of change that the iterator
  9492   9853   ** currently points to is either [SQLITE_DELETE] or [SQLITE_UPDATE]. Otherwise,
................................................................................
  9509   9870     sqlite3_changeset_iter *pIter,  /* Changeset iterator */
  9510   9871     int iVal,                       /* Column number */
  9511   9872     sqlite3_value **ppValue         /* OUT: Old value (or NULL pointer) */
  9512   9873   );
  9513   9874   
  9514   9875   /*
  9515   9876   ** CAPI3REF: Obtain new.* Values From A Changeset Iterator
         9877  +** METHOD: sqlite3_changeset_iter
  9516   9878   **
  9517   9879   ** The pIter argument passed to this function may either be an iterator
  9518   9880   ** passed to a conflict-handler by [sqlite3changeset_apply()], or an iterator
  9519   9881   ** created by [sqlite3changeset_start()]. In the latter case, the most recent
  9520   9882   ** call to [sqlite3changeset_next()] must have returned SQLITE_ROW. 
  9521   9883   ** Furthermore, it may only be called if the type of change that the iterator
  9522   9884   ** currently points to is either [SQLITE_UPDATE] or [SQLITE_INSERT]. Otherwise,
................................................................................
  9542   9904     sqlite3_changeset_iter *pIter,  /* Changeset iterator */
  9543   9905     int iVal,                       /* Column number */
  9544   9906     sqlite3_value **ppValue         /* OUT: New value (or NULL pointer) */
  9545   9907   );
  9546   9908   
  9547   9909   /*
  9548   9910   ** CAPI3REF: Obtain Conflicting Row Values From A Changeset Iterator
         9911  +** METHOD: sqlite3_changeset_iter
  9549   9912   **
  9550   9913   ** This function should only be used with iterator objects passed to a
  9551   9914   ** conflict-handler callback by [sqlite3changeset_apply()] with either
  9552   9915   ** [SQLITE_CHANGESET_DATA] or [SQLITE_CHANGESET_CONFLICT]. If this function
  9553   9916   ** is called on any other iterator, [SQLITE_MISUSE] is returned and *ppValue
  9554   9917   ** is set to NULL.
  9555   9918   **
................................................................................
  9569   9932     sqlite3_changeset_iter *pIter,  /* Changeset iterator */
  9570   9933     int iVal,                       /* Column number */
  9571   9934     sqlite3_value **ppValue         /* OUT: Value from conflicting row */
  9572   9935   );
  9573   9936   
  9574   9937   /*
  9575   9938   ** CAPI3REF: Determine The Number Of Foreign Key Constraint Violations
         9939  +** METHOD: sqlite3_changeset_iter
  9576   9940   **
  9577   9941   ** This function may only be called with an iterator passed to an
  9578   9942   ** SQLITE_CHANGESET_FOREIGN_KEY conflict handler callback. In this case
  9579   9943   ** it sets the output variable to the total number of known foreign key
  9580   9944   ** violations in the destination database and returns SQLITE_OK.
  9581   9945   **
  9582   9946   ** In all other cases this function returns SQLITE_MISUSE.
................................................................................
  9585   9949     sqlite3_changeset_iter *pIter,  /* Changeset iterator */
  9586   9950     int *pnOut                      /* OUT: Number of FK violations */
  9587   9951   );
  9588   9952   
  9589   9953   
  9590   9954   /*
  9591   9955   ** CAPI3REF: Finalize A Changeset Iterator
         9956  +** METHOD: sqlite3_changeset_iter
  9592   9957   **
  9593   9958   ** This function is used to finalize an iterator allocated with
  9594   9959   ** [sqlite3changeset_start()].
  9595   9960   **
  9596   9961   ** This function should only be called on iterators created using the
  9597   9962   ** [sqlite3changeset_start()] function. If an application calls this
  9598   9963   ** function with an iterator passed to a conflict-handler by
................................................................................
  9601   9966   **
  9602   9967   ** If an error was encountered within a call to an sqlite3changeset_xxx()
  9603   9968   ** function (for example an [SQLITE_CORRUPT] in [sqlite3changeset_next()] or an 
  9604   9969   ** [SQLITE_NOMEM] in [sqlite3changeset_new()]) then an error code corresponding
  9605   9970   ** to that error is returned by this function. Otherwise, SQLITE_OK is
  9606   9971   ** returned. This is to allow the following pattern (pseudo-code):
  9607   9972   **
         9973  +** <pre>
  9608   9974   **   sqlite3changeset_start();
  9609   9975   **   while( SQLITE_ROW==sqlite3changeset_next() ){
  9610   9976   **     // Do something with change.
  9611   9977   **   }
  9612   9978   **   rc = sqlite3changeset_finalize();
  9613   9979   **   if( rc!=SQLITE_OK ){
  9614   9980   **     // An error has occurred 
  9615   9981   **   }
         9982  +** </pre>
  9616   9983   */
  9617   9984   SQLITE_API int sqlite3changeset_finalize(sqlite3_changeset_iter *pIter);
  9618   9985   
  9619   9986   /*
  9620   9987   ** CAPI3REF: Invert A Changeset
  9621   9988   **
  9622   9989   ** This function is used to "invert" a changeset object. Applying an inverted
................................................................................
  9656  10023   ** single changeset. The result is a changeset equivalent to applying
  9657  10024   ** changeset A followed by changeset B. 
  9658  10025   **
  9659  10026   ** This function combines the two input changesets using an 
  9660  10027   ** sqlite3_changegroup object. Calling it produces similar results as the
  9661  10028   ** following code fragment:
  9662  10029   **
        10030  +** <pre>
  9663  10031   **   sqlite3_changegroup *pGrp;
  9664  10032   **   rc = sqlite3_changegroup_new(&pGrp);
  9665  10033   **   if( rc==SQLITE_OK ) rc = sqlite3changegroup_add(pGrp, nA, pA);
  9666  10034   **   if( rc==SQLITE_OK ) rc = sqlite3changegroup_add(pGrp, nB, pB);
  9667  10035   **   if( rc==SQLITE_OK ){
  9668  10036   **     rc = sqlite3changegroup_output(pGrp, pnOut, ppOut);
  9669  10037   **   }else{
  9670  10038   **     *ppOut = 0;
  9671  10039   **     *pnOut = 0;
  9672  10040   **   }
        10041  +** </pre>
  9673  10042   **
  9674  10043   ** Refer to the sqlite3_changegroup documentation below for details.
  9675  10044   */
  9676  10045   SQLITE_API int sqlite3changeset_concat(
  9677  10046     int nA,                         /* Number of bytes in buffer pA */
  9678  10047     void *pA,                       /* Pointer to buffer containing changeset A */
  9679  10048     int nB,                         /* Number of bytes in buffer pB */
................................................................................
  9681  10050     int *pnOut,                     /* OUT: Number of bytes in output changeset */
  9682  10051     void **ppOut                    /* OUT: Buffer containing output changeset */
  9683  10052   );
  9684  10053   
  9685  10054   
  9686  10055   /*
  9687  10056   ** CAPI3REF: Changegroup Handle
        10057  +**
        10058  +** A changegroup is an object used to combine two or more 
        10059  +** [changesets] or [patchsets]
  9688  10060   */
  9689  10061   typedef struct sqlite3_changegroup sqlite3_changegroup;
  9690  10062   
  9691  10063   /*
  9692  10064   ** CAPI3REF: Create A New Changegroup Object
        10065  +** CONSTRUCTOR: sqlite3_changegroup
  9693  10066   **
  9694  10067   ** An sqlite3_changegroup object is used to combine two or more changesets
  9695  10068   ** (or patchsets) into a single changeset (or patchset). A single changegroup
  9696  10069   ** object may combine changesets or patchsets, but not both. The output is
  9697  10070   ** always in the same format as the input.
  9698  10071   **
  9699  10072   ** If successful, this function returns SQLITE_OK and populates (*pp) with
................................................................................
  9723  10096   ** sqlite3changegroup_output() functions, also available are the streaming
  9724  10097   ** versions sqlite3changegroup_add_strm() and sqlite3changegroup_output_strm().
  9725  10098   */
  9726  10099   SQLITE_API int sqlite3changegroup_new(sqlite3_changegroup **pp);
  9727  10100   
  9728  10101   /*
  9729  10102   ** CAPI3REF: Add A Changeset To A Changegroup
        10103  +** METHOD: sqlite3_changegroup
  9730  10104   **
  9731  10105   ** Add all changes within the changeset (or patchset) in buffer pData (size
  9732  10106   ** nData bytes) to the changegroup. 
  9733  10107   **
  9734  10108   ** If the buffer contains a patchset, then all prior calls to this function
  9735  10109   ** on the same changegroup object must also have specified patchsets. Or, if
  9736  10110   ** the buffer contains a changeset, so must have the earlier calls to this
................................................................................
  9800  10174   **
  9801  10175   ** If no error occurs, SQLITE_OK is returned.
  9802  10176   */
  9803  10177   SQLITE_API int sqlite3changegroup_add(sqlite3_changegroup*, int nData, void *pData);
  9804  10178   
  9805  10179   /*
  9806  10180   ** CAPI3REF: Obtain A Composite Changeset From A Changegroup
        10181  +** METHOD: sqlite3_changegroup
  9807  10182   **
  9808  10183   ** Obtain a buffer containing a changeset (or patchset) representing the
  9809  10184   ** current contents of the changegroup. If the inputs to the changegroup
  9810  10185   ** were themselves changesets, the output is a changeset. Or, if the
  9811  10186   ** inputs were patchsets, the output is also a patchset.
  9812  10187   **
  9813  10188   ** As with the output of the sqlite3session_changeset() and
................................................................................
  9830  10205     sqlite3_changegroup*,
  9831  10206     int *pnData,                    /* OUT: Size of output buffer in bytes */
  9832  10207     void **ppData                   /* OUT: Pointer to output buffer */
  9833  10208   );
  9834  10209   
  9835  10210   /*
  9836  10211   ** CAPI3REF: Delete A Changegroup Object
        10212  +** DESTRUCTOR: sqlite3_changegroup
  9837  10213   */
  9838  10214   SQLITE_API void sqlite3changegroup_delete(sqlite3_changegroup*);
  9839  10215   
  9840  10216   /*
  9841  10217   ** CAPI3REF: Apply A Changeset To A Database
  9842  10218   **
  9843         -** Apply a changeset to a database. This function attempts to update the
  9844         -** "main" database attached to handle db with the changes found in the
  9845         -** changeset passed via the second and third arguments.
        10219  +** Apply a changeset or patchset to a database. These functions attempt to
        10220  +** update the "main" database attached to handle db with the changes found in
        10221  +** the changeset passed via the second and third arguments. 
  9846  10222   **
  9847         -** The fourth argument (xFilter) passed to this function is the "filter
        10223  +** The fourth argument (xFilter) passed to these functions is the "filter
  9848  10224   ** callback". If it is not NULL, then for each table affected by at least one
  9849  10225   ** change in the changeset, the filter callback is invoked with
  9850  10226   ** the table name as the second argument, and a copy of the context pointer
  9851         -** passed as the sixth argument to this function as the first. If the "filter
  9852         -** callback" returns zero, then no attempt is made to apply any changes to 
  9853         -** the table. Otherwise, if the return value is non-zero or the xFilter
  9854         -** argument to this function is NULL, all changes related to the table are
  9855         -** attempted.
        10227  +** passed as the sixth argument as the first. If the "filter callback"
        10228  +** returns zero, then no attempt is made to apply any changes to the table.
        10229  +** Otherwise, if the return value is non-zero or the xFilter argument to
        10230  +** is NULL, all changes related to the table are attempted.
  9856  10231   **
  9857  10232   ** For each table that is not excluded by the filter callback, this function 
  9858  10233   ** tests that the target database contains a compatible table. A table is 
  9859  10234   ** considered compatible if all of the following are true:
  9860  10235   **
  9861  10236   ** <ul>
  9862  10237   **   <li> The table has the same name as the name recorded in the 
................................................................................
  9893  10268   ** actions are taken by sqlite3changeset_apply() depending on the value
  9894  10269   ** returned by each invocation of the conflict-handler function. Refer to
  9895  10270   ** the documentation for the three 
  9896  10271   ** [SQLITE_CHANGESET_OMIT|available return values] for details.
  9897  10272   **
  9898  10273   ** <dl>
  9899  10274   ** <dt>DELETE Changes<dd>
  9900         -**   For each DELETE change, this function checks if the target database 
        10275  +**   For each DELETE change, the function checks if the target database 
  9901  10276   **   contains a row with the same primary key value (or values) as the 
  9902  10277   **   original row values stored in the changeset. If it does, and the values 
  9903  10278   **   stored in all non-primary key columns also match the values stored in 
  9904  10279   **   the changeset the row is deleted from the target database.
  9905  10280   **
  9906  10281   **   If a row with matching primary key values is found, but one or more of
  9907  10282   **   the non-primary key fields contains a value different from the original
................................................................................
  9938  10313   **   violation (e.g. NOT NULL or UNIQUE), the conflict handler function is 
  9939  10314   **   invoked with the second argument set to [SQLITE_CHANGESET_CONSTRAINT].
  9940  10315   **   This includes the case where the INSERT operation is re-attempted because 
  9941  10316   **   an earlier call to the conflict handler function returned 
  9942  10317   **   [SQLITE_CHANGESET_REPLACE].
  9943  10318   **
  9944  10319   ** <dt>UPDATE Changes<dd>
  9945         -**   For each UPDATE change, this function checks if the target database 
        10320  +**   For each UPDATE change, the function checks if the target database 
  9946  10321   **   contains a row with the same primary key value (or values) as the 
  9947  10322   **   original row values stored in the changeset. If it does, and the values 
  9948  10323   **   stored in all modified non-primary key columns also match the values
  9949  10324   **   stored in the changeset the row is updated within the target database.
  9950  10325   **
  9951  10326   **   If a row with matching primary key values is found, but one or more of
  9952  10327   **   the modified non-primary key fields contains a value different from an
................................................................................
  9969  10344   ** </dl>
  9970  10345   **
  9971  10346   ** It is safe to execute SQL statements, including those that write to the
  9972  10347   ** table that the callback related to, from within the xConflict callback.
  9973  10348   ** This can be used to further customize the applications conflict
  9974  10349   ** resolution strategy.
  9975  10350   **
  9976         -** All changes made by this function are enclosed in a savepoint transaction.
        10351  +** All changes made by these functions are enclosed in a savepoint transaction.
  9977  10352   ** If any other error (aside from a constraint failure when attempting to
  9978  10353   ** write to the target database) occurs, then the savepoint transaction is
  9979  10354   ** rolled back, restoring the target database to its original state, and an 
  9980  10355   ** SQLite error code returned.
        10356  +**
        10357  +** If the output parameters (ppRebase) and (pnRebase) are non-NULL and
        10358  +** the input is a changeset (not a patchset), then sqlite3changeset_apply_v2()
        10359  +** may set (*ppRebase) to point to a "rebase" that may be used with the 
        10360  +** sqlite3_rebaser APIs buffer before returning. In this case (*pnRebase)
        10361  +** is set to the size of the buffer in bytes. It is the responsibility of the
        10362  +** caller to eventually free any such buffer using sqlite3_free(). The buffer
        10363  +** is only allocated and populated if one or more conflicts were encountered
        10364  +** while applying the patchset. See comments surrounding the sqlite3_rebaser
        10365  +** APIs for further details.
        10366  +**
        10367  +** The behavior of sqlite3changeset_apply_v2() and its streaming equivalent
        10368  +** may be modified by passing a combination of
        10369  +** [SQLITE_CHANGESETAPPLY_NOSAVEPOINT | supported flags] as the 9th parameter.
        10370  +**
        10371  +** Note that the sqlite3changeset_apply_v2() API is still <b>experimental</b>
        10372  +** and therefore subject to change.
  9981  10373   */
  9982  10374   SQLITE_API int sqlite3changeset_apply(
  9983  10375     sqlite3 *db,                    /* Apply change to "main" db of this handle */
  9984  10376     int nChangeset,                 /* Size of changeset in bytes */
  9985  10377     void *pChangeset,               /* Changeset blob */
  9986  10378     int(*xFilter)(
  9987  10379       void *pCtx,                   /* Copy of sixth arg to _apply() */
................................................................................
  9990  10382     int(*xConflict)(
  9991  10383       void *pCtx,                   /* Copy of sixth arg to _apply() */
  9992  10384       int eConflict,                /* DATA, MISSING, CONFLICT, CONSTRAINT */
  9993  10385       sqlite3_changeset_iter *p     /* Handle describing change and conflict */
  9994  10386     ),
  9995  10387     void *pCtx                      /* First argument passed to xConflict */
  9996  10388   );
        10389  +SQLITE_API int sqlite3changeset_apply_v2(
        10390  +  sqlite3 *db,                    /* Apply change to "main" db of this handle */
        10391  +  int nChangeset,                 /* Size of changeset in bytes */
        10392  +  void *pChangeset,               /* Changeset blob */
        10393  +  int(*xFilter)(
        10394  +    void *pCtx,                   /* Copy of sixth arg to _apply() */
        10395  +    const char *zTab              /* Table name */
        10396  +  ),
        10397  +  int(*xConflict)(
        10398  +    void *pCtx,                   /* Copy of sixth arg to _apply() */
        10399  +    int eConflict,                /* DATA, MISSING, CONFLICT, CONSTRAINT */
        10400  +    sqlite3_changeset_iter *p     /* Handle describing change and conflict */
        10401  +  ),
        10402  +  void *pCtx,                     /* First argument passed to xConflict */
        10403  +  void **ppRebase, int *pnRebase, /* OUT: Rebase data */
        10404  +  int flags                       /* Combination of SESSION_APPLY_* flags */
        10405  +);
        10406  +
        10407  +/*
        10408  +** CAPI3REF: Flags for sqlite3changeset_apply_v2
        10409  +**
        10410  +** The following flags may passed via the 9th parameter to
        10411  +** [sqlite3changeset_apply_v2] and [sqlite3changeset_apply_v2_strm]:
        10412  +**
        10413  +** <dl>
        10414  +** <dt>SQLITE_CHANGESETAPPLY_NOSAVEPOINT <dd>
        10415  +**   Usually, the sessions module encloses all operations performed by
        10416  +**   a single call to apply_v2() or apply_v2_strm() in a [SAVEPOINT]. The
        10417  +**   SAVEPOINT is committed if the changeset or patchset is successfully
        10418  +**   applied, or rolled back if an error occurs. Specifying this flag
        10419  +**   causes the sessions module to omit this savepoint. In this case, if the
        10420  +**   caller has an open transaction or savepoint when apply_v2() is called, 
        10421  +**   it may revert the partially applied changeset by rolling it back.
        10422  +*/
        10423  +#define SQLITE_CHANGESETAPPLY_NOSAVEPOINT   0x0001
  9997  10424   
  9998  10425   /* 
  9999  10426   ** CAPI3REF: Constants Passed To The Conflict Handler
 10000  10427   **
 10001  10428   ** Values that may be passed as the second argument to a conflict-handler.
 10002  10429   **
 10003  10430   ** <dl>
................................................................................
 10087  10514   **   and the call to sqlite3changeset_apply() returns SQLITE_ABORT.
 10088  10515   ** </dl>
 10089  10516   */
 10090  10517   #define SQLITE_CHANGESET_OMIT       0
 10091  10518   #define SQLITE_CHANGESET_REPLACE    1
 10092  10519   #define SQLITE_CHANGESET_ABORT      2
 10093  10520   
        10521  +/* 
        10522  +** CAPI3REF: Rebasing changesets
        10523  +** EXPERIMENTAL
        10524  +**
        10525  +** Suppose there is a site hosting a database in state S0. And that
        10526  +** modifications are made that move that database to state S1 and a
        10527  +** changeset recorded (the "local" changeset). Then, a changeset based
        10528  +** on S0 is received from another site (the "remote" changeset) and 
        10529  +** applied to the database. The database is then in state 
        10530  +** (S1+"remote"), where the exact state depends on any conflict
        10531  +** resolution decisions (OMIT or REPLACE) made while applying "remote".
        10532  +** Rebasing a changeset is to update it to take those conflict 
        10533  +** resolution decisions into account, so that the same conflicts
        10534  +** do not have to be resolved elsewhere in the network. 
        10535  +**
        10536  +** For example, if both the local and remote changesets contain an
        10537  +** INSERT of the same key on "CREATE TABLE t1(a PRIMARY KEY, b)":
        10538  +**
        10539  +**   local:  INSERT INTO t1 VALUES(1, 'v1');
        10540  +**   remote: INSERT INTO t1 VALUES(1, 'v2');
        10541  +**
        10542  +** and the conflict resolution is REPLACE, then the INSERT change is
        10543  +** removed from the local changeset (it was overridden). Or, if the
        10544  +** conflict resolution was "OMIT", then the local changeset is modified
        10545  +** to instead contain:
        10546  +**
        10547  +**           UPDATE t1 SET b = 'v2' WHERE a=1;
        10548  +**
        10549  +** Changes within the local changeset are rebased as follows:
        10550  +**
        10551  +** <dl>
        10552  +** <dt>Local INSERT<dd>
        10553  +**   This may only conflict with a remote INSERT. If the conflict 
        10554  +**   resolution was OMIT, then add an UPDATE change to the rebased
        10555  +**   changeset. Or, if the conflict resolution was REPLACE, add
        10556  +**   nothing to the rebased changeset.
        10557  +**
        10558  +** <dt>Local DELETE<dd>
        10559  +**   This may conflict with a remote UPDATE or DELETE. In both cases the
        10560  +**   only possible resolution is OMIT. If the remote operation was a
        10561  +**   DELETE, then add no change to the rebased changeset. If the remote
        10562  +**   operation was an UPDATE, then the old.* fields of change are updated
        10563  +**   to reflect the new.* values in the UPDATE.
        10564  +**
        10565  +** <dt>Local UPDATE<dd>
        10566  +**   This may conflict with a remote UPDATE or DELETE. If it conflicts
        10567  +**   with a DELETE, and the conflict resolution was OMIT, then the update
        10568  +**   is changed into an INSERT. Any undefined values in the new.* record
        10569  +**   from the update change are filled in using the old.* values from
        10570  +**   the conflicting DELETE. Or, if the conflict resolution was REPLACE,
        10571  +**   the UPDATE change is simply omitted from the rebased changeset.
        10572  +**
        10573  +**   If conflict is with a remote UPDATE and the resolution is OMIT, then
        10574  +**   the old.* values are rebased using the new.* values in the remote
        10575  +**   change. Or, if the resolution is REPLACE, then the change is copied
        10576  +**   into the rebased changeset with updates to columns also updated by
        10577  +**   the conflicting remote UPDATE removed. If this means no columns would 
        10578  +**   be updated, the change is omitted.
        10579  +** </dl>
        10580  +**
        10581  +** A local change may be rebased against multiple remote changes 
        10582  +** simultaneously. If a single key is modified by multiple remote 
        10583  +** changesets, they are combined as follows before the local changeset
        10584  +** is rebased:
        10585  +**
        10586  +** <ul>
        10587  +**    <li> If there has been one or more REPLACE resolutions on a
        10588  +**         key, it is rebased according to a REPLACE.
        10589  +**
        10590  +**    <li> If there have been no REPLACE resolutions on a key, then
        10591  +**         the local changeset is rebased according to the most recent
        10592  +**         of the OMIT resolutions.
        10593  +** </ul>
        10594  +**
        10595  +** Note that conflict resolutions from multiple remote changesets are 
        10596  +** combined on a per-field basis, not per-row. This means that in the 
        10597  +** case of multiple remote UPDATE operations, some fields of a single 
        10598  +** local change may be rebased for REPLACE while others are rebased for 
        10599  +** OMIT.
        10600  +**
        10601  +** In order to rebase a local changeset, the remote changeset must first
        10602  +** be applied to the local database using sqlite3changeset_apply_v2() and
        10603  +** the buffer of rebase information captured. Then:
        10604  +**
        10605  +** <ol>
        10606  +**   <li> An sqlite3_rebaser object is created by calling 
        10607  +**        sqlite3rebaser_create().
        10608  +**   <li> The new object is configured with the rebase buffer obtained from
        10609  +**        sqlite3changeset_apply_v2() by calling sqlite3rebaser_configure().
        10610  +**        If the local changeset is to be rebased against multiple remote
        10611  +**        changesets, then sqlite3rebaser_configure() should be called
        10612  +**        multiple times, in the same order that the multiple
        10613  +**        sqlite3changeset_apply_v2() calls were made.
        10614  +**   <li> Each local changeset is rebased by calling sqlite3rebaser_rebase().
        10615  +**   <li> The sqlite3_rebaser object is deleted by calling
        10616  +**        sqlite3rebaser_delete().
        10617  +** </ol>
        10618  +*/
        10619  +typedef struct sqlite3_rebaser sqlite3_rebaser;
        10620  +
        10621  +/*
        10622  +** CAPI3REF: Create a changeset rebaser object.
        10623  +** EXPERIMENTAL
        10624  +**
        10625  +** Allocate a new changeset rebaser object. If successful, set (*ppNew) to
        10626  +** point to the new object and return SQLITE_OK. Otherwise, if an error
        10627  +** occurs, return an SQLite error code (e.g. SQLITE_NOMEM) and set (*ppNew) 
        10628  +** to NULL. 
        10629  +*/
        10630  +SQLITE_API int sqlite3rebaser_create(sqlite3_rebaser **ppNew);
        10631  +
        10632  +/*
        10633  +** CAPI3REF: Configure a changeset rebaser object.
        10634  +** EXPERIMENTAL
        10635  +**
        10636  +** Configure the changeset rebaser object to rebase changesets according
        10637  +** to the conflict resolutions described by buffer pRebase (size nRebase
        10638  +** bytes), which must have been obtained from a previous call to
        10639  +** sqlite3changeset_apply_v2().
        10640  +*/
        10641  +SQLITE_API int sqlite3rebaser_configure(
        10642  +  sqlite3_rebaser*, 
        10643  +  int nRebase, const void *pRebase
        10644  +); 
        10645  +
        10646  +/*
        10647  +** CAPI3REF: Rebase a changeset
        10648  +** EXPERIMENTAL
        10649  +**
        10650  +** Argument pIn must point to a buffer containing a changeset nIn bytes
        10651  +** in size. This function allocates and populates a buffer with a copy
        10652  +** of the changeset rebased rebased according to the configuration of the
        10653  +** rebaser object passed as the first argument. If successful, (*ppOut)
        10654  +** is set to point to the new buffer containing the rebased changset and 
        10655  +** (*pnOut) to its size in bytes and SQLITE_OK returned. It is the
        10656  +** responsibility of the caller to eventually free the new buffer using
        10657  +** sqlite3_free(). Otherwise, if an error occurs, (*ppOut) and (*pnOut)
        10658  +** are set to zero and an SQLite error code returned.
        10659  +*/
        10660  +SQLITE_API int sqlite3rebaser_rebase(
        10661  +  sqlite3_rebaser*,
        10662  +  int nIn, const void *pIn, 
        10663  +  int *pnOut, void **ppOut 
        10664  +);
        10665  +
        10666  +/*
        10667  +** CAPI3REF: Delete a changeset rebaser object.
        10668  +** EXPERIMENTAL
        10669  +**
        10670  +** Delete the changeset rebaser object and all associated resources. There
        10671  +** should be one call to this function for each successful invocation
        10672  +** of sqlite3rebaser_create().
        10673  +*/
        10674  +SQLITE_API void sqlite3rebaser_delete(sqlite3_rebaser *p); 
        10675  +
 10094  10676   /*
 10095  10677   ** CAPI3REF: Streaming Versions of API functions.
 10096  10678   **
 10097  10679   ** The six streaming API xxx_strm() functions serve similar purposes to the 
 10098  10680   ** corresponding non-streaming API functions:
 10099  10681   **
 10100  10682   ** <table border=1 style="margin-left:8ex;margin-right:8ex">
 10101  10683   **   <tr><th>Streaming function<th>Non-streaming equivalent</th>
 10102  10684   **   <tr><td>sqlite3changeset_apply_strm<td>[sqlite3changeset_apply] 
        10685  +**   <tr><td>sqlite3changeset_apply_strm_v2<td>[sqlite3changeset_apply_v2] 
 10103  10686   **   <tr><td>sqlite3changeset_concat_strm<td>[sqlite3changeset_concat] 
 10104  10687   **   <tr><td>sqlite3changeset_invert_strm<td>[sqlite3changeset_invert] 
 10105  10688   **   <tr><td>sqlite3changeset_start_strm<td>[sqlite3changeset_start] 
 10106  10689   **   <tr><td>sqlite3session_changeset_strm<td>[sqlite3session_changeset] 
 10107  10690   **   <tr><td>sqlite3session_patchset_strm<td>[sqlite3session_patchset] 
 10108  10691   ** </table>
 10109  10692   **
................................................................................
 10190  10773     ),
 10191  10774     int(*xConflict)(
 10192  10775       void *pCtx,                   /* Copy of sixth arg to _apply() */
 10193  10776       int eConflict,                /* DATA, MISSING, CONFLICT, CONSTRAINT */
 10194  10777       sqlite3_changeset_iter *p     /* Handle describing change and conflict */
 10195  10778     ),
 10196  10779     void *pCtx                      /* First argument passed to xConflict */
        10780  +);
        10781  +SQLITE_API int sqlite3changeset_apply_v2_strm(
        10782  +  sqlite3 *db,                    /* Apply change to "main" db of this handle */
        10783  +  int (*xInput)(void *pIn, void *pData, int *pnData), /* Input function */
        10784  +  void *pIn,                                          /* First arg for xInput */
        10785  +  int(*xFilter)(
        10786  +    void *pCtx,                   /* Copy of sixth arg to _apply() */
        10787  +    const char *zTab              /* Table name */
        10788  +  ),
        10789  +  int(*xConflict)(
        10790  +    void *pCtx,                   /* Copy of sixth arg to _apply() */
        10791  +    int eConflict,                /* DATA, MISSING, CONFLICT, CONSTRAINT */
        10792  +    sqlite3_changeset_iter *p     /* Handle describing change and conflict */
        10793  +  ),
        10794  +  void *pCtx,                     /* First argument passed to xConflict */
        10795  +  void **ppRebase, int *pnRebase,
        10796  +  int flags
 10197  10797   );
 10198  10798   SQLITE_API int sqlite3changeset_concat_strm(
 10199  10799     int (*xInputA)(void *pIn, void *pData, int *pnData),
 10200  10800     void *pInA,
 10201  10801     int (*xInputB)(void *pIn, void *pData, int *pnData),
 10202  10802     void *pInB,
 10203  10803     int (*xOutput)(void *pOut, const void *pData, int nData),
................................................................................
 10227  10827   SQLITE_API int sqlite3changegroup_add_strm(sqlite3_changegroup*, 
 10228  10828       int (*xInput)(void *pIn, void *pData, int *pnData),
 10229  10829       void *pIn
 10230  10830   );
 10231  10831   SQLITE_API int sqlite3changegroup_output_strm(sqlite3_changegroup*,
 10232  10832       int (*xOutput)(void *pOut, const void *pData, int nData), 
 10233  10833       void *pOut
        10834  +);
        10835  +SQLITE_API int sqlite3rebaser_rebase_strm(
        10836  +  sqlite3_rebaser *pRebaser,
        10837  +  int (*xInput)(void *pIn, void *pData, int *pnData),
        10838  +  void *pIn,
        10839  +  int (*xOutput)(void *pOut, const void *pData, int nData),
        10840  +  void *pOut
 10234  10841   );
 10235  10842   
 10236  10843   
 10237  10844   /*
 10238  10845   ** Make sure we can call this stuff from C++.
 10239  10846   */
 10240  10847   #ifdef __cplusplus

Changes to src/style.c.

   541    541     ** creation of the submenu until the end so that we can add elements
   542    542     ** to the submenu while generating page text.
   543    543     */
   544    544     cgi_destination(CGI_HEADER);
   545    545     if( nSubmenu+nSubmenuCtrl>0 ){
   546    546       int i;
   547    547       if( nSubmenuCtrl ){
   548         -      cgi_printf("<form id='f01' method='GET' action='%R/%s'>", g.zPath);
          548  +      @ <form id='f01' method='GET' action='%R/%s(g.zPath)'>
          549  +      @ <input type='hidden' name='udc' value='1'>
   549    550       }
   550    551       @ <div class="submenu">
   551    552       if( nSubmenu>0 ){
   552    553         qsort(aSubmenu, nSubmenu, sizeof(aSubmenu[0]), submenuCompare);
   553    554         for(i=0; i<nSubmenu; i++){
   554    555           struct Submenu *p = &aSubmenu[i];
   555    556           if( p->zLink==0 ){
................................................................................
   831    832   
   832    833   /*
   833    834   ** WEBPAGE: builtin
   834    835   ** URL:  builtin/FILENAME
   835    836   **
   836    837   ** Return the built-in text given by FILENAME.  This is used internally 
   837    838   ** by many Fossil web pages to load built-in javascript files.
          839  +**
          840  +** If the id= query parameter is present, then Fossil assumes that the
          841  +** result is immutable and sets a very large cache retention time (1 year).
   838    842   */
   839    843   void page_builtin_text(void){
   840    844     Blob out;
   841    845     const char *zName = P("name");
   842    846     const char *zTxt = 0;
          847  +  const char *zId = P("id");
          848  +  int nId;
   843    849     if( zName ) zTxt = builtin_text(zName);
   844    850     if( zTxt==0 ){
   845    851       cgi_set_status(404, "Not Found");
   846    852       @ File "%h(zName)" not found
   847    853       return;
   848    854     }
   849    855     if( sqlite3_strglob("*.js", zName)==0 ){
   850    856       cgi_set_content_type("application/javascript");
   851    857     }else{
   852    858       cgi_set_content_type("text/plain");
          859  +  }
          860  +  if( zId && (nId = (int)strlen(zId))>=8 && strncmp(zId,MANIFEST_UUID,nId)==0 ){
          861  +    g.isConst = 1;
          862  +  }else{
          863  +    etag_check(0,0);
   853    864     }
   854    865     blob_init(&out, zTxt, -1);
   855    866     cgi_set_content(&out);
   856         -  g.isConst = 1;
   857    867   }
   858    868   
   859    869   
   860    870   /*
   861    871   ** WEBPAGE: test_env
   862    872   **
   863    873   ** Display CGI-variables and other aspects of the run-time
................................................................................
   868    878     int i;
   869    879     int showAll;
   870    880     char zCap[30];
   871    881     static const char *const azCgiVars[] = {
   872    882       "COMSPEC", "DOCUMENT_ROOT", "GATEWAY_INTERFACE",
   873    883       "HTTP_ACCEPT", "HTTP_ACCEPT_CHARSET", "HTTP_ACCEPT_ENCODING",
   874    884       "HTTP_ACCEPT_LANGUAGE", "HTTP_AUTHENICATION",
   875         -    "HTTP_CONNECTION", "HTTP_HOST", "HTTP_IF_NONE_MATCH",
          885  +    "HTTP_CONNECTION", "HTTP_HOST",
          886  +    "HTTP_IF_NONE_MATCH", "HTTP_IF_MODIFIED_SINCE",
   876    887       "HTTP_USER_AGENT", "HTTP_REFERER", "PATH_INFO", "PATH_TRANSLATED",
   877    888       "QUERY_STRING", "REMOTE_ADDR", "REMOTE_PORT",
   878    889       "REMOTE_USER", "REQUEST_METHOD",
   879    890       "REQUEST_URI", "SCRIPT_FILENAME", "SCRIPT_NAME", "SERVER_PROTOCOL",
   880    891       "HOME", "FOSSIL_HOME", "USERNAME", "USER", "FOSSIL_USER",
   881    892       "SQLITE_TMPDIR", "TMPDIR",
   882    893       "TEMP", "TMP", "FOSSIL_VFS",

Changes to src/tag.c.

   513    513           blob_append_sql(&sql,
   514    514             "%s"
   515    515             "  AND event.type GLOB '%q'"
   516    516             "  AND blob.rid IN ("
   517    517                       " SELECT rid FROM tagxref"
   518    518                       "  WHERE tagtype>0 AND tagid=%d"
   519    519                       ")"
   520         -          " ORDER BY event.mtime DESC",
          520  +          " ORDER BY event.mtime DESC /*sort*/",
   521    521             timeline_query_for_tty(), zType, tagid
   522    522           );
   523    523           db_prepare(&q, "%s", blob_sql_text(&sql));
   524    524           blob_reset(&sql);
   525    525           print_timeline(&q, nFindLimit, 79, 0);
   526    526           db_finalize(&q);
   527    527         }
................................................................................
   699    699     login_anonymous_available();
   700    700     @ <h2>Check-ins with non-propagating tags:</h2>
   701    701     db_prepare(&q,
   702    702       "%s AND blob.rid IN (SELECT rid FROM tagxref"
   703    703       "                     WHERE tagtype=1 AND srcid>0"
   704    704       "                       AND tagid IN (SELECT tagid FROM tag "
   705    705       "                                      WHERE tagname GLOB 'sym-*'))"
   706         -    " ORDER BY event.mtime DESC",
          706  +    " ORDER BY event.mtime DESC /*sort*/",
   707    707       timeline_query_for_www()
   708    708     );
   709    709     www_print_timeline(&q, 0, 0, 0, 0, 0);
   710    710     db_finalize(&q);
   711    711     @ <br />
   712    712     style_footer();
   713    713   }

Changes to src/tar.c.

   644    644     }
   645    645     tarball_of_checkin(rid, &tarball, zName, pInclude, pExclude);
   646    646     glob_free(pInclude);
   647    647     glob_free(pExclude);
   648    648     blob_write_to_file(&tarball, g.argv[3]);
   649    649     blob_reset(&tarball);
   650    650   }
          651  +
          652  +/*
          653  +** Check to see if the input string is of the form:
          654  +**
          655  +**        checkin-name/filename.ext
          656  +**
          657  +** In other words, check to see if the input contains a single '/'
          658  +** character that separates a valid check-in name from a filename.
          659  +**
          660  +** If the condition is true, return the check-in name and set the
          661  +** input string to be the filename.
          662  +**
          663  +** If the condition is false, return NULL
          664  +*/
          665  +char *tar_uuid_from_name(char **pzName){
          666  +  char *zName = *pzName;
          667  +  int i, n;
          668  +  for(i=n=0; zName[i]; i++){
          669  +    if( zName[i]=='/' ){
          670  +      if( n==0 ) n = i;
          671  +      else return 0;
          672  +    }
          673  +  }
          674  +  if( n==0 ) return 0;
          675  +  if( zName[n+1]==0 ) return 0;
          676  +  zName[n] = 0;
          677  +  *pzName = fossil_strdup(&zName[n+1]);
          678  +  return zName;
          679  +}
   651    680   
   652    681   /*
   653    682   ** WEBPAGE: tarball
   654    683   ** URL: /tarball
   655    684   **
   656    685   ** Generate a compressed tarball for the check-in specified by the "r"
   657    686   ** query parameter.  Return that compressed tarball as the HTTP reply
   658    687   ** content.
          688  +**
          689  +** The r= and name= query parameters can be specified as extensions to the
          690  +** URI.  Example, the following URIs are all equivalent:
          691  +**
          692  +**      /tarball/release/xyz.tar.gz
          693  +**      /tarball?r=release&name=xyz.tar.gz
          694  +**      /tarball/xyz.tar.gz?r=release
          695  +**      /tarball?name=release/xyz.tar.gz
   659    696   **
   660    697   ** Query parameters:
   661    698   **
   662    699   **   name=NAME[.tar.gz]  The base name of the output file.  The default
   663    700   **                       value is a configuration parameter in the project
   664    701   **                       settings.  A prefix of the name, omitting the
   665    702   **                       extension, is used as the top-most directory name.
   666    703   **
   667    704   **   r=TAG               The check-in that is turned into a compressed tarball.
   668    705   **                       Defaults to "trunk".  This query parameter used to
   669    706   **                       be called "uuid" and "uuid" is still accepted for
   670         -**                       backwards compatibility.  If omitted, the default
   671         -**                       check-in name is "trunk".
          707  +**                       backwards compatibility.  If the name= query parameter
          708  +**                       contains one "/" character then the part before the /
          709  +**                       is the TAG and the part after the / is the true name.
          710  +**                       If no TAG is specified by any of the above means, then
          711  +**                       "trunk" is used as the default.
   672    712   **
   673    713   **   in=PATTERN          Only include files that match the comma-separate
   674    714   **                       list of GLOB patterns in PATTERN, as with ex=
   675    715   **
   676    716   **   ex=PATTERN          Omit any file that match PATTERN.  PATTERN is a
   677    717   **                       comma-separated list of GLOB patterns, where each
   678    718   **                       pattern can optionally be quoted using ".." or '..'.
................................................................................
   689    729     Glob *pExclude = 0;           /* The compiled ex= glob pattern */
   690    730     Blob tarball;                 /* Tarball accumulated here */
   691    731     const char *z;
   692    732   
   693    733     login_check_credentials();
   694    734     if( !g.perm.Zip ){ login_needed(g.anon.Zip); return; }
   695    735     load_control();
   696         -  zName = mprintf("%s", PD("name",""));
   697         -  nName = strlen(zName);
          736  +  zName = fossil_strdup(PD("name",""));
   698    737     z = P("r");
   699    738     if( z==0 ) z = P("uuid");
          739  +  if( z==0 ) z = tar_uuid_from_name(&zName);
   700    740     if( z==0 ) z = "trunk";
   701    741     g.zOpenRevision = zRid = fossil_strdup(z);
   702    742     nRid = strlen(zRid);
   703    743     zInclude = P("in");
   704    744     if( zInclude ) pInclude = glob_create(zInclude);
   705    745     zExclude = P("ex");
   706    746     if( zExclude ) pExclude = glob_create(zExclude);
          747  +  nName = strlen(zName);
   707    748     if( nName>7 && fossil_strcmp(&zName[nName-7], ".tar.gz")==0 ){
   708    749       /* Special case:  Remove the ".tar.gz" suffix.  */
   709    750       nName -= 7;
   710    751       zName[nName] = 0;
   711    752     }else{
   712    753       /* If the file suffix is not ".tar.gz" then just remove the
   713    754       ** suffix up to and including the last "." */
................................................................................
   729    770     /* Compute a unique key for the cache entry based on query parameters */
   730    771     blob_init(&cacheKey, 0, 0);
   731    772     blob_appendf(&cacheKey, "/tarball/%z", rid_to_uuid(rid));
   732    773     blob_appendf(&cacheKey, "/%q", zName);
   733    774     if( zInclude ) blob_appendf(&cacheKey, ",in=%Q", zInclude);
   734    775     if( zExclude ) blob_appendf(&cacheKey, ",ex=%Q", zExclude);
   735    776     zKey = blob_str(&cacheKey);
          777  +  etag_check(ETAG_HASH, zKey);
   736    778   
   737    779     if( P("debug")!=0 ){
   738    780       style_header("Tarball Generator Debug Screen");
   739    781       @ zName = "%h(zName)"<br />
   740    782       @ rid = %d(rid)<br />
   741    783       if( zInclude ){
   742    784         @ zInclude = "%h(zInclude)"<br />

Changes to src/th_main.c.

   747    747   ** "useTclStubs"     = USE_TCL_STUBS
   748    748   ** "tclStubs"        = FOSSIL_ENABLE_TCL_STUBS
   749    749   ** "tclPrivateStubs" = FOSSIL_ENABLE_TCL_PRIVATE_STUBS
   750    750   ** "json"            = FOSSIL_ENABLE_JSON
   751    751   ** "markdown"        = FOSSIL_ENABLE_MARKDOWN
   752    752   ** "unicodeCmdLine"  = !BROKEN_MINGW_CMDLINE
   753    753   ** "dynamicBuild"    = FOSSIL_DYNAMIC_BUILD
          754  +** "mman"            = USE_MMAN_H
   754    755   ** "see"             = USE_SEE
   755    756   **
   756    757   ** Specifying an unknown feature will return a value of false, it will not
   757    758   ** raise a script error.
   758    759   */
   759    760   static int hasfeatureCmd(
   760    761     Th_Interp *interp,
................................................................................
   827    828       rc = 1;
   828    829     }
   829    830   #endif
   830    831   #if defined(FOSSIL_DYNAMIC_BUILD)
   831    832     else if( 0 == fossil_strnicmp( zArg, "dynamicBuild\0", 13 ) ){
   832    833       rc = 1;
   833    834     }
          835  +#endif
          836  +#if defined(USE_MMAN_H)
          837  +  else if( 0 == fossil_strnicmp( zArg, "mman\0", 5 ) ){
          838  +    rc = 1;
          839  +  }
   834    840   #endif
   835    841   #if defined(USE_SEE)
   836    842     else if( 0 == fossil_strnicmp( zArg, "see\0", 4 ) ){
   837    843       rc = 1;
   838    844     }
   839    845   #endif
   840    846     else if( 0 == fossil_strnicmp( zArg, "markdown\0", 9 ) ){

Changes to src/th_tcl.c.

   129    129   #    elif defined(__APPLE__)
   130    130   #      ifndef TCL_LIBRARY_NAME
   131    131   #        define TCL_LIBRARY_NAME "libtcl8.6.dylib\0"
   132    132   #      endif
   133    133   #      ifndef TCL_MINOR_OFFSET
   134    134   #        define TCL_MINOR_OFFSET (8)
   135    135   #      endif
          136  +#    elif defined(__FreeBSD__)
          137  +#      ifndef TCL_LIBRARY_NAME
          138  +#        define TCL_LIBRARY_NAME "libtcl86.so\0"
          139  +#      endif
          140  +#      ifndef TCL_MINOR_OFFSET
          141  +#        define TCL_MINOR_OFFSET (7)
          142  +#      endif
   136    143   #    else
   137    144   #      ifndef TCL_LIBRARY_NAME
   138    145   #        define TCL_LIBRARY_NAME "libtcl8.6.so\0"
   139    146   #      endif
   140    147   #      ifndef TCL_MINOR_OFFSET
   141    148   #        define TCL_MINOR_OFFSET (8)
   142    149   #      endif

Changes to src/timeline.c.

    89     89     }
    90     90   }
    91     91   
    92     92   /*
    93     93   ** Allowed flags for the tmFlags argument to www_print_timeline
    94     94   */
    95     95   #if INTERFACE
    96         -#define TIMELINE_ARTID    0x00001  /* Show artifact IDs on non-check-in lines */
    97         -#define TIMELINE_LEAFONLY 0x00002  /* Show "Leaf" but not "Merge", "Fork" etc */
    98         -#define TIMELINE_BRIEF    0x00004  /* Combine adjacent elements of same obj */
    99         -#define TIMELINE_GRAPH    0x00008  /* Compute a graph */
   100         -#define TIMELINE_DISJOINT 0x00010  /* Elements are not contiguous */
   101         -#define TIMELINE_FCHANGES 0x00020  /* Detail file changes */
   102         -#define TIMELINE_BRCOLOR  0x00040  /* Background color by branch name */
   103         -#define TIMELINE_UCOLOR   0x00080  /* Background color by user */
   104         -#define TIMELINE_FRENAMES 0x00100  /* Detail only file name changes */
   105         -#define TIMELINE_UNHIDE   0x00200  /* Unhide check-ins with "hidden" tag */
   106         -#define TIMELINE_SHOWRID  0x00400  /* Show RID values in addition to UUIDs */
   107         -#define TIMELINE_BISECT   0x00800  /* Show supplimental bisect information */
   108         -#define TIMELINE_COMPACT  0x01000  /* Use the "compact" view style */
   109         -#define TIMELINE_VERBOSE  0x02000  /* Use the "detailed" view style */
   110         -#define TIMELINE_MODERN   0x04000  /* Use the "modern" view style */
   111         -#define TIMELINE_COLUMNAR 0x08000  /* Use the "columns view style */
   112         -#define TIMELINE_VIEWS    0x0f000  /* Mask for all of the view styles */
   113         -#define TIMELINE_NOSCROLL 0x10000  /* Don't scroll to the selection */
   114         -#define TIMELINE_FILEDIFF 0x20000  /* Show File differences, not ckin diffs */
           96  +#define TIMELINE_ARTID    0x000001  /* Show artifact IDs on non-check-in lines */
           97  +#define TIMELINE_LEAFONLY 0x000002  /* Show "Leaf" but not "Merge", "Fork" etc */
           98  +#define TIMELINE_BRIEF    0x000004  /* Combine adjacent elements of same obj */
           99  +#define TIMELINE_GRAPH    0x000008  /* Compute a graph */
          100  +#define TIMELINE_DISJOINT 0x000010  /* Elements are not contiguous */
          101  +#define TIMELINE_FCHANGES 0x000020  /* Detail file changes */
          102  +#define TIMELINE_BRCOLOR  0x000040  /* Background color by branch name */
          103  +#define TIMELINE_UCOLOR   0x000080  /* Background color by user */
          104  +#define TIMELINE_FRENAMES 0x000100  /* Detail only file name changes */
          105  +#define TIMELINE_UNHIDE   0x000200  /* Unhide check-ins with "hidden" tag */
          106  +#define TIMELINE_SHOWRID  0x000400  /* Show RID values in addition to UUIDs */
          107  +#define TIMELINE_BISECT   0x000800  /* Show supplimental bisect information */
          108  +#define TIMELINE_COMPACT  0x001000  /* Use the "compact" view style */
          109  +#define TIMELINE_VERBOSE  0x002000  /* Use the "detailed" view style */
          110  +#define TIMELINE_MODERN   0x004000  /* Use the "modern" view style */
          111  +#define TIMELINE_COLUMNAR 0x008000  /* Use the "columns" view style */
          112  +#define TIMELINE_CLASSIC  0x010000  /* Use the "classic" view style */
          113  +#define TIMELINE_VIEWS    0x01f000  /* Mask for all of the view styles */
          114  +#define TIMELINE_NOSCROLL 0x100000  /* Don't scroll to the selection */
          115  +#define TIMELINE_FILEDIFF 0x200000  /* Show File differences, not ckin diffs */
   115    116   #endif
   116    117   
   117    118   /*
   118    119   ** Hash a string and use the hash to determine a background color.
   119    120   */
   120    121   char *hash_color(const char *z){
   121    122     int i;                       /* Loop counter */
................................................................................
   273    274     }
   274    275     if( tmFlags & TIMELINE_COLUMNAR ){
   275    276       zStyle = "Columnar";
   276    277     }else if( tmFlags & TIMELINE_COMPACT ){
   277    278       zStyle = "Compact";
   278    279     }else if( tmFlags & TIMELINE_VERBOSE ){
   279    280       zStyle = "Verbose";
          281  +  }else if( tmFlags & TIMELINE_CLASSIC ){
          282  +    zStyle = "Classic";
   280    283     }else{
   281    284       zStyle = "Modern";
   282    285     }
   283    286     zDateFmt = P("datefmt");
   284    287     if( zDateFmt ) dateFormat = atoi(zDateFmt);
   285    288     if( tmFlags & TIMELINE_GRAPH ){
   286    289       pGraph = graph_init();
................................................................................
   463    466         db_bind_int(&bisectQuery, ":rid", rid);
   464    467         if( db_step(&bisectQuery)==SQLITE_ROW ){
   465    468           @ <b>%s(db_column_text(&bisectQuery,1))</b>
   466    469           @ (%d(db_column_int(&bisectQuery,0)))
   467    470         }
   468    471         db_reset(&bisectQuery);
   469    472       }
   470         -    drawDetailEllipsis = (tmFlags & TIMELINE_COMPACT)!=0;
          473  +    drawDetailEllipsis = (tmFlags & (TIMELINE_COMPACT))!=0;
   471    474       db_column_blob(pQuery, commentColumn, &comment);
   472    475       if( tmFlags & TIMELINE_COMPACT ){
   473    476         @ <span class='timelineCompactComment' data-id='%d(rid)'>
   474    477       }else{
   475    478         @ <span class='timeline%s(zStyle)Comment'>
   476    479       }
   477         -    if( (tmFlags & TIMELINE_VERBOSE)!=0 ){
          480  +    if( (tmFlags & TIMELINE_CLASSIC)!=0 ){
   478    481         if( zType[0]=='c' ){
   479    482           hyperlink_to_uuid(zUuid);
   480    483           if( isLeaf ){
   481    484             if( db_exists("SELECT 1 FROM tagxref"
   482    485                           " WHERE rid=%d AND tagid=%d AND tagtype>0",
   483    486                           rid, TAG_CLOSED) ){
   484    487               @ <span class="timelineLeaf">Closed-Leaf:</span>
................................................................................
   547    550           @ <td class="timelineDetailCell">
   548    551         }
   549    552       }
   550    553       if( tmFlags & TIMELINE_COMPACT ){