Fossil Forum

Repurposing a Fossil repository as a document

Repurposing a Fossil repository as a document

Repurposing a Fossil repository as a document

(1) By Richard Hipp (drh) on 2020-04-07 19:30:32 [source]

Over on the SQLite Forum there is a thread that began as a request for PDF of the SQLite documentation. Upon questioning the OP further, we discover that what he really wants is just a single file that captures all of the SQLite documentation, so that he doesn't have to lug around a directory hierarchy full of HTML, image files, and other accessory files needed by a website.

In other words, he wants a way to capture a large document consisting of a multitude of HTML pages into a single file on disk, in such a way that it can be easily viewed.

It occurs to me that Fossil could meet this need.

Proposed: The Web Document Format (WDF)

A WDF document is just a Fossil repository that contains the document as its only check-in. The document is accessed using the embedded documentation mechanism.

No changes or enhancements are needed in Fossil to prototype this idea. To create a new WDF just do something like these steps:

  1. fossil new somedoc.wdf
  2. fossil open somedoc.wdf
  3. fossil add *
  4. fossil commit -m "All the files that implement the document"
  5. fossil ui
  6. Make whatever style adjustments seem appropriate.
  7. fossil sql "PRAGMA page_size=512; VACUUM;"

Then you ship the somedoc.wdf as the document. To read the document, simply run "fossil ui somedoc.wdf". Presumably you configured the repository so that its home pages is the first page of the document when you did the configuration in step 6 above. So running "fossil ui" automatically brings up a web-browser viewing your document starting on the first page.

Advantages Of WDF

  • Documents use a human-readable format (HTML or Markdown).

  • A huge variety of tools are available for creating documents.

  • Adding full-text search to your document seems like it would be easy.

  • Your document could be written in HTML (of course), or it could be Markdown. Or a mix. Whatever works well for you.

  • Built-in document versioning!

Possible Enhancements To Fossil That Might Make This Work Better

  • Fossil should have a built-in skin that is specifically designed for WDF files. The WDF skin would work hard to hide all of the other functionality of Fossil, and make the "ui" command work like any other document viewer, without the VCS baggage cluttering up the screen.

  • For displaying a WDF document, it would be better to say just "fossil somedoc.wdf". In other words, just give the name of the document as the only argument to Fossil and endow Fossil with enough sense to recognize this as a WDF document and automatically kick off the usual "ui" processing.

  • It would be good to add a single command that automatically creates, populates, and configures the document. The idea is that if you have an existing HTML-based document in a subdiretory doc, you could just type: "fossil make-wdf doc mydoc.wdf" and it would automatically make the mydoc.wdf repository, fill it with content, and configure it as a document.

(2) By Richard Hipp (drh) on 2020-04-07 19:42:07 in reply to 1 [link] [source]

Here is a sample WDF document that contains the (latest snapshot) of the SQLite documentation:

  1. Download (16MB)
  2. Run "fossil ui sqlite-docs-20200407.wdf"

(3) By sean (jungleboogie) on 2020-04-07 20:24:07 in reply to 2 [link] [source]

Nice idea! Doesn't require pandoc or extra tools, but it would require Fossil and a web browser.

If you had not provided the wdf file, what steps are needed to make it?

Is it basically the .fossil of the repo with this new extension?

(4.1) By Larry Brasfield (LarryBrasfield) on 2020-04-07 20:37:26 edited from 4.0 in reply to 3 [link] [source]

If you had not provided the wdf file, what steps are needed to make it?

Steps 1-4 in Richard's initial post

Is it basically the .fossil of the repo with this new extension?

Yes, for some value of "basically". Some tools or other non-viewable stuff would be left out.

(5) By sean (jungleboogie) on 2020-04-07 20:37:44 in reply to 4.0 [link] [source]

Steps 1-7 in Richard's initial post

Well in the case of the sqlite documentation, I wouldn't create a new repo - I'd clone an existing repo.

(6) By Andreas Kupries (aku) on 2020-04-07 20:53:42 in reply to 1 [link] [source]

This feels to me to be running in the same space as various ebook formats.

Notably E-PUB. The link goes to the implementation section on the wikipedia page.

The description there is quite similar to what you wrote here.

It might be worthwhile to get a variety of .epub's and look at their structure (They are just zip files with html, other assets, and meta data in XML).

(7.1) By Larry Brasfield (LarryBrasfield) on 2020-04-08 23:37:33 edited from 7.0 in reply to 1 [link] [source]

Because it took some non-trivial time (for me) to interpret this step,

`6. Make whatever style adjustments seem appropriate.

, I offer this experience to guide that "seems appropriate" state of mind:


, as added to the repository and committed, is going to become the initial page served up by 'fossil ui' (or 'fossil server <somedoc.wdf>'), then in fossil's web ui, navigate to Admin / Configuration , and set the Index Page item to

Any other stylistic adjustments that seem appropriate, or setting project names, etc. as Fossil docs and books suggest, may also be done.

(8) By Offray (offray) on 2020-04-13 18:03:06 in reply to 1 [link] [source]

This seems like a pretty useful idea and is similar to what we are using for our Coronapedia, because we need a self contained documentation repository. In our approach we packaged Markdeep and Font Awesome as part of the repository.

The clickable WDF file launching a local server is a good idea, for "end users" of such documentation. It would become almost download and click. It also has advantages over alternative approaches like EPUB, particularly build in collaboration and versioning.

You may consider some issues:

  1. What happens if the repository includes unversioned files (raw images, PDF's and so on)? With this idea of simplicity in mind (download and run), wouldn't be good to provide a way to package such files in the repository? (as discussed in other thread and addressing the different namespaces raised there)

  2. Could Fossil package itself and/or can we have a WDF player (like Wolfram/Mathematica player)? I think that this is already reaching the extension capabilities mentioned in the unversioned files thread, and this maybe something not really needed, as we have now one line installers for Fossil in most common platforms, but for non technical people, installing Fossil can be tricky. I imagine something like the a way to package Fossil and the WDF in the same file, so people can download and click/launch, according to their platform. Other approach would be to distribute something like the Wolfram Player, that allows to read documents in the Computable Document Format. The advantage is that such player is just the Fossil binary in a click and install/run format.

  3. Could we have some config (yaml, toml, json) file for the WDF repository? So we could do something like fossil new mydoc.wdf from config.yaml that essentially would tell which directory to use, the default file to be used as the document "landing page", the repository title and description, if search is enabled by default and so on. This file could be part of the repository, internalizing the setup options that usually are defined via command shell for an easier setup and increased reproducibility (we make something like this in a special node inside Grafoscopio notebooks).

This could have important impacts for reproducible research and publishing. I use Fossil mostly for documentation (software source code is in SmalltalkHub and soon unfortunately on Git :-/ ) and I made a prototype of how to use Fossil to version interactive notebooks about the [Panama Papers as reproducible research]. The WDF would automatize this greatly and improve the overall user experience.

(9) By geburashka on 2020-04-17 05:49:18 in reply to 1 [link] [source]

I really like this idea. I think it could be a great contender as an alternative for simple (and maybe even not so simple) Word/OpenOffice docs - but much easier to edit and of course having history tracking, and even tickets etc.

I envision something like:

  1. double click the 'doc.wdf' file
    • the contents are checked out to folder named 'doc' along side the file
    • fossil ui is launched and browser opened to the main page
  2. make changes to the .md or .html etc files using whatever editor
  3. refresh the browser (maybe it can auto refresh on changes?)
  4. provide a 'Commit' button on the page or a simple shell/bat 'commit' script which will commit all changes with a 'wip' message or just the date.
    • the web ui could provide a box to enter the commit message
    • the button or script means (on Windows especially) super simple, easy and quick updates without opening terminals, navigating to the folder, and typing the commands (especially for non-technical folks).
  5. another button/script to push/pull the changes to/from the source (the URL of which could be stored in a file within the repo).

the fossil scripts could be generated by fossil, further reducing the technical load on document writers.