PATCH: Fossil Automatic Code Highlighting with Prism or HighightJS (and Line Numbering!)
(1.1) By llmII (lmartin92) on 2019-09-26 23:57:29 edited from 1.0 [source]
UPDATE: Fixed a few markdown styling issues. The code is now in a branch in the fossil repo syntax-hl-with-linenumbers.
This builds upon the work in code-in-pre branch by wyoung and brings a few features to the table.
First note, there will be 2 patches provided. Both will be against checkout 28b15b4816.
The first patch slightly modifies the work wyoung has done. The second (coming later) will dodge the use of an SQLITE query to attain the extension name to resolve one thing he mentioned he wasn't "wild about".
UPDATE: The branch follows the second path provided only.
Keep in mind that I'm writing this only minutes after checking in the first functional version of that feature. I have no reason to expect that it covers all cases. I'm also not wild about how it does a file name lookup in the Fossil SQL DB purely for this purpose; I was hoping Fossil had already gathered that info somehow above the HTML code I added to the Fossil source, but I couldn't see anywhere I could snag it, so I gave up and added my one-off SQL.
The features brought in enable syntax highlighting with line numbering. This will be mostly a repeat of an old post I made.
Patch introduces a new setting, syntax-hl. This enables the code in info.c to detect if the repo owner
wants to utilize syntax highlighting and as such passes on the responsibility from fossil to the owner of the
repo to provide a js file in the theme that will handle line numbering. This means fossil will bypass numbering
whatsoever if the setting is enabled, allowing a user provided javascript to take over.
You can refer to both of the following links for a how to use as how wyoung does and how I do are different but both likely work just fine.
The main key is to cause either fossil or your own web server to serve the js and/or css files from the same domain that the fossil content is hosted under.
The example from my prior post as referenced above as to how to enable the syntax-hl detection in your theme
still applies.
A full example of what to modify in the th1 code for a theme file is as follows:
added to header above <div class="header">:
<th1>
if {[setting "syntax-hl"]} {
html "<link rel='stylesheet' href='https://code.amlegion.org/agate.min.css'>\n"
html "<script src='https://code.amlegion.org/highlight.min.js'></script>\n"
html "<script src='https://code.amlegion.org/hljsln.js'></script>\n"
html "<script src='https://code.amlegion.org/hljsenable.js'></script>\n"
if {[getParameter "ln"] ne ""} {
html "<script src='https://code.amlegion.org/hljsenableln.js'></script>\n"
}
}
</th1>
Note that wyoung uses Prism and I use HighlightJS. I like HighlightJS because of its license and ease of use but albeit my examples pertain strictly to HighlightJS, it should work similarly for Prism. You have to enable something for Prism to do line numbering is all.
The added benefit however to using HighlightJS at the moment is that I've taken the line numbering extension for HighlightJS and modified it to do line numbering very much how fossil does it (the link formats for non-highlighting and highlighting enabled repos should remain similar or function between).
The current working one may be found at https://code.amlegion.org/hljsln.js and https://code.amlegion.org/hljsenable.js. The repository for the HighlightJS related stuff is at https://code.amlegion.org/hljs_line_numbers/.
If line numbering selection links do not work as per the old way fossil did it that is a bug and feel free to inform or submit patches. I'll continue improving the HighlightJS side of things as I have time. Someone will need to modify Prism's line numbering to support fossil style selection links if they wish to use it or just have to bare in mind that their selection links will not be compatible with the fossil way if they enable and utlize Prism for highlighting.
To see line numbering with syntax highlighting in action take a gander at https://code.amlegion.org/hljs_line_numbers/, and on the home page it provides some handy links at the bottom so as to see this in action, working properly.
I'm open to suggestions as to improving either/or/both patches for inclusion in Fossil.
(2) By llmII (lmartin92) on 2019-09-26 16:05:55 in reply to 1.0 [link] [source]
First Diff as below:
Index: src/db.c
==================================================================
--- src/db.c
+++ src/db.c
@@ -3600,10 +3600,15 @@
** A shell command used to launch your preferred
** web browser when given a URL as an argument.
** Defaults to "start" on windows, "open" on Mac,
** and "firefox" on Unix.
*/
+/*
+** SETTING: syntax-hl boolean default=off
+** This is a boolean set such that themes can use TH1 code
+** to detect if they should enable syntax highlighting.
+*/
/*
** Look up a control setting by its name. Return a pointer to the Setting
** object, or NULL if there is no such setting.
**
Index: src/info.c
==================================================================
--- src/info.c
+++ src/info.c
@@ -2338,27 +2338,24 @@
}
blob_to_utf8_no_bom(&content, 0);
zMime = mimetype_from_content(&content);
@ <blockquote>
if( zMime==0 ){
- const char *z, *zFileName, *zExt;
+ const char *z, *zFileName, *zExt, *zLanguage, *tmp;
z = blob_str(&content);
zFileName = db_text(0,
"SELECT name FROM mlink, filename"
" WHERE filename.fnid=mlink.fnid"
" AND mlink.fid=%d",
rid);
- zExt = zFileName ? strrchr(zFileName, '.') : 0;
- if( zLn ){
+ zExt = (tmp = strrchr(zFileName, '.')) == NULL ? "" : tmp+1;
+ zLanguage = strrchr(zFileName, '.') == NULL ? "" : "language-";
+ if( zLn && (db_get_boolean("syntax-hl", 0) == 0) ) {
output_text_with_line_numbers(z, zLn);
- }else if( zExt && zExt[1] ){
- @ <pre>
- @ <code class="language-%s(zExt+1)">%h(z)</code>
- @ </pre>
}else{
@ <pre>
- @ %h(z)
+ @ <code class="%s(zLanguage)%s(zExt)">%h(z)</code>
@ </pre>
}
}else if( strncmp(zMime, "image/", 6)==0 ){
@ <p>(file is %d(blob_size(&content)) bytes of image data)</i></p>
@ <p><img src="%R/raw/%s(zUuid)?m=%s(zMime)"></p>
(3) By llmII (lmartin92) on 2019-09-26 16:14:17 in reply to 1.0 [link] [source]
This one removes the need for an SQL query so as to pull the extension out of the file name. Second diff as below (this is actually the one I prefer):
Index: src/db.c
==================================================================
--- src/db.c
+++ src/db.c
@@ -3600,10 +3600,15 @@
** A shell command used to launch your preferred
** web browser when given a URL as an argument.
** Defaults to "start" on windows, "open" on Mac,
** and "firefox" on Unix.
*/
+/*
+** SETTING: syntax-hl boolean default=off
+** This is a boolean set such that themes can use TH1 code
+** to detect if they should enable syntax highlighting.
+*/
/*
** Look up a control setting by its name. Return a pointer to the Setting
** object, or NULL if there is no such setting.
**
Index: src/info.c
==================================================================
--- src/info.c
+++ src/info.c
@@ -2338,27 +2338,20 @@
}
blob_to_utf8_no_bom(&content, 0);
zMime = mimetype_from_content(&content);
@ <blockquote>
if( zMime==0 ){
- const char *z, *zFileName, *zExt;
+ const char *z, *zFileName, *zExt, *zLanguage, *tmp;
z = blob_str(&content);
- zFileName = db_text(0,
- "SELECT name FROM mlink, filename"
- " WHERE filename.fnid=mlink.fnid"
- " AND mlink.fid=%d",
- rid);
- zExt = zFileName ? strrchr(zFileName, '.') : 0;
- if( zLn ){
+ zFileName = blob_str(&downloadName);
+ zExt = (tmp = strrchr(zFileName, '.')) == NULL ? "" : tmp+1;
+ zLanguage = strrchr(zFileName, '.') == NULL ? "" : "language-";
+ if( zLn && (db_get_boolean("syntax-hl", 0) == 0) ) {
output_text_with_line_numbers(z, zLn);
- }else if( zExt && zExt[1] ){
- @ <pre>
- @ <code class="language-%s(zExt+1)">%h(z)</code>
- @ </pre>
}else{
@ <pre>
- @ %h(z)
+ @ <code class="%s(zLanguage)%s(zExt)">%h(z)</code>
@ </pre>
}
}else if( strncmp(zMime, "image/", 6)==0 ){
@ <p>(file is %d(blob_size(&content)) bytes of image data)</i></p>
@ <p><img src="%R/raw/%s(zUuid)?m=%s(zMime)"></p>
(4) By Warren Young (wyoung) on 2019-09-26 20:08:25 in reply to 1.0 [link] [source]
2 patches provided.
Unless your intent is that people apply your patches to their local Fossil instances and rebuild, you'll need to follow through with the contribution guide's steps. Until then, those of us with commit access — who have followed those steps — cannot inspect your patch with an eye to applying it to the fossil-scm.org/fossil repo for you, since to apply anything like what you've posted would basically be a violation of your copyright.
If these were trivial patches, such as to fix a type casting warning, then it wouldn't be an issue, since there is no creative value in the patch. My skim of your prose — having skipped over the code entirely, per above — says this is not a trivial change.
You may have interacted with other projects that do accept non-trivial drive-by patches on the mailing list or forum, but these are generally using a viral license so that it automatically attaches to such contributions, or they're run by people who don't care about the legal details. Neither applies here.
(5) By llmII (lmartin92) on 2019-09-26 21:02:54 in reply to 4 [link] [source]
I'm fairly certain in the past, at least on the mailing list, if not on the forum, I've explained I've signed a contributor agreement. As to following steps, wherein the contributors guide explains to send the patch to drh, I've done that as well.
So to make it exceedingly clear, possibly again - The code is provided under the BSD2 License as linked within the fossil repo. In addition the copyrights are, when commited, be it under a review branch or into fossil, or in any way touches fossil's repos, assigned as per the Contributors Agreement.
I've supplied the patches for review by the people who use fossil, and this forum, as I didn't have commit access at the time of development. Now I do and will update the top post with a link to that branch when I finally get it that done.
So it seems I qualify as:
@wyoung
us with commit access — who have followed those steps
So my code should be of no issue for you to peruse. To be safe, wait for it to arrive on a review/feature branch in a bit.
@wyoung
My skim of your prose — having skipped over the code entirely, per above — says this is not a trivial change
It is as trivial a change as your change was actually in that your change requires users to make modifications to themes built in to fossil just as mine does. I only add one knob that basically hints to fossil to not even attempt to number lines and let a javascript file that the user specifies (by modifying their theme) handle it. That said, fossil remains as simple as it was before in that the knob is off by default and one has to look to enable it. Fossil functions fairly much the exact same way as it did before without the knob when the knob is off. I haven't tried it but I would almost bet it even functions the same as before the knob and where a user patches the theme as per your instructions, with the exception that line numbering remains controlled by fossil and no syntax highlighting with line numbers, as it was with your changes. So it would seem that it is as simple as your changes, in that both these changes require a user's intervention to enable properly. You are free to disagree on this but I would ask you to point out how it is in fact non-trivial so that I may make an effort to make it trivial. I am not looking to make things complex.
(6) By Warren Young (wyoung) on 2019-09-26 21:19:00 in reply to 5 [link] [source]
I'm fairly certain in the past, at least on the mailing list, if not on the forum, I've explained I've signed a contributor agreement.
I sent my reply after checking the list of repo contributors via the /reports page and finding none from an lmartin92 user. You say you've sent in the contribution agreement and gotten a login, so why are you sending the patch to the forum instead of checking it in on a branch, per section 3.0 in the contribution guide?
Those of us without admin access on the Fossil repo can't check the list of actual users, as that's protected, on purpose. We can only inspect the list of users who have made changes to the repository. The two lists are mutually non-overlapping: you can have someone on either list and not the other. (Yes, you can have check-ins ascribed to a user without a login! Ask me how I know!)
Your declaration of license here in a forum post isn't sufficient. There's also a copyright assignment agreement, which needs to be on file in drh's office, which semi-outsiders like me also don't have access to.
You clear up all of this uncertainty by checking the change in. The very fact that you can do that proves you've got a copyright assignment on file.
It is as trivial a change as your change was actually
My vague sense is that your change is more substantial than both of mine on this topic, combined. But we'll see, once the necessary proofs are carried out. :)
(7) By llmII (lmartin92) on 2019-09-27 00:15:22 in reply to 6 [link] [source]
@wyoung
I sent my reply after checking the list of repo contributors via the /reports page and finding none from an lmartin92 user. You say you've sent in the contribution agreement and gotten a login, so why are you sending the patch to the forum instead of checking it in on a branch, per section 3.0 in the contribution guide?
Prior to posting this on the forum and emailing drh referencing the post I did not have a login. Thus the order of events are post, email, login acquisition, branch, commit. Section 3.0 didn't apply until I received my login, by which point I had already made the post. I can't comply with something I didn't know I would have in the future, and it was important for people to have a chance to test it and find ways to possibly break it that I haven't. I found an issue in the hljs line numbering modification I used this morning (that are not part of the commit and remain entirely separate from fossil) and I've been running the basis of the code submitted for about a year in production now and haven't been able to break it. It needed eyes, and I had no way to acquire any, and no one knew (or ignored) that this work was available for use prior. That said the check-in checklist part of the fossil contributor's guide was applied.
(Yes, you can have check-ins ascribed to a user without a login! Ask me how I know!)
I know this as well, I've done it in my own repos when my local username differed from the username I wanted.
Your declaration of license here in a forum post isn't sufficient.
One can declare a license for their code wherever, excepting where the rules of a place where they post their code mention the automatic application of a license (usually to the site owner) for usage in any manner they see fit in which case they at the very least dual license it if not have their license stripped by the TOS.
You clear up all of this uncertainty by checking the change in.
Done.
My vague sense is that your change is more substantial than both of mine on this topic, combined.
I await you to point out where it is non-trivial. Functionality of fossil fairly much remains the same unless you turn on the syntax-hl knob in which case you need to modify your themes appropriately. Your change needed a modification of themes as well to enable syntax highlighting. That's from what I draw my conclusion of that it is trivial. The other source of that conclusion is that the actual changes to the code in fossil are small (very few lines touched).