Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
43 check-ins for the month beginning 2022-08-01 by user wyoung
Following month ↑|
2022-08-30
| ||
| 01:49 | Embroidered the "make container-run" target to make it more convenient. ... (check-in: bc09e28a user: wyoung tags: trunk) | |
|
2022-08-29
| ||
| 18:21 | The container doc bit on raw sockets now covers the other three Busybox utilities we left out previously. Today's removal of ping and traceroute merely completes the set; it wasn't complete in itself. ... (check-in: b429bd71 user: wyoung tags: trunk) | |
| 18:07 | Clarified the points in §5.2.1 of the Docker container build doc regarding the reason why the server parent process runs as root. ... (check-in: c2eaa60d user: wyoung tags: trunk) | |
| 17:54 | Researched, tested, and documented the set of "docker create --cap-drop" options we can add to strip away unnecessary root privileges inside the container without harming normal operation. Belt-and-suspenders: if any bad actor ever got into the container with root privileges, this would help prevent them from affecting anything outside the container. Added that set to the "make container-run" target so they get applied by default in the easy case. ... (check-in: f715add9 user: wyoung tags: trunk) | |
| 17:32 | Removed ping and traceroute commands from the Docker container. They require raw sockets support, which means if anyone broke into the container and managed a root privilege escalation, they could do a wide array of bad things on any network the container is bound to. ... (check-in: f00a88f8 user: wyoung tags: trunk) | |
| 16:01 | Polishing pass on §5.2 of the container build doc, "Why Chroot?" ... (check-in: e9860314 user: wyoung tags: trunk) | |
|
2022-08-28
| ||
| 17:58 | Clarified the parent process user ID vs the child process in the explanation of how the chroot feature interacts with the custom user feature of the Docker container. ... (check-in: f9ddd38e user: wyoung tags: trunk) | |
| 17:52 | Made a better distinction between bind mounts and Docker volumes in the new Docker section of the build doc. ... (check-in: 958a6af9 user: wyoung tags: trunk) | |
|
2022-08-17
| ||
| 05:30 | Removed a digression in the gitusers doc about Fossil's new clone-and-open mechanisms. That got moved to the ckout-workflows doc quite some time back, and we already point to it from that same section. There's no reason for the redundancy. Also cleaned up some grammar and typos while in there. ... (check-in: f43eaf01 user: wyoung tags: trunk) | |
|
2022-08-16
| ||
| 11:05 | Changed the "fossil server --user" flag's argument back to "admin" from "fossil" for the container: I was confusing the Unix user name with the default Fossil repo user name. The new "adduser fossil" stuff doesn't help here; we still want it to be called "admin". ... (check-in: 72d820f3 user: wyoung tags: trunk) | |
| 11:04 |
ARM build fixes for the container:
| |
| 09:39 | Minor fixes to the Docker container build process ... (check-in: 454397b0 user: wyoung tags: trunk) | |
| 07:14 | URL fix necessitated from the Dockerfile.in rename ... (check-in: 2f67bf94 user: wyoung tags: trunk) | |
| 07:03 | Carved the Docker container image size down still further by stripping out all but two of the stock skins (d* so we get default and darkmode) and packing Fossil and BusyBox with UPX. ... (check-in: e20d044c user: wyoung tags: trunk) | |
| 06:49 | Fixed an Obi Wan error in the new Fossil version prefix stuff in auto.def: it was extracting the first 13 characters of the hash, not the first 12. ... (check-in: 7ecd23e0 user: wyoung tags: trunk) | |
|
2022-08-15
| ||
| 23:21 | Added the container-image and container-run top-level build targets to manage dependencies better and to auto-version the build products. ... (check-in: 67386c75 user: wyoung tags: trunk) | |
| 23:07 | Put the "--user fossil" bit back into the fossil server command for the container. Just ran into a situations where it's still needed. ... (check-in: 4c8cc804 user: wyoung tags: trunk) | |
| 22:13 | Polishing pass on the container repo storage section of the build docs. ... (check-in: 3e332637 user: wyoung tags: trunk) | |
| 22:02 | Changed several of the Docker environment variables to build arguments so the user an override them at build time rather than container creation time, and documented them in build.wiki. Using this new mechanism to pull the Fossil source tarball in such a way that we can use the Docker artifact cache without getting stale builds. You can now pass one of the new build args to force the old behavior if you want it. This required generating Dockerfile from Dockerfile.in at configure time, to inject the current Fossil checkin ID. (This busts the Docker cache when the source tree changes.) ... (check-in: f9384383 user: wyoung tags: trunk) | |
| 15:32 | Adding the BusyBox tarball to the container image with an ADD command rather than wget to avoid triggering GitHub throttling. Unlike the Fossil repo URL, it has a version number baked into it, so it's safe to give it over to Docker's caching behavior. ... (check-in: d06d7c46 user: wyoung tags: trunk) | |
| 14:48 | Noted the container size shrinkage in the fossil-v-git doc ... (check-in: f21de33e user: wyoung tags: trunk) | |
| 14:42 | The container now builds Busybox from source so we can remove utilities that are unhelpful inside the container. We leave a lot behind for expansion (e.g. the runit init system, crond, inetd…) but we remove things that have no possible justification, such as modprobe. We remove everything from /bin that's a shell builtin (echo, printf, test…) and we replace a few BusyBox commands (sha[13]sum) with wrapper shell scripts that call Fossil builtins. We cap that off by adding a "sqlite3" wrapper that calls "fossil sqlite3 --no-repository", just for fun. All together, this trims about a meg of fat. ... (check-in: 953f367e user: wyoung tags: trunk) | |
|
2022-08-14
| ||
| 19:53 | The chown -R bit added to the Dockerfile touches /jail/bin/fossil, which causes "docker build" to promote it back into a new layer, nearly doubling the container size. Doing a chown now only on two directories, restoring it to its sub-9M size. ... (check-in: 00cc9c3e user: wyoung tags: trunk) | |
| 19:42 |
Fossil's chroot feature drops root permissions based on file ownership,
but since the container was built with everything-root, its HTTP hit
handling children would run as whatever host-side UID/GID pair you used
for file ownership. What happened next was complex.
If you let the container create the repo internally, it would be owned as root, so it would drop root permissions for…root! This isn't super-bad, since Fossil is presumed secure and is double-jailed besides. The risk is, if anyone works out an RCE for Fossil, they might be able to get it to create raw sockets or do various other types of escapes despite the double-jail dance. Attaching a Docker volume brings external permisssions into the container. We were recommending a "chown 0" command on the shared volume to make it similar to the in-container case, but that opens you to the same risks above. If you ignored this and used host-side UID/GID pairs, Fossil would then be left running under IDs that didn't exist internally, which could cause assorted weirdness. We're now creating an explicit "fossil" user/group pair inside the container and recommending that Docker volumes use these IDs for copied-in files to batten down something that shouldn't've been left flapping. Updated build.wiki to cover all this. ... (check-in: ba21bc0b user: wyoung tags: trunk) | |
| 18:48 | Moved the SIGTERM handler up before the "fossil server" HTTP hit handler. We had it clustered with the other signal() calls, but those are to handle signals intended to occur only during CGI processing. This one will normally occur while we're blocked, waiting for the HTTP hit to occur, so it had no useful effect where it was. ... (check-in: d3c55fe0 user: wyoung tags: trunk) | |
| 18:01 | Changed previous to call fossil_exit() instead of exit(3) so we close our databases before dying. ... (check-in: 7c857d22 user: wyoung tags: trunk) | |
| 17:59 | The parent process now handles SIGTERM with an explicit exit(3) call when its PID is 1, as when it's running as "fossil server" in a Docker container. Without this, the container host's shutdown process takes a long time because it's waiting on PID 1 to die and eventually has to time out and kill it. ... (check-in: 1d09e607 user: wyoung tags: trunk) | |
| 16:19 | Markup fix ... (check-in: cf149787 user: wyoung tags: trunk) | |
| 16:18 | Clarified the fact that the "docker cp" command is changing the name of the repository DB file. ... (check-in: f0b15a37 user: wyoung tags: trunk) | |
| 16:15 | Slight emphasis fix in previous ... (check-in: 1441c2e6 user: wyoung tags: trunk) | |
| 16:13 | Edit pass on §5.1 of build.wiki, fixing a number of unclear bits, particularly with regard to images vs containers. ... (check-in: e2b9114b user: wyoung tags: trunk) | |
|
2022-08-13
| ||
| 23:39 | Using the preceding --chroot fixes to make the Docker container serve the repo from /jail/museum/repo.fossil rather than from the chroot dir, /jail. This then allows us to mount a Docker volume at /jail/museum, which has an independent persistence from the container proper, so we can now rebuild the container without destroying the presumably precious repo. Updated build.wiki to track this change and document the lessons gleaned from doing all of this. ... (check-in: f76e762f user: wyoung tags: trunk) | |
| 22:15 | Moved the chdir() call within enter_chroot_jail() down below the new repo name canonicalization code to allow use of relative path names. Before, you had to give an absolute path to the repo, since we'd cd'd away from that directory before we started to validate the path. ... (check-in: e9462118 user: wyoung tags: trunk) | |
| 22:14 | Moved the setting of g.fJail flag into the repo = "/" case since it exists only to communicate the chroot status to --repolist mode. (This confirms the speculation in the prior commit's comment: the prior behavior existed to serve repolist mode only.) ... (check-in: 324d232c user: wyoung tags: trunk) | |
| 21:21 | Fixed the --chroot flag to "fossil server" and "fossil http" to allow it to work in conjunction with the single-repository case. Before, it blindly assumed --repolist mode. ... (check-in: 6f92ad99 user: wyoung tags: trunk) | |
|
2022-08-12
| ||
| 17:01 | Fixed pointless use of interwiki link in the new section 2.2 material of fossil-v-git. ... (check-in: 73c95307 user: wyoung tags: trunk) | |
|
2022-08-06
| ||
| 22:13 | Fixed a few stray parens in the new material in the fossil-v-git doc, left behind from a prior edit. ... (check-in: ea13701c user: wyoung tags: trunk) | |
| 22:08 | Typo fix ... (check-in: b628a883 user: wyoung tags: trunk) | |
| 20:30 | Fixed a problem in image naming in the new Docker container doc in build.wiki reported on the forum. ... (check-in: 509447a2 user: wyoung tags: trunk) | |
| 19:56 | Did away with the temporary src.tar.gz file in the new Docker container by streaming the output of wget straight into tar's stdin. This cuts the build time by about five seconds, presumably due to the saving from unnecessary file I/O. Also replaced the explicit "cd src" afterward with an out-of-tree build configuration, since it doesn't matter if we clutter the first stage's /tmp dir. ... (check-in: 289c9b50 user: wyoung tags: trunk) | |
| 19:34 | The build docs for "./configure --static" now reference the section further down on Docker, since you may need to use this indirection to get --static to produce something suitable. ... (check-in: 7bfd7413 user: wyoung tags: trunk) | |
| 04:24 |
Replaced Jan Nijtman's Dockerfile with a new one that does a 2-stage
build. The first stage runs atop Alpine Linux instead of Fedora,
reducing the initial build from ~635 MiB to about 16.
Rather than stop there, I then made it multi-stage, copying two key static binaries — Fossil and Busybox — over from the first stage into a fresh-from-scratch container and set it up to run the former jailed away from the latter. The result is under 9 MiB, and it's as secure as one can hope, given that it starts up in "PUBLIC" mode. The new build doesn't have all the extra features turned on that the old one did, but it seems right to build the container with Fossil in its default configuration. If you want something else, copy the Dockerfile, hack it, and make it do what you want instead. Having done all this, I replaced the one-off Dockerfile inline in section 5.0 of the build doc with a reference to this new Dockerfile and rewrote the section in terms of the new capabilities. Finally, this lets us brag on how small the container can be, as compared to the Gitlab-CE container. Before, we were comparing a standalone binary to the container, which wan't entirely fair. (The desire to produce such a container was the spark that kicked this project off.) ... (check-in: 77d603c6 user: wyoung tags: trunk) | |
|
2022-08-05
| ||
| 12:05 | Assorted improvements to the first few sections of the fossil-v-git doc, mainly in updating them to track changes to world facts and to clarify the presentation. ... (check-in: c7afd68b user: wyoung tags: trunk) | |