Fossil

Check-in [2ef70be2]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Reworked the documentation for email alerts to put all the information in a single document: a) Moved the "quick email setup" material from the forums.wiki document into `alerts.md`; b) linked to that moved material from its original location; c) greatly expanded the material previously present in the `alerts.md` document; c) merged the `emaildesign.md` document into `alerts.md`, so we don't have to link out to two separate documents which previously partially repeated each other. This pass also fixed a number of grammar errors and broken links.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256:2ef70be2f754941baba5d5dd849528079cd810ebac92817c9a7334f52995c653
User & Date: wyoung 2018-09-17 02:57:14
Context
2018-09-17
02:58
Updated the documentation permuted index to include a grammar fix checked in with the previous checkin. check-in: d8d979fe user: wyoung tags: trunk
02:57
Reworked the documentation for email alerts to put all the information in a single document: a) Moved the "quick email setup" material from the forums.wiki document into `alerts.md`; b) linked to that moved material from its original location; c) greatly expanded the material previously present in the `alerts.md` document; c) merged the `emaildesign.md` document into `alerts.md`, so we don't have to link out to two separate documents which previously partially repeated each other. This pass also fixed a number of grammar errors and broken links. check-in: 2ef70be2 user: wyoung tags: trunk
2018-09-16
21:14
Replaced the WIP and indefinite future paragraphs in www/alerts.md with definite statements now that we're approaching release. check-in: 05b40829 user: wyoung tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to www/alerts.md.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
































































































































































































































































































































































































































































































































































































Email Alerts
============

Email Alerts And Notifications
------------------------------

Beginning with version 2.7, Fossil supports the ability to send
email messages to subscribers alerting them to changes in the repository.
Subscribers can request an email notification of the following kinds
of changes:

  *  New check-ins
  *  Changes to any ticket
  *  Changes to any wiki page
  *  New forum posts
  *  Announcements

Subscribers can either elect to receive emails as soon as these events happen,
or they can receive a daily digest of the events instead.

Email alerts are sent by a [Fossil server](./server.wiki).  You must
have a server [set up to make use of email alerts](#setup).  Email
alerts do not currently work if you are only using Fossil from the
command line.

Users and Subscribers
---------------------

Fossil makes a distinction between "users" and "subscribers".  A
user is someone with a username and password — someone who can
log in.  A subscriber is someone who receives email alerts.  Users
can also be subscribers and subscribers can be users, but that does
not have to be the case.  It is possible to be a user without being
a subscriber and to be a subscriber without being a user.

In the repository database file, users are tracked with the USER table
and subscribers are tracked via the SUBSCRIBER table.

<a id="setup"></a>
Activating Email Alerts
-----------------------

Email alerts are turned off by default.  To activate them, log into
the Fossil server as an administrator and visit the 
[Admin/Notification](/setup_notification)
setup page. ([`/setup_notification`](/setup_notification))

Important:  Email alerts are configured using Admin/Notification, not
Admin/Email-Server.  The Email-Server setup screen is used to configure
a different subsystem within Fossil.

The Admin/Notification setup screen lets you configure how Fossil should
send email for alerts.  There are some required fields at the top of the
screen for elements such as the "From:" address for outgoing emails,
the URL for the Fossil server, and a nickname for the repository that
will appear in the "Subject:" line of outgoing emails.  But the key
setup parameter is the "Email Send Method".

Fossil supports multiple methods for sending email alerts:

  1.  Pipe the email message text into a command, such as `sendmail`.
  2.  Store email messages as individual files in a directory and let
      some other process set up by the administrator take care of
      reading and forwarding those files.
  3.  Store email messages as entries in an SQLite database where
      some external process and read and forward the emails.
  4.  Send emails to an SMTP Relay.
  5.  Send emails directly to the recipients via SMTP.

Fossil 2.7 does not support method (5). Work on supporting that method
is still in progress.

The self-hosting Fossil repository at <https://www.fossil-scm.org/> currently
uses method (3).  Outgoing email messages are added to an SQLite database
file.  A separate daemon process continously monitors that database file,
extracts email messages as they are added, and hands them off to 
"procmail" to be sent on to the final recipient.  The self-hosting
Fossil repository uses this technique rather than method (1) because
it is running inside of a restrictive chroot jail which is unable to
hand off messages to "procmail" directly.  The daemon that monitors the
email database is a [short TCL script](/file/tools/email-sender.tcl).
That daemon is started automatically by adding this line:

      /usr/bin/tclsh /home/www/fossil/email-sender.tcl &

To the `/etc/rc.local` file of the Ubuntu server that hosts the
repository.

After making necessary changes to the Admin/Notification page, test
those changes by clicking the "[Send Announcement](/announce)" link
at the top of the page.  Fill in your email address in the "To:"
line and a test message below, and press "Send Message" to verify that
outgoing email is working.

<a id="cap7"></a>
Once email notification is working, one must also adjust user permissions
to allow users to subscribe to email notification.  On the 
Setup/User page, under the permissions for each user, is a new capability
called "Email Alerts".  The corresponding capability letter is "7".
That new "7" capability must be enabled in order for
users to be able to become subscribers.  To allow anonymous passers-by
on the internet to subscribe, simply enable "Email Alerts" for the
"nobody" user category. To require that the user solve a simple CAPTCHA
first, add it to the "anonymous" user category instead.

Signing Up For Email Notification
---------------------------------

Users and/or anonymous passers-by can visit the 
[`/subscribe`](/subscribe) page to sign
up for email notification.

If your users are getting the following complaint from Fossil:

<blockquote>
  Use a different login with greater privilege than FOO to access
  /subscribe
</blockquote>

...then you forgot to [give capability 7](#cap7) to that user or to a
user category that the user is a member of.

After signing up, a single verification email
is sent.  The new subscriber must click a link on that email in order to
activate the subscription.

Subscription verification emails are only sent once.  This is a defense
against malicious robots that try to harass innocent internet users
by having subscription pages send multiple verification emails.
If the initial subscription verification does not go through correctly,
an administrator must intervene to reset the subscription.

Every subscriber has a long random hexadecimal security code that serves
as their password.  All email notifications contain a link back to the
Fossil server, incorporating this security code, which allows the 
subscriber to adjust their subscription options.

Administrator Activities
------------------------

The repository administrator has unlimited control over individual
subscriptions.  The "[List Subscribers](/subscribers)" button at the top
of the Setup/Notification screen gives a list of subscribers.  Clicking on
any subscriber link allows the administrator to adjust the subscription.

To unsubscribe, first select the "unsubscribe" checkbox, then press the
"Unsubscribe" button.

The "verified" checkbox determines whether or not an email address has
been verified.  This can be enabled or disabled manually by the
administrator.

Cloning, Syncing, and Backups
-----------------------------

The Setup/Notification settings are not replicated using clone or sync.
In a network of peer repositories, you only want one repository sending
email notifications.  If you were to replicate the email notification
settings to a separate repository, then subscribers would get multiple
notifications for each event, which would be bad.

However, the subscriber list can be synced for backup purposes.  Use the
[`fossil config pull subscriber`](/help?cmd=configuration) command to
pull the latest subscriber list from a server into a backup repository.
































































































































































































































































































































































































































































































































































































|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
# Email Alerts

## Overview

Beginning with version 2.7, Fossil can send email messages to
subscribers to alert them to changes in the repository:

  *  New [checkins](/help?cmd=ci)
  *  [Ticket](./tickets.wiki) changes
  *  [Wiki](./wikitheory.wiki) page changes
  *  New and edited [forum](./forum.wiki) posts
  *  Announcements

Subscribers can elect to receive emails as soon as these events happen,
or they can receive a daily digest of the events instead.

Email alerts are sent by a [Fossil server](./server.wiki), which must be
[set up](#quick) by the Fossil administrator to send email.

Email alerts do not currently work if you are only using Fossil from the
command line.

A bit of terminology: Fossil uses the terms "email alerts" and
"notifications" interchangeably. We stick to the former term in this
document except when referring to parts of the Fossil UI still using the
latter term.


## Setup Prerequisites

Much of this document describes how to set up Fossil's email alert
system. To follow this guide, you will need a Fossil UI browser window
open to the [Admin → Notification](/setup_notification) Fossil UI screen
on the the Fossil server that will be sending these email alerts, logged
in as a user with Admin capability. It is not possible to work on a
clone of the server's repository and push the configuration changes up
to that repo as an Admin user, [on purpose](#backup).

**Important:** Do not confuse that screen with Admin → Email-Server,
which sets up a different subsystem within Fossil. That feature is
related to this document's topic, but it is currently incomplete, so we
do not cover it at this time.

<a id="cd"></a>
You will also need a CLI window open with its working directory changed
to a checkout directory of the Fossil repository you are setting up to
send email.  If you don't `cd` to such a checkout directory first,
you'll need to add `-R /path/to/repo.fossil` to each `fossil` command
below to tell Fossil which repository you mean it to apply the command
to.

There are other prerequisites for email service, but since they vary
depending on the configuration you choose, we'll cover these inline
below.


<a id="quick"></a>
## Quick Email Service Setup

If you've already got a working Postfix, Exim, or Sendmail server on the
machine running your Fossil instance(s), and you aren't using Fossil's
`chroot` feature to wall Fossil off from the rest of the machine, it's
fairly simple to set up email alerts.

(Otherwise, skip [ahead](#advanced) to the sections on advanced email
service setup.)

This is our "quick setup" option even though setting up an SMTP mail
server is not trival, because there are many other reasons to have such
a server set up already: internal project email service, `cron`
notifications, server status monitoring notifications...

With that out of the way, the Fossil-specific steps are easy:

1.  Go to [Admin → Notification](/setup_notification) and fill out all
    of the **Required** fields:

    * **Canonical server URL** — Use the suggested URL

    * **"From" email address** — `forum-bounces@example.com` is
      traditional, but suit yourself

    * **Repository nickname** — See the suggested examples on the web page.

2.  Set "Email Send Method" to "Pipe to a command"

3.  Set the "Administrator email address" to a suitable valid email
    address on that machine.  It could be the same value you used for
    the "From" address above, or it could be a different value like
    `admin@example.com`.

Save your changes.

At the command line, say

      $ fossil set email-send-command

If that gives a blank value instead of `sendmail -ti`, say

      $ fossil set email-send-command "sendmail -ti"

to force the setting. That works around a [known
bug](https://fossil-scm.org/forum/forumpost/840b676410) which may be
squished by the time you read this.

If you're running Postfix or Exim, you might think that command is
wrong, since you aren't running Sendmail. These mail servers provide a
`sendmail` command for compatibility with software like Fossil that has
no good reason to care exactly which SMTP server implementation is
running at a given site. There may be other SMTP servers that also
provide a compatible `sendmail` command, in which case they may work
with Fossil using the same steps as above.

<a id="status"></a>
If you reload the Admin → Notification page, the Status section at the
top should show:

      Outgoing Email: Piped to command "sendmail -ti"
      Pending Alerts: 0 normal, 0 digest
      Subscribers:    0 active, 0 total

Before you move on to the next section, you might like to read up on
[some subtleties](#pipe) with the "pipe to a command" method that we did
not cover above.


<a id="usage"></a>
## Usage and Testing

Now that email service from Fossil is set up, you can test it and begin
using it.


<a id="sub" name="subscribe"></a>
### Subscribing to Alerts

In the Status output above, we saw that there are no subscribers, so the
next step is to add the first one.

Go to the `/subscribe` page on your Fossil instance to sign up for email
alerts.  At the very least, you will need to sign up for "Forum Posts"
and "Announcements" to complete the testing steps below.

If you're logged in with a Fossil repository user account and put the
same user name and email address into this forum as you used for your
user information under Admin → Users, Fossil will simply tie your alert
preferences to your login record, and the email address in your user's
Contact Info field will be considered already-verified.  Otherwise,
Fossil will create an alert-only record, and you will have to verify the
email address before Fossil will send alerts to it.

This shows a key aspect of the way Fossil's email alerts system works,
by the way: a user can be signed up for email alerts without having a
full-fledged Fossil user account. Only when both user names are the same
are the two records tied together under the hood. For more on this, see
[Users vs Subscribers below](#uvs).

If you are seeing the following complaint from Fossil:

<blockquote>
  Use a different login with greater privilege than FOO to access
  /subscribe
</blockquote>

...then the repository's administrator forgot to [give the Alerts capability](#cap7)
to that user or to a user category that the user is a member of.

After a subscriber signs up for alerts for the first time, a single
verification email is sent to that subscriber's given email address.
The new subscriber must click a link in that email in order to activate
the subscription.

Subscription verification emails are only sent once.  This is a defense
against malicious robots that try to harass innocent Internet users by
having subscription pages send multiple verification emails.  If the
initial subscription verification does not go through correctly, an
administrator must [intervene](#admin) to reset the subscription.

Every subscriber-only email address has a [long random hexadecimal
security code](#scode) that serves in place of a password.  All email
alerts contain a link in their footer back to the Fossil server,
incorporating this security code, which allows the subscriber to adjust
their subscription options.  If a user doesn't have any of those emails,
they can request a link via email by visiting the `/alerts` or
`/unsubscribe` page on the repository.

Those with Fossil repository logins can adjust their email alert
settings by visiting the `/alerts` page on the repository.  With the
default skin, you can get there by clicking the "Logout" link in the
upper right corner of any Fossil UI page then clicking the "Email
Alerts" link.  That link is also available via the Sitemap (`/sitemap`)
and via the default skin's hamburger menu (&#9776;).


<a id="unsub" name="unsubscribe"></a>
### Unsubscribing

To unsubscribe from alerts, visit the `/alerts` page on the repository,
click the "Unsubscribe" button, then check the "Unsbuscribe" checkbox to
verify your action and press the "Unsubscribe" button a second time.

This interlock is intended to prevent accidental unsubscription.


<a id="test"></a>
### Test Email Service

The easiest way to test email sending from Fossil is via the "[Send
Announcement](/announce)" link at the top of the "Email Notification
Setup" page.  Put your email address in the "To:" line and a test
message below, then press "Send Message" to verify that outgoing email
is working.

Another method is from the command line:

      $ fossil alerts test-message you@example.com --body README.md --subject Test

That should send you an email with "Test" in the subject line and the
contents of your project's `README.md` file in the body.

That command assumes that your project contains a "readme" file, but of
course it does, because you have followed the [Programming Style Guide
Checklist][cl], right? Right.

[cl]: https://sendgrid.com/blog/programming-style-guide-checklist/


<a id="cap7"></a>
### User Capabilities

Once email alerts are working, one must also adjust user permissions to
allow users to subscribe to email alerts.  In the capability list for
each user on the Admin → Users page is a new capability called "Email
Alerts".  The corresponding capability letter is "7", which you must
give to any user that needs to use the subscription setup pages,
`/subscribe` and `/alerts`.

To allow any passer-by on the Internet to subscribe, give the "Email
Alerts" capability to the "nobody" user category.  To require that a
person solve a simple CAPTCHA first, give that capability to the
"anonymous" user category instead.


<a id="first" name="frist"></a>
### First Post

I suggest taking the time to compose a suitable introductory message
especially for your project's forum, one which a new user would find
helpful.

Wait a few seconds, and you should receive an email alert with the
post's subject and body text in the email.


<a id="trouble"></a>
### Troubleshooting

If email alerts aren't working, there are several useful commands you
can give to figure out why.

(Be sure to [`cd` into a repo checkout directory](#cd) first!)

      $ fossil alerts status

This should give much the same information as you saw [above](#status).
One difference is that, since you've created a forum post, the
`pending-alerts` value should only be zero if you did in fact get the
requested email alert. If it's zero, check your mailer's spam folder. If
it's nonzero, continue with these troubleshooting steps.

      $ fossil backoffice

That forces Fossil to run its ["back office" process](./backoffice.md).
Its only purpose at the time of this writing is to push out alert
emails, but it might do other things later. Sometimes it can get stuck
and needs to be kicked. For that reason, you might want to set up a
crontab entry to make sure it runs occasionally.

      $ fossil alerts send

This should also kick off the backoffice processing, if there are any
pending alerts to send out.

      $ fossil alert pending

Show any pending alerts. The number of lines output here should equal
the [status output above](#status).

      $ fossil test-add-alerts f5900
      $ fossil alert send

Manually create an email alert and push it out immediately.

The `f` in the first command's final parameter means you're scheduling a
"forum" alert. The integer is the ID of a forum post, which you can find
by visiting `/timeline?showid` on your Fossil instance.

The second command above is necessary because the `test-add-alerts`
command doesn't kick off a backoffice run.

      $ fossil ale send

This only does the same thing as the final command above, rather than
send you an ale, as you might be hoping. Sorry.


<a id="advanced"></a>
## Advanced Email Setups

Fossil offers several methods of sending email:

  1.  Pipe the email message text into a command.
  2.  Store email messages as entries in a SQLite database.
  3.  Store email messages as individual files in a directory.
  4.  Send emails to an SMTP relay.
  5.  Send emails directly to the recipients via SMTP.

This wide range of options allows Fossil to talk to pretty much any
SMTP setup.

The first four options let Fossil delegate email handling to an existing
[MTA][mta] so that Fossil does not need to implement the [roughly two
dozen][mprotos] separate [RFCs][rfcs] required in order to properly
support SMTP email in this complex world we've built.  As well, this
design choice means you do not need to do duplicate configuration, such
as to point Fossil at your server's TLS certificate in order to support
users behind mail servers that require STARTTLS encryption.

[mprotos]: http://sqlite.1065341.n5.nabble.com/Many-ML-emails-going-to-GMail-s-SPAM-tp98685p98722.html
[rfcs]:    https://en.wikipedia.org/wiki/Request_for_Comments


<a id="pipe"></a>
### Method 1: Pipe to a Command

This is our ["quick setup" option](#quick) above, but there are some
details we ignored which we'll cover now.

Fossil pipes the email message in [RFC 822 format][rfc822] to the
standard input of the command you gave as the "Email Send Method",
defaulting to `sendmail -ti`. This constitutes a protocol between Fossil
and the SMTP [message transfer agent (MTA)][mta]. Any other MTA which
speaks the same protocol can be used in place of the most common
options: Sendmail, Exim, and Postfix.

The `-t` option tells the command to expect the list of email recipients
in a `To` header in the RFC 822 message presented on its standard input.
Without this option, the `sendmail` command expects to receive the
recipient list on the command line, but that's not possible with the
current design of this email sending method. Therefore, if you're
attempting to use a less common MTA which cannot parse the recipient
list from the `To` header in the email message, you might need to look
for a different MTA.

The `-i` option is only needed for MTAs that take a dot/period at the
beginning of a line of standard input text as "end of message." Fossil
doesn't attempt to escape such dots, so if the line wrapping happens to
occur such that a dot or period in an alert message is at the beginning
of a line, you'll get a truncated email message without this option.
Statistically, this will happen about once every 70 or so messages, so
it is important to give this option if your MTA treats leading dots on a
line this way.

<a id="msmtp"></a>
We believe the [`msmtp`][msmtp] SMTP client is compatible with this
protocol if you give it the `-t` option. To our knowledge, this remains
untested, but if it works, this would be a useful option on a server
hosting a Fossil repository which doesn't otherwise require a separate
SMTP server for other purposes.

It is probably also possible to configure [`procmail`][pmdoc] to work
with this protocol. If you know how to do it, a patch to this document
or a how-to on [the Fossil forum][ff] would be appreciated.

[ff]:     https://fossil-scm.org/forum/
[msmtp]:  https://marlam.de/msmtp/
[mta]:    https://en.wikipedia.org/wiki/Message_transfer_agent
[pmdoc]:  http://pm-doc.sourceforge.net/doc/
[rfc822]: https://www.w3.org/Protocols/rfc822/


<a id="db"></a>
### Method 2: Store in a Database

The self-hosting Fossil repository at <https://www.fossil-scm.org/>
currently uses this method rather than [the pipe method](#pipe) because
it is running inside of a restrictive [chroot jail][cj] which is unable
to hand off messages to the local MTA directly.

When you configure a Fossil server this way, it adds outgoing email
messages to a SQLite database file.  A separate daemon process can then
extract those messages for further disposition.

Fossil includes a copy of [the daemon](/file/tools/email-sender.tcl)
used on `fossil-scm.org`: it is just a short Tcl script that
continuously monitors this database for new messages and hands any that
it finds off to a local MTA using the same [pipe to MTA protocol](#pipe)
as above.

In this way, outbound email alerts escape the chroot jail without
requiring that we insert a separate MTA configuration inside that jail.
We only need to arrange that the same SQLite DB file be visible both
inside and outside the chroot jail, which we do by naming the database
file in the "Store Emails In This Database" setting under Admin →
Notification.  The Tcl script has this path hard-coded as
`/home/www/fossil/emailqueue.db`, but you will probably need to adjust
that for your local purposes.

This method may work with other similar technologies besides `chroot`:
Docker containers, LXC containers, BSD jails, Solaris zones, etc.

With suitable file share mappings, this method may even work with
virtual machine or distributed computing setups where the MTA and Fossil
servers are not on the same machine, though beware the [risk of DB
corruption][rdbc] if used with a file sharing technology that doesn't
use proper file locking.

You can start this Tcl script as a daemon automatically on most Unix and
Unix-like systems by adding the following line to the `/etc/rc.local`
file of the server that hosts the repository sending email alerts:

      /usr/bin/tclsh /home/www/fossil/email-sender.tcl &

[cj]:   https://en.wikipedia.org/wiki/Chroot
[rdbc]: https://www.sqlite.org/howtocorrupt.html#_filesystems_with_broken_or_missing_lock_implementations


<a id="dir"></a>
### Method 3: Store in a Directory

This method is functionally very similar to [the DB method](#db),
differing only in that messages are written to a directory in the
filesystem.  You should therefore read that section and make the minor
adjustments required by the storage method.

This method may work over a file sharing mechanism that doesn't do file
locking properly, as long as the reading process is somehow restricted
from reading a message file as it's being written.

It might be useful in testing and debugging to temporarily switch to
this method, since you can easily read the generated email messages
without needing to involve [an MTA][mta].


<a id="relay"></a>
### Method 4: SMTP Relay

In this configuration, the Fossil server contacts an open SMTP relay and
sends the messages to it. This method is only appropriate when:

1. You have a local MTA that doesn't accept [the pipe
   protocol](#pipe).

2. The MTA is willing to accept anonymous submissions, since Fossil
   currently has no way to authenticate itself to the MTA.  This is [an
   unsafe configuration][omr] in most cases, but some SMTP servers make
   an exception for connections coming from a `localhost` or LAN
   address, choosing to accept such submissions as inherently safe.

If you have a local MTA meeting criterion #1 but not #2, we'd suggest
using a more powerful SMTP client such as [msmtp](#msmtp) along with one
of the other methods above.

[omr]: https://en.wikipedia.org/wiki/Open_mail_relay


<a id="direct"></a>
### Method 5: Direct SMTP Send

As of Fossil 2.7, the code to support this method is incomplete, so you
cannot currently select it as an option in Admin → Notification.


<a id="uvs"></a>
## Users vs Subscribers

Fossil makes a distinction between "users" and "subscribers".  A user is
someone with a username and password: that is, someone who can log into
the Fossi repository.  A subscriber is someone who receives email
alerts.  Users can also be subscribers and subscribers can be users, but
that does not have to be the case.  It is possible to be a user without
being a subscriber and to be a subscriber without being a user.

In the repository database file, users are tracked with the `user` table
and subscribers are tracked via the `subscriber` table.


<a id="admin"></a>
## Administrator Activities

The "[List Subscribers](/subscribers)" button at the top of the Admin →
Notification screen gives a list of subscribers, which gives a Fossil
server administrator a lot of power over those subscriptions.

Clicking an email address in this subscriber list opens the same
`/alerts` page that the user can see for their own subscription, but
with more information and functionality than normal users get:

*  Subscription creation and modification timestamps.

*  The IP address the user had when they last made a change via either
   `/subscribe` or `/alert`.

*  The user's login name, if they are not [a mere subscriber](#uvs).  A
   Fossil Admin user is allowed to modify this, either to tie a
   subscription-only record to an existing Fossil user account or to
   break that tie.

*  The "Do not call" checkbox allows a Fossil Admin user to mark a given
   email address so that Fossil never sends email to that address.  This
   is distinct from unsubscribing that email address because it prevents
   Fossil from accepting a new subscription for that address.

*  The Verified checkbox is initially unchecked for subscriber-only
   email addresses until the user clicks the link in the verification
   email. This checkbox lets the Fossil Admin user manually verify the
   user, such as in the case where the verification email message got
   lost.  Unchecking this box does not cause another verification email
   to be sent.

This screen also allows a Fossil Admin user to perform other activities
on behalf of a subscriber which they could do themselves, such as to
[unsubscribe](#unsub) them.


<a id="backup"></a>
## Cloning, Syncing, and Backups

The Admin → Notification settings are not replicated using clone or
sync, and it is not possible to push such settings from one repository
to another.  In a network of peer repositories, you only want one
repository sending email alerts.  If you were to replicate the email
alert settings to a separate repository, then subscribers would get
multiple alerts for each event, which would be bad.

However, the subscriber list can be synced for backup purposes.  Use the
[`fossil config pull subscriber`](/help?cmd=configuration) command to
pull the latest subscriber list from a server into a backup repository.

The `push`, `export`, and `import` commands all work similarly.


<a id="pages" name="commands"></a>
## Controlling the Email Alert System

This section collects the list of Fossil UI pages and CLI commands that
control the email alert system, some of which have not been mentioned so
far:

Commands:

   *  The [`alerts`](/help?cmd=alerts) command
   *  The [`test-alert`](/help?cmd=test-alert) command
   *  The [`test-add-alerts`](/help?cmd=test-add-alerts) command

Web pages available to users and subscribers:

   *  The [`/subscribe`](/help?cmd=/subscribe) page
   *  The [`/alerts`](/help?cmd=/alerts) page
   *  The [`/unsubscribe`](/help?cmd=/unsubscribe) page
   *  The [`/contact_admin`](/help?cmd=/contact_admin) page

Administrator-only web pages:

   *  The [`/setup_notification`](/help?cmd=/setup_notification) page
   *  The [`/subscribers`](/help?cmd=/subscribers) page


<a id="design"></a>
## Design of Email Alerts

This section describes the low-level design of the email alert system in
Fossil.  This expands on the high-level administion focused material
above with minimal repetition.

This section assumes expert-level systems knowledge. If the material
above sufficed for your purposes, feel free to skip this section, which
runs to the end of this document.


<a id="datades"></a>
### Data Design

There are three new tables in the repository database, starting with
Fossil 2.7.  These tables are not created in new repositories by
default.  The tables only come into existence as needed when email
alerts are configured and used.


  *  <b>SUBSCRIBER</b> →
     The subscriber table records the email address for people who
     want to receive email notifications.  Each subscriber has a
     `subscriberCode` which is a random 32-byte blob that uniquely
     identifies the subscriber.  There are also fields to indicate
     what kinds of notifications the subscriber wishes to receive,
     whether or not the email address of the subscriber has been
     verified, etc.

  *  <b>PENDING\_ALERT</b> →
     The PENDING\_ALERT table contains records that define events
     about which alert emails might need to be sent.
     A pending\_alert always refers to an entry in the
     EVENT table.  The EVENT table is part of the standard schema
     and records timeline entries.  In other words, there is one
     row in the EVENT table for each possible timeline entry.  The
     PENDING\_ALERT table refers to EVENT table entries for which
     we might need to send alert emails.

  *  <b>EMAIL\_BOUNCE</b> →
     This table is intended to record email bounce history so that
     subscribers with excessive bounces can be turned off.  That
     logic has not yet been implemented so the EMAIL\_BOUNCE table
     is currently unused.

As pointed out above, ["subscribers" are distinct from "users"](#uvs).
The SUBSCRIBER.SUNAME field is the optional linkage between users and
subscribers.


<a id="stdout"></a>
### The "stdout" Method

The [list of mail sending methods](#advanced) above left out an
internal-only method called "stdout" which simply writes the text of the
email message on standard output.  The "stdout" method is used for
testing and debugging.  If you need something similar and can't modify
your local Fossil instance to use this method, you might temporarily
switch to [the "dir" method](#dir) instead.


<a id="msgfmt"></a>
### Message Format

The email messages generated by Fossil have a [well-formed
header][rfc822].  The downstream processing is expected to extract the
"To:", "From:", "Subject:" and whatever other attributes it needs from
the email header text.

These emails use the `text/plain` MIME type with the UTF-8 character
set.  We currently use a transfer encoding of `quoted-printable`, but
there is commented-out code in Fossil to switch to `base64` encoding,
which Fossil used in the early days leading up to the 2.7 release.

If you switch Fossil back to `base64` mode, you may want to build a
utility program that ships in the Fossil source tree named
["tools/decode-email.c"](/file/tools/decode-email.c) which can decode
these messages into a human-readable format.


<a id="inbound" name="bounces"></a>
### Dealing with Inbound Email

Inbound email messages — for example, bounces from failed alert emails —
should be relayed to the `fossil email inbound` command.  That command
is currently a no-op place-holder.  At some point, we will need to
design and write a bounce-message processing system for Fossil.


<a id="password" name="scode" name="verification"></a>
### Passwords vs Subscriber Codes

When anonymous passers-by on the Internet sign up for email alerts,
their email address must first be verified.  An email message is sent to
the address supplied inviting the user to click on a link.  The link
includes a pseudorandom 128-bit blob encoded as 32 hexadecimal digits,
which serves in place of a password for that email address.  (This is
stored in the database as `subscriber.subscriberCode`.) If anyone visits
the link, the email address is verified.

Knowledge of the `subscriberCode` is sufficient to control a
subscription.

Because this code is included in plain text in email alert messages, it
is not as secure as a separate password, but it has several virtues:

*   It is easier for the average subscriber to deal with in that they
    don't have to come up with yet another password and store it safely.

*   If the `subscriberCode` is stolen, the worst that can happen is that
    the thief can change that email address's subscription settings.
    Contrast a password which may be shared with other services, which
    then compromises those other services.

*   No PII other than the subscriber's email address is available to an
    attacker with the `subscriberCode`.  Nor can knowledge of the
    `subscriberCode` lead to a email flood or other annoyance attack, as
    far as I can see.

If the `subscriberCodes` for a Fossil repository are ever compromised,
new ones can be generated as follows:

        UPDATE subscriber SET subscriberCode=randomblob(32);

Since this then affects all new email alerts going out from Fossil, your
end users may never even realize that they're getting new codes, as long
as they don't click on the URLs in the footer of old alert messages.

With that in mind, a Fossil server administrator could choose to
randomize the `subscriberCodes` periodically, such as just before the
daily digest emails are sent out each day.

**Important:** All of the above is distinct from the passwords for users
with a Fossil repository login. Such users also have subscriber codes,
but those codes can only be used to modify the user's email alert
settings. That code cannot allow a user to log into the user's Fossil
repository account.


<a id="processing"></a>
### Internal Processing Flow

Almost all of the email alert code is found in the
[`src/alerts.c`](/file/src/alerts.c) source file.

When email alerts are enabled, a trigger is created in the schema
(`email_trigger1`) that adds a new entry to the `PENDING_ALERT` table
every time a row is added to the `EVENT` table.  During a `fossil
rebuild`, the `EVENT` table is rebuilt from scratch; since we do not
want users to get alerts for every historical check-in, the trigger is
disabled during `rebuild`.

Email alerts are sent out by the `alert_send_alerts()` function, which
is normally called automatically due to the `email-autoexec` setting,
which defaults to enabled. If that setting is disabled or if the user
simply wants to force email alerts to be sent immediately, they can give
a `fossil alert send` command, such as via a `cron` script.  Each time
this function is called, the alert messages are moved further down the
chain, so you cannot cause duplicate alerts by calling it too often.

Digests are handled by recording the time of the last digest in the
`email-last-digest` setting, and only sending a new digest if the
current time is one day or later after the last digest.

Individual emails are sent to each subscriber.  I (drh) ran tests and
found that I could send about 1200 emails/second, which is fast enough
that I do not need to resort to trying to notify multiple subscribers
with a single email.  Because each subscriber gets a separate email, the
system can include information in the email that is unique to the
subscriber, such as a link to the page to edit their subscription.  That
link includes the `subscriberCode`.

Deleted www/emaildesign.md.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
Design of Email Notification
============================

This document contains high-level design notes for the email
notification system in Fossil.  Use this document to get a better
understanding of how Fossil handles email notification, to help
with doing custom configurations, or to help contribute features.

This document assumes expert-level systems knowledge.  A separate
tutorial for setting up email notification by non-experts will be
generated once the email notification system stabilizes.

Email notification is under active development as of this writing
(2018-06-25).  Check back frequently for updates.

Data Design
-----------

There are three new tables in the repository database.  These tables
are not created in new repositories by default.  The tables only
come into existance if email notification is configured and used.


  *  <b>SUBSCRIBER</b> &rarr;
     The subscriber table records the email address for people who
     want to receive email notifications.  Each subscriber has a
     `subscriberCode` which is a random 32-byte blob that uniquely
     identifies the subscriber.  There are also fields to indicate
     what kinds of notifications the subscriber wishes to receive,
     whether or not the email address of the subscriber has been
     verified, etc.

  *  <b>PENDING\_ALERT</b> &rarr;
     The PENDING\_ALERT table contains records that define events
     about which notification emails might need to be sent.
     A pending\_alert always refers to an entry in the
     EVENT table.  The EVENT table is part of the standard schema
     and records timeline entries.  In other words, there is one
     row in the EVENT table for each possible timeline entry.  The
     PENDING\_ALERT table refers to EVENT table entries for which
     we might need to send notification emails.

  *  <b>EMAIL\_BOUNCE</b> &rarr;
     This table is intended to record email bounce history so that
     subscribers with excessive bounces can be turned off.  That
     logic has not yet been implemented so the EMAIL\_BOUNCE table
     is currently unused.

Note that "subscribers" are distinct from "users" in the USER table.
A "user" is someone who has a login and password.  A "subscriber" is
an email address that receives notification events.  Users can be
subscribers, and there is a SUBSCRIBER.SUNAME field that records
the linkage between users and subscribers.  But it is also possible
to be a user without being a subscriber, or to be a subscriber without
being a user.

Sending Email Messages
----------------------

Fossil expects to interact with an external [mail transfer agent][MTA].
There are currently three different methods for sending outbound
email messages from Fossil to the external mail agent:

  1.  <b>"pipe"</b> &rarr; Invoke an external command that accepts
      the email message on standard input.  This is useful if the
      host computer has a command like /usr/sbin/sendmail that will
      accept well-formed email messages from standard input and forward
      them to the appropriate destination.

  2.  <b>"db"</b> &rarr; Write outgoing email messages into an
      SQLite database file.  The self-hosting Fossil website uses
      this technique because Fossil runs inside a reduced-privilege
      chroot jail and cannot invoke commands like /usr/sbin/sendmail.
      A separate TCL script running outside of the jail monitors
      the email queue database and forwards email messages to the
      Postfix mail transfer agent.  There is an example TCL script in the
      [tools/email-sender.tcl](/file/tools/email-sender.tcl) file
      of the source tree that shows how this is done.

  3.  <b>"dir"</b> &rarr; Write outgoing email messages as individual
      files in a designated directory.  This might be useful for
      testing and debugging.

Internally, there is a fourth email sending method named "stdout"
which simply writes the text of the email message on standard output.
The "stdout" method is used for testing and debugging.

Perhaps we will add an "smtp" sending method in the future.  The
main problem with an "smtp" delivery method is that front-line Fossil
running inside the privilege jail would need to deal with all kinds
of errors from SMTP, such as unable to connect, or connection resets,
etc.  SMTP expects the sender to have the ability to retry, does it
not?

The emails transmitted have a well-formed header.  The downstream
processing is expected to extract the "To:", "From:", "Subject:" and
whatever other attributes it needs from the email header text.

All emails are text/plain and use a transfer-encoding of base64.

There is a utility command-line program named 
["tools/decode-email.c"](/file/tools/decode-email.c) in
the Fossil source tree.  If you compile this program, you can use it
to convert the base64 transfer-encoding into human-readable output for
testing and debugging.

[MTA]: https://en.wikipedia.org/wiki/Message_transfer_agent


Receiving Email Messages
------------------------

Inbound email messages (for example bounces from failed notification
emails) should be relayed to the "fossil email inbound" command.  That
command is currently a no-op place-holder.  At some point, we will need
to design and write a bounce-message processing system for Fossil.

Controlling The Setup
---------------------

Commands:

   *  The [email](/help?cmd=email) command

Web pages:

   *  The [/subscribe](/help?cmd=/subscribe) page
   *  The [/alerts](/help?cmd=/alerts) page
   *  The [/unsubscribe](/help?cmd=/unsubscribe) page
   *  The [/msgtoadmin](/help?cmd=/msgtoadmin) page

Web pages for administrators only:

   *  The [/setup_email](/help?cmd=/setup_email) page
   *  The [/subscribers](/help?cmd=/subscribers) page

Test command:

   *  The [test-alert](/help?cmd=test-alert) command
   *  The [test-add-alerts](/help?cmd=test-add-alerts) command

Email Address Verification
--------------------------

When anonymous passers-by on the internet sign up for email notifications,
their email address must first be verified.  An email message is sent to
the address supplied inviting the user to click on a link.  The link includes
the random 32-byte `subscriberCode` in hex.  If anyone visits the link, the
email address is verified.

There is no password.  Knowledge of the `subscriberCode` is sufficient to
control the subscription.  This is not a secure as a separate password,
but on the other hand it is easier for the average subscriber to deal
with in that they don't have to come up with yet another password.  Also,
even if the `subscriberCode` is stolen, the worst that can happens is that
the thief can change your subscription settings.  No PII (other than
the subscriber's email address) is available to an attacker with the
`subscriberCode`.  Nor can knowledge of the `subscriberCode` lead to a
email flood or other annoyance attack, as far as I can see.

If subscriberCodes are ever compromised, new ones can be generated
as follows:

        UPDATE subscriber SET subscriberCode=randomblob(32);

Perhaps the system be enhanced to randomize the
`subscriberCodes` periodically - say just before each daily digest
is sent out?

User Control Of Their Subscription
----------------------------------

If a user has a separate account with a login and password for
the repository, then their subscription is linked to their account.
On the /login page is a link to a page to control their subscription.

For users without logins, they can request a link to a page for
controling their subscription on the `/alerts` or `/unsubscribe` page.
The link is sent via email, and includes the `subscriberCode`.

Internal Processing Flow
------------------------

Almost all of the email notification code is found in the `src/email.c`
source file.

When email notifications are enabled, a trigger is created in the schema
(the `email_trigger1` trigger) that adds a new entry to the
`PENDING_ALERT` table every time a row is added to the `EVENT` table.
During a `fossil rebuild`, the `EVENT` table is rebuilt from scratch; since we
do not want users to get notifications for every historical check-in,
the trigger is disabled during `rebuild`.

Email notifications are sent out by the `email_send_alerts()` function.
This function is can be called by having a cron job invoke the
`fossil email exec` command.  Or, if the email-autoexec setting is
enabled, then `email_send_alerts()` is invoked automatically after each
successful webpage is generated.  The latter approach is used on the
Fossil self-hosting repository.  The `email_send_alerts()` function is
a no-op (obviously) if there are no pending events to be sent.

Digests are handled by recording the time of the last digest in the
`email-last-digest` setting, and only sending a new digest if the
current time is one day or later after the last digest.

Individual emails are sent to each subscriber.  I ran tests and found
that I could send about 1200 emails/second, which is fast enough that
I do not need to resort to trying to notify multiple subscribers with
a single email.  Because each subscriber gets a separate email, the
system can include information in the email that is unique to the
subscriber, such as a link to the page to edit their subscription.  That
link includes the `subscriberCode`., 

Other Notes
-----------

The `fossil configuration pull subscriber` command pulls down the content
of the `SUBSCRIBER` table.  This is intended to as a backup-only.  It
is not desirable to have two or more systems sending emails to the
same people for the same repository, as that would mean users would
receive duplicate emails.  Hence, the settings that control email 
notifications are not transmitted with the pull.  The `push`, `export`,
and `import` commands all work similarly.
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






























































































































































































































































































































































































































































Changes to www/forum.wiki.

95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
...
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
...
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
...
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
...
321
322
323
324
325
326
327
328
329
330
331
332

333
334

335
336
337
338
339
340
341

342
343
344
345
346
347
348
349
350
351
352






353
354
355
356
357
358
359
360





361

362
363
364
365
366
367


368

369
370
371
372
373
374
375
376

377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
     restrict posts to plain text only or to allow wild-west
     HTML-formatted MIME email. Fossil's lightweight markup language
     formatting features give you a middle path, providing your users
     enough formatting power to communicate complex ideas well without
     providing so much power as to risk
     [https://wonko.com/post/html-escaping | security problems].

  *  <b>Easy Notification Emails:</b> It is easy to configure Fossil to
     send email notifications of new posts to interested forum users via
     your Fossil server's existing
     [https://en.wikipedia.org/wiki/Message_transfer_agent | message
     transfer agent] (MTA): Postfix, Exim, Sendmail... Notification
     emails include the complete message content for the benefit of
     those that prefer to visit the forum only when they need to post
     something. Notifications are optional, and each user gets the
     choice of immediate or daily digest delivery.

  *  <b>Talks to Everyone:</b> Because Fossil delegates email handling
     to your existing MTA, it does not need to implement the
     [http://sqlite.1065341.n5.nabble.com/Many-ML-emails-going-to-GMail-s-SPAM-tp98685p98722.html
     | roughly two dozen]
     [https://en.wikipedia.org/wiki/Request_for_Comments | RFCs]
     required in order to properly support SMTP email in this complex
     world we've built.  As well, this design choice means you do not
     need to do duplicate configuration, such as to point Fossil at your
     server's TLS certificate private key in order to support users
     behind mail servers that require STARTTLS encryption.


<h2 id="setup">Setting up a Fossil Forum</h2>

<h3 id="caps">Capabilities</h3>

Fossil forums use the same role-based access control mechanism as
................................................................................
     showing the list of pending moderation tasks.

  *  <b>Supervise Forum</b> (<tt>6</tt>): User can grant or revoke
     <b>WriteTrusted</b> capability for other users. (Currently
     unimplemented.)

  *  <b>Email Alerts</b> (<tt>7</tt>): User can sign themselves up for
     email notifications, a.k.a. alerts.

By default, no Fossil user has permission to use the forums except for
users with Setup and Admin capabilities, which get these as part of the
large package of other capabilities they get.

For public Fossil repositories that wish to accept new users without
involving a human, go into Admin &rarr; Access and enable the "Allow
................................................................................

For either type of repository, you are likely to want to give at least
the WriteTrusted capability (4) to users in the <tt>developer</tt>
category. If you did not give the Read Forum capability (2) to
<tt>anonymous</tt> above, you should give <tt>developer</tt> that
capability here if you choose to give it capability 3 or 4.

If you want to use the email notification feature, by default only those
users in the Setup and Admin user categories can make use of it. Grant
the Email Alerts capability (7) to give others access to this feature.
Alternately, you can handle notification signups outside of Fossil, with
a Setup or Admin users manually signing users up via Admin &rarr;
Notification. You'll want to grant this capability to the
<tt>nobody</tt> user category if you want anyone to sign up without any
restrictions.  Give it to <tt>anonymous</tt> instead if you want the
user to solve a simple CAPTCHA before signing up. Or, give it to
<tt>reader</tt> or <tt>developer</tt> if you want only users with Fossil
logins to have this ability. (That's assuming you give one or both of
................................................................................
The remainder of this section summarizes the differences you're expected
to see when taking option #2.

The first thing is that you'll need to add something like the following
to the Header part of the skin to create the navbar link:

<verbatim>
  if {[anycap 234567] || [anoncap 2] || [anoncap 3]} {
    menulink /forum Forum
  }
</verbatim>

These rules say that any logged-in user with any forum-related
capability (2-7 inclusive, as of this writing) or an anonymous user with
read or write capability on the forum (2, 3) will see the "Forum" navbar
link, which just takes you to <tt>/forum</tt>.

The exact code you need here varies depending on which skin you're
using. Follow the style you see for the other navbar links.

The new forum feature also brings many new CSS styles to the table. If
................................................................................
project's Fossil-hosted forum. Fossil has a feature to solve this
problem which is probably less well known than it should be, and which
has been a feature of Fossil since April of 2011: Admin &rarr;
Login-Group. This allows one Fossil repository to recognize users
authorized on a different Fossil repository.


<h3 id="alerts">Email Notifications (Alerts)</h3>

The detailed information for this feature of Fossil is in two other
documents:


  *  [./alerts.md | Email Alerts]
  *  [./emaildesign.md | Email Notification Design]


You may want to study those in detail, since email is complicated. Some
of that complexity necessarily leaks out to the configuration of email
notification in Fossil.


<h4 id="quick">Quick Setup</h4>


However, if you've already got a working Postfix, Exim, or Sendmail
server on the machine running your Fossil instance(s), and you aren't
using Fossil's <tt>chroot</tt> feature to wall it off from the rest of
the machine, it's fairly simple to set up email notifications:

<ol>
    <li>Go to Admin &rarr; Notification and fill out all of the
    <b>Required</b> fields:</li>

    <ul>






        <li><b>Canonical server URL</b> — Use the suggested URL</li>
        <li><b>"From" email address</b> — <tt>forum-bounces@example.com</tt>
        is traditional, but do what you like</li>
        <li><b>Repository nickname</b> — See the suggested examples on
        the web page.</li>
    </ul>

    <li>Set "Email Send Method" to "Pipe to a command"</li>







    <li>Set the "Administrator email address" to a suitable email address.
    It could be the same value you used for the "From" address above,
    or it could be a different value like
    <tt>admin@example.com</tt>.</li>
</ol>



Save your changes.


<a id="cd"></a>For all of the following <tt>fossil</tt> CLI commands,
we're going to assume that you've <tt>cd</tt>'d into a checkout
directory for your repository on the central server.  Otherwise, we'd
need to add <tt>-R /path/to/repo.fossil</tt> to each of them.

Give this command:


<verbatim>
    $ fossil set email-send-command
</verbatim>

If that gives a blank value instead of <tt>sendmail -ti</tt>, say

<verbatim>
    $ fossil set email-send-command "sendmail -ti"
</verbatim>

to force the setting. That's a
[https://fossil-scm.org/forum/forumpost/840b676410 | known bug] which
should be squished soon.

This is the correct command even if you're running Postfix or Exim
rather than Sendmail. These mail servers (and perhaps others) provide a
<tt>sendmail</tt> command for this very sort of reason: it lets programs
send email without having to care exactly which server implementation is
running at a given site.

<a id="status"></a>If you reload the Email Notification Setup page, the
Status section at the top should show:

<verbatim>
    Outgoing Email: Piped to command "sendmail -ti"
    Pending Alerts: 0 normal, 0 digest
    Subscribers:    0 active, 0 total
</verbatim>


<h4 id="subscribe">Subscribe to Alerts</h4>

Above, we see that there are no subscribers, so the next step is to add
one.

Go to the <tt>/alerts</tt> page on your Fossil instance and sign your
regular Fossil user up for email alerts. (Alternate path: click the user
name or login/logout link in the upper right corner, depending on the
skin you're using, then go to "Email Alerts".) You will need "Forum
Posts" checked at the least for the testing steps below.

If you use the same user name and email address as you used for your
normal user login, Fossil will simply tie your alert preferences to your
login record, and the email address in your user's Contact Info field
will be considered already-verified. Otherwise, Fossil will create an
alert-only record, and you will have to verify the email address before
Fossil will send notifications to it.

This shows a key aspect of the way Fossil's email alerts system works,
by the way: a user can be signed up for email alerts without having a
full-fledged Fossil user account. Only when both user names are the same
are the two records tied together under the hood.


<h4 id="alert-test">Test the Email Subsystem</h4>

If you'd rather not create an inane "testing" post in your Fossil
instance just to force out an email alert, you can test the email
subsystem separately from the rest of the Fossil email alerts system
with the following command:

<verbatim>
    $ fossil alerts test-message you@example.com --body README.md --subject Test
</verbatim>

That should send you an email with "Test" in the subject line and the
contents of your project's <tt>README.md</tt> file in the body. If it
works, the file's contents may be truncated. I'm uncertain whether
that's due to an intentionally-small buffer size or if it's a bug.

That command assumes that your project [./foss-cklist.wiki | contains a
readme file]. Of course it does. Why would it not?


<h4 id="frist" name="first">First Post</h4>

Since you've already edited the skin per [#skin | the instructions
above], you can click the "Forum" link in the navbar and create a new
post. I suggest taking the time to compose a suitable introductory
message especially for your project's forum, one which a new user would
find helpful.

Wait a few seconds, and you should receive a notification email with the
post's subject and body text in the email.


<h4 id="trouble">Troubleshooting</h4>

If email alerts aren't working, there are several useful commands you
can give to figure out why.

(Be sure to [#cd | <tt>cd</tt> into a repo checkout directory] first!)

<verbatim>
    $ fossil alerts status
</verbatim>

This should give much the same information as you saw [#status | above].
One difference is that, since you've created a forum post, the
<tt>pending-alerts</tt> value should only be zero if you did in fact get
the requested email alert. If it's zero, check your mailer's spam
folder. If it's nonzero, continue with these troubleshooting steps.

<verbatim>
    $ fossil backoffice
</verbatim>

That forces Fossil to run its [./backoffice.md | "back office" process].
Its only purpose at the time of this writing is to push out alert
emails, but it might do other things later. Sometimes it can get stuck
and needs to be kicked. For that reason, you might want to set up a
crontab entry to make sure it runs occasionally.

<verbatim>
    $ fossil alerts send
</verbatim>

This should also kick off the backoffice processing, if there are any
pending alerts to send out.

<verbatim>
    $ fossil alert pending
</verbatim>

Show any pending alerts. The number of lines output here should equal
the [#status | status output above].

<verbatim>
    $ fossil test-add-alerts f5900
    $ fossil alert send
</verbatim>

Manually create an email alert and push it out immediately.

The <tt>f</tt> in the first command's final parameter means you're
scheduling a "forum" alert. The integer is the ID of a forum post, which
you can find by visiting <tt>/timeline?showid</tt> on your Fossil
instance.

The second command above is necessary because the
<tt>test-add-alerts</tt> command doesn't kick off a backoffice run.

<verbatim>
    $ fossil ale send
</verbatim>

This only does the same thing as the final command above, rather than
send you an ale, as you might be hoping. Sorry.


<h2 id="moderation">How Moderation Works</h2>

In this section, we're going to call all of the following a "forum
update:"








|
|
<
<
<
|
|
|
|
<
<
<
<
<
<
<
<
<
<
<







 







|







 







|


|







 







|





|







 







|

|
|
<
>
|
|
>



|


<
>

<
|
<
<

<
<
<
<
|
>
>
>
>
>
>
|
<
<
<
<
<

<
>
>
>
>
>

>
|
|
|
<
<

>
>
|
>

<
|
|
|

|
<
>
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







95
96
97
98
99
100
101
102
103



104
105
106
107











108
109
110
111
112
113
114
...
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
...
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
...
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
...
307
308
309
310
311
312
313
314
315
316
317

318
319
320
321
322
323
324
325
326
327

328
329

330


331




332
333
334
335
336
337
338
339





340

341
342
343
344
345
346
347
348
349
350


351
352
353
354
355
356

357
358
359
360
361

362
363



















































































































































364
365
366
367
368
369
370
     restrict posts to plain text only or to allow wild-west
     HTML-formatted MIME email. Fossil's lightweight markup language
     formatting features give you a middle path, providing your users
     enough formatting power to communicate complex ideas well without
     providing so much power as to risk
     [https://wonko.com/post/html-escaping | security problems].

  *  <b>Easy Email Alerts:</b> You can configure Fossil to
     [./alerts.md | send email alerts]. Forum post emails include the



     complete message content for the benefit of those that prefer to
     visit the forum only when they need to post something.  Alerts are
     optional, and each user gets the choice of immediate or daily
     digest delivery.













<h2 id="setup">Setting up a Fossil Forum</h2>

<h3 id="caps">Capabilities</h3>

Fossil forums use the same role-based access control mechanism as
................................................................................
     showing the list of pending moderation tasks.

  *  <b>Supervise Forum</b> (<tt>6</tt>): User can grant or revoke
     <b>WriteTrusted</b> capability for other users. (Currently
     unimplemented.)

  *  <b>Email Alerts</b> (<tt>7</tt>): User can sign themselves up for
     email alerts, a.k.a. notifications.

By default, no Fossil user has permission to use the forums except for
users with Setup and Admin capabilities, which get these as part of the
large package of other capabilities they get.

For public Fossil repositories that wish to accept new users without
involving a human, go into Admin &rarr; Access and enable the "Allow
................................................................................

For either type of repository, you are likely to want to give at least
the WriteTrusted capability (4) to users in the <tt>developer</tt>
category. If you did not give the Read Forum capability (2) to
<tt>anonymous</tt> above, you should give <tt>developer</tt> that
capability here if you choose to give it capability 3 or 4.

If you want to use the email alert feature, by default only those
users in the Setup and Admin user categories can make use of it. Grant
the Email Alerts capability (7) to give others access to this feature.
Alternately, you can handle alert signups outside of Fossil, with
a Setup or Admin users manually signing users up via Admin &rarr;
Notification. You'll want to grant this capability to the
<tt>nobody</tt> user category if you want anyone to sign up without any
restrictions.  Give it to <tt>anonymous</tt> instead if you want the
user to solve a simple CAPTCHA before signing up. Or, give it to
<tt>reader</tt> or <tt>developer</tt> if you want only users with Fossil
logins to have this ability. (That's assuming you give one or both of
................................................................................
The remainder of this section summarizes the differences you're expected
to see when taking option #2.

The first thing is that you'll need to add something like the following
to the Header part of the skin to create the navbar link:

<verbatim>
  if {[anycap 23456] || [anoncap 2] || [anoncap 3]} {, 
    menulink /forum Forum
  }
</verbatim>

These rules say that any logged-in user with any forum-related
capability (2-6 inclusive, as of this writing) or an anonymous user with
read or write capability on the forum (2, 3) will see the "Forum" navbar
link, which just takes you to <tt>/forum</tt>.

The exact code you need here varies depending on which skin you're
using. Follow the style you see for the other navbar links.

The new forum feature also brings many new CSS styles to the table. If
................................................................................
project's Fossil-hosted forum. Fossil has a feature to solve this
problem which is probably less well known than it should be, and which
has been a feature of Fossil since April of 2011: Admin &rarr;
Login-Group. This allows one Fossil repository to recognize users
authorized on a different Fossil repository.


<h3 id="alerts">Email Alerts (a.k.a. Notifications)</h3>

Internet email service has become rather complicated since its initial
simple and insecure implementation decades ago. Fossil's role in all of

this is rather small at the moment, but the details of the integration
are complex enough to justify [./alerts.md | a separate document].
(Fossil's email alerts system also gets used by subsystems other than
the Fossil forum feature.)

You may want to study those in detail, since email is complicated. Some
of that complexity necessarily leaks out to the configuration of email
alerts in Fossil.



<h2 id="access">Accessing the Forum</h2>


There are many paths to a repository's Fossil forum:







<ul>
  <li>
    <p>If you're using the default Fossil skin as shipped with Fossil
    2.7 or one updated to include the changes since 2.6 or prior, there
    is a Forum button in the navbar which appears for users with any of
    the forum-related user capabilities: 2 through 6 inclusive for those
    with repository logins, or caps 2 and 3 for users without a user
    account but who have solved the Anonymous user CAPTCHA.</p>







    <p>This button will not appear in the default skin for such users if
    their browser window is not greater than 1200 pixels wide.  The
    Fossil admin can adjust this limit in the skin's CSS section, down
    near the bottom in the definition of the `wideonly` style.</p>
  </li>

  <li>The other stock skins have this button in them as of 2.7 as well,
  without the screen width restriction, since the navbar in those skins
  wraps on narrow screens more gracefully than the default skin
  does.</li>



  <li>Users who set up their Fossil repository under prior versions and
  who now have local skin changes they don't want to overwrite by
  reverting to the stock 2.7 version of the skin they chose to start
  with can easily [#skin | edit their skin] to include these links.</li>


  <li>A "Forum" link appears in the drop-down panel when you click the
  default skin's hamburger menu (&#9776;) for any user with one or more
  of the capabilities listed above.</li>

  <li>That same link also appears on the repository's `/sitemap` page,

  since it provides the content for the hamburger menu's panel.</li>
</ul>





















































































































































<h2 id="moderation">How Moderation Works</h2>

In this section, we're going to call all of the following a "forum
update:"

Changes to www/mkindex.tcl.

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#

set doclist {
  aboutcgi.wiki {How CGI Works In Fossil}
  aboutdownload.wiki {How The Download Page Works}
  adding_code.wiki {Adding New Features To Fossil}
  adding_code.wiki {Hacking Fossil}
  alerts.md {Email Alerts And Notification}
  antibot.wiki {Defense against Spiders and Bots}
  backoffice.md {The "Backoffice" mechanism of Fossil}
  blame.wiki {The Annotate/Blame Algorithm Of Fossil}
  branching.wiki {Branching, Forking, Merging, and Tagging}
  bugtheory.wiki {Bug Tracking In Fossil}
  build.wiki {Compiling and Installing Fossil}
  changes.wiki {Fossil Changelog}







|







7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#

set doclist {
  aboutcgi.wiki {How CGI Works In Fossil}
  aboutdownload.wiki {How The Download Page Works}
  adding_code.wiki {Adding New Features To Fossil}
  adding_code.wiki {Hacking Fossil}
  alerts.md {Email Alerts And Notifications}
  antibot.wiki {Defense against Spiders and Bots}
  backoffice.md {The "Backoffice" mechanism of Fossil}
  blame.wiki {The Annotate/Blame Algorithm Of Fossil}
  branching.wiki {Branching, Forking, Merging, and Tagging}
  bugtheory.wiki {Bug Tracking In Fossil}
  build.wiki {Compiling and Installing Fossil}
  changes.wiki {Fossil Changelog}