Fossil

Check-in [aa04ac9d]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Extended the collection of revisions for a file with a separate blob store to manage the text ranges of revisions and their dependencies separate from the revisions. This will be used later (upcoming) to properly expand a file even if revisions were removed as irrelevant during the collection and filter passes.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:aa04ac9d10aab85fa43e75be859ae52c839c7ed9
User & Date: aku 2008-02-03 00:04:37
Context
2008-02-03
00:45
Modified filtering pass to drop the blobs of excluded revisions, for them we can be sure that we will not need them later, as whole branches are dropped. Modified the file import to use the new blob information to setup the expansion graph. This fixes the trouble we had with archive files in bwidget. They are now properly expanded. check-in: 83d75a6c user: aku tags: trunk
00:04
Extended the collection of revisions for a file with a separate blob store to manage the text ranges of revisions and their dependencies separate from the revisions. This will be used later (upcoming) to properly expand a file even if revisions were removed as irrelevant during the collection and filter passes. check-in: aa04ac9d user: aku tags: trunk
2008-02-02
22:59
Reworked the storage of the text range a bit, split immediately (settext) and store the pieces, instead of splitting everytime a piece is needed. check-in: f94f7e17 user: aku tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Added tools/cvs2fossil/lib/c2f_blobstore.tcl.























































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Blob storage. Each instance stores the blob data of a single rcs
## archive file, i.e. which file, all text ranges, delta dependencies,
## and associated revisions (as object references). The data is
## persistent and used by the import pass(es) to expand the revisions
## of a file.

# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                             ; # Required runtime.
package require snit                                ; # OO system.
package require vc::fossil::import::cvs::state      ; # State storage.
package require vc::fossil::import::cvs::integrity  ; # State integrity checks.
package require vc::tools::trouble                  ; # Error reporting.
package require vc::tools::log                      ; # User feedback
#package require vc::tools::misc                     ; # Text formatting

# # ## ### ##### ######## ############# #####################
##

snit::type ::vc::fossil::import::cvs::blobstore {
    # # ## ### ##### ######## #############
    ## Public API

    constructor {fid} {
	set       myfile $fid
	array set myparent {}
	array set myblob   {}
	return
    }

    method setid {id} {
	integrity assert {$myfile eq ""} {Already has an id, '$myfile'}
	set myfile $id
	return
    }

    # Remember the file revision object for the revision REVNR.

    method add {revnr rev} {
	set myblob($revnr) $rev
	return
    }

    # Remember that the DELTA revision is specified as a delta against
    # the BASE revision. Both are specified as revision numbers.

    method delta {delta base} {
	set myparent($delta) $base
	return
    }

    # Specify the text range in the archive file for the data of the
    # revision identified by REVNR.

    method extend {revnr textrange} {
	struct::list assign $textrange coff end
	set clen [expr {$end - $coff}]
	lappend myblob($revnr) $coff $clen
	return
    }

    # Write the stored information into the persistent state.

    method persist {} {
	array set bids {}
	state transaction {
	    # Phase I: Store the basic blob information.

	    foreach revnr [lsort [array names myblob]] {
		struct::list assign $myblob($revnr) rev coff clen
		state run {
		    INSERT INTO blob (bid, rid,   fid,     coff,  clen,  pid)
		    VALUES           (NULL, NULL, $myfile, $coff, $clen, NULL)
		}
		set current [state id]
		set bids($revnr) $current

		# Ia. Set the reference to the revision of the blob,
		# if applicable. We can have blobs without revisions,
		# their revisions were removed as irrelevant. We need
		# them however for the proper delta ordering and patch
		# application when expanding a file (-> Import passes).

		set rid [$rev id]
		if {$rid eq ""} continue
		state run {
		    UPDATE blob
		    SET    rid = $rid
		    WHERE  bid = $current
		}
	    }

	    # Phase II: Set the parent links for deltas.
	    foreach revnr [array names myparent] {
		set bid $bids($revnr)
		set pid $bids($myparent($revnr))

		state run {
		    UPDATE blob
		    SET    pid = $pid
		    WHERE  bid = $bid
		}
	    }
	}
	return
    }

    # # ## ### ##### ######## #############
    ## State

    variable myfile          {} ; # Id of the file the blobs belong to.
    variable myparent -array {} ; # Map delta-encoded revision numbers
				  # to their baseline revisions.
    variable myblob   -array {} ; # Map revision numbers to associated
				  # file revision object and text
				  # range.

    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hastypeinfo    no  ; # no type introspection
    pragma -hasinfo        no  ; # no object introspection
    pragma -hastypemethods no  ; # type is not relevant.

    # # ## ### ##### ######## #############
}

namespace eval ::vc::fossil::import::cvs {
    namespace export blobstore
    namespace eval blobstore {
	namespace import ::vc::tools::trouble
	namespace import ::vc::tools::log
	namespace import ::vc::fossil::import::cvs::state
	namespace import ::vc::fossil::import::cvs::integrity
    }
}

# # ## ### ##### ######## ############# #####################
## Ready

package provide vc::fossil::import::cvs::blobstore 1.0
return

Changes to tools/cvs2fossil/lib/c2f_file.tcl.

16
17
18
19
20
21
22

23
24
25
26
27
28
29
..
39
40
41
42
43
44
45

46
47
48
49
50
51

52
53
54
55
56
57
58
..
92
93
94
95
96
97
98

99
100
101
102
103
104
105
...
154
155
156
157
158
159
160











161
162
163
164
165
166
167
...
209
210
211
212
213
214
215


216
217
218
219
220
221
222
...
573
574
575
576
577
578
579




580
581
582
583
584
585
586
....
1387
1388
1389
1390
1391
1392
1393

1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                             ; # Required runtime.
package require snit                                ; # OO system.
package require struct::set                         ; # Set operations.
package require struct::list                        ; # Higher order operations.

package require vc::fossil::import::cvs::file::rev  ; # CVS per file revisions.
package require vc::fossil::import::cvs::file::sym  ; # CVS per file symbols.
package require vc::fossil::import::cvs::state      ; # State storage.
package require vc::fossil::import::cvs::integrity  ; # State integrity checks.
package require vc::fossil::import::cvs::gtcore     ; # Graph traversal core.
package require vc::tools::trouble                  ; # Error reporting.
package require vc::tools::log                      ; # User feedback
................................................................................
    constructor {id path usrpath executable project} {
	set myid         $id
	set mypath       $path
	set myusrpath    $usrpath
	set myexecutable $executable
	set myproject    $project
	set mytrunk      [$myproject trunk]

	return
    }

    method setid {id} {
	integrity assert {$myid eq ""} {File '$mypath' already has an id, '$myid'}
	set myid $id

	return
    }

    method id      {} { return $myid }
    method path    {} { return $mypath }
    method usrpath {} { return $myusrpath }
    method project {} { return $myproject }
................................................................................
	struct::list assign [$self Active] revisions symbols
	foreach rev $revisions { $rev defid }
	foreach sym $symbols   { $sym defid }

	state transaction {
	    foreach rev $revisions { $rev persist }
	    foreach sym $symbols   { $sym persist }

	}
	return
    }

    method drop {} {
	foreach {_ rev}    [array get myrev]      { $rev destroy }
	foreach {_ branch} [array get mybranches] { $branch destroy }
................................................................................
	if {[info exists myrev($revnr)]} {
	    trouble fatal "File $mypath contains duplicate definitions for revision $revnr."
	    return
	}

	set myaid($revnr) [$myproject defauthor $author]
	set myrev($revnr) [rev %AUTO% $revnr $date $state $self]












	$self RecordBasicDependencies $revnr $next
	return
    }

    method defdone {} {
	# This is all done after the revision tree has been extracted
................................................................................
	# and/or branches).

	set lod [$self GetLOD $revnr]

	$rev setmeta [$myproject defmeta [$lod id] $myaid($revnr) $cmid]
	$rev settext $textrange
	$rev setlod  $lod



	# If this is revision 1.1, we have to determine whether the
	# file seems to have been created through 'cvs add' instead of
	# 'cvs import'. This can be done by looking at the un-
	# adulterated commit message, as CVS uses a hardwired magic
	# message for the latter, i.e. "Initial revision\n", no
	# period.  (This fact also helps us when the time comes to
................................................................................
			     # 'Created later'.

    variable mytrunk {} ; # Direct reference to myproject -> trunk.
    variable myroots {} ; # List of roots in the forest of
			  # lod's. Object references to revisions and
			  # branches. The latter can appear when they
			  # are severed from their parent.





    # # ## ### ##### ######## #############
    ## Internal methods

    method RecordBranchCommits {branches} {
	foreach branchrevnr $branches {
	    if {[catch {
................................................................................
    namespace eval file {
	# Import not required, already a child namespace.
	# namespace import ::vc::fossil::import::cvs::file::rev
	# namespace import ::vc::fossil::import::cvs::file::sym
	namespace import ::vc::tools::misc::*
	namespace import ::vc::tools::trouble
	namespace import ::vc::tools::log

	namespace import ::vc::fossil::import::cvs::state
	namespace import ::vc::fossil::import::cvs::integrity
	namespace import ::vc::fossil::import::cvs::gtcore
    }
}

# # ## ### ##### ######## ############# #####################
## Ready

package provide vc::fossil::import::cvs::file 1.0
return







>







 







>






>







 







>







 







>
>
>
>
>
>
>
>
>
>
>







 







>
>







 







>
>
>
>







 







>











16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
..
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
..
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
...
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
...
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
...
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
....
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                             ; # Required runtime.
package require snit                                ; # OO system.
package require struct::set                         ; # Set operations.
package require struct::list                        ; # Higher order operations.
package require vc::fossil::import::cvs::blobstore  ; # Blob storage.
package require vc::fossil::import::cvs::file::rev  ; # CVS per file revisions.
package require vc::fossil::import::cvs::file::sym  ; # CVS per file symbols.
package require vc::fossil::import::cvs::state      ; # State storage.
package require vc::fossil::import::cvs::integrity  ; # State integrity checks.
package require vc::fossil::import::cvs::gtcore     ; # Graph traversal core.
package require vc::tools::trouble                  ; # Error reporting.
package require vc::tools::log                      ; # User feedback
................................................................................
    constructor {id path usrpath executable project} {
	set myid         $id
	set mypath       $path
	set myusrpath    $usrpath
	set myexecutable $executable
	set myproject    $project
	set mytrunk      [$myproject trunk]
	set myblob       [blobstore ${selfns}::%AUTO% $id]
	return
    }

    method setid {id} {
	integrity assert {$myid eq ""} {File '$mypath' already has an id, '$myid'}
	set myid $id
	$myblob setid $id
	return
    }

    method id      {} { return $myid }
    method path    {} { return $mypath }
    method usrpath {} { return $myusrpath }
    method project {} { return $myproject }
................................................................................
	struct::list assign [$self Active] revisions symbols
	foreach rev $revisions { $rev defid }
	foreach sym $symbols   { $sym defid }

	state transaction {
	    foreach rev $revisions { $rev persist }
	    foreach sym $symbols   { $sym persist }
	    $myblob persist
	}
	return
    }

    method drop {} {
	foreach {_ rev}    [array get myrev]      { $rev destroy }
	foreach {_ branch} [array get mybranches] { $branch destroy }
................................................................................
	if {[info exists myrev($revnr)]} {
	    trouble fatal "File $mypath contains duplicate definitions for revision $revnr."
	    return
	}

	set myaid($revnr) [$myproject defauthor $author]
	set myrev($revnr) [rev %AUTO% $revnr $date $state $self]

	$myblob add $revnr $myrev($revnr)

	if {$next ne ""} {
	    # parent revision NEXT is a delta of current.
	    $myblob delta $next $revnr
	}
	foreach b $branches {
	    # branch child revision B is a delta of current.
	    $myblob delta $b $revnr
	}

	$self RecordBasicDependencies $revnr $next
	return
    }

    method defdone {} {
	# This is all done after the revision tree has been extracted
................................................................................
	# and/or branches).

	set lod [$self GetLOD $revnr]

	$rev setmeta [$myproject defmeta [$lod id] $myaid($revnr) $cmid]
	$rev settext $textrange
	$rev setlod  $lod

	$myblob extend $revnr $textrange

	# If this is revision 1.1, we have to determine whether the
	# file seems to have been created through 'cvs add' instead of
	# 'cvs import'. This can be done by looking at the un-
	# adulterated commit message, as CVS uses a hardwired magic
	# message for the latter, i.e. "Initial revision\n", no
	# period.  (This fact also helps us when the time comes to
................................................................................
			     # 'Created later'.

    variable mytrunk {} ; # Direct reference to myproject -> trunk.
    variable myroots {} ; # List of roots in the forest of
			  # lod's. Object references to revisions and
			  # branches. The latter can appear when they
			  # are severed from their parent.

    variable myblob {} ; # Reference to the object managing the blob
			 # information (textrange of revisions, and
			 # delta dependencies) of this file.

    # # ## ### ##### ######## #############
    ## Internal methods

    method RecordBranchCommits {branches} {
	foreach branchrevnr $branches {
	    if {[catch {
................................................................................
    namespace eval file {
	# Import not required, already a child namespace.
	# namespace import ::vc::fossil::import::cvs::file::rev
	# namespace import ::vc::fossil::import::cvs::file::sym
	namespace import ::vc::tools::misc::*
	namespace import ::vc::tools::trouble
	namespace import ::vc::tools::log
	namespace import ::vc::fossil::import::cvs::blobstore
	namespace import ::vc::fossil::import::cvs::state
	namespace import ::vc::fossil::import::cvs::integrity
	namespace import ::vc::fossil::import::cvs::gtcore
    }
}

# # ## ### ##### ######## ############# #####################
## Ready

package provide vc::fossil::import::cvs::file 1.0
return

Changes to tools/cvs2fossil/lib/c2f_pcollrev.tcl.

141
142
143
144
145
146
147


























148
149
150
151
152
153
154
	    state TEXT     NOT NULL,
	    mid   INTEGER  NOT NULL  REFERENCES meta,
	    coff  INTEGER  NOT NULL,
	    clen  INTEGER  NOT NULL,

	    UNIQUE (fid, rev) -- The DTN is unique within the revision's file.
	}



























	state extend optype {
	    oid   INTEGER  NOT NULL  PRIMARY KEY,
	    name  TEXT     NOT NULL,
	    UNIQUE(name)
	}
	state run {







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
	    state TEXT     NOT NULL,
	    mid   INTEGER  NOT NULL  REFERENCES meta,
	    coff  INTEGER  NOT NULL,
	    clen  INTEGER  NOT NULL,

	    UNIQUE (fid, rev) -- The DTN is unique within the revision's file.
	}

	# Blobs contain the information needed to extract revisions
	# from rcs archive files. As such each revision has an
	# associated blob. However we can have blobs without
	# revisions. This happens if a logically irrelevant revision
	# is removed. We may however still need its blob to correctly
	# expand other revisions, both its contents and for the
	# ordering.

	state extend blob {
	    bid  INTEGER  NOT NULL  PRIMARY KEY AUTOINCREMENT,
	    rid  INTEGER            REFERENCES revision,
	    fid  INTEGER  NOT NULL  REFERENCES file,   -- File owning blob.

	    -- The text content is an (offset,length) pair into the
	    -- rcs archive. For deltas we additionally refer to the
	    -- parent blob the delta is made against.

	    coff INTEGER  NOT NULL,
	    clen INTEGER  NOT NULL,
	    pid  INTEGER            REFERENCES blob,

	    UNIQUE (rid)
	} { fid }
	# Index on owning file to collect all blobs of a file when the
	# time for its expansion comes.

	state extend optype {
	    oid   INTEGER  NOT NULL  PRIMARY KEY,
	    name  TEXT     NOT NULL,
	    UNIQUE(name)
	}
	state run {

Changes to tools/cvs2fossil/lib/pkgIndex.tcl.

1
2
3
4
5
6

7
8
9
10
11
12
13
# # ## ### ##### ######## ############# #####################
## Package management.
## Index of the local packages required by cvs2fossil
# # ## ### ##### ######## ############# #####################
if {![package vsatisfies [package require Tcl] 8.4]} return
package ifneeded vc::fossil::import::cvs                    1.0 [list source [file join $dir cvs2fossil.tcl]]

package ifneeded vc::fossil::import::cvs::file              1.0 [list source [file join $dir c2f_file.tcl]]
package ifneeded vc::fossil::import::cvs::file::rev         1.0 [list source [file join $dir c2f_frev.tcl]]
package ifneeded vc::fossil::import::cvs::file::sym         1.0 [list source [file join $dir c2f_fsym.tcl]]
package ifneeded vc::fossil::import::cvs::file::trunk       1.0 [list source [file join $dir c2f_ftrunk.tcl]]
package ifneeded vc::fossil::import::cvs::fossil            1.0 [list source [file join $dir c2f_fossil.tcl]]
package ifneeded vc::fossil::import::cvs::option            1.0 [list source [file join $dir c2f_option.tcl]]
package ifneeded vc::fossil::import::cvs::integrity         1.0 [list source [file join $dir c2f_integrity.tcl]]






>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
# # ## ### ##### ######## ############# #####################
## Package management.
## Index of the local packages required by cvs2fossil
# # ## ### ##### ######## ############# #####################
if {![package vsatisfies [package require Tcl] 8.4]} return
package ifneeded vc::fossil::import::cvs                    1.0 [list source [file join $dir cvs2fossil.tcl]]
package ifneeded vc::fossil::import::cvs::blobstore         1.0 [list source [file join $dir c2f_blobstore.tcl]]
package ifneeded vc::fossil::import::cvs::file              1.0 [list source [file join $dir c2f_file.tcl]]
package ifneeded vc::fossil::import::cvs::file::rev         1.0 [list source [file join $dir c2f_frev.tcl]]
package ifneeded vc::fossil::import::cvs::file::sym         1.0 [list source [file join $dir c2f_fsym.tcl]]
package ifneeded vc::fossil::import::cvs::file::trunk       1.0 [list source [file join $dir c2f_ftrunk.tcl]]
package ifneeded vc::fossil::import::cvs::fossil            1.0 [list source [file join $dir c2f_fossil.tcl]]
package ifneeded vc::fossil::import::cvs::option            1.0 [list source [file join $dir c2f_option.tcl]]
package ifneeded vc::fossil::import::cvs::integrity         1.0 [list source [file join $dir c2f_integrity.tcl]]