View Ticket
Ticket UUID: 708eab9d48592746b30974c580d353c777c8c609
Title: Message "killed" opening large repository
Status: Fixed Type: Code_Defect
Severity: Critical Priority:
Subsystem: Resolution: Fixed
Last Modified: 2010-01-20 16:56:19
Version Found In: c16017374e4318ade54ee0c1e269a0bc2ddde355
Took a large SVN repp (6.8G in over 70K files).  Created new fossil repo, and added and committed the data.  That went well, creating a 1.4G fossil repo.

Created a new directory, and "fossil open"ed the new repo.  Went ok for a while, then got a "killed" message and only some of the files were actually checked out (46,401 files).

The "fossil rstat" reports:
 Number of Artifacts: 12536
  12536 full text + 0 delta blobs
 112758 bytes average, 1413538178 bytes total

  Number Of Checkins: 1
     Number Of Files: 55323
Number Of Wiki Pages: 0
   Number Of Tickets: 0
 Duration Of Project: 1 days

<hr><i>anonymous added on 2010-01-14 13:16:05:</i><br>
This is on Linux (kernel 2.6.32).  The "open" leaves behind a "_FOSSIL_-journal" as well

<hr><i>drh added on 2010-01-14 14:15:38:</i><br>
You say your repository has over 70K files but fossil is only showing
12536 artifacts and only 55323 files.  If some of the files are duplicates
of one another, then the duplicates will only count as a single artifact
and so that would explain why the artifact count is much lower than the
file count.  (Please confirm that the project contains many duplicate
files.)  But Fossil's file count should be accurate and 55323 is less
than 70K.  Are you sure the initial check-in worked?

<hr><i>anonymous added on 2010-01-14 17:45:20:</i><br>
I just reran the test, first copying the SVN repository over and removing all '.svn' directories.  Then I created a repo "fossil new t", changed dir to the interesting dir and "fossil open ../t; fossil add *; fossil ci -m 'test'"

Then I created another directory, and did "fossil open ../t" there.  The result is the same: a "Killed" message.

However, something interesting: I ran "top" in another window, and saw the memory usage of fossil go upwards of 1.5G before it was finally killed. 

So I think the problem is there is a (bad) memory leak in the "open" scenario (and perhaps generally).  For I cannot see any reason for the memory usage to ramp up so high.

I also did "find . -type f | wc" in that dir, the result is 52,218.  The fossil rstats reported 51,192 files ... I'm not sure what the discrepancy would be besides the _FOSSIL_ and two manifest files.

<hr><i>anonymous added on 2010-01-14 20:02:09:</i><br>
I just ran under 'valgrind' with the command:

<tt>valgrind --log-file=../fossil.log --leak-check=full --show-reachable=yes --track-origins=yes  ../fossil/fossil open ~/fossil/torah</tt>

This is a quite small project, so the valgrind command was able to finish quickly.

The logfile is here: -- it really needs someone who understands the code to determine whether or not particular problems are real problems.

I hope this helps ...

<hr><i>ron added on 2010-01-19 06:39:49:</i><br>
This particular issue prevents me deploying Fossil at work for production support file management :(

<hr><i>drh added on 2010-01-20 15:51:30:</i><br>
Check-in [726a13d29689f4b92749072c2a47771767960373] fixes a memory leak
which should resolve this issue.

<hr><i>ron added on 2010-01-20 16:56:19:</i><br>
Thank you!  I confirm the bug is fixed.