mercurial/revlog.py
author Matt Mackall <mpm@selenic.com>
Thu, 11 Oct 2007 00:46:47 -0500
changeset 5445 64cf1c853674
parent 5368 61462e7d62ed
child 5448 e038738714fd
permissions -rw-r--r--
revlog: reduce memory usage in addgroup - use a buffer to extract the delta from a chunk - avoid concatenating to a compressed delta - use a buffer to directly extra full text from a trivial delta - delete chunk and delta objects after use
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     1
"""
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     2
revlog.py - storage back-end for mercurial
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     3
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     4
This provides efficient delta storage with O(1) retrieve and append
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     5
and O(changes) merge between branches
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     6
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4497
diff changeset
     7
Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     8
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
     9
This software may be used and distributed according to the terms
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    10
of the GNU General Public License, incorporated herein by reference.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    11
"""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    12
1089
142b5d5ec9cc Break apart hg.py
mpm@selenic.com
parents: 1083
diff changeset
    13
from node import *
3891
6b4127c7d52a Simplify i18n imports
Matt Mackall <mpm@selenic.com>
parents: 3877
diff changeset
    14
from i18n import _
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3755
diff changeset
    15
import binascii, changegroup, errno, ancestor, mdiff, os
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents: 3755
diff changeset
    16
import sha, struct, util, zlib
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
    17
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    18
_pack = struct.pack
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    19
_unpack = struct.unpack
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    20
_compress = zlib.compress
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    21
_decompress = zlib.decompress
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    22
_sha = sha.new
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    23
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    24
# revlog flags
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
    25
REVLOGV0 = 0
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
    26
REVLOGNG = 1
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
    27
REVLOGNGINLINEDATA = (1 << 16)
2222
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    28
REVLOG_DEFAULT_FLAGS = REVLOGNGINLINEDATA
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    29
REVLOG_DEFAULT_FORMAT = REVLOGNG
c9e264b115e6 Use revlogng and inlined data files by default
mason@suse.com
parents: 2177
diff changeset
    30
REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
    31
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    32
class RevlogError(Exception):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    33
    pass
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    34
class LookupError(RevlogError):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    35
    pass
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    36
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    37
def getoffset(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    38
    return int(q >> 16)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    39
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    40
def gettype(q):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    41
    return int(q & 0xFFFF)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    42
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    43
def offset_type(offset, type):
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    44
    return long(long(offset) << 16 | type)
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
    45
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    46
def hash(text, p1, p2):
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    47
    """generate a hash from the given text and its parent hashes
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    48
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    49
    This hash combines both the current file contents and its history
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    50
    in a manner that makes it easy to distinguish nodes with the same
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    51
    content in the revision graph.
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    52
    """
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    53
    l = [p1, p2]
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    54
    l.sort()
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    55
    s = _sha(l[0])
1091
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    56
    s.update(l[1])
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    57
    s.update(text)
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    58
    return s.digest()
d62130f99a73 Move hash function back to revlog from node
mpm@selenic.com
parents: 1089
diff changeset
    59
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    60
def compress(text):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    61
    """ generate a possibly-compressed representation of text """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    62
    if not text:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    63
        return ("", text)
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
    64
    if len(text) < 44:
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    65
        if text[0] == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    66
            return ("", text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    67
        return ('u', text)
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    68
    bin = _compress(text)
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
    69
    if len(bin) > len(text):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    70
        if text[0] == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    71
            return ("", text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    72
        return ('u', text)
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
    73
    return ("", bin)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    74
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    75
def decompress(bin):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    76
    """ decompress the given input """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    77
    if not bin:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    78
        return bin
112
aea6562add6c Make compression more intelligent:
mpm@selenic.com
parents: 98
diff changeset
    79
    t = bin[0]
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    80
    if t == '\0':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    81
        return bin
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    82
    if t == 'x':
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
    83
        return _decompress(bin)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    84
    if t == 'u':
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
    85
        return bin[1:]
1853
5ac811b720de Fix some problems when working on broken repositories:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1784
diff changeset
    86
    raise RevlogError(_("unknown compression type %r") % t)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
    87
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
    88
class lazyparser(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    89
    """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    90
    this class avoids the need to parse the entirety of large indices
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
    91
    """
2250
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    92
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    93
    # lazyparser is not safe to use on windows if win32 extensions not
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    94
    # available. it keeps file handle open, which make it not possible
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    95
    # to break hardlinks on local cloned repos.
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    96
    safe_to_use = os.name != 'nt' or (not util.is_win_9x() and
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    97
                                      hasattr(util, 'win32api'))
45aef5ddcdbe windows: revlog.lazyparser not always safe to use.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2222
diff changeset
    98
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
    99
    def __init__(self, dataf, size):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   100
        self.dataf = dataf
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   101
        self.s = struct.calcsize(indexformatng)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   102
        self.datasize = size
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   103
        self.l = size/self.s
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   104
        self.index = [None] * self.l
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   105
        self.map = {nullid: nullrev}
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   106
        self.allmap = 0
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   107
        self.all = 0
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   108
        self.mapfind_count = 0
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   109
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   110
    def loadmap(self):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   111
        """
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   112
        during a commit, we need to make sure the rev being added is
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   113
        not a duplicate.  This requires loading the entire index,
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   114
        which is fairly slow.  loadmap can load up just the node map,
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   115
        which takes much less time.
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   116
        """
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   117
        if self.allmap:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   118
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   119
        end = self.datasize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   120
        self.allmap = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   121
        cur = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   122
        count = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   123
        blocksize = self.s * 256
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   124
        self.dataf.seek(0)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   125
        while cur < end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   126
            data = self.dataf.read(blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   127
            off = 0
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   128
            for x in xrange(256):
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   129
                n = data[off + ngshaoffset:off + ngshaoffset + 20]
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   130
                self.map[n] = count
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   131
                count += 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   132
                if count >= self.l:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   133
                    break
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   134
                off += self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   135
            cur += blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   136
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   137
    def loadblock(self, blockstart, blocksize, data=None):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   138
        if self.all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   139
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   140
        if data is None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   141
            self.dataf.seek(blockstart)
3078
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   142
            if blockstart + blocksize > self.datasize:
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   143
                # the revlog may have grown since we've started running,
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   144
                # but we don't have space in self.index for more entries.
baa3873eb387 don't let lazyparser read more data than it can handle
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2890
diff changeset
   145
                # limit blocksize so that we don't get too much data.
3089
e7fc04dc6349 Avoid negative block sizes in lazyparser.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3078
diff changeset
   146
                blocksize = max(self.datasize - blockstart, 0)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   147
            data = self.dataf.read(blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   148
        lend = len(data) / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   149
        i = blockstart / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   150
        off = 0
4061
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   151
        # lazyindex supports __delitem__
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   152
        if lend > len(self.index) - i:
40030c1b6bc6 lazyindex: handle __delitem__ in loadblock
Brendan Cully <brendan@kublai.com>
parents: 3930
diff changeset
   153
            lend = len(self.index) - i
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   154
        for x in xrange(lend):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   155
            if self.index[i + x] == None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   156
                b = data[off : off + self.s]
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   157
                self.index[i + x] = b
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   158
                n = b[ngshaoffset:ngshaoffset + 20]
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   159
                self.map[n] = i + x
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   160
            off += self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   161
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   162
    def findnode(self, node):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   163
        """search backwards through the index file for a specific node"""
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   164
        if self.allmap:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   165
            return None
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   166
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   167
        # hg log will cause many many searches for the manifest
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   168
        # nodes.  After we get called a few times, just load the whole
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   169
        # thing.
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   170
        if self.mapfind_count > 8:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   171
            self.loadmap()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   172
            if node in self.map:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   173
                return node
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   174
            return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   175
        self.mapfind_count += 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   176
        last = self.l - 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   177
        while self.index[last] != None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   178
            if last == 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   179
                self.all = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   180
                self.allmap = 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   181
                return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   182
            last -= 1
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   183
        end = (last + 1) * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   184
        blocksize = self.s * 256
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   185
        while end >= 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   186
            start = max(end - blocksize, 0)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   187
            self.dataf.seek(start)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   188
            data = self.dataf.read(end - start)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   189
            findend = end - start
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   190
            while True:
4994
d36310dd51d7 lazyparser.findnode: fix typo and s/rfind/find/
Matt Mackall <mpm@selenic.com>
parents: 4993
diff changeset
   191
                # we're searching backwards, so we have to make sure
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   192
                # we don't find a changeset where this node is a parent
4994
d36310dd51d7 lazyparser.findnode: fix typo and s/rfind/find/
Matt Mackall <mpm@selenic.com>
parents: 4993
diff changeset
   193
                off = data.find(node, 0, findend)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   194
                findend = off
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   195
                if off >= 0:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   196
                    i = off / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   197
                    off = i * self.s
4976
79c39cc9ff69 revlog: only allow lazy parsing with revlogng files
Matt Mackall <mpm@selenic.com>
parents: 4975
diff changeset
   198
                    n = data[off + ngshaoffset:off + ngshaoffset + 20]
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   199
                    if n == node:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   200
                        self.map[n] = i + start / self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   201
                        return node
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   202
                else:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   203
                    break
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   204
            end -= blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   205
        return None
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   206
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   207
    def loadindex(self, i=None, end=None):
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   208
        if self.all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   209
            return
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   210
        all = False
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   211
        if i == None:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   212
            blockstart = 0
4992
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   213
            blocksize = (65536 / self.s) * self.s
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   214
            end = self.datasize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   215
            all = True
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   216
        else:
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   217
            if end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   218
                blockstart = i * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   219
                end = end * self.s
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   220
                blocksize = end - blockstart
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   221
            else:
4992
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   222
                blockstart = (i & ~1023) * self.s
0a676643687b lazyparser: up the blocksize from 512 bytes to 64k
Matt Mackall <mpm@selenic.com>
parents: 4991
diff changeset
   223
                blocksize = self.s * 1024
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   224
                end = blockstart + blocksize
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   225
        while blockstart < end:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   226
            self.loadblock(blockstart, blocksize)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   227
            blockstart += blocksize
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   228
        if all:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   229
            self.all = True
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   230
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   231
class lazyindex(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   232
    """a lazy version of the index array"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   233
    def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   234
        self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   235
    def __len__(self):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   236
        return len(self.p.index)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
   237
    def load(self, pos):
1403
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
   238
        if pos < 0:
bc3e66edb04c lazyindex fix, make load handle negative indexes properly.
Eric Hopper <hopper@omnifarious.org>
parents: 1402
diff changeset
   239
            pos += len(self.p.index)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   240
        self.p.loadindex(pos)
115
39b438eeb25a Make lazyindex load slightly faster
mpm@selenic.com
parents: 112
diff changeset
   241
        return self.p.index[pos]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   242
    def __getitem__(self, pos):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   243
        return _unpack(indexformatng, self.p.index[pos] or self.load(pos))
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   244
    def __setitem__(self, pos, item):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   245
        self.p.index[pos] = _pack(indexformatng, *item)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   246
    def __delitem__(self, pos):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   247
        del self.p.index[pos]
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   248
    def insert(self, pos, e):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   249
        self.p.index.insert(pos, _pack(indexformatng, *e))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   250
    def append(self, e):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   251
        self.p.index.append(_pack(indexformatng, *e))
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   252
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   253
class lazymap(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   254
    """a lazy version of the node map"""
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   255
    def __init__(self, parser):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   256
        self.p = parser
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   257
    def load(self, key):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   258
        n = self.p.findnode(key)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   259
        if n == None:
1214
34706a835d4a Smarter handling of revlog key errors
mpm@selenic.com
parents: 1201
diff changeset
   260
            raise KeyError(key)
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   261
    def __contains__(self, key):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   262
        if key in self.p.map:
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   263
            return True
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   264
        self.p.loadmap()
323
c6f0673ab7e9 lazyparser speed ups
mpm@selenic.com
parents: 306
diff changeset
   265
        return key in self.p.map
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
   266
    def __iter__(self):
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
   267
        yield nullid
97
7a2abee6b0c2 Add iterator to the lazymap code
mpm@selenic.com
parents: 94
diff changeset
   268
        for i in xrange(self.p.l):
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   269
            ret = self.p.index[i]
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   270
            if not ret:
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   271
                self.p.loadindex(i)
2080
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   272
                ret = self.p.index[i]
1cbb14c048cb Reduce index memory usage by storing the bare string instead of tuples
mason@suse.com
parents: 2079
diff changeset
   273
            if isinstance(ret, str):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   274
                ret = _unpack(indexformatng, ret)
4978
93d48a8fa496 revlog: change accesses to index entry elements to use positive offsets
Matt Mackall <mpm@selenic.com>
parents: 4977
diff changeset
   275
            yield ret[7]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   276
    def __getitem__(self, key):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   277
        try:
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   278
            return self.p.map[key]
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   279
        except KeyError:
86
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   280
            try:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   281
                self.load(key)
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   282
                return self.p.map[key]
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   283
            except KeyError:
1b945e8ba67b Friendlier exceptions for unknown node errors
mpm@selenic.com
parents: 84
diff changeset
   284
                raise KeyError("node " + hex(key))
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   285
    def __setitem__(self, key, val):
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   286
        self.p.map[key] = val
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   287
    def __delitem__(self, key):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
   288
        del self.p.map[key]
76
d993ebd69d28 Add lazy{parser,index,map} to speed up processing of index files
mpm@selenic.com
parents: 73
diff changeset
   289
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   290
indexformatv0 = ">4l20s20s20s"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   291
v0shaoffset = 56
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   292
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   293
class revlogoldio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   294
    def __init__(self):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   295
        self.size = struct.calcsize(indexformatv0)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   296
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   297
    def parseindex(self, fp, inline):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   298
        s = self.size
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   299
        index = []
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   300
        nodemap =  {nullid: nullrev}
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   301
        n = off = 0
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   302
        data = fp.read()
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   303
        l = len(data)
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   304
        while off + s <= l:
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   305
            cur = data[off:off + s]
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   306
            off += s
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   307
            e = _unpack(indexformatv0, cur)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   308
            # transform to revlogv1 format
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   309
            e2 = (offset_type(e[0], 0), e[1], -1, e[2], e[3],
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   310
                  nodemap[e[4]], nodemap[e[5]], e[6])
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   311
            index.append(e2)
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   312
            nodemap[e[6]] = n
4973
a386a6e4fe46 revlog: simplify the v0 parser
Matt Mackall <mpm@selenic.com>
parents: 4972
diff changeset
   313
            n += 1
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   314
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   315
        return index, nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   316
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   317
    def packentry(self, entry, node, version, rev):
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   318
        e2 = (getoffset(entry[0]), entry[1], entry[3], entry[4],
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   319
              node(entry[5]), node(entry[6]), entry[7])
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   320
        return _pack(indexformatv0, *e2)
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   321
4987
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   322
# index ng:
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   323
# 6 bytes offset
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   324
# 2 bytes flags
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   325
# 4 bytes compressed length
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   326
# 4 bytes uncompressed length
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   327
# 4 bytes: base rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   328
# 4 bytes link rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   329
# 4 bytes parent 1 rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   330
# 4 bytes parent 2 rev
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   331
# 32 bytes: nodeid
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   332
indexformatng = ">Qiiiiii20s12x"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   333
ngshaoffset = 32
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   334
versionformat = ">I"
8d30004ada40 revlog: some basic code reordering
Matt Mackall <mpm@selenic.com>
parents: 4986
diff changeset
   335
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   336
class revlogio(object):
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   337
    def __init__(self):
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   338
        self.size = struct.calcsize(indexformatng)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   339
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   340
    def parseindex(self, fp, inline):
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   341
        try:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   342
            size = util.fstat(fp).st_size
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   343
        except AttributeError:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   344
            size = 0
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   345
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   346
        if lazyparser.safe_to_use and not inline and size > 1000000:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   347
            # big index, let's parse it on demand
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   348
            parser = lazyparser(fp, size)
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   349
            index = lazyindex(parser)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   350
            nodemap = lazymap(parser)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   351
            e = list(index[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   352
            type = gettype(e[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   353
            e[0] = offset_type(0, type)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   354
            index[0] = e
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   355
            return index, nodemap, None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   356
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   357
        s = self.size
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   358
        cache = None
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   359
        index = []
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   360
        nodemap =  {nullid: nullrev}
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   361
        n = off = 0
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   362
        # if we're not using lazymap, always read the whole index
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   363
        data = fp.read()
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   364
        l = len(data) - s
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   365
        append = index.append
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   366
        if inline:
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   367
            cache = (0, data)
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   368
            while off <= l:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   369
                e = _unpack(indexformatng, data[off:off + s])
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   370
                nodemap[e[7]] = n
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   371
                append(e)
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   372
                n += 1
4975
8b7e480a7603 revlog: simplify the v1 immediate parser
Matt Mackall <mpm@selenic.com>
parents: 4974
diff changeset
   373
                if e[1] < 0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   374
                    break
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   375
                off += e[1] + s
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   376
        else:
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   377
            while off <= l:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   378
                e = _unpack(indexformatng, data[off:off + s])
4990
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   379
                nodemap[e[7]] = n
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   380
                append(e)
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   381
                n += 1
4491125c0f21 revlogio: speed up parsing
Matt Mackall <mpm@selenic.com>
parents: 4989
diff changeset
   382
                off += s
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   383
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   384
        e = list(index[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   385
        type = gettype(e[0])
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   386
        e[0] = offset_type(0, type)
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   387
        index[0] = e
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   388
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   389
        return index, nodemap, cache
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   390
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   391
    def packentry(self, entry, node, version, rev):
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   392
        p = _pack(indexformatng, *entry)
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   393
        if rev == 0:
5007
3addf4531643 revlog: localize some fastpath functions
Matt Mackall <mpm@selenic.com>
parents: 5006
diff changeset
   394
            p = _pack(versionformat, version) + p[4:]
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   395
        return p
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   396
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 1551
diff changeset
   397
class revlog(object):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   398
    """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   399
    the underlying revision storage object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   400
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   401
    A revlog consists of two parts, an index and the revision data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   402
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   403
    The index is a file with a fixed record size containing
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   404
    information on each revision, includings its nodeid (hash), the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   405
    nodeids of its parents, the position and offset of its data within
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   406
    the data file, and the revision it's based on. Finally, each entry
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   407
    contains a linkrev entry that can serve as a pointer to external
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   408
    data.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   409
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   410
    The revision data itself is a linear collection of data chunks.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   411
    Each chunk represents a revision and is usually represented as a
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   412
    delta against the previous chunk. To bound lookup time, runs of
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   413
    deltas are limited to about 2 times the length of the original
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   414
    version data. This makes retrieval of a version proportional to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   415
    its size, or O(1) relative to the number of revisions.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   416
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   417
    Both pieces of the revlog are written to in an append-only
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   418
    fashion, which means we never need to rewrite a file to insert or
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   419
    remove data, and can use some simple techniques to avoid the need
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   420
    for locking while reading.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   421
    """
4258
b11a2fb59cf5 revlog: simplify revlog version handling
Matt Mackall <mpm@selenic.com>
parents: 4257
diff changeset
   422
    def __init__(self, opener, indexfile):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   423
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   424
        create a revlog object
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   425
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   426
        opener is a function that abstracts the file opening operation
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   427
        and can be used to implement COW semantics or the like.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   428
        """
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   429
        self.indexfile = indexfile
4257
1b5c38e9d7aa revlog: don't pass datafile as an argument
Matt Mackall <mpm@selenic.com>
parents: 4224
diff changeset
   430
        self.datafile = indexfile[:-2] + ".d"
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   431
        self.opener = opener
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   432
        self._cache = None
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   433
        self._chunkcache = None
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   434
        self.nodemap = {nullid: nullrev}
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   435
        self.index = []
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   436
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   437
        v = REVLOG_DEFAULT_VERSION
4258
b11a2fb59cf5 revlog: simplify revlog version handling
Matt Mackall <mpm@selenic.com>
parents: 4257
diff changeset
   438
        if hasattr(opener, "defversion"):
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   439
            v = opener.defversion
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   440
            if v & REVLOGNG:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   441
                v |= REVLOGNGINLINEDATA
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
   442
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   443
        i = ""
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   444
        try:
1784
2e0a288ca93e revalidate revlog data after locking the repo (issue132)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1749
diff changeset
   445
            f = self.opener(self.indexfile)
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   446
            i = f.read(4)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   447
            f.seek(0)
4918
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   448
            if len(i) > 0:
e017d3a82e1d revlog: raise offset/type helpers to global scope
Matt Mackall <mpm@selenic.com>
parents: 4746
diff changeset
   449
                v = struct.unpack(versionformat, i)[0]
1322
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   450
        except IOError, inst:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   451
            if inst.errno != errno.ENOENT:
b3d44e9b3092 Make revlog constructor more discerning in its treatment of errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1232
diff changeset
   452
                raise
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   453
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   454
        self.version = v
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   455
        self._inline = v & REVLOGNGINLINEDATA
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   456
        flags = v & ~0xFFFF
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   457
        fmt = v & 0xFFFF
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   458
        if fmt == REVLOGV0 and flags:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   459
            raise RevlogError(_("index %s unknown flags %#04x for format v0")
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   460
                              % (self.indexfile, flags >> 16))
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   461
        elif fmt == REVLOGNG and flags & ~REVLOGNGINLINEDATA:
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   462
            raise RevlogError(_("index %s unknown flags %#04x for revlogng")
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   463
                              % (self.indexfile, flags >> 16))
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   464
        elif fmt > REVLOGNG:
3743
3a099154b110 Make revlog error slightly less scary
Matt Mackall <mpm@selenic.com>
parents: 3683
diff changeset
   465
            raise RevlogError(_("index %s unknown format %d")
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   466
                              % (self.indexfile, fmt))
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   467
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   468
        self._io = revlogio()
4971
3e6dae278c99 revlog: regroup parsing code
Matt Mackall <mpm@selenic.com>
parents: 4920
diff changeset
   469
        if self.version == REVLOGV0:
4972
8d0cf46e0dc6 revlog: add revlogio interface
Matt Mackall <mpm@selenic.com>
parents: 4971
diff changeset
   470
            self._io = revlogoldio()
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   471
        if i:
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   472
            d = self._io.parseindex(f, self._inline)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   473
            self.index, self.nodemap, self._chunkcache = d
4985
e6525e459157 revlog: simplify revlog.__init__
Matt Mackall <mpm@selenic.com>
parents: 4984
diff changeset
   474
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   475
        # add the magic null revision at -1
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   476
        self.index.append((0, 0, 0, -1, -1, -1, -1, nullid))
116
e484cd5ec282 Only use lazy indexing for big indices and avoid the overhead of the
mpm@selenic.com
parents: 115
diff changeset
   477
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   478
    def _loadindex(self, start, end):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   479
        """load a block of indexes all at once from the lazy parser"""
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   480
        if isinstance(self.index, lazyindex):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   481
            self.index.p.loadindex(start, end)
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   482
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   483
    def _loadindexmap(self):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   484
        """loads both the map and the index from the lazy parser"""
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   485
        if isinstance(self.index, lazyindex):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   486
            p = self.index.p
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   487
            p.loadindex()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   488
            self.nodemap = p.map
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   489
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   490
    def _loadmap(self):
2079
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   491
        """loads the map from the lazy parser"""
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   492
        if isinstance(self.nodemap, lazymap):
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   493
            self.nodemap.p.loadmap()
ee96ca273f32 New lazy index code for revlogs.
mason@suse.com
parents: 2078
diff changeset
   494
            self.nodemap = self.nodemap.p.map
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   495
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   496
    def tip(self):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   497
        return self.node(len(self.index) - 2)
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   498
    def count(self):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   499
        return len(self.index) - 1
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   500
1201
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   501
    def rev(self, node):
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   502
        try:
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   503
            return self.nodemap[node]
59bfbdbc38f6 revlog: raise informative exception if file is missing.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1099
diff changeset
   504
        except KeyError:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   505
            raise LookupError(_('%s: no node %s') % (self.indexfile, hex(node)))
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   506
    def node(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   507
        return self.index[rev][7]
2642
6414ee2eb688 correct the handling of linkrev with nullid
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2641
diff changeset
   508
    def linkrev(self, node):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   509
        return self.index[self.rev(node)][4]
2
ecf3fd948051 Handle nullid better for ancestor
mpm@selenic.com
parents: 0
diff changeset
   510
    def parents(self, node):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   511
        d = self.index[self.rev(node)][5:7]
3508
0aef94f45ebf revlog.py: always return tuples from parents and parentrevs
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3490
diff changeset
   512
        return (self.node(d[0]), self.node(d[1]))
2489
568e58eed096 Add revlog.parentrevs function.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2354
diff changeset
   513
    def parentrevs(self, rev):
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
   514
        return self.index[rev][5:7]
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   515
    def start(self, rev):
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   516
        return int(self.index[rev][0] >> 16)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   517
    def end(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   518
        return self.start(rev) + self.length(rev)
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   519
    def length(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   520
        return self.index[rev][1]
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   521
    def base(self, rev):
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   522
        return self.index[rev][3]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   523
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   524
    def size(self, rev):
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   525
        """return the length of the uncompressed text for a given revision"""
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   526
        l = self.index[rev][2]
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   527
        if l >= 0:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   528
            return l
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   529
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   530
        t = self.revision(self.node(rev))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   531
        return len(t)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   532
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   533
        # alternate implementation, The advantage to this code is it
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   534
        # will be faster for a single revision.  But, the results are not
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   535
        # cached, so finding the size of every revision will be slower.
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   536
        """
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   537
        if self.cache and self.cache[1] == rev:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   538
            return len(self.cache[2])
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   539
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   540
        base = self.base(rev)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   541
        if self.cache and self.cache[1] >= base and self.cache[1] < rev:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   542
            base = self.cache[1]
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   543
            text = self.cache[2]
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   544
        else:
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   545
            text = self.revision(self.node(base))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   546
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   547
        l = len(text)
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   548
        for x in xrange(base + 1, rev + 1):
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   549
            l = mdiff.patchedsize(l, self.chunk(x))
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   550
        return l
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   551
        """
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
   552
3630
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   553
    def reachable(self, node, stop=None):
3683
b0fd43effdb3 add docstring to reachable
Matt Mackall <mpm@selenic.com>
parents: 3680
diff changeset
   554
        """return a hash of all nodes ancestral to a given node, including
b0fd43effdb3 add docstring to reachable
Matt Mackall <mpm@selenic.com>
parents: 3680
diff changeset
   555
         the node itself, stopping when stop is matched"""
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   556
        reachable = {}
3630
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   557
        visit = [node]
508036290b00 revlog: reachable actually takes a node
Matt Mackall <mpm@selenic.com>
parents: 3585
diff changeset
   558
        reachable[node] = 1
1074
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   559
        if stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   560
            stopn = self.rev(stop)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   561
        else:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   562
            stopn = 0
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   563
        while visit:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   564
            n = visit.pop(0)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   565
            if n == stop:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   566
                continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   567
            if n == nullid:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   568
                continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   569
            for p in self.parents(n):
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   570
                if self.rev(p) < stopn:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   571
                    continue
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   572
                if p not in reachable:
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   573
                    reachable[p] = 1
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   574
                    visit.append(p)
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   575
        return reachable
55bf5cfde69e Add revlog.reachable to find a graph of ancestors for a given rev
mason@suse.com
parents: 1073
diff changeset
   576
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   577
    def nodesbetween(self, roots=None, heads=None):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   578
        """Return a tuple containing three elements. Elements 1 and 2 contain
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   579
        a final list bases and heads after all the unreachable ones have been
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   580
        pruned.  Element 0 contains a topologically sorted list of all
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   581
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   582
        nodes that satisfy these constraints:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   583
        1. All nodes must be descended from a node in roots (the nodes on
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   584
           roots are considered descended from themselves).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   585
        2. All nodes must also be ancestors of a node in heads (the nodes in
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   586
           heads are considered to be their own ancestors).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   587
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   588
        If roots is unspecified, nullid is assumed as the only root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   589
        If heads is unspecified, it is taken to be the output of the
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   590
        heads method (i.e. a list of all nodes in the repository that
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   591
        have no children)."""
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   592
        nonodes = ([], [], [])
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   593
        if roots is not None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   594
            roots = list(roots)
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   595
            if not roots:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   596
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   597
            lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   598
        else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   599
            roots = [nullid] # Everybody's a descendent of nullid
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   600
            lowestrev = nullrev
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   601
        if (lowestrev == nullrev) and (heads is None):
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   602
            # We want _all_ the nodes!
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   603
            return ([self.node(r) for r in xrange(0, self.count())],
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   604
                    [nullid], list(self.heads()))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   605
        if heads is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   606
            # All nodes are ancestors, so the latest ancestor is the last
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   607
            # node.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   608
            highestrev = self.count() - 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   609
            # Set ancestors to None to signal that every node is an ancestor.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   610
            ancestors = None
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   611
            # Set heads to an empty dictionary for later discovery of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   612
            heads = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   613
        else:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   614
            heads = list(heads)
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   615
            if not heads:
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   616
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   617
            ancestors = {}
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   618
            # Turn heads into a dictionary so we can remove 'fake' heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   619
            # Also, later we will be using it to filter out the heads we can't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   620
            # find from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   621
            heads = dict.fromkeys(heads, 0)
3360
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
   622
            # Start at the top and keep marking parents until we're done.
ef8307585b41 nodesbetween: fix a bug with duplicate heads
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3335
diff changeset
   623
            nodestotag = heads.keys()
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   624
            # Remember where the top was so we can use it as a limit later.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   625
            highestrev = max([self.rev(n) for n in nodestotag])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   626
            while nodestotag:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   627
                # grab a node to tag
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   628
                n = nodestotag.pop()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   629
                # Never tag nullid
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   630
                if n == nullid:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   631
                    continue
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   632
                # A node's revision number represents its place in a
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   633
                # topologically sorted list of nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   634
                r = self.rev(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   635
                if r >= lowestrev:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   636
                    if n not in ancestors:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   637
                        # If we are possibly a descendent of one of the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   638
                        # and we haven't already been marked as an ancestor
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   639
                        ancestors[n] = 1 # Mark as ancestor
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   640
                        # Add non-nullid parents to list of nodes to tag.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   641
                        nodestotag.extend([p for p in self.parents(n) if
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   642
                                           p != nullid])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   643
                    elif n in heads: # We've seen it before, is it a fake head?
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   644
                        # So it is, real heads should not be the ancestors of
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   645
                        # any other heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   646
                        heads.pop(n)
1459
106fdec8e1fb Fix small bug in nodesbetween if heads is [nullid].
Eric Hopper <hopper@omnifarious.org>
parents: 1458
diff changeset
   647
            if not ancestors:
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   648
                return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   649
            # Now that we have our set of ancestors, we want to remove any
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   650
            # roots that are not ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   651
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   652
            # If one of the roots was nullid, everything is included anyway.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   653
            if lowestrev > nullrev:
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   654
                # But, since we weren't, let's recompute the lowest rev to not
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   655
                # include roots that aren't ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   656
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   657
                # Filter out roots that aren't ancestors of heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   658
                roots = [n for n in roots if n in ancestors]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   659
                # Recompute the lowest revision
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   660
                if roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   661
                    lowestrev = min([self.rev(n) for n in roots])
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   662
                else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   663
                    # No more roots?  Return empty list
1463
26e73acc0cdf Fix to handle case of empty list for roots or heads in nodesbetween.
Eric Hopper <hopper@omnifarious.org>
parents: 1459
diff changeset
   664
                    return nonodes
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   665
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   666
                # We are descending from nullid, and don't need to care about
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   667
                # any other roots.
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   668
                lowestrev = nullrev
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   669
                roots = [nullid]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   670
        # Transform our roots list into a 'set' (i.e. a dictionary where the
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   671
        # values don't matter.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   672
        descendents = dict.fromkeys(roots, 1)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   673
        # Also, keep the original roots so we can filter out roots that aren't
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   674
        # 'real' roots (i.e. are descended from other roots).
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   675
        roots = descendents.copy()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   676
        # Our topologically sorted list of output nodes.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   677
        orderedout = []
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   678
        # Don't start at nullid since we don't want nullid in our output list,
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   679
        # and if nullid shows up in descedents, empty parents will look like
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   680
        # they're descendents.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   681
        for r in xrange(max(lowestrev, 0), highestrev + 1):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   682
            n = self.node(r)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   683
            isdescendent = False
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
   684
            if lowestrev == nullrev:  # Everybody is a descendent of nullid
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   685
                isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   686
            elif n in descendents:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   687
                # n is already a descendent
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   688
                isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   689
                # This check only needs to be done here because all the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   690
                # will start being marked is descendents before the loop.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   691
                if n in roots:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   692
                    # If n was a root, check if it's a 'real' root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   693
                    p = tuple(self.parents(n))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   694
                    # If any of its parents are descendents, it's not a root.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   695
                    if (p[0] in descendents) or (p[1] in descendents):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   696
                        roots.pop(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   697
            else:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   698
                p = tuple(self.parents(n))
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   699
                # A node is a descendent if either of its parents are
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   700
                # descendents.  (We seeded the dependents list with the roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   701
                # up there, remember?)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   702
                if (p[0] in descendents) or (p[1] in descendents):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   703
                    descendents[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   704
                    isdescendent = True
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   705
            if isdescendent and ((ancestors is None) or (n in ancestors)):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   706
                # Only include nodes that are both descendents and ancestors.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   707
                orderedout.append(n)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   708
                if (ancestors is not None) and (n in heads):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   709
                    # We're trying to figure out which heads are reachable
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   710
                    # from roots.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   711
                    # Mark this head as having been reached
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   712
                    heads[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   713
                elif ancestors is None:
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   714
                    # Otherwise, we're trying to discover the heads.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   715
                    # Assume this is a head because if it isn't, the next step
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   716
                    # will eventually remove it.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   717
                    heads[n] = 1
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   718
                    # But, obviously its parents aren't.
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   719
                    for p in self.parents(n):
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   720
                        heads.pop(p, None)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   721
        heads = [n for n in heads.iterkeys() if heads[n] != 0]
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   722
        roots = roots.keys()
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   723
        assert orderedout
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   724
        assert roots
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   725
        assert heads
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   726
        return (orderedout, roots, heads)
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
   727
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   728
    def heads(self, start=None, stop=None):
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   729
        """return the list of all nodes that have no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   730
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   731
        if start is specified, only heads that are descendants of
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   732
        start will be returned
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   733
        if stop is specified, it will consider all the revs from stop
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   734
        as if they had no children
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   735
        """
4991
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   736
        if start is None and stop is None:
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   737
            count = self.count()
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   738
            if not count:
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   739
                return [nullid]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   740
            ishead = [1] * (count + 1)
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   741
            index = self.index
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   742
            for r in xrange(count):
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   743
                e = index[r]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   744
                ishead[e[5]] = ishead[e[6]] = 0
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   745
            return [self.node(r) for r in xrange(count) if ishead[r]]
9c8c42bcf17a revlog: implement a fast path for heads
Matt Mackall <mpm@selenic.com>
parents: 4990
diff changeset
   746
1551
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   747
        if start is None:
e793cbc8be00 Fixes to "hg heads -r FOO":
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1550
diff changeset
   748
            start = nullid
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   749
        if stop is None:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   750
            stop = []
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   751
        stoprevs = dict.fromkeys([self.rev(n) for n in stop])
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   752
        startrev = self.rev(start)
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   753
        reachable = {startrev: 1}
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   754
        heads = {startrev: 1}
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   755
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   756
        parentrevs = self.parentrevs
1550
ccb9b62de892 add a -r/--rev option to heads to show only heads descendant from rev
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1535
diff changeset
   757
        for r in xrange(startrev + 1, self.count()):
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   758
            for p in parentrevs(r):
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   759
                if p in reachable:
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   760
                    if r not in stoprevs:
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   761
                        reachable[r] = 1
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   762
                    heads[r] = 1
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   763
                if p in heads and p not in stoprevs:
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   764
                    del heads[p]
3923
27230c29bfec fix calculation of new heads added during push with -r
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3755
diff changeset
   765
2490
6ff82ec1f4b8 Change revlog.heads to walk the revision graph using revision numbers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2489
diff changeset
   766
        return [self.node(r) for r in heads]
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   767
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   768
    def children(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   769
        """find the children of a given node"""
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   770
        c = []
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   771
        p = self.rev(node)
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   772
        for r in range(p + 1, self.count()):
4746
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   773
            prevs = [pr for pr in self.parentrevs(r) if pr != nullrev]
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   774
            if prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   775
                for pr in prevs:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   776
                    if pr == p:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   777
                        c.append(self.node(r))
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   778
            elif p == nullrev:
62c56d8f368b Fix revlog.children so the real children of the null revision can be calculated.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4635
diff changeset
   779
                c.append(self.node(r))
370
c90385d82aec revlog: add a children function
mpm@selenic.com
parents: 330
diff changeset
   780
        return c
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   781
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   782
    def _match(self, id):
3210
7240f9e47144 correctly find the type of 'id' in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3157
diff changeset
   783
        if isinstance(id, (long, int)):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   784
            # rev
2641
156fb1feab62 lookup should allow -1 to represent nullid (if passed an int as arg)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2600
diff changeset
   785
            return self.node(id)
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   786
        if len(id) == 20:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   787
            # possibly a binary node
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   788
            # odds of a binary node being all hex in ASCII are 1 in 10**25
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   789
            try:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   790
                node = id
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   791
                r = self.rev(node) # quick search the index
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   792
                return node
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   793
            except LookupError:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   794
                pass # may be partial hex id
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   795
        try:
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   796
            # str(rev)
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   797
            rev = int(id)
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   798
            if str(rev) != id:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   799
                raise ValueError
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   800
            if rev < 0:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   801
                rev = self.count() + rev
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   802
            if rev < 0 or rev >= self.count():
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   803
                raise ValueError
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   804
            return self.node(rev)
469
e205194ca7ef Various node id lookup tweaks
mpm@selenic.com
parents: 451
diff changeset
   805
        except (ValueError, OverflowError):
3156
d01e4cb2f5f2 cleanups in revlog.lookup
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3139
diff changeset
   806
            pass
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   807
        if len(id) == 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   808
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   809
                # a full hex nodeid?
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   810
                node = bin(id)
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   811
                r = self.rev(node)
3157
4fe41a9e4591 optimize revlog.lookup when passed hex(node)[:...]
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3156
diff changeset
   812
                return node
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   813
            except TypeError:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   814
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   815
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   816
    def _partialmatch(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   817
        if len(id) < 40:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   818
            try:
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   819
                # hex(node)[:...]
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   820
                bin_id = bin(id[:len(id) & ~1]) # grab an even number of digits
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   821
                node = None
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   822
                for n in self.nodemap:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   823
                    if n.startswith(bin_id) and hex(n).startswith(id):
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   824
                        if node is not None:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   825
                            raise LookupError(_("Ambiguous identifier"))
3438
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   826
                        node = n
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   827
                if node is not None:
b17f9d3eda74 revlog.lookup tweaks
Matt Mackall <mpm@selenic.com>
parents: 3390
diff changeset
   828
                    return node
3453
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   829
            except TypeError:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   830
                pass
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   831
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   832
    def lookup(self, id):
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   833
        """locate a node based on:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   834
            - revision number or str(revision number)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   835
            - nodeid or subset of hex nodeid
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   836
        """
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   837
        n = self._match(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   838
        if n is not None:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   839
            return n
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   840
        n = self._partialmatch(id)
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   841
        if n:
dba3cadef789 Only look up tags and branches as a last resort
Matt Mackall <mpm@selenic.com>
parents: 3438
diff changeset
   842
            return n
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   843
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
   844
        raise LookupError(_("No match found"))
36
da28286bf6b7 Add smart node lookup by substring or by rev number
mpm@selenic.com
parents: 26
diff changeset
   845
2890
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   846
    def cmp(self, node, text):
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   847
        """compare text with a given file revision"""
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   848
        p1, p2 = self.parents(node)
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   849
        return hash(text, p1, p2) != node
5df3e5cf16bc Move cmp bits from filelog to revlog
Matt Mackall <mpm@selenic.com>
parents: 2859
diff changeset
   850
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   851
    def chunk(self, rev, df=None):
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   852
        def loadcache(df):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   853
            if not df:
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   854
                if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   855
                    df = self.opener(self.indexfile)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   856
                else:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   857
                    df = self.opener(self.datafile)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   858
            df.seek(start)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   859
            self._chunkcache = (start, df.read(cache_length))
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   860
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   861
        start, length = self.start(rev), self.length(rev)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   862
        if self._inline:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   863
            start += (rev + 1) * self._io.size
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   864
        end = start + length
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   865
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   866
        offset = 0
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   867
        if not self._chunkcache:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   868
            cache_length = max(65536, length)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   869
            loadcache(df)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   870
        else:
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   871
            cache_start = self._chunkcache[0]
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   872
            cache_length = len(self._chunkcache[1])
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   873
            cache_end = cache_start + cache_length
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   874
            if start >= cache_start and end <= cache_end:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   875
                # it is cached
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   876
                offset = start - cache_start
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   877
            else:
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   878
                cache_length = max(65536, length)
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   879
                loadcache(df)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   880
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   881
        # avoid copying large chunks
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   882
        c = self._chunkcache[1]
5006
c2febf5420e9 revlog: minor chunk speed-up
Matt Mackall <mpm@selenic.com>
parents: 5005
diff changeset
   883
        if cache_length != length:
4988
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   884
            c = c[offset:offset + length]
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   885
14486eea8e7a revlog: speed up chunkcache
Matt Mackall <mpm@selenic.com>
parents: 4987
diff changeset
   886
        return decompress(c)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   887
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   888
    def delta(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   889
        """return or calculate a delta between a node and its predecessor"""
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   890
        r = self.rev(node)
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   891
        return self.revdiff(r - 1, r)
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   892
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   893
    def revdiff(self, rev1, rev2):
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   894
        """return or calculate a delta between two revisions"""
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   895
        if rev1 + 1 == rev2 and self.base(rev1) == self.base(rev2):
1941
7518823709a2 revlog.py: factorization and fixes for rev < 0 (nullid)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1853
diff changeset
   896
            return self.chunk(rev2)
5005
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   897
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   898
        return mdiff.textdiff(self.revision(self.node(rev1)),
72082bfced9a revlog: minor revdiff reorganization
Matt Mackall <mpm@selenic.com>
parents: 5004
diff changeset
   899
                              self.revision(self.node(rev2)))
119
c7a66f9752a4 Add code to retrieve or construct a revlog delta
mpm@selenic.com
parents: 117
diff changeset
   900
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   901
    def revision(self, node):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   902
        """return an uncompressed revision of a given"""
4980
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   903
        if node == nullid:
fc44c8df9d99 revlog: some codingstyle cleanups
Matt Mackall <mpm@selenic.com>
parents: 4979
diff changeset
   904
            return ""
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   905
        if self._cache and self._cache[0] == node:
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   906
            return self._cache[2]
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   907
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   908
        # look up what we need to read
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   909
        text = None
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   910
        rev = self.rev(node)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   911
        base = self.base(rev)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   912
5004
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   913
        # check rev flags
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   914
        if self.index[rev][0] & 0xFFFF:
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
   915
            raise RevlogError(_('incompatible revision flag %x') %
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
   916
                              (self.index[rev][0] & 0xFFFF))
5004
825516d16b25 revlog: move flag checking out of the offset fastpath
Matt Mackall <mpm@selenic.com>
parents: 4996
diff changeset
   917
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   918
        if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   919
            # we probably have the whole chunk cached
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   920
            df = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   921
        else:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   922
            df = self.opener(self.datafile)
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   923
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   924
        # do we have useful data cached?
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   925
        if self._cache and self._cache[1] >= base and self._cache[1] < rev:
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   926
            base = self._cache[1]
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   927
            text = self._cache[2]
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   928
            self._loadindex(base, rev + 1)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   929
        else:
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
   930
            self._loadindex(base, rev + 1)
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
   931
            text = self.chunk(base, df=df)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   932
4989
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
   933
        bins = [self.chunk(r, df) for r in xrange(base + 1, rev + 1)]
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
   934
        text = mdiff.patches(text, bins)
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
   935
        p1, p2 = self.parents(node)
26
9cf83bf9ad38 Simplify integrity checking
mpm@selenic.com
parents: 14
diff changeset
   936
        if node != hash(text, p1, p2):
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
   937
            raise RevlogError(_("integrity check failed on %s:%d")
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   938
                              % (self.datafile, rev))
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   939
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
   940
        self._cache = (node, rev, text)
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
   941
        return text
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
   942
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   943
    def checkinlinesize(self, tr, fp=None):
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   944
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   945
            return
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   946
        if not fp:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
   947
            fp = self.opener(self.indexfile, 'r')
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2081
diff changeset
   948
            fp.seek(0, 2)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   949
        size = fp.tell()
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   950
        if size < 131072:
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   951
            return
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   952
        trinfo = tr.find(self.indexfile)
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   953
        if trinfo == None:
3680
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   954
            raise RevlogError(_("%s not found in the transaction")
69cf255a55a1 Indentation cleanups for 2956948b81f3.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3679
diff changeset
   955
                              % self.indexfile)
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   956
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   957
        trindex = trinfo[2]
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   958
        dataoff = self.start(trindex)
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   959
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   960
        tr.add(self.datafile, dataoff)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   961
        df = self.opener(self.datafile, 'w')
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
   962
        calc = self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   963
        for r in xrange(self.count()):
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   964
            start = self.start(r) + (r + 1) * calc
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   965
            length = self.length(r)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   966
            fp.seek(start)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   967
            d = fp.read(length)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   968
            df.write(d)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   969
        fp.close()
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   970
        df.close()
2076
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   971
        fp = self.opener(self.indexfile, 'w', atomictemp=True)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   972
        self.version &= ~(REVLOGNGINLINEDATA)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   973
        self._inline = False
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
   974
        for i in xrange(self.count()):
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
   975
            e = self._io.packentry(self.index[i], self.node, self.version, i)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   976
            fp.write(e)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   977
2076
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   978
        # if we don't call rename, the temp file will never replace the
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   979
        # real index
d007df6daf8e Create an atomic opener that does not automatically rename on close
mason@suse.com
parents: 2075
diff changeset
   980
        fp.rename()
2084
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   981
Chris Mason <mason@suse.com>
parents: 2082
diff changeset
   982
        tr.replace(self.indexfile, trindex * calc)
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
   983
        self._chunkcache = None
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
   984
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
   985
    def addrevision(self, text, transaction, link, p1, p2, d=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   986
        """add a revision to the log
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   987
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   988
        text - the revision data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   989
        transaction - the transaction object used for rollback
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   990
        link - the linkrev data to add
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   991
        p1, p2 - the parent nodeids of the revision
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   992
        d - an optional precomputed delta
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
   993
        """
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
   994
        dfh = None
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
   995
        if not self._inline:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
   996
            dfh = self.opener(self.datafile, "a")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
   997
        ifh = self.opener(self.indexfile, "a+")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
   998
        return self._addrevision(text, transaction, link, p1, p2, d, ifh, dfh)
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
   999
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1000
    def _addrevision(self, text, transaction, link, p1, p2, d, ifh, dfh):
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1001
        node = hash(text, p1, p2)
301
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1002
        if node in self.nodemap:
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1003
            return node
5add718d92db revlog: allow duplicates
mpm@selenic.com
parents: 243
diff changeset
  1004
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1005
        curr = self.count()
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1006
        prev = curr - 1
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1007
        base = self.base(prev)
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1008
        offset = self.end(prev)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1009
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1010
        if curr:
644
6ebe118280bd Performance enhancements for manifest.add()
mason@suse.com
parents: 547
diff changeset
  1011
            if not d:
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1012
                ptext = self.revision(self.node(prev))
4989
1aaed3d69772 revlog: eliminate diff and patches functions
Matt Mackall <mpm@selenic.com>
parents: 4988
diff changeset
  1013
                d = mdiff.textdiff(ptext, text)
98
3dde7c87e36d Add paranoia to diff code
mpm@selenic.com
parents: 97
diff changeset
  1014
            data = compress(d)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
  1015
            l = len(data[1]) + len(data[0])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1016
            dist = l + offset - self.start(base)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1017
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1018
        # full versions are inserted when the needed deltas
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1019
        # become comparable to the uncompressed text
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1020
        if not curr or dist > len(text) * 2:
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1021
            data = compress(text)
1533
3d11f81c9145 Reduce string duplication in compression code
mason@suse.com
parents: 1509
diff changeset
  1022
            l = len(data[1]) + len(data[0])
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1023
            base = curr
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1024
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1025
        e = (offset_type(offset, 0), l, len(text),
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1026
             base, link, self.rev(p1), self.rev(p2), node)
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1027
        self.index.insert(-1, e)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1028
        self.nodemap[node] = curr
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1029
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
  1030
        entry = self._io.packentry(e, self.node, self.version, curr)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1031
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1032
            transaction.add(self.datafile, offset)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1033
            transaction.add(self.indexfile, curr * len(entry))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1034
            if data[0]:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1035
                dfh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1036
            dfh.write(data[1])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1037
            dfh.flush()
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1038
            ifh.write(entry)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1039
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1040
            offset += curr * self._io.size
5324
8409a2e3a78d revlog: fix inlined revision transaction extra data (issue 749)
Patrick Mezard <pmezard@gmail.com>
parents: 5007
diff changeset
  1041
            transaction.add(self.indexfile, offset, curr)
4981
e7131935fbb3 revlog: simplify addrevision
Matt Mackall <mpm@selenic.com>
parents: 4980
diff changeset
  1042
            ifh.write(entry)
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1043
            ifh.write(data[0])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1044
            ifh.write(data[1])
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1045
            self.checkinlinesize(transaction, ifh)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1046
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
  1047
        self._cache = (node, curr, text)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1048
        return node
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1049
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1050
    def ancestor(self, a, b):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1051
        """calculate the least common ancestor of nodes a and b"""
2081
416d8b2a75b8 Speedup revlog.ancestors for the linear case
Chris Mason <mason@suse.com>
parents: 2080
diff changeset
  1052
3139
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1053
        def parents(rev):
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1054
            return [p for p in self.parentrevs(rev) if p != nullrev]
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1055
3139
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1056
        c = ancestor.ancestor(self.rev(a), self.rev(b), parents)
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1057
        if c is None:
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1058
            return nullid
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1059
1fd1cdcc4200 Switch revlog.ancestor to use revisions rather than nodeids
Matt Mackall <mpm@selenic.com>
parents: 3135
diff changeset
  1060
        return self.node(c)
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
  1061
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1062
    def group(self, nodelist, lookup, infocollect=None):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1063
        """calculate a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1064
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1065
        Given a list of changeset revs, return a set of deltas and
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1066
        metadata corresponding to nodes. the first delta is
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1067
        parent(nodes[0]) -> nodes[0] the receiver is guaranteed to
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1068
        have this parent as it has all history before these
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1069
        changesets. parent is parent[0]
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1070
        """
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1071
        revs = [self.rev(n) for n in nodelist]
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1072
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1073
        # if we don't have any revisions touched by these changesets, bail
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1074
        if not revs:
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1941
diff changeset
  1075
            yield changegroup.closechunk()
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1076
            return
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1077
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1078
        # add the parent of the first rev
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1079
        p = self.parents(self.node(revs[0]))[0]
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1080
        revs.insert(0, self.rev(p))
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1081
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1082
        # build deltas
71
47c9a869adee Add mdiff.patches to speed up applying thousands of patches to the manifest
mpm@selenic.com
parents: 67
diff changeset
  1083
        for d in xrange(0, len(revs) - 1):
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1084
            a, b = revs[d], revs[d + 1]
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1085
            nb = self.node(b)
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1086
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1087
            if infocollect is not None:
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1088
                infocollect(nb)
1458
1033892bbb87 This changes the revlog.group and re-implements the localrepo.changeroup
Eric Hopper <hopper@omnifarious.org>
parents: 1457
diff changeset
  1089
1598
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1090
            p = self.parents(nb)
14d1f1868bf6 cleanup of revlog.group when repository is local
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1559
diff changeset
  1091
            meta = nb + p[0] + p[1] + lookup(nb)
5367
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1092
            if a == -1:
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1093
                d = self.revision(nb)
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1094
                meta += mdiff.trivialdiffheader(len(d))
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1095
            else:
7530334bf301 revlog: generate trivial deltas against null revision
Matt Mackall <mpm@selenic.com>
parents: 5338
diff changeset
  1096
                d = self.revdiff(a, b)
5368
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 5367
diff changeset
  1097
            yield changegroup.chunkheader(len(meta) + len(d))
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 5367
diff changeset
  1098
            yield meta
61462e7d62ed changegroup: avoid large copies
Matt Mackall <mpm@selenic.com>
parents: 5367
diff changeset
  1099
            yield d
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1100
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1941
diff changeset
  1101
        yield changegroup.closechunk()
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1102
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 902
diff changeset
  1103
    def addgroup(self, revs, linkmapper, transaction, unique=0):
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1104
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1105
        add a delta group
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1106
1083
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1107
        given a set of deltas, add them to the revision log. the
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1108
        first delta is against its parent, which should be in our
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1109
        log, the rest are against the previous delta.
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1110
        """
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1111
30974cf73435 Add some docstrings to revlog.py
mpm@selenic.com
parents: 1074
diff changeset
  1112
        #track the base of the current delta log
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1113
        r = self.count()
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1114
        t = r - 1
2002
4aab906517c6 Calling revlog.addgroup with an empty changegroup now raises RevlogError.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1981
diff changeset
  1115
        node = None
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 484
diff changeset
  1116
3578
3b4e00cba57a Define and use nullrev (revision of nullid) instead of -1.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3508
diff changeset
  1117
        base = prev = nullrev
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1118
        start = end = textlen = 0
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1119
        if r:
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1120
            end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1121
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1122
        ifh = self.opener(self.indexfile, "a+")
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1123
        isize = r * self._io.size
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1124
        if self._inline:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1125
            transaction.add(self.indexfile, end + isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1126
            dfh = None
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1127
        else:
4996
a0d37976cd5b revlog: avoid some unnecessary seek/tell syscalls
Matt Mackall <mpm@selenic.com>
parents: 4994
diff changeset
  1128
            transaction.add(self.indexfile, isize, r)
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1129
            transaction.add(self.datafile, end)
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1130
            dfh = self.opener(self.datafile, "a")
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1131
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1132
        # loop through our set of deltas
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1133
        chain = None
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1134
        for chunk in revs:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1135
            node, p1, p2, cs = struct.unpack("20s20s20s20s", chunk[:80])
94
7daef883134f Refactor merge code
mpm@selenic.com
parents: 86
diff changeset
  1136
            link = linkmapper(cs)
77
bed15e766511 Fix bug in lazymap code
mpm@selenic.com
parents: 76
diff changeset
  1137
            if node in self.nodemap:
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
  1138
                # this can happen if two branches make the same change
1218
cde6818e082a Add preliminary support for the bundle and unbundle commands
mpm@selenic.com
parents: 1214
diff changeset
  1139
                # if unique:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
  1140
                #    raise RevlogError(_("already have %s") % hex(node[:4]))
653
94cdd02792b5 Fix corruption resulting from skipping parts of a revision group
Matt Mackall <mpm@selenic.com>
parents: 651
diff changeset
  1141
                chain = node
224
ccbcc4d76f81 fix bad assumption about uniqueness of file versions
mpm@selenic.com
parents: 221
diff changeset
  1142
                continue
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1143
            delta = buffer(chunk, 80)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1144
            del chunk
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1145
1509
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1146
            for p in (p1, p2):
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1147
                if not p in self.nodemap:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
  1148
                    raise LookupError(_("unknown parent %s") % short(p))
1509
46a07392cf28 Add safety check for addgroup
Matt Mackall <mpm@selenic.com>
parents: 1494
diff changeset
  1149
192
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1150
            if not chain:
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1151
                # retrieve the parent revision of the delta chain
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1152
                chain = p1
5d8553352d2e Changes to network protocol
mpm@selenic.com
parents: 155
diff changeset
  1153
                if not chain in self.nodemap:
3930
01d98d68d697 Add revlog.LookupError exception, and use it instead of RevlogError.
Brendan Cully <brendan@kublai.com>
parents: 3928
diff changeset
  1154
                    raise LookupError(_("unknown base %s") % short(chain[:4]))
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1155
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1156
            # full versions are inserted when the needed deltas become
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1157
            # comparable to the uncompressed text or when the previous
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1158
            # version is not the one we have a delta against. We use
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1159
            # the size of the previous full rev as a proxy for the
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1160
            # current size.
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1161
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1162
            if chain == prev:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1163
                cdelta = compress(delta)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1164
                cdeltalen = len(cdelta[0]) + len(cdelta[1])
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1165
                textlen = mdiff.patchedsize(textlen, delta)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1166
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1167
            if chain != prev or (end - start + cdeltalen) > textlen * 2:
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1168
                # flush our writes here so we can read it in revision
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1169
                if dfh:
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1170
                    dfh.flush()
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1171
                ifh.flush()
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
  1172
                text = self.revision(chain)
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1173
                if len(text) == 0:
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1174
                    # skip over trivial delta header
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1175
                    text = buffer(delta, 12)
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1176
                else:
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1177
                    text = mdiff.patches(text, [delta])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1178
                del delta
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1179
                chk = self._addrevision(text, transaction, link, p1, p2, None,
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1180
                                        ifh, dfh)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1181
                if not dfh and not self._inline:
3390
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1182
                    # addrevision switched from inline to conventional
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1183
                    # reopen the index
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1184
                    dfh = self.opener(self.datafile, "a")
a74addddd092 make revlog.addgroup pass its file handles to addrevision
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3360
diff changeset
  1185
                    ifh = self.opener(self.indexfile, "a")
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1186
                if chk != node:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
  1187
                    raise RevlogError(_("consistency error adding group"))
2078
441ea218414e Fill in the uncompressed size during revlog.addgroup
mason@suse.com
parents: 2077
diff changeset
  1188
                textlen = len(text)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1189
            else:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1190
                e = (offset_type(end, 0), cdeltalen, textlen, base,
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1191
                     link, self.rev(p1), self.rev(p2), node)
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1192
                self.index.insert(-1, e)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1193
                self.nodemap[node] = r
5338
f87685355c9c revlog: fix revlogio.packentry corner case
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5325
diff changeset
  1194
                entry = self._io.packentry(e, self.node, self.version, r)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1195
                if self._inline:
4986
58cc017ec7e0 revlog: abstract out index entry packing
Matt Mackall <mpm@selenic.com>
parents: 4985
diff changeset
  1196
                    ifh.write(entry)
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1197
                    ifh.write(cdelta[0])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1198
                    ifh.write(cdelta[1])
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2073
diff changeset
  1199
                    self.checkinlinesize(transaction, ifh)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1200
                    if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1201
                        dfh = self.opener(self.datafile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1202
                        ifh = self.opener(self.indexfile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1203
                else:
5445
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1204
                    dfh.write(cdelta[0])
64cf1c853674 revlog: reduce memory usage in addgroup
Matt Mackall <mpm@selenic.com>
parents: 5368
diff changeset
  1205
                    dfh.write(cdelta[1])
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1206
                    ifh.write(entry)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1207
65
d40cc5aacc31 Fix up a bunch of bugs in the new merge code
mpm@selenic.com
parents: 64
diff changeset
  1208
            t, r, chain, prev = r, r + 1, node, node
1749
d457fec76ab0 fix warnings from pychecker (unused variables and shadowing)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1711
diff changeset
  1209
            base = self.base(t)
d457fec76ab0 fix warnings from pychecker (unused variables and shadowing)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1711
diff changeset
  1210
            start = self.start(base)
46
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1211
            end = self.end(t)
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1212
93e868fa0db8 Add changegroup support
mpm@selenic.com
parents: 45
diff changeset
  1213
        return node
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1214
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1215
    def strip(self, rev, minlink):
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1216
        if self.count() == 0 or rev >= self.count():
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1217
            return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1218
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1219
        if isinstance(self.index, lazyindex):
4920
ee983d0dbea8 revlog: privatize some methods
Matt Mackall <mpm@selenic.com>
parents: 4919
diff changeset
  1220
            self._loadindexmap()
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1221
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1222
        # When stripping away a revision, we need to make sure it
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1223
        # does not actually belong to an older changeset.
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1224
        # The minlink parameter defines the oldest revision
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1225
        # we're allowed to strip away.
4978
93d48a8fa496 revlog: change accesses to index entry elements to use positive offsets
Matt Mackall <mpm@selenic.com>
parents: 4977
diff changeset
  1226
        while minlink > self.index[rev][4]:
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1227
            rev += 1
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1228
            if rev >= self.count():
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1229
                return
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1230
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1231
        # first truncate the files on disk
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1232
        end = self.start(rev)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1233
        if not self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1234
            df = self.opener(self.datafile, "a")
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1235
            df.truncate(end)
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1236
            end = rev * self._io.size
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1237
        else:
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1238
            end += rev * self._io.size
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1239
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1240
        indexf = self.opener(self.indexfile, "a")
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1241
        indexf.truncate(end)
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1242
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1243
        # then reset internal state in memory to forget those revisions
4984
b4066fcbd6ba revlog: mark cache private
Matt Mackall <mpm@selenic.com>
parents: 4983
diff changeset
  1244
        self._cache = None
4983
4dbcfc6e359e revlog: pull chunkcache back into revlog
Matt Mackall <mpm@selenic.com>
parents: 4982
diff changeset
  1245
        self._chunkcache = None
2072
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1246
        for x in xrange(rev, self.count()):
74d3f5336b66 Implement revlogng.
mason@suse.com
parents: 2002
diff changeset
  1247
            del self.nodemap[self.node(x)]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1248
4979
06abdaf78788 revlog: add a magic null revision to our index
Matt Mackall <mpm@selenic.com>
parents: 4978
diff changeset
  1249
        del self.index[rev:-1]
1535
7ae0ce7a3dc4 Add revlog.strip to truncate away revisions.
mason@suse.com
parents: 1533
diff changeset
  1250
1493
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1251
    def checksize(self):
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1252
        expected = 0
1a216cb4ee64 verify: add check for mismatch of index and data length
Matt Mackall <mpm@selenic.com>
parents: 1469
diff changeset
  1253
        if self.count():
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1254
            expected = max(0, self.end(self.count() - 1))
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1255
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1256
        try:
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1257
            f = self.opener(self.datafile)
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1258
            f.seek(0, 2)
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1259
            actual = f.tell()
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1260
            dd = actual - expected
1494
249ca10d37f4 Handle empty logs in repo.checksize
Matt Mackall <mpm@selenic.com>
parents: 1493
diff changeset
  1261
        except IOError, inst:
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1262
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1263
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1264
            dd = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1265
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1266
        try:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1267
            f = self.opener(self.indexfile)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1268
            f.seek(0, 2)
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1269
            actual = f.tell()
4977
6cb30bc4ca32 revlog: parse revlogv0 indexes into v1 internally
Matt Mackall <mpm@selenic.com>
parents: 4976
diff changeset
  1270
            s = self._io.size
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1271
            i = max(0, actual / s)
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1272
            di = actual - (i * s)
4982
9672e3c42b0c revlog: change _inline from a function to a variable
Matt Mackall <mpm@selenic.com>
parents: 4981
diff changeset
  1273
            if self._inline:
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1274
                databytes = 0
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1275
                for r in xrange(self.count()):
5312
fb070713ff36 revlog: more robust for damaged indexes
Matt Mackall <mpm@selenic.com>
parents: 5007
diff changeset
  1276
                    databytes += max(0, self.length(r))
2073
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1277
                dd = 0
1e6745f78989 Implement data inlined with the index file
mason@suse.com
parents: 2072
diff changeset
  1278
                di = actual - self.count() * s - databytes
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1279
        except IOError, inst:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1280
            if inst.errno != errno.ENOENT:
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1281
                raise
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1282
            di = 0
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1283
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
  1284
        return (dd, di)