mercurial/pure/parsers.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Wed, 15 Jan 2020 15:48:09 +0100
changeset 44313 6f9e8e142cea
parent 44034 ab595920de0e
child 44314 7f4f7ef3133e
permissions -rw-r--r--
nodemap: add a (python) index class for persistent nodemap testing Using the persistent nodemap require a compeling performance boost and an existing implementation. The benefit of the persistent nodemap for pure python code is unclear and we don't have a C implementation for it. Yet we would like to actually start testing it in more details and define an API for using that persistent nodemap. We introduce a new `devel` config option to use an index class dedicated to Nodemap Testing. This feature is "pure" only because having using a pure-python index with the `cext` policy proved more difficult than I would like. There is nothing going on in that class for now, but the coming changeset will change that. Differential Revision: https://phab.mercurial-scm.org/D7840
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
     1
# parsers.py - Python implementation of parsers.c
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
     2
#
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
     3
# Copyright 2009 Matt Mackall <mpm@selenic.com> and others
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
     4
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 7945
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 8225
diff changeset
     6
# GNU General Public License version 2 or any later version.
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
     7
27339
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
     8
from __future__ import absolute_import
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
     9
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
    10
import struct
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
    11
import zlib
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
    12
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    13
from ..node import nullid, nullrev
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    14
from .. import (
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    15
    pycompat,
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    16
    util,
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    17
)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
    18
44034
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
    19
from ..revlogutils import nodemap as nodemaputil
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
    20
36958
644a02f6b34f util: prefer "bytesio" to "stringio"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34331
diff changeset
    21
stringio = pycompat.bytesio
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    22
31220
37596c980662 parsers: alias long to int on Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29133
diff changeset
    23
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    24
_pack = struct.pack
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    25
_unpack = struct.unpack
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    26
_compress = zlib.compress
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    27
_decompress = zlib.decompress
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
    28
21809
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    29
# Some code below makes tuples directly because it's more convenient. However,
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    30
# code outside this module should always use dirstatetuple.
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    31
def dirstatetuple(*x):
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    32
    # x is a tuple
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    33
    return x
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
    34
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
    35
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    36
indexformatng = b">Qiiiiii20s12x"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    37
indexfirst = struct.calcsize(b'Q')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    38
sizeint = struct.calcsize(b'i')
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    39
indexsize = struct.calcsize(indexformatng)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    40
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
    41
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    42
def gettype(q):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    43
    return int(q & 0xFFFF)
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
    44
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
    45
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    46
def offset_type(offset, type):
31529
61ff3852f6ed pure: use int instead of long
Martin von Zweigbergk <martinvonz@google.com>
parents: 31220
diff changeset
    47
    return int(int(offset) << 16 | type)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    48
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
    49
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    50
class BaseIndexObject(object):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    51
    @property
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    52
    def nodemap(self):
43619
c207c46a86b9 py3: pass a bytes value for "msg" to nouideprecwarn()
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 43574
diff changeset
    53
        msg = b"index.nodemap is deprecated, use index.[has_node|rev|get_rev]"
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    54
        util.nouideprecwarn(msg, b'5.3', stacklevel=2)
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    55
        return self._nodemap
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    56
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    57
    @util.propertycache
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    58
    def _nodemap(self):
44034
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
    59
        nodemap = nodemaputil.NodeMap({nullid: nullrev})
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    60
        for r in range(0, len(self)):
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    61
            n = self[r][7]
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    62
            nodemap[n] = r
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    63
        return nodemap
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    64
43534
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
    65
    def has_node(self, node):
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
    66
        """return True if the node exist in the index"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    67
        return node in self._nodemap
43534
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
    68
43552
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
    69
    def rev(self, node):
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
    70
        """return a revision for a node
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
    71
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
    72
        If the node is unknown, raise a RevlogError"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    73
        return self._nodemap[node]
43552
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
    74
43554
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
    75
    def get_rev(self, node):
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
    76
        """return a revision for a node
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
    77
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
    78
        If the node is unknown, return None"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    79
        return self._nodemap.get(node)
43554
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
    80
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
    81
    def _stripnodes(self, start):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    82
        if '_nodemap' in vars(self):
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
    83
            for r in range(start, len(self)):
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
    84
                n = self[r][7]
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    85
                del self._nodemap[n]
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
    86
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    87
    def clearcaches(self):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    88
        self.__dict__.pop('_nodemap', None)
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
    89
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    90
    def __len__(self):
38851
781b2720d2ac index: don't include nullid in len()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38850
diff changeset
    91
        return self._lgt + len(self._extra)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    92
38850
6104b203bec8 index: replace insert(-1, e) method by append(e) method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38848
diff changeset
    93
    def append(self, tup):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    94
        if '_nodemap' in vars(self):
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
    95
            self._nodemap[tup[7]] = len(self)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    96
        self._extra.append(tup)
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
    97
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
    98
    def _check_index(self, i):
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
    99
        if not isinstance(i, int):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   100
            raise TypeError(b"expecting int indexes")
39216
ec6d5a9d1631 index: don't include nullid in boundary check in pure code
Martin von Zweigbergk <martinvonz@google.com>
parents: 39047
diff changeset
   101
        if i < 0 or i >= len(self):
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   102
            raise IndexError
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
   103
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   104
    def __getitem__(self, i):
39047
a1f934573c0b parsers: adjust pure-python version to mimic a3dacabd476b
Augie Fackler <augie@google.com>
parents: 38851
diff changeset
   105
        if i == -1:
38847
f3d394ea17db index: handle index[-1] as nullid more explicitly
Martin von Zweigbergk <martinvonz@google.com>
parents: 36958
diff changeset
   106
            return (0, 0, 0, -1, -1, -1, -1, nullid)
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
   107
        self._check_index(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   108
        if i >= self._lgt:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   109
            return self._extra[i - self._lgt]
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   110
        index = self._calculate_index(i)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   111
        r = struct.unpack(indexformatng, self._data[index : index + indexsize])
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   112
        if i == 0:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   113
            e = list(r)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   114
            type = gettype(e[0])
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   115
            e[0] = offset_type(0, type)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   116
            return tuple(e)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   117
        return r
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   118
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   119
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   120
class IndexObject(BaseIndexObject):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   121
    def __init__(self, data):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   122
        assert len(data) % indexsize == 0
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   123
        self._data = data
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   124
        self._lgt = len(data) // indexsize
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   125
        self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   126
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   127
    def _calculate_index(self, i):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   128
        return i * indexsize
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
   129
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   130
    def __delitem__(self, i):
34331
531332502568 style: always use `x is not None` instead of `not x is None`
Alex Gaynor <agaynor@mozilla.com>
parents: 32372
diff changeset
   131
        if not isinstance(i, slice) or not i.stop == -1 or i.step is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   132
            raise ValueError(b"deleting slices only supports a:-1 with step 1")
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
   133
        i = i.start
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
   134
        self._check_index(i)
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
   135
        self._stripnodes(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   136
        if i < self._lgt:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   137
            self._data = self._data[: i * indexsize]
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   138
            self._lgt = i
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   139
            self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   140
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   141
            self._extra = self._extra[: i - self._lgt]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   142
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   143
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   144
class PersistentNodeMapIndexObject(IndexObject):
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   145
    """a Debug oriented class to test persistent nodemap
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   146
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   147
    We need a simple python object to test API and higher level behavior. See
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   148
    the Rust implementation for  more serious usage. This should be used only
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   149
    through the dedicated `devel.persistent-nodemap` config.
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   150
    """
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   151
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   152
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   153
class InlinedIndexObject(BaseIndexObject):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   154
    def __init__(self, data, inline=0):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   155
        self._data = data
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   156
        self._lgt = self._inline_scan(None)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   157
        self._inline_scan(self._lgt)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   158
        self._extra = []
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   159
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   160
    def _inline_scan(self, lgt):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   161
        off = 0
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   162
        if lgt is not None:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   163
            self._offsets = [0] * lgt
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   164
        count = 0
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   165
        while off <= len(self._data) - indexsize:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   166
            (s,) = struct.unpack(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   167
                b'>i', self._data[off + indexfirst : off + sizeint + indexfirst]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   168
            )
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   169
            if lgt is not None:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   170
                self._offsets[count] = off
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   171
            count += 1
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   172
            off += indexsize + s
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   173
        if off != len(self._data):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   174
            raise ValueError(b"corrupted data")
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   175
        return count
14421
639f26cab2f5 pure parsers: properly detect corrupt index files
Augie Fackler <durin42@gmail.com>
parents: 14064
diff changeset
   176
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   177
    def __delitem__(self, i):
34331
531332502568 style: always use `x is not None` instead of `not x is None`
Alex Gaynor <agaynor@mozilla.com>
parents: 32372
diff changeset
   178
        if not isinstance(i, slice) or not i.stop == -1 or i.step is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   179
            raise ValueError(b"deleting slices only supports a:-1 with step 1")
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
   180
        i = i.start
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
   181
        self._check_index(i)
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
   182
        self._stripnodes(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   183
        if i < self._lgt:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   184
            self._offsets = self._offsets[:i]
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   185
            self._lgt = i
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   186
            self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   187
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   188
            self._extra = self._extra[: i - self._lgt]
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   189
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   190
    def _calculate_index(self, i):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   191
        return self._offsets[i]
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   192
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   193
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   194
def parse_index2(data, inline):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   195
    if not inline:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   196
        return IndexObject(data), None
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
   197
    return InlinedIndexObject(data, inline), (0, data)
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   198
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   199
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   200
def parse_index_devel_nodemap(data, inline):
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   201
    """like parse_index2, but alway return a PersistentNodeMapIndexObject
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   202
    """
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   203
    return PersistentNodeMapIndexObject(data), None
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   204
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
   205
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   206
def parse_dirstate(dmap, copymap, st):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   207
    parents = [st[:20], st[20:40]]
17425
e95ec38f86b0 fix wording and not-completely-trivial spelling errors and bad docstrings
Mads Kiilerich <mads@kiilerich.com>
parents: 14995
diff changeset
   208
    # dereference fields so they will be local in loop
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   209
    format = b">cllll"
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
   210
    e_size = struct.calcsize(format)
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   211
    pos1 = 40
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   212
    l = len(st)
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   213
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   214
    # the inner loop
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   215
    while pos1 < l:
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   216
        pos2 = pos1 + e_size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   217
        e = _unpack(b">cllll", st[pos1:pos2])  # a literal here is faster
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   218
        pos1 = pos2 + e[4]
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   219
        f = st[pos2:pos1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   220
        if b'\0' in f:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   221
            f, c = f.split(b'\0')
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   222
            copymap[f] = c
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   223
        dmap[f] = e[:4]
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
   224
    return parents
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   225
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
   226
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   227
def pack_dirstate(dmap, copymap, pl, now):
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   228
    now = int(now)
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 27339
diff changeset
   229
    cs = stringio()
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   230
    write = cs.write
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   231
    write(b"".join(pl))
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   232
    for f, e in pycompat.iteritems(dmap):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   233
        if e[0] == b'n' and e[3] == now:
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   234
            # The file was last modified "simultaneously" with the current
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   235
            # write to dirstate (i.e. within the same second for file-
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   236
            # systems with a granularity of 1 sec). This commonly happens
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   237
            # for at least a couple of files on 'update'.
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   238
            # The user could change the file without changing its size
19652
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
   239
            # within the same second. Invalidate the file's mtime in
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   240
            # dirstate, forcing future 'status' calls to compare the
19652
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
   241
            # contents of the file if the size is the same. This prevents
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
   242
            # mistakenly treating such files as clean.
21809
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
   243
            e = dirstatetuple(e[0], e[1], e[2], -1)
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   244
            dmap[f] = e
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   245
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   246
        if f in copymap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   247
            f = b"%s\0%s" % (f, copymap[f])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   248
        e = _pack(b">cllll", e[0], e[1], e[2], e[3], len(f))
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   249
        write(e)
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   250
        write(f)
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
   251
    return cs.getvalue()