mercurial/revlogutils/constants.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Mon, 05 Apr 2021 12:21:12 +0200
changeset 46858 85e3a630cad9
parent 46857 cc65cea90edb
child 46859 c6e23fb4bfb4
permissions -rw-r--r--
revlog: move the details of revlog "v2" index inside revlog.utils.constants the revlog module is quite large and this kind of format information would handy for other module. So let us start to gather this information about the format in a more appropriate place. We update various reference to this information to use the new "source of truth" in the process. Differential Revision: https://phab.mercurial-scm.org/D10305
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     1
# revlogdeltas.py - constant used for revlog logic
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     2
#
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 46704
diff changeset
     3
# Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     4
# Copyright 2018 Octobus <contact@octobus.net>
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     5
#
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     6
# This software may be used and distributed according to the terms of the
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     7
# GNU General Public License version 2 or any later version.
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     8
"""Helper class to compute deltas stored inside revlogs"""
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     9
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    10
from __future__ import absolute_import
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    11
46856
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    12
import struct
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    13
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
    14
from ..interfaces import repository
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    15
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    16
### main revlog header
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    17
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    18
## revlog version
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    19
REVLOGV0 = 0
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    20
REVLOGV1 = 1
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    21
# Dummy value until file format is finalized.
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    22
REVLOGV2 = 0xDEAD
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    23
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    24
##  global revlog header flags
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
    25
# Shared across v1 and v2.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
    26
FLAG_INLINE_DATA = 1 << 16
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
    27
# Only used by v1, implied by v2.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
    28
FLAG_GENERALDELTA = 1 << 17
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    29
REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    30
REVLOG_DEFAULT_FORMAT = REVLOGV1
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    31
REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    32
REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA
41202
e7a2cc84dbc0 revlog: always enable generaldelta on version 2 revlogs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40047
diff changeset
    33
REVLOGV2_FLAGS = FLAG_INLINE_DATA
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    34
46855
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    35
### individual entry
aba724bf550e revlog: add some comment in the header sections
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
    36
46856
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    37
## index v0:
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    38
#  4 bytes: offset
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    39
#  4 bytes: compressed length
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    40
#  4 bytes: base rev
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    41
#  4 bytes: link rev
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    42
# 20 bytes: parent 1 nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    43
# 20 bytes: parent 2 nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    44
# 20 bytes: nodeid
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    45
INDEX_ENTRY_V0 = struct.Struct(b">4l20s20s20s")
34e1fa4b548a revlog: move the details of revlog "v0" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46855
diff changeset
    46
46857
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    47
## index v1
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    48
#  6 bytes: offset
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    49
#  2 bytes: flags
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    50
#  4 bytes: compressed length
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    51
#  4 bytes: uncompressed length
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    52
#  4 bytes: base rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    53
#  4 bytes: link rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    54
#  4 bytes: parent 1 rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    55
#  4 bytes: parent 2 rev
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    56
# 32 bytes: nodeid
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    57
INDEX_ENTRY_V1 = struct.Struct(b">Qiiiiii20s12x")
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    58
assert INDEX_ENTRY_V1.size == 32 * 2
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46856
diff changeset
    59
46858
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    60
#  6 bytes: offset
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    61
#  2 bytes: flags
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    62
#  4 bytes: compressed length
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    63
#  4 bytes: uncompressed length
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    64
#  4 bytes: base rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    65
#  4 bytes: link rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    66
#  4 bytes: parent 1 rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    67
#  4 bytes: parent 2 rev
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    68
# 32 bytes: nodeid
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    69
#  8 bytes: sidedata offset
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    70
#  4 bytes: sidedata compressed length
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    71
#  20 bytes: Padding to align to 96 bytes (see RevlogV2Plan wiki page)
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    72
INDEX_ENTRY_V2 = struct.Struct(b">Qiiiiii20s12xQi20x")
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    73
assert INDEX_ENTRY_V2.size == 32 * 3
85e3a630cad9 revlog: move the details of revlog "v2" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46857
diff changeset
    74
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    75
# revlog index flags
40047
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    76
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    77
# For historical reasons, revlog's internal flags were exposed via the
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    78
# wire protocol and are even exposed in parts of the storage APIs.
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    79
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    80
# revision has censor metadata, must be verified
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    81
REVIDX_ISCENSORED = repository.REVISION_FLAG_CENSORED
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    82
# revision hash does not match data (narrowhg)
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    83
REVIDX_ELLIPSIS = repository.REVISION_FLAG_ELLIPSIS
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    84
# revision data is stored externally
8e398628a3f2 repository: define and use revision flag constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39506
diff changeset
    85
REVIDX_EXTSTORED = repository.REVISION_FLAG_EXTSTORED
43032
a12a9af7536c sidedata: add a new revision flag constant for side data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
    86
# revision data contains extra metadata not part of the official digest
a12a9af7536c sidedata: add a new revision flag constant for side data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
    87
REVIDX_SIDEDATA = repository.REVISION_FLAG_SIDEDATA
45671
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    88
# revision changes files in a way that could affect copy tracing.
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    89
REVIDX_HASCOPIESINFO = repository.REVISION_FLAG_HASCOPIESINFO
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    90
REVIDX_DEFAULT_FLAGS = 0
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    91
# stable order in which flags need to be processed and their processors applied
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    92
REVIDX_FLAGS_ORDER = [
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    93
    REVIDX_ISCENSORED,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    94
    REVIDX_ELLIPSIS,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    95
    REVIDX_EXTSTORED,
43032
a12a9af7536c sidedata: add a new revision flag constant for side data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
    96
    REVIDX_SIDEDATA,
45671
2d6aea053153 copies: add a HASCOPIESINFO flag to highlight rev with useful data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    97
    REVIDX_HASCOPIESINFO,
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    98
]
42730
92ac6b1697a7 flagutil: move REVIDX_KNOWN_FLAGS source of truth in flagutil (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41202
diff changeset
    99
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   100
# bitmark for flags that could cause rawdata content change
43032
a12a9af7536c sidedata: add a new revision flag constant for side data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
   101
REVIDX_RAWTEXT_CHANGING_FLAGS = (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43032
diff changeset
   102
    REVIDX_ISCENSORED | REVIDX_EXTSTORED | REVIDX_SIDEDATA
43032
a12a9af7536c sidedata: add a new revision flag constant for side data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42813
diff changeset
   103
)
39506
b66ea3fc3a86 sparse-revlog: set max delta chain length to on thousand
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
   104
b66ea3fc3a86 sparse-revlog: set max delta chain length to on thousand
Boris Feld <boris.feld@octobus.net>
parents: 39330
diff changeset
   105
SPARSE_REVLOG_MAX_CHAIN_LENGTH = 1000