hgext/sqlitestore.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Wed, 09 Nov 2022 13:54:15 -0500
branchstable
changeset 49789 df750b81272f
parent 49284 d44e3c45f0e4
child 49845 e0c0545e2e55
permissions -rw-r--r--
sqlitestore: add an `ancestors` method We will need it during bundling. The implementation mirror the one in revlog.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# sqlitestore.py - Storage backend that uses SQLite
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
"""store repository data in SQLite (EXPERIMENTAL)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    10
The sqlitestore extension enables the storage of repository data in SQLite.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    11
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    12
This extension is HIGHLY EXPERIMENTAL. There are NO BACKWARDS COMPATIBILITY
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    13
GUARANTEES. This means that repositories created with this extension may
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    14
only be usable with the exact version of this extension/Mercurial that was
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
used. The extension attempts to enforce this in order to prevent repository
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
corruption.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    18
In addition, several features are not yet supported or have known bugs:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    19
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
* Only some data is stored in SQLite. Changeset, manifest, and other repository
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
  data is not yet stored in SQLite.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
* Transactions are not robust. If the process is aborted at the right time
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
  during transaction close/rollback, the repository could be in an inconsistent
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
  state. This problem will diminish once all repository data is tracked by
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
  SQLite.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
* Bundle repositories do not work (the ability to use e.g.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
  `hg -R <bundle-file> log` to automatically overlay a bundle on top of the
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
  existing repository).
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
* Various other features don't work.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
This extension should work for basic clone/pull, update, and commit workflows.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
Some history rewriting operations may fail due to lack of support for bundle
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
repositories.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
To use, activate the extension and set the ``storage.new-repo-backend`` config
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
option to ``sqlite`` to enable new repositories to use SQLite for storage.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    37
"""
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    38
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    39
# To run the test suite with repos using SQLite by default, execute the
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
# following:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
#
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
# HGREPOFEATURES="sqlitestore" run-tests.py \
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    43
#     --extra-config-opt extensions.sqlitestore= \
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
#     --extra-config-opt storage.new-repo-backend=sqlite
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    45
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    46
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
import sqlite3
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
import struct
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    49
import threading
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    50
import zlib
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
from mercurial.i18n import _
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
from mercurial.node import (
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
    nullrev,
46780
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46715
diff changeset
    55
    sha1nodeconstants,
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    56
    short,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
    58
from mercurial.thirdparty import attr
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    59
from mercurial import (
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    60
    ancestor,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
    dagop,
40410
3b782669561d py3: make sure we pass sysstr in sqlite3.connect()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40392
diff changeset
    62
    encoding,
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
    error,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
    extensions,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
    localrepo,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
    mdiff,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
    pycompat,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    68
    registrar,
45372
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44062
diff changeset
    69
    requirements,
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
    util,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    71
    verify,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    72
)
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42772
diff changeset
    73
from mercurial.interfaces import (
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42772
diff changeset
    74
    repository,
42814
2c4f656c8e9f interfaceutil: move to interfaces/
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42813
diff changeset
    75
    util as interfaceutil,
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42772
diff changeset
    76
)
44062
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
    77
from mercurial.utils import (
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
    78
    hashutil,
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
    79
    storageutil,
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
    80
)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
try:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
    from mercurial import zstd
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
    84
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
    zstd.__version__
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
except ImportError:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
    zstd = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
configtable = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
configitem = registrar.configitem(configtable)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    92
# experimental config: storage.sqlite.compression
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
    93
configitem(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    94
    b'storage',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    95
    b'sqlite.compression',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    96
    default=b'zstd' if zstd else b'zlib',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
    97
    experimental=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
    98
)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
# be specifying the version(s) of Mercurial they are tested with, or
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
# leave the attribute unspecified.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   104
testedwith = b'ships-with-hg-core'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   105
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   106
REQUIREMENT = b'exp-sqlite-001'
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   107
REQUIREMENT_ZSTD = b'exp-sqlite-comp-001=zstd'
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   108
REQUIREMENT_ZLIB = b'exp-sqlite-comp-001=zlib'
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   109
REQUIREMENT_NONE = b'exp-sqlite-comp-001=none'
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
   110
REQUIREMENT_SHALLOW_FILES = b'exp-sqlite-shallow-files'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   111
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   112
CURRENT_SCHEMA_VERSION = 1
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   113
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   114
COMPRESSION_NONE = 1
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   115
COMPRESSION_ZSTD = 2
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   116
COMPRESSION_ZLIB = 3
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   117
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   118
FLAG_CENSORED = 1
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   119
FLAG_MISSING_P1 = 2
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   120
FLAG_MISSING_P2 = 4
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   121
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   122
CREATE_SCHEMA = [
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   123
    # Deltas are stored as content-indexed blobs.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   124
    # compression column holds COMPRESSION_* constant for how the
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   125
    # delta is encoded.
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   126
    'CREATE TABLE delta ('
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   127
    '    id INTEGER PRIMARY KEY, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   128
    '    compression INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   129
    '    hash BLOB UNIQUE ON CONFLICT ABORT, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   130
    '    delta BLOB NOT NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   131
    ')',
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
    # Tracked paths are denormalized to integers to avoid redundant
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   133
    # storage of the path name.
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   134
    'CREATE TABLE filepath ('
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   135
    '    id INTEGER PRIMARY KEY, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   136
    '    path BLOB NOT NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   137
    ')',
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   138
    'CREATE UNIQUE INDEX filepath_path ON filepath (path)',
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   139
    # We have a single table for all file revision data.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   140
    # Each file revision is uniquely described by a (path, rev) and
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   141
    # (path, node).
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   142
    #
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   143
    # Revision data is stored as a pointer to the delta producing this
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
    # revision and the file revision whose delta should be applied before
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   145
    # that one. One can reconstruct the delta chain by recursively following
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   146
    # the delta base revision pointers until one encounters NULL.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   147
    #
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   148
    # flags column holds bitwise integer flags controlling storage options.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   149
    # These flags are defined by the FLAG_* constants.
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   150
    'CREATE TABLE fileindex ('
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   151
    '    id INTEGER PRIMARY KEY, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   152
    '    pathid INTEGER REFERENCES filepath(id), '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   153
    '    revnum INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   154
    '    p1rev INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   155
    '    p2rev INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   156
    '    linkrev INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   157
    '    flags INTEGER NOT NULL, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   158
    '    deltaid INTEGER REFERENCES delta(id), '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   159
    '    deltabaseid INTEGER REFERENCES fileindex(id), '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   160
    '    node BLOB NOT NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   161
    ')',
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   162
    'CREATE UNIQUE INDEX fileindex_pathrevnum '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   163
    '    ON fileindex (pathid, revnum)',
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   164
    'CREATE UNIQUE INDEX fileindex_pathnode ON fileindex (pathid, node)',
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   165
    # Provide a view over all file data for convenience.
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   166
    'CREATE VIEW filedata AS '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   167
    'SELECT '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   168
    '    fileindex.id AS id, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   169
    '    filepath.id AS pathid, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   170
    '    filepath.path AS path, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   171
    '    fileindex.revnum AS revnum, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   172
    '    fileindex.node AS node, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   173
    '    fileindex.p1rev AS p1rev, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   174
    '    fileindex.p2rev AS p2rev, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   175
    '    fileindex.linkrev AS linkrev, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   176
    '    fileindex.flags AS flags, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   177
    '    fileindex.deltaid AS deltaid, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   178
    '    fileindex.deltabaseid AS deltabaseid '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   179
    'FROM filepath, fileindex '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   180
    'WHERE fileindex.pathid=filepath.id',
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   181
    'PRAGMA user_version=%d' % CURRENT_SCHEMA_VERSION,
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   184
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   185
def resolvedeltachain(db, pathid, node, revisioncache, stoprids, zstddctx=None):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   186
    """Resolve a delta chain for a file node."""
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   187
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
    # TODO the "not in ({stops})" here is possibly slowing down the query
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
    # because it needs to perform the lookup on every recursive invocation.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   190
    # This could possibly be faster if we created a temporary query with
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   191
    # baseid "poisoned" to null and limited the recursive filter to
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   192
    # "is not null".
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   193
    res = db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   194
        'WITH RECURSIVE '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   195
        '    deltachain(deltaid, baseid) AS ('
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   196
        '        SELECT deltaid, deltabaseid FROM fileindex '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   197
        '            WHERE pathid=? AND node=? '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   198
        '        UNION ALL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   199
        '        SELECT fileindex.deltaid, deltabaseid '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   200
        '            FROM fileindex, deltachain '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   201
        '            WHERE '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   202
        '                fileindex.id=deltachain.baseid '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   203
        '                AND deltachain.baseid IS NOT NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   204
        '                AND fileindex.id NOT IN ({stops}) '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   205
        '    ) '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   206
        'SELECT deltachain.baseid, compression, delta '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   207
        'FROM deltachain, delta '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   208
        'WHERE delta.id=deltachain.deltaid'.format(
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   209
            stops=','.join(['?'] * len(stoprids))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   210
        ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   211
        tuple([pathid, node] + list(stoprids.keys())),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   212
    )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   213
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   214
    deltas = []
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   215
    lastdeltabaseid = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   216
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
    for deltabaseid, compression, delta in res:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
        lastdeltabaseid = deltabaseid
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
        if compression == COMPRESSION_ZSTD:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   221
            delta = zstddctx.decompress(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   222
        elif compression == COMPRESSION_NONE:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   223
            delta = delta
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   224
        elif compression == COMPRESSION_ZLIB:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   225
            delta = zlib.decompress(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   226
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   227
            raise SQLiteStoreError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   228
                b'unhandled compression type: %d' % compression
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   229
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   230
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   231
        deltas.append(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   232
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   233
    if lastdeltabaseid in stoprids:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   234
        basetext = revisioncache[stoprids[lastdeltabaseid]]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   235
    else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   236
        basetext = deltas.pop()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   237
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   238
    deltas.reverse()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   239
    fulltext = mdiff.patches(basetext, deltas)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   240
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   241
    # SQLite returns buffer instances for blob columns on Python 2. This
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   242
    # type can propagate through the delta application layer. Because
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   243
    # downstream callers assume revisions are bytes, cast as needed.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   244
    if not isinstance(fulltext, bytes):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   245
        fulltext = bytes(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   246
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   247
    return fulltext
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   248
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   249
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   250
def insertdelta(db, compression, hash, delta):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   251
    try:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   252
        return db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   253
            'INSERT INTO delta (compression, hash, delta) VALUES (?, ?, ?)',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   254
            (compression, hash, delta),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   255
        ).lastrowid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   256
    except sqlite3.IntegrityError:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   257
        return db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   258
            'SELECT id FROM delta WHERE hash=?', (hash,)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   259
        ).fetchone()[0]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   260
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   261
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   262
class SQLiteStoreError(error.StorageError):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   263
    pass
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   264
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   265
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   266
@attr.s
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   267
class revisionentry:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   268
    rid = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   269
    rev = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   270
    node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   271
    p1rev = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   272
    p2rev = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   273
    p1node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   274
    p2node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   275
    linkrev = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   276
    flags = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   277
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   278
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   279
@interfaceutil.implementer(repository.irevisiondelta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   280
@attr.s(slots=True)
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   281
class sqliterevisiondelta:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   282
    node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   283
    p1node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   284
    p2node = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   285
    basenode = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   286
    flags = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   287
    baserevisionsize = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   288
    revision = attr.ib()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   289
    delta = attr.ib()
46712
e8c11a2c96c0 delta: add sidedata field to revision delta
Raphaël Gomès <rgomes@octobus.net>
parents: 46711
diff changeset
   290
    sidedata = attr.ib()
47077
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata
Raphaël Gomès <rgomes@octobus.net>
parents: 47012
diff changeset
   291
    protocol_flags = attr.ib()
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
    linknode = attr.ib(default=None)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   293
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   294
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   295
@interfaceutil.implementer(repository.iverifyproblem)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   296
@attr.s(frozen=True)
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   297
class sqliteproblem:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   298
    warning = attr.ib(default=None)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   299
    error = attr.ib(default=None)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   300
    node = attr.ib(default=None)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   301
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   302
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   303
@interfaceutil.implementer(repository.ifilestorage)
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   304
class sqlitefilestore:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   305
    """Implements storage for an individual tracked path."""
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   306
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   307
    def __init__(self, db, path, compression):
46780
6266d19556ad node: introduce nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46715
diff changeset
   308
        self.nullid = sha1nodeconstants.nullid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   309
        self._db = db
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   310
        self._path = path
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   311
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   312
        self._pathid = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   313
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   314
        # revnum -> node
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   315
        self._revtonode = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   316
        # node -> revnum
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   317
        self._nodetorev = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
        # node -> data structure
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   319
        self._revisions = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   320
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   321
        self._revisioncache = util.lrucachedict(10)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   322
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   323
        self._compengine = compression
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   324
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   325
        if compression == b'zstd':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   326
            self._cctx = zstd.ZstdCompressor(level=3)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   327
            self._dctx = zstd.ZstdDecompressor()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   328
        else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   329
            self._cctx = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   330
            self._dctx = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   331
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   332
        self._refreshindex()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   333
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   334
    def _refreshindex(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   335
        self._revtonode = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   336
        self._nodetorev = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   337
        self._revisions = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   338
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   339
        res = list(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   340
            self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   341
                'SELECT id FROM filepath WHERE path=?', (self._path,)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   342
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   343
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   344
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   345
        if not res:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   346
            self._pathid = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   347
            return
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   348
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   349
        self._pathid = res[0][0]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   350
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   351
        res = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   352
            'SELECT id, revnum, node, p1rev, p2rev, linkrev, flags '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   353
            'FROM fileindex '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   354
            'WHERE pathid=? '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   355
            'ORDER BY revnum ASC',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   356
            (self._pathid,),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   357
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   358
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   359
        for i, row in enumerate(res):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   360
            rid, rev, node, p1rev, p2rev, linkrev, flags = row
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   361
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   362
            if i != rev:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   363
                raise SQLiteStoreError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43077
diff changeset
   364
                    _(b'sqlite database has inconsistent revision numbers')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   365
                )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   366
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   367
            if p1rev == nullrev:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   368
                p1node = sha1nodeconstants.nullid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   369
            else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   370
                p1node = self._revtonode[p1rev]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   371
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   372
            if p2rev == nullrev:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   373
                p2node = sha1nodeconstants.nullid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   374
            else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   375
                p2node = self._revtonode[p2rev]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   376
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   377
            entry = revisionentry(
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   378
                rid=rid,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   379
                rev=rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   380
                node=node,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   381
                p1rev=p1rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   382
                p2rev=p2rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   383
                p1node=p1node,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   384
                p2node=p2node,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   385
                linkrev=linkrev,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   386
                flags=flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   387
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   388
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   389
            self._revtonode[rev] = node
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   390
            self._nodetorev[node] = rev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   391
            self._revisions[node] = entry
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   392
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   393
    # Start of ifileindex interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   394
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   395
    def __len__(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   396
        return len(self._revisions)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   397
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   398
    def __iter__(self):
49284
d44e3c45f0e4 py3: replace `pycompat.xrange` by `range`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
   399
        return iter(range(len(self._revisions)))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   400
40387
f1a39128da95 filelog: add a hasnode() method (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40326
diff changeset
   401
    def hasnode(self, node):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   402
        if node == sha1nodeconstants.nullid:
40387
f1a39128da95 filelog: add a hasnode() method (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40326
diff changeset
   403
            return False
f1a39128da95 filelog: add a hasnode() method (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40326
diff changeset
   404
f1a39128da95 filelog: add a hasnode() method (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40326
diff changeset
   405
        return node in self._nodetorev
f1a39128da95 filelog: add a hasnode() method (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40326
diff changeset
   406
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   407
    def revs(self, start=0, stop=None):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   408
        return storageutil.iterrevs(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   409
            len(self._revisions), start=start, stop=stop
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   410
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   411
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   412
    def parents(self, node):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   413
        if node == sha1nodeconstants.nullid:
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   414
            return sha1nodeconstants.nullid, sha1nodeconstants.nullid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   415
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   416
        if node not in self._revisions:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   417
            raise error.LookupError(node, self._path, _(b'no node'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   418
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   419
        entry = self._revisions[node]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   420
        return entry.p1node, entry.p2node
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   421
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   422
    def parentrevs(self, rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   423
        if rev == nullrev:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   424
            return nullrev, nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   425
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   426
        if rev not in self._revtonode:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   427
            raise IndexError(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   428
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   429
        entry = self._revisions[self._revtonode[rev]]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   430
        return entry.p1rev, entry.p2rev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   431
49789
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   432
    def ancestors(self, revs, stoprev=0, inclusive=False):
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   433
        """Generate the ancestors of 'revs' in reverse revision order.
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   434
        Does not generate revs lower than stoprev.
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   435
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   436
        See the documentation for ancestor.lazyancestors for more details."""
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   437
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   438
        # first, make sure start revisions aren't filtered
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   439
        revs = list(revs)
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   440
        checkrev = self.node
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   441
        for r in revs:
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   442
            checkrev(r)
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   443
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   444
        return ancestor.lazyancestors(
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   445
            self.parentrevs,
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   446
            revs,
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   447
            stoprev=stoprev,
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   448
            inclusive=inclusive,
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   449
        )
df750b81272f sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49284
diff changeset
   450
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   451
    def rev(self, node):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   452
        if node == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   453
            return nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   454
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   455
        if node not in self._nodetorev:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   456
            raise error.LookupError(node, self._path, _(b'no node'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   457
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   458
        return self._nodetorev[node]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   459
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   460
    def node(self, rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   461
        if rev == nullrev:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   462
            return sha1nodeconstants.nullid
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   463
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   464
        if rev not in self._revtonode:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   465
            raise IndexError(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   466
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   467
        return self._revtonode[rev]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   468
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   469
    def lookup(self, node):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   470
        return storageutil.fileidlookup(self, node, self._path)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   471
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   472
    def linkrev(self, rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   473
        if rev == nullrev:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   474
            return nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   475
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   476
        if rev not in self._revtonode:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   477
            raise IndexError(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   478
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   479
        entry = self._revisions[self._revtonode[rev]]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   480
        return entry.linkrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   481
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   482
    def iscensored(self, rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   483
        if rev == nullrev:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   484
            return False
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   485
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   486
        if rev not in self._revtonode:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   487
            raise IndexError(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   488
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   489
        return self._revisions[self._revtonode[rev]].flags & FLAG_CENSORED
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   490
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   491
    def commonancestorsheads(self, node1, node2):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   492
        rev1 = self.rev(node1)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   493
        rev2 = self.rev(node2)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   494
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   495
        ancestors = ancestor.commonancestorsheads(self.parentrevs, rev1, rev2)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   496
        return pycompat.maplist(self.node, ancestors)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   497
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   498
    def descendants(self, revs):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   499
        # TODO we could implement this using a recursive SQL query, which
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   500
        # might be faster.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   501
        return dagop.descendantrevs(revs, self.revs, self.parentrevs)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   502
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   503
    def heads(self, start=None, stop=None):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   504
        if start is None and stop is None:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   505
            if not len(self):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   506
                return [sha1nodeconstants.nullid]
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   507
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   508
        startrev = self.rev(start) if start is not None else nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   509
        stoprevs = {self.rev(n) for n in stop or []}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   510
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   511
        revs = dagop.headrevssubset(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   512
            self.revs, self.parentrevs, startrev=startrev, stoprevs=stoprevs
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   513
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   514
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   515
        return [self.node(rev) for rev in revs]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   516
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   517
    def children(self, node):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   518
        rev = self.rev(node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   519
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   520
        res = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   521
            'SELECT'
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   522
            '  node '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   523
            '  FROM filedata '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   524
            '  WHERE path=? AND (p1rev=? OR p2rev=?) '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   525
            '  ORDER BY revnum ASC',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   526
            (self._path, rev, rev),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   527
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   528
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   529
        return [row[0] for row in res]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   530
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   531
    # End of ifileindex interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   532
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   533
    # Start of ifiledata interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   534
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   535
    def size(self, rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   536
        if rev == nullrev:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   537
            return 0
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   538
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   539
        if rev not in self._revtonode:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   540
            raise IndexError(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   541
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   542
        node = self._revtonode[rev]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   543
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   544
        if self.renamed(node):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   545
            return len(self.read(node))
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   546
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   547
        return len(self.revision(node))
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   548
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   549
    def revision(self, node, raw=False, _verifyhash=True):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   550
        if node in (sha1nodeconstants.nullid, nullrev):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   551
            return b''
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   552
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   553
        if isinstance(node, int):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   554
            node = self.node(node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   555
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   556
        if node not in self._nodetorev:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   557
            raise error.LookupError(node, self._path, _(b'no node'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   558
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   559
        if node in self._revisioncache:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   560
            return self._revisioncache[node]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   561
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   562
        # Because we have a fulltext revision cache, we are able to
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   563
        # short-circuit delta chain traversal and decompression as soon as
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   564
        # we encounter a revision in the cache.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   565
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   566
        stoprids = {self._revisions[n].rid: n for n in self._revisioncache}
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   567
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   568
        if not stoprids:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   569
            stoprids[-1] = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   570
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   571
        fulltext = resolvedeltachain(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   572
            self._db,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   573
            self._pathid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   574
            node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   575
            self._revisioncache,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   576
            stoprids,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   577
            zstddctx=self._dctx,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   578
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   579
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   580
        # Don't verify hashes if parent nodes were rewritten, as the hash
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   581
        # wouldn't verify.
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   582
        if self._revisions[node].flags & (FLAG_MISSING_P1 | FLAG_MISSING_P2):
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   583
            _verifyhash = False
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   584
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   585
        if _verifyhash:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   586
            self._checkhash(fulltext, node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   587
            self._revisioncache[node] = fulltext
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   588
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   589
        return fulltext
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   590
42722
c9f3f4c8999a rawdata: implement `rawdata` for `sqlitestore` too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 40460
diff changeset
   591
    def rawdata(self, *args, **kwargs):
c9f3f4c8999a rawdata: implement `rawdata` for `sqlitestore` too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 40460
diff changeset
   592
        return self.revision(*args, **kwargs)
c9f3f4c8999a rawdata: implement `rawdata` for `sqlitestore` too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 40460
diff changeset
   593
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   594
    def read(self, node):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   595
        return storageutil.filtermetadata(self.revision(node))
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   596
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   597
    def renamed(self, node):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   598
        return storageutil.filerevisioncopied(self, node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   599
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   600
    def cmp(self, node, fulltext):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   601
        return not storageutil.filedataequivalent(self, node, fulltext)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   602
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   603
    def emitrevisions(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   604
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   605
        nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   606
        nodesorder=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   607
        revisiondata=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   608
        assumehaveparentrevisions=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   609
        deltamode=repository.CG_DELTAMODE_STD,
46715
45f0d5297698 changegroupv4: add sidedata helpers
Raphaël Gomès <rgomes@octobus.net>
parents: 46712
diff changeset
   610
        sidedata_helpers=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   611
    ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   612
        if nodesorder not in (b'nodes', b'storage', b'linear', None):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   613
            raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   614
                b'unhandled value for nodesorder: %s' % nodesorder
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   615
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   616
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   617
        nodes = [n for n in nodes if n != sha1nodeconstants.nullid]
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   618
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   619
        if not nodes:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   620
            return
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   621
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   622
        # TODO perform in a single query.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   623
        res = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   624
            'SELECT revnum, deltaid FROM fileindex '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   625
            'WHERE pathid=? '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   626
            '    AND node in (%s)' % (','.join(['?'] * len(nodes))),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   627
            tuple([self._pathid] + nodes),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   628
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   629
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   630
        deltabases = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   631
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   632
        for rev, deltaid in res:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   633
            res = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   634
                'SELECT revnum from fileindex WHERE pathid=? AND deltaid=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   635
                (self._pathid, deltaid),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   636
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   637
            deltabases[rev] = res.fetchone()[0]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   638
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   639
        # TODO define revdifffn so we can use delta from storage.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   640
        for delta in storageutil.emitrevisions(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   641
            self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   642
            nodes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   643
            nodesorder,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   644
            sqliterevisiondelta,
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   645
            deltaparentfn=deltabases.__getitem__,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   646
            revisiondata=revisiondata,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   647
            assumehaveparentrevisions=assumehaveparentrevisions,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   648
            deltamode=deltamode,
46715
45f0d5297698 changegroupv4: add sidedata helpers
Raphaël Gomès <rgomes@octobus.net>
parents: 46712
diff changeset
   649
            sidedata_helpers=sidedata_helpers,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   650
        ):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   651
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   652
            yield delta
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   653
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   654
    # End of ifiledata interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   655
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   656
    # Start of ifilemutation interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   657
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   658
    def add(self, filedata, meta, transaction, linkrev, p1, p2):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   659
        if meta or filedata.startswith(b'\x01\n'):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   660
            filedata = storageutil.packmeta(meta, filedata)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   661
46508
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   662
        rev = self.addrevision(filedata, transaction, linkrev, p1, p2)
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   663
        return self.node(rev)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   664
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   665
    def addrevision(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   666
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   667
        revisiondata,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   668
        transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   669
        linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   670
        p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   671
        p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   672
        node=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   673
        flags=0,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   674
        cachedelta=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   675
    ):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   676
        if flags:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   677
            raise SQLiteStoreError(_(b'flags not supported on revisions'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   678
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   679
        validatehash = node is not None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   680
        node = node or storageutil.hashrevisionsha1(revisiondata, p1, p2)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   681
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   682
        if validatehash:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   683
            self._checkhash(revisiondata, node, p1, p2)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   684
46508
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   685
        rev = self._nodetorev.get(node)
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   686
        if rev is not None:
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   687
            return rev
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   688
46508
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   689
        rev = self._addrawrevision(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   690
            node, revisiondata, transaction, linkrev, p1, p2
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   691
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   692
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   693
        self._revisioncache[node] = revisiondata
46508
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
   694
        return rev
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   695
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   696
    def addgroup(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   697
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   698
        deltas,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   699
        linkmapper,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   700
        transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   701
        addrevisioncb=None,
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   702
        duplicaterevisioncb=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   703
        maybemissingparents=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   704
    ):
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   705
        empty = True
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   706
46711
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   707
        for (
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   708
            node,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   709
            p1,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   710
            p2,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   711
            linknode,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   712
            deltabase,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   713
            delta,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   714
            wireflags,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   715
            sidedata,
a41565bef69f changegroup: add v4 changegroup for revlog v2 exchange
Raphaël Gomès <rgomes@octobus.net>
parents: 46509
diff changeset
   716
        ) in deltas:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   717
            storeflags = 0
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   718
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   719
            if wireflags & repository.REVISION_FLAG_CENSORED:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   720
                storeflags |= FLAG_CENSORED
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   721
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   722
            if wireflags & ~repository.REVISION_FLAG_CENSORED:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   723
                raise SQLiteStoreError(b'unhandled revision flag')
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   724
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   725
            if maybemissingparents:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   726
                if p1 != sha1nodeconstants.nullid and not self.hasnode(p1):
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   727
                    p1 = sha1nodeconstants.nullid
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   728
                    storeflags |= FLAG_MISSING_P1
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   729
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   730
                if p2 != sha1nodeconstants.nullid and not self.hasnode(p2):
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   731
                    p2 = sha1nodeconstants.nullid
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   732
                    storeflags |= FLAG_MISSING_P2
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   733
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   734
            baserev = self.rev(deltabase)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   735
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   736
            # If base is censored, delta must be full replacement in a single
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   737
            # patch operation.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   738
            if baserev != nullrev and self.iscensored(baserev):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   739
                hlen = struct.calcsize(b'>lll')
42772
5c2e8a661418 rawdata: update callers in sqlitestore
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42760
diff changeset
   740
                oldlen = len(self.rawdata(deltabase, _verifyhash=False))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   741
                newlen = len(delta) - hlen
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   742
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   743
                if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   744
                    raise error.CensoredBaseError(self._path, deltabase)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   745
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   746
            if not (storeflags & FLAG_CENSORED) and storageutil.deltaiscensored(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   747
                delta, baserev, lambda x: len(self.rawdata(x))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   748
            ):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   749
                storeflags |= FLAG_CENSORED
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   750
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   751
            linkrev = linkmapper(linknode)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   752
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   753
            if node in self._revisions:
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   754
                # Possibly reset parents to make them proper.
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   755
                entry = self._revisions[node]
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   756
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   757
                if (
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   758
                    entry.flags & FLAG_MISSING_P1
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   759
                    and p1 != sha1nodeconstants.nullid
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   760
                ):
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   761
                    entry.p1node = p1
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   762
                    entry.p1rev = self._nodetorev[p1]
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   763
                    entry.flags &= ~FLAG_MISSING_P1
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   764
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   765
                    self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   766
                        'UPDATE fileindex SET p1rev=?, flags=? WHERE id=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   767
                        (self._nodetorev[p1], entry.flags, entry.rid),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   768
                    )
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   769
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   770
                if (
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   771
                    entry.flags & FLAG_MISSING_P2
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   772
                    and p2 != sha1nodeconstants.nullid
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   773
                ):
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   774
                    entry.p2node = p2
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   775
                    entry.p2rev = self._nodetorev[p2]
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   776
                    entry.flags &= ~FLAG_MISSING_P2
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   777
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   778
                    self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   779
                        'UPDATE fileindex SET p2rev=?, flags=? WHERE id=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   780
                        (self._nodetorev[p1], entry.flags, entry.rid),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   781
                    )
40392
595641bd8404 sqlitestore: support for storing revisions without their parents
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40390
diff changeset
   782
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   783
                if duplicaterevisioncb:
46509
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46508
diff changeset
   784
                    duplicaterevisioncb(self, self.rev(node))
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   785
                empty = False
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   786
                continue
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   787
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
   788
            if deltabase == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   789
                text = mdiff.patch(b'', delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   790
                storedelta = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   791
            else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   792
                text = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   793
                storedelta = (deltabase, delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   794
46509
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46508
diff changeset
   795
            rev = self._addrawrevision(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   796
                node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   797
                text,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   798
                transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   799
                linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   800
                p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   801
                p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   802
                storedelta=storedelta,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   803
                flags=storeflags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   804
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   805
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   806
            if addrevisioncb:
46509
7a93b7b3dc2d revlog: change addgroup callbacks to take revision numbers
Joerg Sonnenberger <joerg@bec.de>
parents: 46508
diff changeset
   807
                addrevisioncb(self, rev)
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   808
            empty = False
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   809
45788
a5206e71c536 revlog: extend addgroup() with callback for duplicates
Joerg Sonnenberger <joerg@bec.de>
parents: 45372
diff changeset
   810
        return not empty
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   811
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   812
    def censorrevision(self, tr, censornode, tombstone=b''):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   813
        tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   814
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   815
        # This restriction is cargo culted from revlogs and makes no sense for
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   816
        # SQLite, since columns can be resized at will.
42772
5c2e8a661418 rawdata: update callers in sqlitestore
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42760
diff changeset
   817
        if len(tombstone) > len(self.rawdata(censornode)):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   818
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43077
diff changeset
   819
                _(b'censor tombstone must be no longer than censored data')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   820
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   821
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   822
        # We need to replace the censored revision's data with the tombstone.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   823
        # But replacing that data will have implications for delta chains that
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   824
        # reference it.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   825
        #
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   826
        # While "better," more complex strategies are possible, we do something
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   827
        # simple: we find delta chain children of the censored revision and we
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   828
        # replace those incremental deltas with fulltexts of their corresponding
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   829
        # revision. Then we delete the now-unreferenced delta and original
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   830
        # revision and insert a replacement.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   831
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   832
        # Find the delta to be censored.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   833
        censoreddeltaid = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   834
            'SELECT deltaid FROM fileindex WHERE id=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   835
            (self._revisions[censornode].rid,),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   836
        ).fetchone()[0]
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   837
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   838
        # Find all its delta chain children.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   839
        # TODO once we support storing deltas for !files, we'll need to look
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   840
        # for those delta chains too.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   841
        rows = list(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   842
            self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   843
                'SELECT id, pathid, node FROM fileindex '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   844
                'WHERE deltabaseid=? OR deltaid=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   845
                (censoreddeltaid, censoreddeltaid),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   846
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   847
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   848
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   849
        for row in rows:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   850
            rid, pathid, node = row
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   851
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   852
            fulltext = resolvedeltachain(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   853
                self._db, pathid, node, {}, {-1: None}, zstddctx=self._dctx
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   854
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   855
44062
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
   856
            deltahash = hashutil.sha1(fulltext).digest()
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   857
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   858
            if self._compengine == b'zstd':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   859
                deltablob = self._cctx.compress(fulltext)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   860
                compression = COMPRESSION_ZSTD
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   861
            elif self._compengine == b'zlib':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   862
                deltablob = zlib.compress(fulltext)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   863
                compression = COMPRESSION_ZLIB
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   864
            elif self._compengine == b'none':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   865
                deltablob = fulltext
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   866
                compression = COMPRESSION_NONE
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   867
            else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   868
                raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   869
                    b'unhandled compression engine: %s' % self._compengine
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   870
                )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   871
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   872
            if len(deltablob) >= len(fulltext):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   873
                deltablob = fulltext
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   874
                compression = COMPRESSION_NONE
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   875
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   876
            deltaid = insertdelta(self._db, compression, deltahash, deltablob)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   877
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   878
            self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   879
                'UPDATE fileindex SET deltaid=?, deltabaseid=NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   880
                'WHERE id=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   881
                (deltaid, rid),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   882
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   883
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   884
        # Now create the tombstone delta and replace the delta on the censored
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   885
        # node.
44062
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
   886
        deltahash = hashutil.sha1(tombstone).digest()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   887
        tombstonedeltaid = insertdelta(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   888
            self._db, COMPRESSION_NONE, deltahash, tombstone
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   889
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   890
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   891
        flags = self._revisions[censornode].flags
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   892
        flags |= FLAG_CENSORED
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   893
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   894
        self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   895
            'UPDATE fileindex SET flags=?, deltaid=?, deltabaseid=NULL '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   896
            'WHERE pathid=? AND node=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   897
            (flags, tombstonedeltaid, self._pathid, censornode),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   898
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   899
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   900
        self._db.execute('DELETE FROM delta WHERE id=?', (censoreddeltaid,))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   901
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   902
        self._refreshindex()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   903
        self._revisioncache.clear()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   904
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   905
    def getstrippoint(self, minlink):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   906
        return storageutil.resolvestripinfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   907
            minlink,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   908
            len(self) - 1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   909
            [self.rev(n) for n in self.heads()],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   910
            self.linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   911
            self.parentrevs,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   912
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   913
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   914
    def strip(self, minlink, transaction):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   915
        if not len(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   916
            return
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   917
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   918
        rev, _ignored = self.getstrippoint(minlink)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   919
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   920
        if rev == len(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   921
            return
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   922
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   923
        for rev in self.revs(rev):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   924
            self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   925
                'DELETE FROM fileindex WHERE pathid=? AND node=?',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   926
                (self._pathid, self.node(rev)),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   927
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   928
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   929
        # TODO how should we garbage collect data in delta table?
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   930
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   931
        self._refreshindex()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   932
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   933
    # End of ifilemutation interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   934
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   935
    # Start of ifilestorage interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   936
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   937
    def files(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   938
        return []
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   939
46712
e8c11a2c96c0 delta: add sidedata field to revision delta
Raphaël Gomès <rgomes@octobus.net>
parents: 46711
diff changeset
   940
    def sidedata(self, nodeorrev, _df=None):
e8c11a2c96c0 delta: add sidedata field to revision delta
Raphaël Gomès <rgomes@octobus.net>
parents: 46711
diff changeset
   941
        # Not supported for now
e8c11a2c96c0 delta: add sidedata field to revision delta
Raphaël Gomès <rgomes@octobus.net>
parents: 46711
diff changeset
   942
        return {}
e8c11a2c96c0 delta: add sidedata field to revision delta
Raphaël Gomès <rgomes@octobus.net>
parents: 46711
diff changeset
   943
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   944
    def storageinfo(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   945
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   946
        exclusivefiles=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   947
        sharedfiles=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   948
        revisionscount=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   949
        trackedsize=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   950
        storedsize=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   951
    ):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   952
        d = {}
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   953
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   954
        if exclusivefiles:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   955
            d[b'exclusivefiles'] = []
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   956
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   957
        if sharedfiles:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   958
            # TODO list sqlite file(s) here.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   959
            d[b'sharedfiles'] = []
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   960
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   961
        if revisionscount:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   962
            d[b'revisionscount'] = len(self)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   963
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   964
        if trackedsize:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   965
            d[b'trackedsize'] = sum(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   966
                len(self.revision(node)) for node in self._nodetorev
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   967
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   968
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   969
        if storedsize:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   970
            # TODO implement this?
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   971
            d[b'storedsize'] = None
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   972
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   973
        return d
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   974
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   975
    def verifyintegrity(self, state):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   976
        state[b'skipread'] = set()
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   977
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   978
        for rev in self:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   979
            node = self.node(rev)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   980
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   981
            try:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   982
                self.revision(node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   983
            except Exception as e:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   984
                yield sqliteproblem(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   985
                    error=_(b'unpacking %s: %s') % (short(node), e), node=node
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
   986
                )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   987
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   988
                state[b'skipread'].add(node)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   989
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   990
    # End of ifilestorage interface.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   991
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   992
    def _checkhash(self, fulltext, node, p1=None, p2=None):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   993
        if p1 is None and p2 is None:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   994
            p1, p2 = self.parents(node)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   995
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   996
        if node == storageutil.hashrevisionsha1(fulltext, p1, p2):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   997
            return
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   998
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   999
        try:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1000
            del self._revisioncache[node]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1001
        except KeyError:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1002
            pass
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1003
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1004
        if storageutil.iscensoredtext(fulltext):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1005
            raise error.CensoredNodeError(self._path, node, fulltext)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1006
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1007
        raise SQLiteStoreError(_(b'integrity check failed on %s') % self._path)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1008
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1009
    def _addrawrevision(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1010
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1011
        node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1012
        revisiondata,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1013
        transaction,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1014
        linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1015
        p1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1016
        p2,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1017
        storedelta=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1018
        flags=0,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1019
    ):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1020
        if self._pathid is None:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1021
            res = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1022
                'INSERT INTO filepath (path) VALUES (?)', (self._path,)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1023
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1024
            self._pathid = res.lastrowid
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1025
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1026
        # For simplicity, always store a delta against p1.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1027
        # TODO we need a lot more logic here to make behavior reasonable.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1028
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1029
        if storedelta:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1030
            deltabase, delta = storedelta
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1031
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1032
            if isinstance(deltabase, int):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1033
                deltabase = self.node(deltabase)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1034
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1035
        else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1036
            assert revisiondata is not None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1037
            deltabase = p1
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1038
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
  1039
            if deltabase == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1040
                delta = revisiondata
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1041
            else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1042
                delta = mdiff.textdiff(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1043
                    self.revision(self.rev(deltabase)), revisiondata
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1044
                )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1045
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1046
        # File index stores a pointer to its delta and the parent delta.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1047
        # The parent delta is stored via a pointer to the fileindex PK.
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
  1048
        if deltabase == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1049
            baseid = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1050
        else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1051
            baseid = self._revisions[deltabase].rid
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1052
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1053
        # Deltas are stored with a hash of their content. This allows
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1054
        # us to de-duplicate. The table is configured to ignore conflicts
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1055
        # and it is faster to just insert and silently noop than to look
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1056
        # first.
44062
2d49482d0dd4 hgext: replace references to hashlib.sha1 with hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43507
diff changeset
  1057
        deltahash = hashutil.sha1(delta).digest()
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1058
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1059
        if self._compengine == b'zstd':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1060
            deltablob = self._cctx.compress(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1061
            compression = COMPRESSION_ZSTD
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1062
        elif self._compengine == b'zlib':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1063
            deltablob = zlib.compress(delta)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1064
            compression = COMPRESSION_ZLIB
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1065
        elif self._compengine == b'none':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1066
            deltablob = delta
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1067
            compression = COMPRESSION_NONE
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1068
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1069
            raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1070
                b'unhandled compression engine: %s' % self._compengine
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1071
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1072
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1073
        # Don't store compressed data if it isn't practical.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1074
        if len(deltablob) >= len(delta):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1075
            deltablob = delta
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1076
            compression = COMPRESSION_NONE
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1077
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1078
        deltaid = insertdelta(self._db, compression, deltahash, deltablob)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1079
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1080
        rev = len(self)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1081
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
  1082
        if p1 == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1083
            p1rev = nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1084
        else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1085
            p1rev = self._nodetorev[p1]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1086
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46780
diff changeset
  1087
        if p2 == sha1nodeconstants.nullid:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1088
            p2rev = nullrev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1089
        else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1090
            p2rev = self._nodetorev[p2]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1091
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1092
        rid = self._db.execute(
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1093
            'INSERT INTO fileindex ('
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1094
            '    pathid, revnum, node, p1rev, p2rev, linkrev, flags, '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1095
            '    deltaid, deltabaseid) '
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1096
            '    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1097
            (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1098
                self._pathid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1099
                rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1100
                node,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1101
                p1rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1102
                p2rev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1103
                linkrev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1104
                flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1105
                deltaid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1106
                baseid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1107
            ),
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1108
        ).lastrowid
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1109
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1110
        entry = revisionentry(
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1111
            rid=rid,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1112
            rev=rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1113
            node=node,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1114
            p1rev=p1rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1115
            p2rev=p2rev,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1116
            p1node=p1,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1117
            p2node=p2,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1118
            linkrev=linkrev,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1119
            flags=flags,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1120
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1121
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1122
        self._nodetorev[node] = rev
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1123
        self._revtonode[rev] = node
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1124
        self._revisions[node] = entry
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1125
46508
f7b61ad3c64a revlog: change addrevision to return the new revision, not node
Joerg Sonnenberger <joerg@bec.de>
parents: 45788
diff changeset
  1126
        return rev
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1127
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1128
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1129
class sqliterepository(localrepo.localrepository):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1130
    def cancopy(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1131
        return False
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1132
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1133
    def transaction(self, *args, **kwargs):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1134
        current = self.currenttransaction()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1135
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1136
        tr = super(sqliterepository, self).transaction(*args, **kwargs)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1137
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1138
        if current:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1139
            return tr
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1140
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1141
        self._dbconn.execute('BEGIN TRANSACTION')
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1142
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1143
        def committransaction(_):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1144
            self._dbconn.commit()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1145
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1146
        tr.addfinalize(b'sqlitestore', committransaction)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1147
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1148
        return tr
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1149
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1150
    @property
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1151
    def _dbconn(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1152
        # SQLite connections can only be used on the thread that created
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1153
        # them. In most cases, this "just works." However, hgweb uses
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1154
        # multiple threads.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1155
        tid = threading.current_thread().ident
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1156
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1157
        if self._db:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1158
            if self._db[0] == tid:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1159
                return self._db[1]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1160
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1161
        db = makedb(self.svfs.join(b'db.sqlite'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1162
        self._db = (tid, db)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1163
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1164
        return db
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1165
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1166
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1167
def makedb(path):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1168
    """Construct a database handle for a database at path."""
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1169
40410
3b782669561d py3: make sure we pass sysstr in sqlite3.connect()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40392
diff changeset
  1170
    db = sqlite3.connect(encoding.strfromlocal(path))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1171
    db.text_factory = bytes
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1172
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1173
    res = db.execute('PRAGMA user_version').fetchone()[0]
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1174
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1175
    # New database.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1176
    if res == 0:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1177
        for statement in CREATE_SCHEMA:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1178
            db.execute(statement)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1179
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1180
        db.commit()
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1181
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1182
    elif res == CURRENT_SCHEMA_VERSION:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1183
        pass
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1184
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1185
    else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1186
        raise error.Abort(_(b'sqlite database has unrecognized version'))
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1187
43507
adde839cc54f sqlitestore: remove superfluous r-prefixes on strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
  1188
    db.execute('PRAGMA journal_mode=WAL')
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1189
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1190
    return db
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1191
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1192
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1193
def featuresetup(ui, supported):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1194
    supported.add(REQUIREMENT)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1195
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1196
    if zstd:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1197
        supported.add(REQUIREMENT_ZSTD)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1198
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1199
    supported.add(REQUIREMENT_ZLIB)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1200
    supported.add(REQUIREMENT_NONE)
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1201
    supported.add(REQUIREMENT_SHALLOW_FILES)
45372
77b8588dd84e requirements: introduce new requirements related module
Pulkit Goyal <7895pulkit@gmail.com>
parents: 44062
diff changeset
  1202
    supported.add(requirements.NARROW_REQUIREMENT)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1203
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1204
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1205
def newreporequirements(orig, ui, createopts):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1206
    if createopts[b'backend'] != b'sqlite':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1207
        return orig(ui, createopts)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1208
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1209
    # This restriction can be lifted once we have more confidence.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1210
    if b'sharedrepo' in createopts:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1211
        raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43077
diff changeset
  1212
            _(b'shared repositories not supported with SQLite store')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1213
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1214
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1215
    # This filtering is out of an abundance of caution: we want to ensure
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1216
    # we honor creation options and we do that by annotating exactly the
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1217
    # creation options we recognize.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1218
    known = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1219
        b'narrowfiles',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1220
        b'backend',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1221
        b'shallowfilestore',
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1222
    }
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1223
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1224
    unsupported = set(createopts) - known
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1225
    if unsupported:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1226
        raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43077
diff changeset
  1227
            _(b'SQLite store does not support repo creation option: %s')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1228
            % b', '.join(sorted(unsupported))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1229
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1230
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1231
    # Since we're a hybrid store that still relies on revlogs, we fall back
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1232
    # to using the revlogv1 backend's storage requirements then adding our
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1233
    # own requirement.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1234
    createopts[b'backend'] = b'revlogv1'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1235
    requirements = orig(ui, createopts)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1236
    requirements.add(REQUIREMENT)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1237
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1238
    compression = ui.config(b'storage', b'sqlite.compression')
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1239
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1240
    if compression == b'zstd' and not zstd:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1241
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1242
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1243
                b'storage.sqlite.compression set to "zstd" but '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1244
                b'zstandard compression not available to this '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1245
                b'Mercurial install'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1246
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1247
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1248
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1249
    if compression == b'zstd':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1250
        requirements.add(REQUIREMENT_ZSTD)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1251
    elif compression == b'zlib':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1252
        requirements.add(REQUIREMENT_ZLIB)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1253
    elif compression == b'none':
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1254
        requirements.add(REQUIREMENT_NONE)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1255
    else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1256
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1257
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1258
                b'unknown compression engine defined in '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1259
                b'storage.sqlite.compression: %s'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1260
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1261
            % compression
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1262
        )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1263
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1264
    if createopts.get(b'shallowfilestore'):
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1265
        requirements.add(REQUIREMENT_SHALLOW_FILES)
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1266
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1267
    return requirements
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1268
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1269
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1270
@interfaceutil.implementer(repository.ilocalrepositoryfilestorage)
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
  1271
class sqlitefilestorage:
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1272
    """Repository file storage backed by SQLite."""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1273
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1274
    def file(self, path):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1275
        if path[0] == b'/':
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1276
            path = path[1:]
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1277
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1278
        if REQUIREMENT_ZSTD in self.requirements:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1279
            compression = b'zstd'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1280
        elif REQUIREMENT_ZLIB in self.requirements:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1281
            compression = b'zlib'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1282
        elif REQUIREMENT_NONE in self.requirements:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1283
            compression = b'none'
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1284
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1285
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1286
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1287
                    b'unable to determine what compression engine '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1288
                    b'to use for SQLite storage'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1289
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1290
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1291
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1292
        return sqlitefilestore(self._dbconn, path, compression)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1293
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1294
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1295
def makefilestorage(orig, requirements, features, **kwargs):
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1296
    """Produce a type conforming to ``ilocalrepositoryfilestorage``."""
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1297
    if REQUIREMENT in requirements:
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1298
        if REQUIREMENT_SHALLOW_FILES in requirements:
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1299
            features.add(repository.REPO_FEATURE_SHALLOW_FILE_STORAGE)
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1300
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1301
        return sqlitefilestorage
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1302
    else:
40390
7e3b6c4f01a2 localrepo: support marking repos as having shallow file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40389
diff changeset
  1303
        return orig(requirements=requirements, features=features, **kwargs)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1304
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1305
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1306
def makemain(orig, ui, requirements, **kwargs):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1307
    if REQUIREMENT in requirements:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1308
        if REQUIREMENT_ZSTD in requirements and not zstd:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1309
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1310
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1311
                    b'repository uses zstandard compression, which '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1312
                    b'is not available to this Mercurial install'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1313
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1314
            )
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1315
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1316
        return sqliterepository
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1317
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1318
    return orig(requirements=requirements, **kwargs)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1319
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1320
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1321
def verifierinit(orig, self, *args, **kwargs):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1322
    orig(self, *args, **kwargs)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1323
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1324
    # We don't care that files in the store don't align with what is
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1325
    # advertised. So suppress these warnings.
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1326
    self.warnorphanstorefiles = False
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1327
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1328
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1329
def extsetup(ui):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1330
    localrepo.featuresetupfuncs.add(featuresetup)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1331
    extensions.wrapfunction(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1332
        localrepo, b'newreporequirements', newreporequirements
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1333
    )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1334
    extensions.wrapfunction(localrepo, b'makefilestorage', makefilestorage)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1335
    extensions.wrapfunction(localrepo, b'makemain', makemain)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1336
    extensions.wrapfunction(verify.verifier, b'__init__', verifierinit)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
  1337
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1338
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1339
def reposetup(ui, repo):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1340
    if isinstance(repo, sqliterepository):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1341
        repo._db = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1342
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
  1343
    # TODO check for bundlerepository?