mercurial/metadata.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 15 Sep 2020 10:55:17 +0200
changeset 45634 9a6b409b8ebc
parent 45623 d31483377673
child 45635 9003e6524f78
permissions -rw-r--r--
changing-files: rework the way we store changed files in side-data We need to store new data so this is a good opportunity to rework this fully. 1) We directly store the list of affected file in the side data: * This avoid having to fetch and parse the `files` list in the revision in addition to the sidedata. Making the data more self sufficient. * This work around situation where that `files` field contains wrong information, and open the way to other bug fixing (eg: issue6219) * The format (fixed initial index, sorted files) allow for fast lookup of filename within the structure. * This unify the storage of affected files and copies sources and destination, limiting the number filename stored redundantly. * This prepare for the fact we should drop the `files` as soon as we do any change affecting the revision schema. * This rely on compression to avoid a significant increase of the changelog.d. More testing on this will be done before we freeze the final format. 2) We can store additional data: * The new "merged" field, * A future "salvaged" set recording files that might have been deleted but have were still present in the final result. Differential Revision: https://phab.mercurial-scm.org/D9090
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     1
# metadata.py -- code related to various metadata computation and access.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     2
#
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     3
# Copyright 2019 Google, Inc <martinvonz@google.com>
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     4
# Copyright 2020 Pierre-Yves David <pierre-yves.david@octobus.net>
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     5
#
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     6
# This software may be used and distributed according to the terms of the
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     7
# GNU General Public License version 2 or any later version.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     8
from __future__ import absolute_import, print_function
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
     9
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    10
import multiprocessing
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
    11
import struct
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    12
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    13
from . import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    14
    error,
44941
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
    15
    node,
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    16
    pycompat,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    17
    util,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    18
)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    19
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    20
from .revlogutils import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    21
    flagutil as sidedataflag,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    22
    sidedata as sidedatamod,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    23
)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    24
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
    25
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    26
class ChangingFiles(object):
45621
646a676f5365 changing-files: fix docstring
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45613
diff changeset
    27
    """A class recording the changes made to files by a changeset
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    28
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    29
    Actions performed on files are gathered into 3 sets:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    30
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    31
    - added:   files actively added in the changeset.
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    32
    - merged:  files whose history got merged
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    33
    - removed: files removed in the revision
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    34
    - touched: files affected by the merge
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    35
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    36
    and copies information is held by 2 mappings
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    37
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    38
    - copied_from_p1: {"<new-name>": "<source-name-in-p1>"} mapping for copies
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    39
    - copied_from_p2: {"<new-name>": "<source-name-in-p2>"} mapping for copies
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    40
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    41
    See their inline help for details.
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    42
    """
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    43
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    44
    def __init__(
45589
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    45
        self,
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    46
        touched=None,
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    47
        added=None,
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    48
        removed=None,
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    49
        merged=None,
45589
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    50
        p1_copies=None,
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    51
        p2_copies=None,
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    52
    ):
45589
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    53
        self._added = set(() if added is None else added)
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    54
        self._merged = set(() if merged is None else merged)
45589
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    55
        self._removed = set(() if removed is None else removed)
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    56
        self._touched = set(() if touched is None else touched)
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    57
        self._touched.update(self._added)
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    58
        self._touched.update(self._merged)
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    59
        self._touched.update(self._removed)
45589
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    60
        self._p1_copies = dict(() if p1_copies is None else p1_copies)
3d5b2b8e93fd changing-files: move default constructor value to None
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45571
diff changeset
    61
        self._p2_copies = dict(() if p2_copies is None else p2_copies)
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    62
45507
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    63
    def __eq__(self, other):
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    64
        return (
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    65
            self.added == other.added
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    66
            and self.merged == other.merged
45507
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    67
            and self.removed == other.removed
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    68
            and self.touched == other.touched
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    69
            and self.copied_from_p1 == other.copied_from_p1
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    70
            and self.copied_from_p2 == other.copied_from_p2
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    71
        )
df87821081ee changing-files: implement equality checking
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45506
diff changeset
    72
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
    73
    @util.propertycache
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    74
    def added(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    75
        """files actively added in the changeset
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    76
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    77
        Any file present in that revision that was absent in all the changeset's
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    78
        parents.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    79
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    80
        In case of merge, this means a file absent in one of the parents but
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    81
        existing in the other will *not* be contained in this set. (They were
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    82
        added by an ancestor)
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
    83
        """
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    84
        return frozenset(self._added)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    85
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    86
    def mark_added(self, filename):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
    87
        if 'added' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
    88
            del self.added
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    89
        self._added.add(filename)
45622
42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45621
diff changeset
    90
        self.mark_touched(filename)
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    91
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    92
    def update_added(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    93
        for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    94
            self.mark_added(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
    95
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
    96
    @util.propertycache
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    97
    def merged(self):
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    98
        """files actively merged during a merge
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
    99
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   100
        Any modified files which had modification on both size that needed merging.
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   101
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   102
        In this case a new filenode was created and it has two parents.
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   103
        """
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   104
        return frozenset(self._merged)
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   105
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   106
    def mark_merged(self, filename):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   107
        if 'merged' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   108
            del self.merged
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   109
        self._merged.add(filename)
45622
42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45621
diff changeset
   110
        self.mark_touched(filename)
45611
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   111
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   112
    def update_merged(self, filenames):
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   113
        for f in filenames:
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   114
            self.mark_merged(f)
e5578dbe36cb changing-files: add the ability to track merged files too
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45589
diff changeset
   115
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   116
    @util.propertycache
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   117
    def removed(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   118
        """files actively removed by the changeset
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   119
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   120
        In case of merge this will only contain the set of files removing "new"
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   121
        content. For any file absent in the current changeset:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   122
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   123
        a) If the file exists in both parents, it is clearly "actively" removed
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   124
        by this changeset.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   125
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   126
        b) If a file exists in only one parent and in none of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   127
        ancestors, then the file was newly added in one of the merged branches
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   128
        and then got "actively" removed.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   129
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   130
        c) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   131
        ancestors using the same filenode, then the file was unchanged on one
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   132
        side and deleted on the other side. The merge "passively" propagated
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   133
        that deletion, but didn't "actively" remove the file. In this case the
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   134
        file is *not* included in the `removed` set.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   135
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   136
        d) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   137
        ancestors using a different filenode, then the file was changed on one
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   138
        side and removed on the other side. The merge process "actively"
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   139
        decided to drop the new change and delete the file. Unlike in the
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   140
        previous case, (c), the file included in the `removed` set.
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   141
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   142
        Summary table for merge:
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   143
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   144
        case | exists in parents | exists in gca || removed
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   145
         (a) |       both        |     *         ||   yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   146
         (b) |       one         |     none      ||   yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   147
         (c) |       one         | same filenode ||   no
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   148
         (d) |       one         |  new filenode ||   yes
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   149
        """
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   150
        return frozenset(self._removed)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   151
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   152
    def mark_removed(self, filename):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   153
        if 'removed' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   154
            del self.removed
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   155
        self._removed.add(filename)
45622
42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45621
diff changeset
   156
        self.mark_touched(filename)
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   157
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   158
    def update_removed(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   159
        for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   160
            self.mark_removed(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   161
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   162
    @util.propertycache
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   163
    def touched(self):
45506
1f50bcc96595 changing-files: document the various sets
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45325
diff changeset
   164
        """files either actively modified, added or removed"""
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   165
        return frozenset(self._touched)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   166
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   167
    def mark_touched(self, filename):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   168
        if 'touched' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   169
            del self.touched
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   170
        self._touched.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   171
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   172
    def update_touched(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   173
        for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   174
            self.mark_touched(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   175
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   176
    @util.propertycache
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   177
    def copied_from_p1(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   178
        return self._p1_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   179
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   180
    def mark_copied_from_p1(self, source, dest):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   181
        if 'copied_from_p1' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   182
            del self.copied_from_p1
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   183
        self._p1_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   184
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   185
    def update_copies_from_p1(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   186
        for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   187
            self.mark_copied_from_p1(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   188
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   189
    @util.propertycache
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   190
    def copied_from_p2(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   191
        return self._p2_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   192
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   193
    def mark_copied_from_p2(self, source, dest):
45623
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   194
        if 'copied_from_p2' in vars(self):
d31483377673 changing-files: cache the various property
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45622
diff changeset
   195
            del self.copied_from_p2
45323
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   196
        self._p2_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   197
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   198
    def update_copies_from_p2(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   199
        for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   200
            self.mark_copied_from_p2(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   201
aea6a812f7cb commitctx: return a richer object from _prepare_files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44942
diff changeset
   202
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   203
def computechangesetfilesadded(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   204
    """return the list of files added in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   205
    """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   206
    added = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   207
    for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   208
        if not any(f in p for p in ctx.parents()):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   209
            added.append(f)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   210
    return added
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   211
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   212
44941
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   213
def get_removal_filter(ctx, x=None):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   214
    """return a function to detect files "wrongly" detected as `removed`
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   215
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   216
    When a file is removed relative to p1 in a merge, this
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   217
    function determines whether the absence is due to a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   218
    deletion from a parent, or whether the merge commit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   219
    itself deletes the file. We decide this by doing a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   220
    simplified three way merge of the manifest entry for
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   221
    the file. There are two ways we decide the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   222
    itself didn't delete a file:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   223
    - neither parent (nor the merge) contain the file
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   224
    - exactly one parent contains the file, and that
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   225
      parent has the same filelog entry as the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   226
      ancestor (or all of them if there two). In other
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   227
      words, that parent left the file unchanged while the
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   228
      other one deleted it.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   229
    One way to think about this is that deleting a file is
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   230
    similar to emptying it, so the list of changed files
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   231
    should be similar either way. The computation
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   232
    described above is not done directly in _filecommit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   233
    when creating the list of changed files, however
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   234
    it does something very similar by comparing filelog
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   235
    nodes.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   236
    """
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   237
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   238
    if x is not None:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   239
        p1, p2, m1, m2 = x
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   240
    else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   241
        p1 = ctx.p1()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   242
        p2 = ctx.p2()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   243
        m1 = p1.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   244
        m2 = p2.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   245
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   246
    @util.cachefunc
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   247
    def mas():
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   248
        p1n = p1.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   249
        p2n = p2.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   250
        cahs = ctx.repo().changelog.commonancestorsheads(p1n, p2n)
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   251
        if not cahs:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   252
            cahs = [node.nullrev]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   253
        return [ctx.repo()[r].manifest() for r in cahs]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   254
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   255
    def deletionfromparent(f):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   256
        if f in m1:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   257
            return f not in m2 and all(
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   258
                f in ma and ma.find(f) == m1.find(f) for ma in mas()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   259
            )
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   260
        elif f in m2:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   261
            return all(f in ma and ma.find(f) == m2.find(f) for ma in mas())
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   262
        else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   263
            return True
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   264
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   265
    return deletionfromparent
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   266
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44940
diff changeset
   267
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   268
def computechangesetfilesremoved(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   269
    """return the list of files removed in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   270
    """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   271
    removed = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   272
    for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   273
        if f not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   274
            removed.append(f)
44942
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
   275
    if removed:
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
   276
        rf = get_removal_filter(ctx)
25512a65cefd metadata: filter the `removed` set to only contains relevant data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44941
diff changeset
   277
        removed = [r for r in removed if not rf(r)]
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   278
    return removed
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   279
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   280
45613
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   281
def computechangesetfilesmerged(ctx):
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   282
    """return the list of files merged in a changeset
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   283
    """
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   284
    merged = []
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   285
    if len(ctx.parents()) < 2:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   286
        return merged
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   287
    for f in ctx.files():
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   288
        if f in ctx:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   289
            fctx = ctx[f]
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   290
            parents = fctx._filelog.parents(fctx._filenode)
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   291
            if parents[1] != node.nullid:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   292
                merged.append(f)
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   293
    return merged
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   294
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45611
diff changeset
   295
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   296
def computechangesetcopies(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   297
    """return the copies data for a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   298
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   299
    The copies data are returned as a pair of dictionnary (p1copies, p2copies).
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   300
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   301
    Each dictionnary are in the form: `{newname: oldname}`
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   302
    """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   303
    p1copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   304
    p2copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   305
    p1 = ctx.p1()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   306
    p2 = ctx.p2()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   307
    narrowmatch = ctx._repo.narrowmatch()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   308
    for dst in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   309
        if not narrowmatch(dst) or dst not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   310
            continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   311
        copied = ctx[dst].renamed()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   312
        if not copied:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   313
            continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   314
        src, srcnode = copied
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   315
        if src in p1 and p1[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   316
            p1copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   317
        elif src in p2 and p2[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   318
            p2copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   319
    return p1copies, p2copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   320
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   321
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   322
def encodecopies(files, copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   323
    items = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   324
    for i, dst in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   325
        if dst in copies:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   326
            items.append(b'%d\0%s' % (i, copies[dst]))
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   327
    if len(items) != len(copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   328
        raise error.ProgrammingError(
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   329
            b'some copy targets missing from file list'
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   330
        )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   331
    return b"\n".join(items)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   332
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   333
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   334
def decodecopies(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   335
    try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   336
        copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   337
        if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   338
            return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   339
        for l in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   340
            strindex, src = l.split(b'\0')
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   341
            i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   342
            dst = files[i]
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   343
            copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   344
        return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   345
    except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   346
        # Perhaps someone had chosen the same key name (e.g. "p1copies") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   347
        # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   348
        return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   349
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   350
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   351
def encodefileindices(files, subset):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   352
    subset = set(subset)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   353
    indices = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   354
    for i, f in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   355
        if f in subset:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   356
            indices.append(b'%d' % i)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   357
    return b'\n'.join(indices)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   358
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   359
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   360
def decodefileindices(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   361
    try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   362
        subset = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   363
        if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   364
            return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   365
        for strindex in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   366
            i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   367
            if i < 0 or i >= len(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   368
                return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   369
            subset.append(files[i])
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   370
        return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   371
    except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   372
        # Perhaps someone had chosen the same key name (e.g. "added") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   373
        # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   374
        return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   375
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   376
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   377
# see mercurial/helptext/internals/revlogs.txt for details about the format
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   378
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   379
ACTION_MASK = int("111" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   380
# note: untouched file used as copy source will as `000` for this mask.
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   381
ADDED_FLAG = int("001" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   382
MERGED_FLAG = int("010" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   383
REMOVED_FLAG = int("011" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   384
# `100` is reserved for future use
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   385
TOUCHED_FLAG = int("101" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   386
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   387
COPIED_MASK = int("11", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   388
COPIED_FROM_P1_FLAG = int("10", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   389
COPIED_FROM_P2_FLAG = int("11", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   390
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   391
# structure is <flag><filename-end><copy-source>
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   392
INDEX_HEADER = struct.Struct(">L")
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   393
INDEX_ENTRY = struct.Struct(">bLL")
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   394
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   395
45569
64d18e9e8508 sidedata: rename `encode_copies_sidedata` to `encode_files_sidedata`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45507
diff changeset
   396
def encode_files_sidedata(files):
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   397
    all_files = set(files.touched)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   398
    all_files.update(files.copied_from_p1.values())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   399
    all_files.update(files.copied_from_p2.values())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   400
    all_files = sorted(all_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   401
    file_idx = {f: i for (i, f) in enumerate(all_files)}
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   402
    file_idx[None] = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   403
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   404
    chunks = [INDEX_HEADER.pack(len(all_files))]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   405
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   406
    filename_length = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   407
    for f in all_files:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   408
        filename_size = len(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   409
        filename_length += filename_size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   410
        flag = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   411
        if f in files.added:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   412
            flag |= ADDED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   413
        elif f in files.merged:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   414
            flag |= MERGED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   415
        elif f in files.removed:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   416
            flag |= REMOVED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   417
        elif f in files.touched:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   418
            flag |= TOUCHED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   419
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   420
        copy = None
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   421
        if f in files.copied_from_p1:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   422
            flag |= COPIED_FROM_P1_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   423
            copy = files.copied_from_p1.get(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   424
        elif f in files.copied_from_p2:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   425
            copy = files.copied_from_p2.get(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   426
            flag |= COPIED_FROM_P2_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   427
        copy_idx = file_idx[copy]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   428
        chunks.append(INDEX_ENTRY.pack(flag, filename_length, copy_idx))
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   429
    chunks.extend(all_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   430
    return {sidedatamod.SD_FILES: b''.join(chunks)}
45325
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
   431
c6eea5804551 commitctx: extract sidedata encoding inside its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45323
diff changeset
   432
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   433
def decode_files_sidedata(changelogrevision, sidedata):
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   434
    md = ChangingFiles()
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   435
    raw = sidedata.get(sidedatamod.SD_FILES)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   436
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   437
    if raw is None:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   438
        return md
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   439
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   440
    copies = []
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   441
    all_files = []
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   442
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   443
    assert len(raw) >= INDEX_HEADER.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   444
    total_files = INDEX_HEADER.unpack_from(raw, 0)[0]
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   445
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   446
    offset = INDEX_HEADER.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   447
    file_offset_base = offset + (INDEX_ENTRY.size * total_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   448
    file_offset_last = file_offset_base
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   449
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   450
    assert len(raw) >= file_offset_base
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   451
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   452
    for idx in range(total_files):
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   453
        flag, file_end, copy_idx = INDEX_ENTRY.unpack_from(raw, offset)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   454
        file_end += file_offset_base
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   455
        filename = raw[file_offset_last:file_end]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   456
        filesize = file_end - file_offset_last
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   457
        assert len(filename) == filesize
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   458
        offset += INDEX_ENTRY.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   459
        file_offset_last = file_end
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   460
        all_files.append(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   461
        if flag & ACTION_MASK == ADDED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   462
            md.mark_added(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   463
        elif flag & ACTION_MASK == MERGED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   464
            md.mark_merged(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   465
        elif flag & ACTION_MASK == REMOVED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   466
            md.mark_removed(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   467
        elif flag & ACTION_MASK == TOUCHED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   468
            md.mark_touched(filename)
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   469
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   470
        copied = None
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   471
        if flag & COPIED_MASK == COPIED_FROM_P1_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   472
            copied = md.mark_copied_from_p1
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   473
        elif flag & COPIED_MASK == COPIED_FROM_P2_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   474
            copied = md.mark_copied_from_p2
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   475
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   476
        if copied is not None:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   477
            copies.append((copied, filename, copy_idx))
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   478
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   479
    for copied, filename, copy_idx in copies:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   480
        copied(all_files[copy_idx], filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   481
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   482
    return md
45571
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   483
7543b5072e84 sidedata: add a `decode_files_sidedata` function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45569
diff changeset
   484
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   485
def _getsidedata(srcrepo, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   486
    ctx = srcrepo[rev]
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   487
    filescopies = computechangesetcopies(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   488
    filesadded = computechangesetfilesadded(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   489
    filesremoved = computechangesetfilesremoved(ctx)
45634
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   490
    filesmerged = computechangesetfilesmerged(ctx)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   491
    files = ChangingFiles()
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   492
    files.update_touched(ctx.files())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   493
    files.update_added(filesadded)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   494
    files.update_removed(filesremoved)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   495
    files.update_merged(filesmerged)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   496
    files.update_copies_from_p1(filescopies[0])
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   497
    files.update_copies_from_p2(filescopies[1])
9a6b409b8ebc changing-files: rework the way we store changed files in side-data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45623
diff changeset
   498
    return encode_files_sidedata(files)
44940
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   499
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   500
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   501
def getsidedataadder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   502
    use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   503
    if pycompat.iswindows or not use_w:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   504
        return _get_simple_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   505
    else:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   506
        return _get_worker_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   507
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   508
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   509
def _sidedata_worker(srcrepo, revs_queue, sidedata_queue, tokens):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   510
    """The function used by worker precomputing sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   511
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   512
    It read an input queue containing revision numbers
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   513
    It write in an output queue containing (rev, <sidedata-map>)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   514
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   515
    The `None` input value is used as a stop signal.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   516
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   517
    The `tokens` semaphore is user to avoid having too many unprocessed
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   518
    entries. The workers needs to acquire one token before fetching a task.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   519
    They will be released by the consumer of the produced data.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   520
    """
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   521
    tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   522
    rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   523
    while rev is not None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   524
        data = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   525
        sidedata_queue.put((rev, data))
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   526
        tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   527
        rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   528
    # processing of `None` is completed, release the token.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   529
    tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   530
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   531
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   532
BUFF_PER_WORKER = 50
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   533
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   534
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   535
def _get_worker_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   536
    """The parallel version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   537
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   538
    This code spawn a pool of worker that precompute a buffer of sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   539
    before we actually need them"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   540
    # avoid circular import copies -> scmutil -> worker -> copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   541
    from . import worker
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   542
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   543
    nbworkers = worker._numworkers(srcrepo.ui)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   544
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   545
    tokens = multiprocessing.BoundedSemaphore(nbworkers * BUFF_PER_WORKER)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   546
    revsq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   547
    sidedataq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   548
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   549
    assert srcrepo.filtername is None
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   550
    # queue all tasks beforehand, revision numbers are small and it make
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   551
    # synchronisation simpler
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   552
    #
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   553
    # Since the computation for each node can be quite expensive, the overhead
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   554
    # of using a single queue is not revelant. In practice, most computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   555
    # are fast but some are very expensive and dominate all the other smaller
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   556
    # cost.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   557
    for r in srcrepo.changelog.revs():
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   558
        revsq.put(r)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   559
    # queue the "no more tasks" markers
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   560
    for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   561
        revsq.put(None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   562
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   563
    allworkers = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   564
    for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   565
        args = (srcrepo, revsq, sidedataq, tokens)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   566
        w = multiprocessing.Process(target=_sidedata_worker, args=args)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   567
        allworkers.append(w)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   568
        w.start()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   569
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   570
    # dictionnary to store results for revision higher than we one we are
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   571
    # looking for. For example, if we need the sidedatamap for 42, and 43 is
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   572
    # received, when shelve 43 for later use.
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   573
    staging = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   574
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   575
    def sidedata_companion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   576
        sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   577
        if util.safehasattr(revlog, b'filteredrevs'):  # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   578
            # Is the data previously shelved ?
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   579
            sidedata = staging.pop(rev, None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   580
            if sidedata is None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   581
                # look at the queued result until we find the one we are lookig
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   582
                # for (shelve the other ones)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   583
                r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   584
                while r != rev:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   585
                    staging[r] = sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   586
                    r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   587
            tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   588
        return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   589
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   590
    return sidedata_companion
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   591
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   592
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   593
def _get_simple_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   594
    """The simple version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   595
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   596
    It just compute it in the same thread on request"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   597
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   598
    def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   599
        sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   600
        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   601
            sidedata = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   602
        return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   603
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   604
    return sidedatacompanion
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   605
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   606
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   607
def getsidedataremover(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   608
    def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   609
        f = ()
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   610
        if util.safehasattr(revlog, 'filteredrevs'):  # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   611
            if revlog.flags(rev) & sidedataflag.REVIDX_SIDEDATA:
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   612
                f = (
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   613
                    sidedatamod.SD_P1COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   614
                    sidedatamod.SD_P2COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   615
                    sidedatamod.SD_FILESADDED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   616
                    sidedatamod.SD_FILESREMOVED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   617
                )
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   618
        return False, f, {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   619
4c1d39215034 metadata: move computation related to files touched in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
   620
    return sidedatacompanion