copies: calculate mergecopies() based on pathcopies() When copies are stored in changesets, we need a changeset-centric version of mergecopies() just like we have a changeset-centric version of pathcopies(). I think the natural way of thinking about mergecopies() is in terms of pathcopies() from the base to each of the commits. So if we can rewrite mergecopies() based on two such pathcopies() calls, we'll get the changeset-centric version for free. That's what this patch does. A nice bonus is that it ends up being a lot simpler. mergecopies() has accumulated a lot of technical debt over time. One good example is the code for dealing with grafts (the "partial/incomplete/dirty" stuff). Since pathcopies() already deals with backwards renames and ping-pong renames, we get that for free. I've run tests with hard-coded debug logging for "fullcopy" and while I haven't looked at every difference it produces, all the ones I have looked at seemed reasonable to me. I'm a little surprised that no more tests fail when run with '--extra-config-opt experimental.copies.read-from=compatibility' compared to before this patch. This patch also fixes the broken cases in test-annotate.t and test-fastannotate.t. It also enables the part of test-copies.t that was previously disabled exactly because mergecopies() needed to get a changeset-centric version. One drawback of the rewritten code is that we may now make remotefilelog prefetch more files. We used to prefetch files that were unique to either side of the merge compared to the other. We now prefetch files that are unique to either side of the merge compared to the base. This means that if you added the same file to each side, we would not prefetch it before, but we would now. Such cases are probably quite rare, but one likely scenario where they happen is when moving from a commit to its successor (or the other way around). The user will probably already have the files in the cache in such cases, so it's probably not a big deal. Some timings for calculating mergecopies between two revisions (revisions shown on each line, all using the common ancestor as base): In the hg repo: 4.8 4.9: 0.21s -> 0.21s 4.0 4.8: 0.35s -> 0.63s In and old copy of the mozilla-unified repo: FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.82s -> 0.82s FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.5s -> 2.6s FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.9s -> 4.1s FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 31s -> 33s So it's measurably slower in most cases. The most significant difference is in the hg repo between revisions 4.0 and 4.8. In that case it seems to come from the fact that pathcopies() uses fctx.isintroducedafter() (in _tracefile), while the old mergecopies() used fctx.linkrev() (in _checkcopies()). That results in a single call to filectx._adjustlinkrev(), which is responsible for the entire difference in time (in my repo). So we pay a performance penalty but we get more correct code (see change in test-mv-cp-st-diff.t). Deleting the "== f.filenode()" in _tracefile() recovers the lost performance in the hg repo. There were are few other optimizations in _checkcopies() that I could not measure any impact from. One was from the "seen" set. Another was from a "continue" when the file was not in the destination manifest (corresponding to "am" in _tracefile). Also note that merge copies are not calculated when updating with a clean working copy, which is probably the most common case. I therefore think the much simpler code is worth the slowdown. Differential Revision: https://phab.mercurial-scm.org/D6255

     1

#require sqlite

     2

     3

  $ cat >> $HGRCPATH <<EOF

     4

  > [extensions]

     5

  > sqlitestore =

     6

  > EOF

     7

     8

New repo should not use SQLite by default

     9

    10

  $ hg init empty-no-sqlite

    11

  $ cat empty-no-sqlite/.hg/requires

    12

  dotencode

    13

  fncache

    14

  generaldelta

    15

  revlogv1

    16

  sparserevlog

    17

  store

    18

    19

storage.new-repo-backend=sqlite is recognized

    20

    21

  $ hg --config storage.new-repo-backend=sqlite init empty-sqlite

    22

  $ cat empty-sqlite/.hg/requires

    23

  dotencode

    24

  exp-sqlite-001

    25

  exp-sqlite-comp-001=zstd (zstd !)

    26

  exp-sqlite-comp-001=$BUNDLE2_COMPRESSIONS$ (no-zstd !)

    27

  fncache

    28

  generaldelta

    29

  revlogv1

    30

  sparserevlog

    31

  store

    32

    33

  $ cat >> $HGRCPATH << EOF

    34

  > [storage]

    35

  > new-repo-backend = sqlite

    36

  > EOF

    37

    38

Can force compression to zlib

    39

    40

  $ hg --config storage.sqlite.compression=zlib init empty-zlib

    41

  $ cat empty-zlib/.hg/requires

    42

  dotencode

    43

  exp-sqlite-001

    44

  exp-sqlite-comp-001=$BUNDLE2_COMPRESSIONS$

    45

  fncache

    46

  generaldelta

    47

  revlogv1

    48

  sparserevlog

    49

  store

    50

    51

Can force compression to none

    52

    53

  $ hg --config storage.sqlite.compression=none init empty-none

    54

  $ cat empty-none/.hg/requires

    55

  dotencode

    56

  exp-sqlite-001

    57

  exp-sqlite-comp-001=none

    58

  fncache

    59

  generaldelta

    60

  revlogv1

    61

  sparserevlog

    62

  store

    63

    64

Can make a local commit

    65

    66

  $ hg init local-commit

    67

  $ cd local-commit

    68

  $ echo 0 > foo

    69

  $ hg commit -A -m initial

    70

  adding foo

    71

    72

That results in a row being inserted into various tables

    73

    74

  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF

    75

  > SELECT * FROM filepath;

    76

  > EOF

    77

  1|foo

    78

    79

  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF

    80

  > SELECT * FROM fileindex;

    81

  > EOF

    82

  1|1|0|-1|-1|0|0|1||6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe (esc)

    83

    84

  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF

    85

  > SELECT * FROM delta;

    86

  > EOF

    87

  1|1|	\xd2\xaf\x8d\xd2"\x01\xdd\x8dH\xe5\xdc\xfc\xae\xd2\x81\xff\x94"\xc7|0 (esc)

    88

    89

    90

Tracking multiple files works

    91

    92

  $ echo 1 > bar

    93

  $ hg commit -A -m 'add bar'

    94

  adding bar

    95

    96

  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF

    97

  > SELECT * FROM filedata ORDER BY id ASC;

    98

  > EOF

    99

  1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1| (esc)

   100

  2|2|bar|0|\xb8\xe0/d3s\x80!\xa0e\xf9Au\xc7\xcd#\xdb_\x05\xbe|-1|-1|1|0|2| (esc)

   101

   102

Multiple revisions of a file works

   103

   104

  $ echo a >> foo

   105

  $ hg commit -m 'modify foo'

   106

   107

  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF

   108

  > SELECT * FROM filedata ORDER BY id ASC;

   109

  > EOF

   110

  1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1| (esc)

   111

  2|2|bar|0|\xb8\xe0/d3s\x80!\xa0e\xf9Au\xc7\xcd#\xdb_\x05\xbe|-1|-1|1|0|2| (esc)

   112

  3|1|foo|1|\xdd\xb3V\xcd\xde1p@\xf7\x8e\x90\xb8*\x8b,\xe9\x0e\xd6j+|0|-1|2|0|3|1 (esc)

   113

   114

  $ cd ..