tests/test-debugbundle.t
author Jun Wu <quark@fb.com>
Fri, 02 Sep 2016 15:20:59 +0100
branchstable
changeset 29861 2f6d5c60f6fc
parent 29593 953839de96ab
child 30211 6b0741d6d234
permissions -rw-r--r--
annotate: pre-calculate the "needed" dictionary (issue5360) The "needed" dict is used as a reference counter to free items in the giant "hist" dict. However, currently it is not very accurate and can lead to dropping "hist" items unnecessarily, for example, with the following DAG, -3- / \ 0--1--2--4-- The current algorithm will visit and calculate rev 1 twice, undesired. And it tries to be smart by clearing rev 1's parents: "pcache[1] = []" at the time hist[1] being accessed (note: hist[1] needs to be used twice, by rev 2 and rev 3). It can result in incorrect results if p1 of rev 4 deletes chunks belonging to rev 0. However, simply removing "needed" is not okay, because it will consume 10x memory: # without any change % HGRCPATH= lrun ./hg annotate mercurial/commands.py -r d130a38 3>&2 [1] MEMORY 49074176 CPUTIME 9.213 REALTIME 9.270 # with "needed" removed MEMORY 637673472 CPUTIME 8.164 REALTIME 8.249 This patch moves "needed" (and "pcache") calculation to a separate DFS to address the issue. It improves perf and fixes issue5360 by correctly reusing hist, while maintaining low memory usage. Some additional attempt has been made to further reduce memory usage, like changing "pcache[f] = []" to "del pcache[f]". Therefore the result can be both faster and lower memory usage: # with this patch applied MEMORY 47575040 CPUTIME 7.870 REALTIME 7.926 [1]: lrun is a lightweight sandbox built on Linux cgroup and namespace. It's used to measure CPU and memory usage here. Source code is available at github.com/quark-zju/lrun.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     1
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     2
Create a test repository:
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     3
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     4
  $ hg init repo
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     5
  $ cd repo
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     6
  $ touch a ; hg add a ; hg ci -ma
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     7
  $ touch b ; hg add b ; hg ci -mb
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
     8
  $ touch c ; hg add c ; hg ci -mc
26864
fb1217cea400 test: enforce v1 in 'test-debugbundle.t'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23748
diff changeset
     9
  $ hg bundle --base 0 --rev tip bundle.hg -v --type v1
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    10
  2 changesets found
23748
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    11
  uncompressed size of bundle content:
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    12
       332 (changelog)
4ab66de46a96 bundle: when verbose, show what takes up the space in the generated bundle
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    13
       282 (manifests)
27711
7a678a12a5cf mdiff: don't emit a diff header for empty trivial deltas
Mike Hommey <mh@glandium.org>
parents: 26864
diff changeset
    14
        93  b
7a678a12a5cf mdiff: don't emit a diff header for empty trivial deltas
Mike Hommey <mh@glandium.org>
parents: 26864
diff changeset
    15
        93  c
29062
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    16
  $ hg bundle --base 0 --rev tip bundle2.hg -v --type none-v2
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    17
  2 changesets found
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    18
  uncompressed size of bundle content:
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    19
       372 (changelog)
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    20
       322 (manifests)
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    21
       113  b
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    22
       113  c
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    23
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    24
Terse output:
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    25
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    26
  $ hg debugbundle bundle.hg
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    27
  0e067c57feba1a5694ca4844f05588bb1bf82342
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    28
  991a3460af53952d10ec8a295d3d2cc2e5fa9690
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    29
29062
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    30
Terse output:
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    31
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    32
  $ hg debugbundle bundle2.hg
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    33
  Stream params: {}
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29592
diff changeset
    34
  changegroup -- "sortdict([('version', '02'), ('nbchanges', '2')])"
29062
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    35
      0e067c57feba1a5694ca4844f05588bb1bf82342
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    36
      991a3460af53952d10ec8a295d3d2cc2e5fa9690
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    37
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    38
Verbose output:
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    39
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    40
  $ hg debugbundle --all bundle.hg
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13724
diff changeset
    41
  format: id, p1, p2, cset, delta base, len(delta)
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    42
  
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    43
  changelog
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13724
diff changeset
    44
  0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 80
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13724
diff changeset
    45
  991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 80
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    46
  
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    47
  manifest
14141
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13724
diff changeset
    48
  686dbf0aeca417636fa26a9121c681eabbb15a20 8515d4bfda768e04af4c13a69a72e28c7effbea7 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 8515d4bfda768e04af4c13a69a72e28c7effbea7 55
bd1cbfe5db5c bundler: make parsechunk return the base revision of the delta
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13724
diff changeset
    49
  ae25a31b30b3490a981e7b96a3238cc69583fda1 686dbf0aeca417636fa26a9121c681eabbb15a20 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 686dbf0aeca417636fa26a9121c681eabbb15a20 55
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    50
  
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    51
  b
27711
7a678a12a5cf mdiff: don't emit a diff header for empty trivial deltas
Mike Hommey <mh@glandium.org>
parents: 26864
diff changeset
    52
  b80de5d138758541c5f05265ad144ab9fa86d1db 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 0
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    53
  
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    54
  c
27711
7a678a12a5cf mdiff: don't emit a diff header for empty trivial deltas
Mike Hommey <mh@glandium.org>
parents: 26864
diff changeset
    55
  b80de5d138758541c5f05265ad144ab9fa86d1db 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0000000000000000000000000000000000000000 0
13724
fe57046e9448 commands: add debugbundle command
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
    56
29062
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    57
  $ hg debugbundle --all bundle2.hg
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    58
  Stream params: {}
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29592
diff changeset
    59
  changegroup -- "sortdict([('version', '02'), ('nbchanges', '2')])"
29062
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    60
      format: id, p1, p2, cset, delta base, len(delta)
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    61
  
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    62
      changelog
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    63
      0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 80
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    64
      991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 80
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    65
  
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    66
      manifest
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    67
      686dbf0aeca417636fa26a9121c681eabbb15a20 8515d4bfda768e04af4c13a69a72e28c7effbea7 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 8515d4bfda768e04af4c13a69a72e28c7effbea7 55
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    68
      ae25a31b30b3490a981e7b96a3238cc69583fda1 686dbf0aeca417636fa26a9121c681eabbb15a20 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 686dbf0aeca417636fa26a9121c681eabbb15a20 55
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    69
  
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    70
      b
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    71
      b80de5d138758541c5f05265ad144ab9fa86d1db 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 0
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    72
  
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    73
      c
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    74
      b80de5d138758541c5f05265ad144ab9fa86d1db 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0000000000000000000000000000000000000000 0
906a1c8a75fd debugbundle: add tests for debugbundle output with bundle2
Mike Hommey <mh@glandium.org>
parents: 27711
diff changeset
    75
16913
f2719b387380 tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents: 14141
diff changeset
    76
  $ cd ..