tests/test-fix-pickle.t
author Pierre-Yves David <pierre-yves.david@octobus.net>
Fri, 05 Apr 2024 11:05:54 +0200
changeset 51576 de5bf3fe0233
parent 45390 7d24201b6447
permissions -rw-r--r--
revset: stop serializing node when using "%ln" Turning hundred of thousand of node from node to hex and back can be slow… what about we stop doing it? In many case were we are using node id we should be using revision id. However this is not a good reason to have a stupidly slow implementation of "%ln". This caught my attention again because the phase discovery during push make an extensive use of "%ln" or huge set. In absolute, that phase discovery probably should use "%ld" and need to improves its algorithmic complexity, but improving "%ln" seems simple and long overdue. This greatly speeds up `hg push` on repository with many drafts. Here are some relevant poulpe benchmarks: ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.push # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = default ## benchmark.variants.revs = any-1-extra-rev before: 44.235070 after: 20.416329 (-53.85%, -23.82) ## benchmark.variants.revs = any-100-extra-rev before: 49.234697 after: 26.519829 (-46.14%, -22.71) ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog before: 10.138396 after: 7.750458 (-23.55%, -2.39) ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog before: 1.263859 after: 0.700229 (-44.60%, -0.56) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog before: 399.484481 after: 346.5089 (-13.26%, -52.98) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog before: 4.540080 after: 3.401700 (-25.07%, -1.14) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog before: 2.975765 after: 1.870798 (-37.13%, -1.10)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
45390
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     1
A script that implements uppercasing all letters in a file.
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     2
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     3
  $ UPPERCASEPY="$TESTTMP/uppercase.py"
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     4
  $ cat > $UPPERCASEPY <<EOF
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     5
  > import sys
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     6
  > from mercurial.utils.procutil import setbinary
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     7
  > setbinary(sys.stdin)
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     8
  > setbinary(sys.stdout)
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
     9
  > sys.stdout.write(sys.stdin.read().upper())
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    10
  > EOF
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    11
  $ TESTLINES="foo\nbar\nbaz\n"
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    12
  $ printf $TESTLINES | "$PYTHON" $UPPERCASEPY
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    13
  FOO
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    14
  BAR
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    15
  BAZ
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    16
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    17
This file attempts to test our workarounds for pickle's lack of
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    18
support for short reads.
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    19
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    20
  $ cat >> $HGRCPATH <<EOF
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    21
  > [extensions]
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    22
  > fix =
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    23
  > [fix]
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    24
  > uppercase-whole-file:command="$PYTHON" $UPPERCASEPY
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    25
  > uppercase-whole-file:pattern=set:**
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    26
  > EOF
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    27
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    28
  $ hg init repo
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    29
  $ cd repo
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    30
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    31
# Create a file that's large enough that it seems to not fit in
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    32
# pickle's buffer, making it use the code path that expects our
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    33
# _blockingreader's read() method to return bytes.
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    34
  $ echo "some stuff" > file
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    35
  $ for i in $($TESTDIR/seq.py 13); do
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    36
  >   cat file file > tmp
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    37
  >   mv -f tmp file
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    38
  > done
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    39
  $ hg commit -Am "add large file"
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    40
  adding file
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    41
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    42
Check that we don't get a crash
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    43
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    44
  $ hg fix -r .
7d24201b6447 worker: don't expose readinto() on _blockingreader since pickle is picky
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
    45
  saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-fix.hg (glob)