contrib/perf-utils/discovery-helper.sh
author Arseniy Alekseyev <aalekseyev@janestreet.com>
Fri, 26 Apr 2024 19:10:35 +0100
changeset 51626 865efc020c33
parent 41928 cae3f7e37623
permissions -rwxr-xr-x
dirstate: remove the python-side whitelist of allowed matchers This whitelist is too permissive because it allows matchers that contain disallowed ones deep inside, for example through `intersectionmatcher`. It is also too restrictive because it doesn't pass through some of the matchers we support, such as `patternmatcher`. It's also unnecessary because unsupported matchers raise `FallbackError` and we fall back anyway. Making this change makes more of the tests use rust code path, and therefore subtly change behavior. For example, rust status in largefiles repos seems to have strange behavior.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     1
#!/bin/bash
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     2
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     3
# produces two repositories with different common and missing subsets
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     4
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     5
#   $ discovery-helper.sh REPO NBHEADS DEPT
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     6
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     7
# The Goal is to produce two repositories with some common part and some
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     8
# exclusive part on each side. Provide a source repository REPO, it will
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
     9
# produce two repositories REPO-left and REPO-right.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    10
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    11
# Each repository will be missing some revisions exclusive to NBHEADS of the
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    12
# repo topological heads. These heads and revisions exclusive to them (up to
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    13
# DEPTH depth) are stripped.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    14
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    15
# The "left" repository will use the NBHEADS first heads (sorted by
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    16
# description). The "right" use the last NBHEADS one.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    17
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    18
# To find out how many topological heads a repo has, use:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    19
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    20
#   $ hg heads -t -T '{rev}\n' | wc -l
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    21
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    22
# Example:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    23
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    24
#  The `pypy-2018-09-01` repository has 192 heads. To produce two repositories
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    25
#  with 92 common heads and ~50 exclusive heads on each side.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    26
#
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    27
#    $ ./discovery-helper.sh pypy-2018-08-01 50 10
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    28
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    29
set -euo pipefail
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    30
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    31
printusage () {
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    32
     echo "usage: `basename $0` REPO NBHEADS DEPTH [left|right]" >&2
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    33
}
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    34
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    35
if [ $# -lt 3 ]; then
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    36
    printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    37
    exit 64
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    38
fi
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    39
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    40
repo="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    41
shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    42
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    43
nbheads="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    44
shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    45
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    46
depth="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    47
shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    48
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    49
doleft=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    50
doright=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    51
if [ $# -gt 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    52
    printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    53
    exit 64
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    54
elif [ $# -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    55
    if [ "$1" == "left" ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    56
        doleft=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    57
        doright=0
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    58
    elif [ "$1" == "right" ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    59
        doleft=0
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    60
        doright=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    61
    else
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    62
        printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    63
        exit 64
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    64
    fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    65
fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    66
41895
c1017bceb6ec discovery-helper: reflect argument value in the name of the results
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41894
diff changeset
    67
leftrepo="${repo}-${nbheads}h-${depth}d-left"
c1017bceb6ec discovery-helper: reflect argument value in the name of the results
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41894
diff changeset
    68
rightrepo="${repo}-${nbheads}h-${depth}d-right"
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    69
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    70
left="first(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    71
right="last(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    72
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    73
leftsubset="ancestors($left, $depth) and only($left, heads(all() - $left))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    74
rightsubset="ancestors($right, $depth) and only($right, heads(all() - $right))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
    75
41894
aea6bc63884d discovery-helper: echo the stripped revsets early
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41893
diff changeset
    76
echo '### creating left/right repositories with missing changesets:'
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    77
if [ $doleft -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    78
    echo '# left  revset:' '"'${leftsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    79
fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    80
if [ $doright -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    81
    echo '# right revset:' '"'${rightsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    82
fi
41894
aea6bc63884d discovery-helper: echo the stripped revsets early
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41893
diff changeset
    83
41926
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
    84
buildone() {
41927
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    85
    side="$1"
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    86
    dest="$2"
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    87
    revset="$3"
41926
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
    88
    echo "### building $side repository: $dest"
41927
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    89
    if [ -e "$dest" ]; then
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    90
        echo "destination repo already exists: $dest" >&2
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    91
        exit 1
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
    92
    fi
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
    93
    echo '# cloning'
41928
cae3f7e37623 discovery-helper: use reflink copy if available
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41927
diff changeset
    94
    if ! cp --recursive --reflink=always ${repo} ${dest}; then
cae3f7e37623 discovery-helper: use reflink copy if available
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41927
diff changeset
    95
        hg clone --noupdate "${repo}" "${dest}"
cae3f7e37623 discovery-helper: use reflink copy if available
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41927
diff changeset
    96
    fi
41926
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
    97
    echo '# stripping' '"'${revset}'"'
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
    98
    hg -R "${dest}" --config extensions.strip= strip --rev "$revset" --no-backup
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
    99
}
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
   100
cb6c9d41a23e discovery-helper: move repository creation in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41925
diff changeset
   101
if [ $doleft -eq 1 ]; then
41927
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
   102
    buildone left "$leftrepo" "$leftsubset"
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
   103
fi
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
   104
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
   105
if [ $doright -eq 1 ]; then
41927
a60e9f163595 discovery-helper: bail out if destination already exists
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41926
diff changeset
   106
    buildone right "$rightrepo" "$rightsubset"
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
   107
fi