tests/test-fuzz-targets.t
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 09 Apr 2024 22:37:15 +0200
changeset 51595 3a6fae3bef35
parent 47063 1d075b857c90
permissions -rw-r--r--
outgoing: add a simple fastpath when there is no common This further speed up case like `hg bundle --all` for larger repository. ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 before: 316.749699 after: 311.165461 (-1.76%, -5.58) There is further work to be done in this area like not doing any outgoing computation in the stream case for example. however the recent changes already gives use a large win for a small amount of local work. ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog pre-%ln-change: 1.263859 the-%ln-change: 0.700229 (-44.60%, -0.56) prev-changeset: 0.496050 (-60.75%, -0.77) this-changeset: 0.495243 (-60.81%, -0.77) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog pre-%ln-change: 2.975765 the-%ln-change: 1.870798 (-37.13%, -1.10) prev-changeset: 1.461583 (-50.88%, -1.51) this-changeset: 1.469185 (-50.63%, -1.51) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog pre-%ln-change: 4.540080 the-%ln-change: 3.401700 (-25.07%, -1.14) prev-changeset: 2.915810 (-35.78%, -1.62) this-changeset: 2.911643 (-35.87%, -1.63) ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog pre-%ln-change: 10.138396 the-%ln-change: 7.750458 (-23.55%, -2.39) prev-changeset: 6.665565 (-34.25%, -3.47) this-changeset: 6.672078 (-34.19%, -3.47) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog pre-%ln-change: 399.484481 the-%ln-change: 346.508952 (-13.26%, -52.98) prev-changeset: 316.749699 (-20.71%, -82.73) this-changeset: 311.165461 (-22.11%, -88.32)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
     1
#require test-repo py3
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
     2
35670
2b9e2415f5b5 contrib: add some basic scaffolding for some fuzz test targets
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
  $ cd $TESTDIR/../contrib/fuzz
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
     4
  $ OUT=$TESTTMP ; export OUT
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
     5
40726
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     6
which(1) could exit nonzero, but that's fine because we'll still end
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     7
up without a valid executable, so we don't need to check $? here.
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     8
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     9
  $ if which gmake >/dev/null 2>&1; then
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    10
  >     MAKE=gmake
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    11
  > else
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    12
  >     MAKE=make
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    13
  > fi
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    14
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    15
  $ havefuzz() {
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    16
  >     cat > $TESTTMP/dummy.cc <<EOF
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    17
  > #include <stdlib.h>
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    18
  > #include <stdint.h>
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    19
  > int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { return 0; }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    20
  > int main(int argc, char **argv) {
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    21
  >     const char data[] = "asdf";
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    22
  >     return LLVMFuzzerTestOneInput((const uint8_t *)data, 4);
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    23
  > }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    24
  > EOF
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    25
  >     $CXX $TESTTMP/dummy.cc -o $TESTTMP/dummy \
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    26
  >        -fsanitize=fuzzer-no-link,address || return 1
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    27
  > }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    28
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    29
Try to find a python3-config that's next to our sys.executable. If
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    30
that doesn't work, fall back to looking for a global python3-config
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    31
and hope that works out for the best.
47063
1d075b857c90 tests: ensure `$PYTHON` is quoted for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 46257
diff changeset
    32
  $ PYBIN=`"$PYTHON" -c 'import sys, os; print(os.path.dirname(sys.executable))'`
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    33
  $ if [ -x "$PYBIN/python3-config" ] ; then
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    34
  >   PYTHON_CONFIG="$PYBIN/python3-config"
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    35
  > else
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    36
  >   PYTHON_CONFIG="`which python3-config`"
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    37
  > fi
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    38
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    39
#if clang-libfuzzer
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    40
  $ CXX=clang++ havefuzz || exit 80
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    41
  $ $MAKE -s clean all PYTHON_CONFIG="$PYTHON_CONFIG"
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    42
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    43
#if no-clang-libfuzzer clang-6.0
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    44
  $ CXX=clang++-6.0 havefuzz || exit 80
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    45
  $ $MAKE -s clean all CC=clang-6.0 CXX=clang++-6.0 PYTHON_CONFIG="$PYTHON_CONFIG"
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    46
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    47
#if no-clang-libfuzzer no-clang-6.0
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    48
  $ exit 80
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    49
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    50
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    51
  $ cd $TESTTMP
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    52
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    53
Run each fuzzer using dummy.cc as a fake input, to make sure it runs
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    54
at all. In the future we should instead unpack the corpus for each
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    55
fuzzer and use that instead.
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    56
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    57
  $ for fuzzer in `ls *_fuzzer | sort` ; do
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    58
  >   echo run $fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    59
  >   ./$fuzzer dummy.cc > /dev/null 2>&1 
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    60
  > done
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    61
  run bdiff_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    62
  run dirs_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    63
  run dirstate_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    64
  run fm1readmarkers_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    65
  run fncache_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    66
  run jsonescapeu8fast_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    67
  run manifest_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    68
  run mpatch_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    69
  run revlog_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    70
  run xdiff_fuzzer...
42948
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    71
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    72
Clean up.
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    73
  $ cd $TESTDIR/../contrib/fuzz
42948
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    74
  $ $MAKE -s clean