contrib/fuzz/fm1readmarkers_corpus.py
author Manuel Jacob <me@manueljacob.de>
Mon, 11 Jul 2022 01:51:20 +0200
branchstable
changeset 49378 094a5fa3cf52
parent 48875 6000f5b25c9b
permissions -rw-r--r--
procutil: make stream detection in make_line_buffered more correct and strict In make_line_buffered(), we don’t want to wrap the stream if we know that lines get flushed to the underlying raw stream already. Previously, the heuristic was too optimistic. It assumed that any stream which is not an instance of io.BufferedIOBase doesn’t need wrapping. However, there are buffered streams that aren’t instances of io.BufferedIOBase, like Mercurial’s own winstdout. The new logic is different in two ways: First, only for the check, if unwraps any combination of WriteAllWrapper and winstdout. Second, it skips wrapping the stream only if it is an instance of io.RawIOBase (or already wrapped). If it is an instance of io.BufferedIOBase, it gets wrapped. In any other case, the function raises an exception. This ensures that, if an unknown stream is passed or we add another wrapper in the future, we don’t wrap the stream if it’s already line buffered or not wrap the stream if it’s not line buffered. In fact, this was already helpful during development of this change. Without it, I possibly would have forgot that WriteAllWrapper needs to be ignored for the check, leading to unnecessary wrapping if stdout is unbuffered. The alternative would have been to always wrap unknown streams. However, I don’t think that anyone would benefit from being less strict. We can expect streams from the standard library to be subclassing either io.RawIOBase or io.BufferedIOBase, so running Mercurial in the standard way should not regress by this change. Py2exe might replace sys.stdout and sys.stderr, but that currently breaks Mercurial anyway and also these streams don’t claim to be interactive, so this function is not called for them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
41024
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
import argparse
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
import zipfile
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
ap = argparse.ArgumentParser()
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
ap.add_argument("out", metavar="some.zip", type=str, nargs=1)
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
args = ap.parse_args()
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
with zipfile.ZipFile(args.out[0], "w", zipfile.ZIP_STORED) as zf:
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
     9
    zf.writestr(
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    10
        'smallish_obsstore',
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    11
        (
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    12
            # header: fm1readmarkers should start at offset 1, and
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    13
            # read until byte 597.
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    14
            '1\x00597\x00'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    15
            # body of obsstore file
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
            '\x01\x00\x00\x00vA\xd7\x02+C\x1a<)\x01,\x00\x00\x01\x03\x03\xe6'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
            '\x92\xde)x\x16\xd1Xph\xc7\xa7[\xe5\xe2\x1a\xab\x1e6e\xaf\xc2\xae'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
            '\xe7\xbc\x83\xe1\x88\xa5\xda\xce>O\xbd\x04\xe9\x03\xc4o\xeb\x03'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
            '\x01\t\x05\x04\x1fef18operationamenduserAugie Fackler <raf@duri'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
            'n42.com>\x00\x00\x00vA\xd7\x02-\x8aD\xaf-\x01,\x00\x00\x01\x03\x03'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
            '\x17*\xca\x8f\x9e}i\xe0i\xbb\xdf\x9fb\x03\xd2XG?\xd3h\x98\x89\x1a'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
            '=2\xeb\xc3\xc5<\xb3\x9e\xcc\x0e;#\xee\xc3\x10ux\x03\x01\t\x05\x04'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
            '\x1fef18operationamenduserAugie Fackler <raf@durin42.com>\x00\x00'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
            '\x00vA\xd7\x02Mn\xd9%\xea\x01,\x00\x00\x01\x03\x03\x98\x89\x1a='
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
            '2\xeb\xc3\xc5<\xb3\x9e\xcc\x0e;#\xee\xc3\x10ux\xe0*\xcaT\x86Z8J'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
            '\x85)\x97\xff7\xcc)\xc1\x7f\x19\x0c\x01\x03\x01\t\x05\x04\x1fef'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
            '18operationamenduserAugie Fackler <raf@durin42.com>\x00\x00\x00'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
            'yA\xd7\x02MtA\xbfj\x01,\x00\x00\x01\x03\x03\xe0*\xcaT\x86Z8J\x85'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
            ')\x97\xff7\xcc)\xc1\x7f\x19\x0c\x01\x00\x94\x01\xa9\n\xf80\x92\xa3'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
            'j\xc5X\xb1\xc9:\xd51\xb8*\xa9\x03\x01\t\x08\x04\x1fef11operatio'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
            'nhistedituserAugie Fackler <raf@durin42.com>\x00\x00\x00yA\xd7\x02'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
            'MtA\xd4\xe1\x01,\x00\x00\x01\x03\x03"\xa5\xcb\x86\xb6\xf4\xbaO\xa0'
6a951f535fee fuzz: new fuzzer for parsers.fm1readmarkers
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
            'sH\xe7?\xcb\x9b\xc2n\xcfI\x9e\x14\xf0D\xf0!\x18DN\xcd\x97\x016\xa5'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41032
diff changeset
    34
            '\xef\xa06\xcb\x884\x8a\x03\x01\t\x08\x04\x1fef14operationhisted'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41032
diff changeset
    35
        ),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41032
diff changeset
    36
    )