stream-clone: disable gc for the initial section for the v3 format The number of small container created turn Python in a gc-frenzy that seriously impact performance. This significantly boost performance. The following number comes from a large private repository using perf::stream-locked-section: base-line: 35.04 seconds prev-change: 24.51 seconds (-30%) prev-change: 20.88 seconds (-40%) this-change: 14.22 seconds (-60% from baseline; -31% from prev)

     1

#!/usr/bin/env python3

     2

#

     3

# A tool to help producing large and poorly compressible files

     4

#

     5

# Usage:

     6

#   $TESTDIR/seq.py 1000 | $TESTDIR/sha256line.py > my-file.txt

     7

     8

     9

import hashlib

    10

import sys

    11

    12

    13

for line in sys.stdin:

    14

    print(hashlib.sha256(line.encode('utf8')).hexdigest())