tests/f
author Raphaël Gomès <rgomes@octobus.net>
Thu, 03 Nov 2022 12:08:02 +0100
branchstable
changeset 49556 9172bd49cedc
parent 48875 6000f5b25c9b
child 49874 e9d06ed64161
permissions -rwxr-xr-x
testlib: add `--raw-sha1` option to `f` This will be used in the patch fixing the ignore hash collision in dirstate-v2 to check the behavior of the new hashing function.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
45830
c102b704edb5 global: use python3 in shebangs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43691
diff changeset
     1
#!/usr/bin/env python3
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     2
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     3
"""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     4
Utility for inspecting files in various ways.
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     5
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     6
This tool is like the collection of tools found in a unix environment but are
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     7
cross platform and stable and suitable for our needs in the test suite.
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     8
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     9
This can be used instead of tools like:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    10
  [
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    11
  dd
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    12
  find
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    13
  head
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    14
  hexdump
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    15
  ls
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    16
  md5sum
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    17
  readlink
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    18
  sha1sum
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    19
  stat
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    20
  tail
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    21
  test
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    22
  readlink.py
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    23
  md5sum.py
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    24
"""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    25
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    26
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    27
import binascii
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    28
import glob
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
    29
import hashlib
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    30
import optparse
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    31
import os
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    32
import re
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    33
import sys
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    34
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    35
# Python 3 adapters
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    36
ispy3 = sys.version_info[0] >= 3
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    37
if ispy3:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    38
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    39
    def iterbytes(s):
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    40
        for i in range(len(s)):
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    41
            yield s[i : i + 1]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    42
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    43
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    44
else:
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    45
    iterbytes = iter
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    46
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    47
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    48
def visit(opts, filenames, outfile):
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    49
    """Process filenames in the way specified in opts, writing output to
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    50
    outfile."""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    51
    for f in sorted(filenames):
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    52
        isstdin = f == '-'
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    53
        if not isstdin and not os.path.lexists(f):
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    54
            outfile.write(b'%s: file not found\n' % f.encode('utf-8'))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    55
            continue
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    56
        quiet = opts.quiet and not opts.recurse or isstdin
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    57
        isdir = os.path.isdir(f)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    58
        islink = os.path.islink(f)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    59
        isfile = os.path.isfile(f) and not islink
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    60
        dirfiles = None
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    61
        content = None
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    62
        facts = []
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    63
        if isfile:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    64
            if opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    65
                facts.append(b'file')
49556
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    66
            needs_reading = (
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    67
                opts.hexdump,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    68
                opts.dump,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    69
                opts.md5,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    70
                opts.sha1,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    71
                opts.raw_sha1,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    72
                opts.sha256,
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    73
            )
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    74
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
    75
            if any(needs_reading):
36948
0585337ea787 cleanup: fix some latent open(path).read() et al calls we previously missed
Augie Fackler <augie@google.com>
parents: 36261
diff changeset
    76
                with open(f, 'rb') as fobj:
0585337ea787 cleanup: fix some latent open(path).read() et al calls we previously missed
Augie Fackler <augie@google.com>
parents: 36261
diff changeset
    77
                    content = fobj.read()
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    78
        elif islink:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    79
            if opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    80
                facts.append(b'link')
40246
be0a5d2d5c78 tests: fix last failure in test-tools.t
Augie Fackler <augie@google.com>
parents: 38364
diff changeset
    81
            content = os.readlink(f).encode('utf8')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    82
        elif isstdin:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    83
            content = getattr(sys.stdin, 'buffer', sys.stdin).read()
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    84
            if opts.size:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    85
                facts.append(b'size=%d' % len(content))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    86
        elif isdir:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    87
            if opts.recurse or opts.type:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    88
                dirfiles = glob.glob(f + '/*')
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    89
                facts.append(b'directory with %d files' % len(dirfiles))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    90
        elif opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    91
            facts.append(b'type unknown')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    92
        if not isstdin:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    93
            stat = os.lstat(f)
23911
593a5cd709a2 tests: teach f not to report directory size
Matt Mackall <mpm@selenic.com>
parents: 23860
diff changeset
    94
            if opts.size and not isdir:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    95
                facts.append(b'size=%d' % stat.st_size)
23912
7d0aa6269ece tests: teach f not to report symlink mode bits
Matt Mackall <mpm@selenic.com>
parents: 23911
diff changeset
    96
            if opts.mode and not islink:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    97
                facts.append(b'mode=%o' % (stat.st_mode & 0o777))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    98
            if opts.links:
38364
06c85cbd6824 py3: use '%d' for os.stat_result.st_nlink instead of '%s'
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36948
diff changeset
    99
                facts.append(b'links=%d' % stat.st_nlink)
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   100
            if opts.newer:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   101
                # mtime might be in whole seconds so newer file might be same
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   102
                if stat.st_mtime >= os.stat(opts.newer).st_mtime:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   103
                    facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   104
                        b'newer than %s' % opts.newer.encode('utf8', 'replace')
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   105
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   106
                else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   107
                    facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   108
                        b'older than %s' % opts.newer.encode('utf8', 'replace')
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   109
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   110
        if opts.md5 and content is not None:
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
   111
            h = hashlib.md5(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   112
            facts.append(b'md5=%s' % binascii.hexlify(h.digest())[: opts.bytes])
49556
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   113
        if opts.raw_sha1 and content is not None:
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   114
            h = hashlib.sha1(content)
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   115
            facts.append(b'raw-sha1=%s' % h.digest()[: opts.bytes])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   116
        if opts.sha1 and content is not None:
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
   117
            h = hashlib.sha1(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   118
            facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   119
                b'sha1=%s' % binascii.hexlify(h.digest())[: opts.bytes]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   120
            )
35472
c1f7037c2ded tests: teach `f` to handle sha256 checksums
Matt Harbison <matt_harbison@yahoo.com>
parents: 35471
diff changeset
   121
        if opts.sha256 and content is not None:
c1f7037c2ded tests: teach `f` to handle sha256 checksums
Matt Harbison <matt_harbison@yahoo.com>
parents: 35471
diff changeset
   122
            h = hashlib.sha256(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   123
            facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   124
                b'sha256=%s' % binascii.hexlify(h.digest())[: opts.bytes]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   125
            )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   126
        if isstdin:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   127
            outfile.write(b', '.join(facts) + b'\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   128
        elif facts:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   129
            outfile.write(b'%s: %s\n' % (f.encode('utf-8'), b', '.join(facts)))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   130
        elif not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   131
            outfile.write(b'%s:\n' % f.encode('utf-8'))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   132
        if content is not None:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   133
            chunk = content
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   134
            if not islink:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   135
                if opts.lines:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   136
                    if opts.lines >= 0:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   137
                        chunk = b''.join(chunk.splitlines(True)[: opts.lines])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   138
                    else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   139
                        chunk = b''.join(chunk.splitlines(True)[opts.lines :])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   140
                if opts.bytes:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   141
                    if opts.bytes >= 0:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   142
                        chunk = chunk[: opts.bytes]
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   143
                    else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   144
                        chunk = chunk[opts.bytes :]
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   145
            if opts.hexdump:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   146
                for i in range(0, len(chunk), 16):
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   147
                    s = chunk[i : i + 16]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   148
                    outfile.write(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   149
                        b'%04x: %-47s |%s|\n'
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   150
                        % (
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   151
                            i,
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   152
                            b' '.join(b'%02x' % ord(c) for c in iterbytes(s)),
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   153
                            re.sub(b'[^ -~]', b'.', s),
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   154
                        )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   155
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   156
            if opts.dump:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   157
                if not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   158
                    outfile.write(b'>>>\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   159
                outfile.write(chunk)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   160
                if not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   161
                    if chunk.endswith(b'\n'):
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   162
                        outfile.write(b'<<<\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   163
                    else:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   164
                        outfile.write(b'\n<<< no trailing newline\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   165
        if opts.recurse and dirfiles:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   166
            assert not isstdin
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   167
            visit(opts, dirfiles, outfile)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   168
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   169
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   170
if __name__ == "__main__":
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   171
    parser = optparse.OptionParser("%prog [options] [filenames]")
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   172
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   173
        "-t",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   174
        "--type",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   175
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   176
        help="show file type (file or directory)",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   177
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   178
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   179
        "-m", "--mode", action="store_true", help="show file mode"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   180
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   181
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   182
        "-l", "--links", action="store_true", help="show number of links"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   183
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   184
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   185
        "-s", "--size", action="store_true", help="show size of file"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   186
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   187
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   188
        "-n", "--newer", action="store", help="check if file is newer (or same)"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   189
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   190
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   191
        "-r", "--recurse", action="store_true", help="recurse into directories"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   192
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   193
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   194
        "-S",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   195
        "--sha1",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   196
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   197
        help="show sha1 hash of the content",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   198
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   199
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   200
        "",
49556
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   201
        "--raw-sha1",
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   202
        action="store_true",
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   203
        help="show raw bytes of the sha1 hash of the content",
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   204
    )
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   205
    parser.add_option(
9172bd49cedc testlib: add `--raw-sha1` option to `f`
Raphaël Gomès <rgomes@octobus.net>
parents: 48875
diff changeset
   206
        "",
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   207
        "--sha256",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   208
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   209
        help="show sha256 hash of the content",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   210
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   211
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   212
        "-M", "--md5", action="store_true", help="show md5 hash of the content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   213
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   214
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   215
        "-D", "--dump", action="store_true", help="dump file content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   216
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   217
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   218
        "-H", "--hexdump", action="store_true", help="hexdump file content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   219
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   220
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   221
        "-B", "--bytes", type="int", help="number of characters to dump"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   222
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   223
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   224
        "-L", "--lines", type="int", help="number of lines to dump"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   225
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   226
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   227
        "-q", "--quiet", action="store_true", help="no default output"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   228
    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   229
    (opts, filenames) = parser.parse_args(sys.argv[1:])
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   230
    if not filenames:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   231
        filenames = ['-']
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   232
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   233
    visit(opts, filenames, getattr(sys.stdout, 'buffer', sys.stdout))