contrib/catapipe.py
author Gregory Szorc <gregory.szorc@gmail.com>
Fri, 31 Aug 2018 15:54:17 -0700
changeset 39412 a40d3da89b7d
parent 39252 9a81f126f9fa
child 39513 e9706686451b
permissions -rwxr-xr-x
cborutil: remove readindefinitebytestringtoiter() This was implemented as part of implementing streaming encoding. It was never used outside of tests. Now that we have a full CBOR decoder, it can be used for incremental decoding of indefinite-length byte strings. This also removes the last use of the vendored cbor2 package from this module. Differential Revision: https://phab.mercurial-scm.org/D4433
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
#!/usr/bin/env python3
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
# Copyright 2018 Google LLC.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
"""Tool read primitive events from a pipe to produce a catapult trace.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     9
For now the event stream supports
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    10
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    11
  START $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    12
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    13
and
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    14
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    15
  END $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
events. Everything after the SESSIONID (which must not contain spaces)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
is used as a label for the event. Events are timestamped as of when
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
they arrive in this process and are then used to produce catapult
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
traces that can be loaded in Chrome's about:tracing utility. It's
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
important that the event stream *into* this process stay simple,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
because we have to emit it from the shell scripts produced by
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
run-tests.py.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
Typically you'll want to place the path to the named pipe in the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
HGCATAPULTSERVERPIPE environment variable, which both run-tests and hg
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
understand.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
"""
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
from __future__ import absolute_import, print_function
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
import argparse
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
import datetime
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
import json
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
import os
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
_TYPEMAP = {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    37
    'START': 'B',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    38
    'END': 'E',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    39
}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    40
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    41
_threadmap = {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
def main():
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
    parser = argparse.ArgumentParser()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
    parser.add_argument('pipe', type=str, nargs=1,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
                        help='Path of named pipe to create and listen on.')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
    parser.add_argument('output', default='trace.json', type=str, nargs='?',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
                        help='Path of named pipe to create and listen on.')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
    parser.add_argument('--debug', default=False, action='store_true',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
                        help='Print useful debug messages')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    51
    args = parser.parse_args()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    52
    fn = args.pipe[0]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    53
    os.mkfifo(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    54
    try:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    55
        with open(fn) as f, open(args.output, 'w') as out:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    56
            out.write('[\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
            start = datetime.datetime.now()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
            while True:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    59
                ev = f.readline().strip()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    60
                if not ev:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    61
                    continue
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    62
                now = datetime.datetime.now()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    63
                if args.debug:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    64
                    print(ev)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    65
                verb, session, label = ev.split(' ', 2)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    66
                if session not in _threadmap:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    67
                    _threadmap[session] = len(_threadmap)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    68
                pid = _threadmap[session]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    69
                ts_micros = (now - start).total_seconds() * 1000000
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    70
                out.write(json.dumps(
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    71
                    {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    72
                        "name": label,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    73
                        "cat": "misc",
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    74
                        "ph": _TYPEMAP[verb],
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    75
                        "ts": ts_micros,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    76
                        "pid": pid,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    77
                        "tid": 1,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    78
                        "args": {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    79
                    }))
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    80
                out.write(',\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    81
    finally:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    82
        os.unlink(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    83
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    84
if __name__ == '__main__':
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    85
    main()