mercurial/utils/cborutil.py
author Augie Fackler <augie@google.com>
Fri, 08 Nov 2019 11:19:20 -0800
changeset 43506 9f70512ae2cf
parent 43117 8ff1ecfadcd1
child 45942 89a2afe31e82
permissions -rw-r--r--
cleanup: remove pointless r-prefixes on single-quoted strings This is the promised second step on single-quoted strings. These had existed because our source transformer didn't turn r'' into b'', so we had tagged some strings as r-strings to get "native" strings on both Pythons. Now that the transformer is gone, we can dispense with this nonsense. Methodology: I ran hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$' | xargs egrep --color=never -n -- \[\^b\]\[\^a-z\]r\'\[\^\'\\\\\]\*\'\[\^\'\ in an emacs grep-mode buffer, and then used a keyboard macro to iterate over the results and remove the r prefix as needed. # skip-blame removing unneeded r prefixes left over from Python 3 migration. Differential Revision: https://phab.mercurial-scm.org/D7306
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# cborutil.py - CBOR extensions
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
from __future__ import absolute_import
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    10
import struct
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    11
import sys
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    12
39456
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
    13
from .. import pycompat
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
    14
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
# Very short very of RFC 7049...
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
#
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
# Each item begins with a byte. The 3 high bits of that byte denote the
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    18
# "major type." The lower 5 bits denote the "subtype." Each major type
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    19
# has its own encoding mechanism.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
#
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
# Most types have lengths. However, bytestring, string, array, and map
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
# can be indefinite length. These are denotes by a subtype with value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
# Sub-components of those types then come afterwards and are terminated
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
# by a "break" byte.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
MAJOR_TYPE_UINT = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
MAJOR_TYPE_NEGINT = 1
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
MAJOR_TYPE_BYTESTRING = 2
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
MAJOR_TYPE_STRING = 3
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
MAJOR_TYPE_ARRAY = 4
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
MAJOR_TYPE_MAP = 5
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
MAJOR_TYPE_SEMANTIC = 6
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
MAJOR_TYPE_SPECIAL = 7
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
SUBTYPE_MASK = 0b00011111
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    37
SUBTYPE_FALSE = 20
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    38
SUBTYPE_TRUE = 21
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    39
SUBTYPE_NULL = 22
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
SUBTYPE_HALF_FLOAT = 25
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
SUBTYPE_SINGLE_FLOAT = 26
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
SUBTYPE_DOUBLE_FLOAT = 27
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    43
SUBTYPE_INDEFINITE = 31
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    45
SEMANTIC_TAG_FINITE_SET = 258
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
    46
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
# Indefinite types begin with their major type ORd with information value 31.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
BEGIN_INDEFINITE_BYTESTRING = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    49
    '>B', MAJOR_TYPE_BYTESTRING << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    50
)
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
BEGIN_INDEFINITE_ARRAY = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    52
    '>B', MAJOR_TYPE_ARRAY << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    53
)
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
BEGIN_INDEFINITE_MAP = struct.pack(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    55
    '>B', MAJOR_TYPE_MAP << 5 | SUBTYPE_INDEFINITE
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    56
)
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    58
ENCODED_LENGTH_1 = struct.Struct('>B')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    59
ENCODED_LENGTH_2 = struct.Struct('>BB')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    60
ENCODED_LENGTH_3 = struct.Struct('>BH')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    61
ENCODED_LENGTH_4 = struct.Struct('>BL')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
    62
ENCODED_LENGTH_5 = struct.Struct('>BQ')
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
# The break ends an indefinite length item.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
BREAK = b'\xff'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
BREAK_INT = 255
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    68
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    69
def encodelength(majortype, length):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
    """Obtain a value encoding the major type and its length."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    71
    if length < 24:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    72
        return ENCODED_LENGTH_1.pack(majortype << 5 | length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    73
    elif length < 256:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    74
        return ENCODED_LENGTH_2.pack(majortype << 5 | 24, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    75
    elif length < 65536:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    76
        return ENCODED_LENGTH_3.pack(majortype << 5 | 25, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    77
    elif length < 4294967296:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    78
        return ENCODED_LENGTH_4.pack(majortype << 5 | 26, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    79
    else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    80
        return ENCODED_LENGTH_5.pack(majortype << 5 | 27, length)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    82
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
def streamencodebytestring(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    84
    yield encodelength(MAJOR_TYPE_BYTESTRING, len(v))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
    yield v
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
    87
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
def streamencodebytestringfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
    """Convert an iterator of chunks to an indefinite bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
    Given an input that is iterable and each element in the iterator is
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    92
    representable as bytes, emit an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
    """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    94
    yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    96
    for chunk in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    97
        yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    98
        yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
    yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   102
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
def streamencodeindefinitebytestring(source, chunksize=65536):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   104
    """Given a large source buffer, emit as an indefinite length bytestring.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   105
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   106
    This is a generator of chunks constituting the encoded CBOR data.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   107
    """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   108
    yield BEGIN_INDEFINITE_BYTESTRING
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   109
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   110
    i = 0
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   111
    l = len(source)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   112
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   113
    while True:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   114
        chunk = source[i : i + chunksize]
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   115
        i += len(chunk)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   116
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   117
        yield encodelength(MAJOR_TYPE_BYTESTRING, len(chunk))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   118
        yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   119
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   120
        if i >= l:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   121
            break
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   122
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   123
    yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   124
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   125
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   126
def streamencodeint(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   127
    if v >= 18446744073709551616 or v < -18446744073709551616:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   128
        raise ValueError(b'big integers not supported')
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   129
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   130
    if v >= 0:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   131
        yield encodelength(MAJOR_TYPE_UINT, v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
    else:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   133
        yield encodelength(MAJOR_TYPE_NEGINT, abs(v) - 1)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   134
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   135
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   136
def streamencodearray(l):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   137
    """Encode a known size iterable to an array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   138
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   139
    yield encodelength(MAJOR_TYPE_ARRAY, len(l))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   140
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   141
    for i in l:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   142
        for chunk in streamencode(i):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   143
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   145
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   146
def streamencodearrayfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   147
    """Encode an iterator of items to an indefinite length array."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   148
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   149
    yield BEGIN_INDEFINITE_ARRAY
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   150
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   151
    for i in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   152
        for chunk in streamencode(i):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   153
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   154
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   155
    yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   156
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   157
37898
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
   158
def _mixedtypesortkey(v):
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
   159
    return type(v).__name__, v
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
   160
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   161
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   162
def streamencodeset(s):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   163
    # https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml defines
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   164
    # semantic tag 258 for finite sets.
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   165
    yield encodelength(MAJOR_TYPE_SEMANTIC, SEMANTIC_TAG_FINITE_SET)
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   166
37898
2ae6a3134362 cborutil: port to Python 3
Augie Fackler <augie@google.com>
parents: 37711
diff changeset
   167
    for chunk in streamencodearray(sorted(s, key=_mixedtypesortkey)):
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   168
        yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   169
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   170
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   171
def streamencodemap(d):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   172
    """Encode dictionary to a generator.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   173
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   174
    Does not supporting indefinite length dictionaries.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   175
    """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   176
    yield encodelength(MAJOR_TYPE_MAP, len(d))
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   177
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   178
    for key, value in sorted(
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
   179
        pycompat.iteritems(d), key=lambda x: _mixedtypesortkey(x[0])
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   180
    ):
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   181
        for chunk in streamencode(key):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
        for chunk in streamencode(value):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   184
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   185
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   186
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   187
def streamencodemapfromiter(it):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
    """Given an iterable of (key, value), encode to an indefinite length map."""
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
    yield BEGIN_INDEFINITE_MAP
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   190
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   191
    for key, value in it:
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   192
        for chunk in streamencode(key):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   193
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   194
        for chunk in streamencode(value):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   195
            yield chunk
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   196
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   197
    yield BREAK
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   198
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   199
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   200
def streamencodebool(b):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   201
    # major type 7, simple value 20 and 21.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   202
    yield b'\xf5' if b else b'\xf4'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   203
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   204
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   205
def streamencodenone(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   206
    # major type 7, simple value 22.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   207
    yield b'\xf6'
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   208
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   209
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   210
STREAM_ENCODERS = {
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   211
    bytes: streamencodebytestring,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   212
    int: streamencodeint,
39456
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39413
diff changeset
   213
    pycompat.long: streamencodeint,
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   214
    list: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   215
    tuple: streamencodearray,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   216
    dict: streamencodemap,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
    set: streamencodeset,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
    bool: streamencodebool,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
    type(None): streamencodenone,
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
}
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   221
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   222
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   223
def streamencode(v):
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   224
    """Encode a value in a streaming manner.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   225
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   226
    Given an input object, encode it to CBOR recursively.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   227
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   228
    Returns a generator of CBOR encoded bytes. There is no guarantee
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   229
    that each emitted chunk fully decodes to a value or sub-value.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   230
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   231
    Encoding is deterministic - unordered collections are sorted.
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   232
    """
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   233
    fn = STREAM_ENCODERS.get(v.__class__)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   234
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   235
    if not fn:
42480
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   236
        # handle subtypes such as encoding.localstr and util.sortdict
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   237
        for ty in STREAM_ENCODERS:
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   238
            if not isinstance(v, ty):
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   239
                continue
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   240
            fn = STREAM_ENCODERS[ty]
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   241
            break
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   242
b6387a65851d cborutil: fix streamencode() to handle subtypes
Yuya Nishihara <yuya@tcha.org>
parents: 40124
diff changeset
   243
    if not fn:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   244
        raise ValueError(b'do not know how to encode %s' % type(v))
37711
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   245
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   246
    return fn(v)
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   247
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   248
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   249
class CBORDecodeError(Exception):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   250
    """Represents an error decoding CBOR."""
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   251
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   252
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   253
if sys.version_info.major >= 3:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   254
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   255
    def _elementtointeger(b, i):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   256
        return b[i]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   257
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   258
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   259
else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   260
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   261
    def _elementtointeger(b, i):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   262
        return ord(b[i])
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   263
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   264
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   265
STRUCT_BIG_UBYTE = struct.Struct('>B')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   266
STRUCT_BIG_USHORT = struct.Struct(b'>H')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   267
STRUCT_BIG_ULONG = struct.Struct(b'>L')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   268
STRUCT_BIG_ULONGLONG = struct.Struct(b'>Q')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   269
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   270
SPECIAL_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   271
SPECIAL_START_INDEFINITE_BYTESTRING = 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   272
SPECIAL_START_ARRAY = 2
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   273
SPECIAL_START_MAP = 3
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   274
SPECIAL_START_SET = 4
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   275
SPECIAL_INDEFINITE_BREAK = 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   276
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   277
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   278
def decodeitem(b, offset=0):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   279
    """Decode a new CBOR value from a buffer at offset.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   280
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   281
    This function attempts to decode up to one complete CBOR value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   282
    from ``b`` starting at offset ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   283
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   284
    The beginning of a collection (such as an array, map, set, or
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   285
    indefinite length bytestring) counts as a single value. For these
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   286
    special cases, a state flag will indicate that a special value was seen.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   287
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   288
    When called, the function either returns a decoded value or gives
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   289
    a hint as to how many more bytes are needed to do so. By calling
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   290
    the function repeatedly given a stream of bytes, the caller can
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   291
    build up the original values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   292
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   293
    Returns a tuple with the following elements:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   294
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   295
    * Bool indicating whether a complete value was decoded.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   296
    * A decoded value if first value is True otherwise None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   297
    * Integer number of bytes. If positive, the number of bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   298
      read. If negative, the number of bytes we need to read to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   299
      decode this value or the next chunk in this value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   300
    * One of the ``SPECIAL_*`` constants indicating special treatment
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   301
      for this value. ``SPECIAL_NONE`` means this is a fully decoded
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   302
      simple value (such as an integer or bool).
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   303
    """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   304
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   305
    initial = _elementtointeger(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   306
    offset += 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   307
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   308
    majortype = initial >> 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   309
    subtype = initial & SUBTYPE_MASK
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   310
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   311
    if majortype == MAJOR_TYPE_UINT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   312
        complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   313
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   314
        if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   315
            return True, value, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   316
        else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   317
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   318
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   319
    elif majortype == MAJOR_TYPE_NEGINT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   320
        # Negative integers are the same as UINT except inverted minus 1.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   321
        complete, value, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   322
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   323
        if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   324
            return True, -value - 1, readcount + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   325
        else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   326
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   327
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   328
    elif majortype == MAJOR_TYPE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   329
        # Beginning of bytestrings are treated as uints in order to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   330
        # decode their length, which may be indefinite.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   331
        complete, size, readcount = decodeuint(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   332
            subtype, b, offset, allowindefinite=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   333
        )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   334
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   335
        # We don't know the size of the bytestring. It must be a definitive
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   336
        # length since the indefinite subtype would be encoded in the initial
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   337
        # byte.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   338
        if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   339
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   340
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   341
        # We know the length of the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   342
        if size is not None:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   343
            # And the data is available in the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   344
            if offset + readcount + size <= len(b):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   345
                value = b[offset + readcount : offset + readcount + size]
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   346
                return True, value, readcount + size + 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   347
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   348
            # And we need more data in order to return the bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   349
            else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   350
                wanted = len(b) - offset - readcount - size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   351
                return False, None, wanted, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   352
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   353
        # It is an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   354
        else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   355
            return True, None, 1, SPECIAL_START_INDEFINITE_BYTESTRING
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   356
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   357
    elif majortype == MAJOR_TYPE_STRING:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   358
        raise CBORDecodeError(b'string major type not supported')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   359
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   360
    elif majortype == MAJOR_TYPE_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   361
        # Beginning of arrays are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   362
        # length. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   363
        complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   364
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   365
        if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   366
            return True, size, readcount + 1, SPECIAL_START_ARRAY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   367
        else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   368
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   369
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   370
    elif majortype == MAJOR_TYPE_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   371
        # Beginning of maps are treated as uints in order to decode their
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   372
        # number of elements. We don't allow indefinite length arrays.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   373
        complete, size, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   374
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   375
        if complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   376
            return True, size, readcount + 1, SPECIAL_START_MAP
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   377
        else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   378
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   379
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   380
    elif majortype == MAJOR_TYPE_SEMANTIC:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   381
        # Semantic tag value is read the same as a uint.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   382
        complete, tagvalue, readcount = decodeuint(subtype, b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   383
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   384
        if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   385
            return False, None, readcount, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   386
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   387
        # This behavior here is a little wonky. The main type being "decorated"
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   388
        # by this semantic tag follows. A more robust parser would probably emit
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   389
        # a special flag indicating this as a semantic tag and let the caller
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   390
        # deal with the types that follow. But since we don't support many
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   391
        # semantic tags, it is easier to deal with the special cases here and
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   392
        # hide complexity from the caller. If we add support for more semantic
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   393
        # tags, we should probably move semantic tag handling into the caller.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   394
        if tagvalue == SEMANTIC_TAG_FINITE_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   395
            if offset + readcount >= len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   396
                return False, None, -1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   397
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   398
            complete, size, readcount2, special = decodeitem(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   399
                b, offset + readcount
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   400
            )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   401
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   402
            if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   403
                return False, None, readcount2, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   404
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   405
            if special != SPECIAL_START_ARRAY:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   406
                raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
   407
                    b'expected array after finite set semantic tag'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   408
                )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   409
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   410
            return True, size, readcount + readcount2 + 1, SPECIAL_START_SET
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   412
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   413
            raise CBORDecodeError(b'semantic tag %d not allowed' % tagvalue)
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   414
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   415
    elif majortype == MAJOR_TYPE_SPECIAL:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   416
        # Only specific values for the information field are allowed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   417
        if subtype == SUBTYPE_FALSE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   418
            return True, False, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   419
        elif subtype == SUBTYPE_TRUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   420
            return True, True, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   421
        elif subtype == SUBTYPE_NULL:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   422
            return True, None, 1, SPECIAL_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   423
        elif subtype == SUBTYPE_INDEFINITE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   424
            return True, None, 1, SPECIAL_INDEFINITE_BREAK
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   425
        # If value is 24, subtype is in next byte.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   426
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   427
            raise CBORDecodeError(b'special type %d not allowed' % subtype)
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   428
    else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   429
        assert False
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   430
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   431
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   432
def decodeuint(subtype, b, offset=0, allowindefinite=False):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   433
    """Decode an unsigned integer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   434
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   435
    ``subtype`` is the lower 5 bits from the initial byte CBOR item
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   436
    "header." ``b`` is a buffer containing bytes. ``offset`` points to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   437
    the index of the first byte after the byte that ``subtype`` was
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   438
    derived from.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   439
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   440
    ``allowindefinite`` allows the special indefinite length value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   441
    indicator.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   442
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   443
    Returns a 3-tuple of (successful, value, count).
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   444
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   445
    The first element is a bool indicating if decoding completed. The 2nd
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   446
    is the decoded integer value or None if not fully decoded or the subtype
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   447
    is 31 and ``allowindefinite`` is True. The 3rd value is the count of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   448
    If positive, it is the number of additional bytes decoded. If negative,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   449
    it is the number of additional bytes needed to decode this value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   450
    """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   451
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   452
    # Small values are inline.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   453
    if subtype < 24:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   454
        return True, subtype, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   455
    # Indefinite length specifier.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   456
    elif subtype == 31:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   457
        if allowindefinite:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   458
            return True, None, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   459
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   460
            raise CBORDecodeError(b'indefinite length uint not allowed here')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   461
    elif subtype >= 28:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   462
        raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   463
            b'unsupported subtype on integer type: %d' % subtype
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   464
        )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   465
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   466
    if subtype == 24:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   467
        s = STRUCT_BIG_UBYTE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   468
    elif subtype == 25:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   469
        s = STRUCT_BIG_USHORT
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   470
    elif subtype == 26:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   471
        s = STRUCT_BIG_ULONG
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   472
    elif subtype == 27:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   473
        s = STRUCT_BIG_ULONGLONG
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   474
    else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   475
        raise CBORDecodeError(b'bounds condition checking violation')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   476
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   477
    if len(b) - offset >= s.size:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   478
        return True, s.unpack_from(b, offset)[0], s.size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   479
    else:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   480
        return False, None, len(b) - offset - s.size
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   481
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   482
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   483
class bytestringchunk(bytes):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   484
    """Represents a chunk/segment in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   485
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   486
    This behaves like a ``bytes`` but in addition has the ``isfirst``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   487
    and ``islast`` attributes indicating whether this chunk is the first
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   488
    or last in an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   489
    """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   490
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   491
    def __new__(cls, v, first=False, last=False):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   492
        self = bytes.__new__(cls, v)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   493
        self.isfirst = first
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   494
        self.islast = last
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   495
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   496
        return self
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   497
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   498
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   499
class sansiodecoder(object):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   500
    """A CBOR decoder that doesn't perform its own I/O.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   501
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   502
    To use, construct an instance and feed it segments containing
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   503
    CBOR-encoded bytes via ``decode()``. The return value from ``decode()``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   504
    indicates whether a fully-decoded value is available, how many bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   505
    were consumed, and offers a hint as to how many bytes should be fed
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   506
    in next time to decode the next value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   507
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   508
    The decoder assumes it will decode N discrete CBOR values, not just
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   509
    a single value. i.e. if the bytestream contains uints packed one after
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   510
    the other, the decoder will decode them all, rather than just the initial
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   511
    one.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   512
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   513
    When ``decode()`` indicates a value is available, call ``getavailable()``
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   514
    to return all fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   515
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   516
    ``decode()`` can partially decode input. It is up to the caller to keep
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   517
    track of what data was consumed and to pass unconsumed data in on the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   518
    next invocation.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   519
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   520
    The decoder decodes atomically at the *item* level. See ``decodeitem()``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   521
    If an *item* cannot be fully decoded, the decoder won't record it as
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   522
    partially consumed. Instead, the caller will be instructed to pass in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   523
    the initial bytes of this item on the next invocation. This does result
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   524
    in some redundant parsing. But the overhead should be minimal.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   525
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   526
    This decoder only supports a subset of CBOR as required by Mercurial.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   527
    It lacks support for:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   528
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   529
    * Indefinite length arrays
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   530
    * Indefinite length maps
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   531
    * Use of indefinite length bytestrings as keys or values within
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   532
      arrays, maps, or sets.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   533
    * Nested arrays, maps, or sets within sets
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   534
    * Any semantic tag that isn't a mathematical finite set
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   535
    * Floating point numbers
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   536
    * Undefined special value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   537
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   538
    CBOR types are decoded to Python types as follows:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   539
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   540
    uint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   541
    negint -> int
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   542
    bytestring -> bytes
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   543
    map -> dict
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   544
    array -> list
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   545
    True -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   546
    False -> bool
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   547
    null -> None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   548
    indefinite length bytestring chunk -> [bytestringchunk]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   549
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   550
    The only non-obvious mapping here is an indefinite length bytestring
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   551
    to the ``bytestringchunk`` type. This is to facilitate streaming
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   552
    indefinite length bytestrings out of the decoder and to differentiate
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   553
    a regular bytestring from an indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   554
    """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   555
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   556
    _STATE_NONE = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   557
    _STATE_WANT_MAP_KEY = 1
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   558
    _STATE_WANT_MAP_VALUE = 2
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   559
    _STATE_WANT_ARRAY_VALUE = 3
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   560
    _STATE_WANT_SET_VALUE = 4
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   561
    _STATE_WANT_BYTESTRING_CHUNK_FIRST = 5
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   562
    _STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT = 6
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   563
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   564
    def __init__(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   565
        # TODO add support for limiting size of bytestrings
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   566
        # TODO add support for limiting number of keys / values in collections
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   567
        # TODO add support for limiting size of buffered partial values
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   568
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   569
        self.decodedbytecount = 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   570
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   571
        self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   572
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   573
        # Stack of active nested collections. Each entry is a dict describing
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   574
        # the collection.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   575
        self._collectionstack = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   576
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   577
        # Fully decoded key to use for the current map.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   578
        self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   579
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   580
        # Fully decoded values available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   581
        self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   582
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   583
    @property
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   584
    def inprogress(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   585
        """Whether the decoder has partially decoded a value."""
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   586
        return self._state != self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   587
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   588
    def decode(self, b, offset=0):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   589
        """Attempt to decode bytes from an input buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   590
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   591
        ``b`` is a collection of bytes and ``offset`` is the byte
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   592
        offset within that buffer from which to begin reading data.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   593
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   594
        ``b`` must support ``len()`` and accessing bytes slices via
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   595
        ``__slice__``. Typically ``bytes`` instances are used.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   596
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   597
        Returns a tuple with the following fields:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   598
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   599
        * Bool indicating whether values are available for retrieval.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   600
        * Integer indicating the number of bytes that were fully consumed,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   601
          starting from ``offset``.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   602
        * Integer indicating the number of bytes that are desired for the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   603
          next call in order to decode an item.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   604
        """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   605
        if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   606
            return bool(self._decodedvalues), 0, 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   607
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   608
        initialoffset = offset
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   609
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   610
        # We could easily split the body of this loop into a function. But
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   611
        # Python performance is sensitive to function calls and collections
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   612
        # are composed of many items. So leaving as a while loop could help
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   613
        # with performance. One thing that may not help is the use of
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   614
        # if..elif versus a lookup/dispatch table. There may be value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   615
        # in switching that.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   616
        while offset < len(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   617
            # Attempt to decode an item. This could be a whole value or a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   618
            # special value indicating an event, such as start or end of a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   619
            # collection or indefinite length type.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   620
            complete, value, readcount, special = decodeitem(b, offset)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   621
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   622
            if readcount > 0:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   623
                self.decodedbytecount += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   624
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   625
            if not complete:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   626
                assert readcount < 0
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   627
                return (
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   628
                    bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   629
                    offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   630
                    -readcount,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   631
                )
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   632
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   633
            offset += readcount
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   634
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   635
            # No nested state. We either have a full value or beginning of a
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   636
            # complex value to deal with.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   637
            if self._state == self._STATE_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   638
                # A normal value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   639
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   640
                    self._decodedvalues.append(value)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   641
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   642
                elif special == SPECIAL_START_ARRAY:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   643
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   644
                        {b'remaining': value, b'v': [],}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   645
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   646
                    self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   647
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   648
                elif special == SPECIAL_START_MAP:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   649
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   650
                        {b'remaining': value, b'v': {},}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   651
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   652
                    self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   653
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   654
                elif special == SPECIAL_START_SET:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   655
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   656
                        {b'remaining': value, b'v': set(),}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   657
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   658
                    self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   659
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   660
                elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   661
                    self._state = self._STATE_WANT_BYTESTRING_CHUNK_FIRST
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   662
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   663
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   664
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   665
                        b'unhandled special state: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   666
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   667
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   668
            # This value becomes an element of the current array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   669
            elif self._state == self._STATE_WANT_ARRAY_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   670
                # Simple values get appended.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   671
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   672
                    c = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   673
                    c[b'v'].append(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   674
                    c[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   675
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   676
                    # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   677
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   678
                # An array nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   679
                elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   680
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   681
                    newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   682
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   683
                    lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   684
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   685
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   686
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   687
                        {b'remaining': value, b'v': newvalue,}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   688
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   689
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   690
                    # self._state doesn't need changed.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   691
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   692
                # A map nested within an array.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   693
                elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   694
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   695
                    newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   696
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   697
                    lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   698
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   699
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   700
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   701
                        {b'remaining': value, b'v': newvalue}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   702
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   703
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   704
                    self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   705
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   706
                elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   707
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   708
                    newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   709
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   710
                    lastc[b'v'].append(newvalue)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   711
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   712
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   713
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   714
                        {b'remaining': value, b'v': newvalue,}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   715
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   716
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   717
                    self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   718
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   719
                elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   720
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   721
                        b'indefinite length bytestrings '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   722
                        b'not allowed as array values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   723
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   724
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   725
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   726
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   727
                        b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   728
                        b'expecting array value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   729
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   730
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   731
            # This value becomes the key of the current map instance.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   732
            elif self._state == self._STATE_WANT_MAP_KEY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   733
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   734
                    self._currentmapkey = value
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   735
                    self._state = self._STATE_WANT_MAP_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   736
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   737
                elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   738
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   739
                        b'indefinite length bytestrings '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   740
                        b'not allowed as map keys'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   741
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   742
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   743
                elif special in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   744
                    SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   745
                    SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   746
                    SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   747
                ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   748
                    raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
   749
                        b'collections not supported as map keys'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   750
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   751
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   752
                # We do not allow special values to be used as map keys.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   753
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   754
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   755
                        b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   756
                        b'expecting map key: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   757
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   758
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   759
            # This value becomes the value of the current map key.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   760
            elif self._state == self._STATE_WANT_MAP_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   761
                # Simple values simply get inserted into the map.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   762
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   763
                    lastc = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   764
                    lastc[b'v'][self._currentmapkey] = value
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   765
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   766
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   767
                    self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   768
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   769
                # A new array is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   770
                elif special == SPECIAL_START_ARRAY:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   771
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   772
                    newvalue = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   773
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   774
                    lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   775
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   776
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   777
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   778
                        {b'remaining': value, b'v': newvalue,}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   779
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   780
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   781
                    self._state = self._STATE_WANT_ARRAY_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   782
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   783
                # A new map is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   784
                elif special == SPECIAL_START_MAP:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   785
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   786
                    newvalue = {}
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   787
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   788
                    lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   789
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   790
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   791
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   792
                        {b'remaining': value, b'v': newvalue,}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   793
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   794
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   795
                    self._state = self._STATE_WANT_MAP_KEY
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   796
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   797
                # A new set is used as the map value.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   798
                elif special == SPECIAL_START_SET:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   799
                    lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   800
                    newvalue = set()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   801
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   802
                    lastc[b'v'][self._currentmapkey] = newvalue
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   803
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   804
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   805
                    self._collectionstack.append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   806
                        {b'remaining': value, b'v': newvalue,}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   807
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   808
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   809
                    self._state = self._STATE_WANT_SET_VALUE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   810
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   811
                elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   812
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   813
                        b'indefinite length bytestrings not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   814
                        b'allowed as map values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   815
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   816
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   817
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   818
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   819
                        b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   820
                        b'expecting map value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   821
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   822
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   823
                self._currentmapkey = None
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   824
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   825
            # This value is added to the current set.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   826
            elif self._state == self._STATE_WANT_SET_VALUE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   827
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   828
                    lastc = self._collectionstack[-1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   829
                    lastc[b'v'].add(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   830
                    lastc[b'remaining'] -= 1
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   831
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   832
                elif special == SPECIAL_START_INDEFINITE_BYTESTRING:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   833
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   834
                        b'indefinite length bytestrings not '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   835
                        b'allowed as set values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   836
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   837
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   838
                elif special in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   839
                    SPECIAL_START_ARRAY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   840
                    SPECIAL_START_MAP,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   841
                    SPECIAL_START_SET,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   842
                ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   843
                    raise CBORDecodeError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
   844
                        b'collections not allowed as set values'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   845
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   846
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   847
                # We don't allow non-trivial types to exist as set values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   848
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   849
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   850
                        b'unhandled special item when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   851
                        b'expecting set value: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   852
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   853
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   854
            # This value represents the first chunk in an indefinite length
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   855
            # bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   856
            elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_FIRST:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   857
                # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   858
                if special == SPECIAL_NONE:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   859
                    self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   860
                        bytestringchunk(value, first=True)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   861
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   862
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   863
                    self._state = self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   864
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   865
                # The end of stream marker. This means it is an empty
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   866
                # indefinite length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   867
                elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   868
                    # We /could/ convert this to a b''. But we want to preserve
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   869
                    # the nature of the underlying data so consumers expecting
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   870
                    # an indefinite length bytestring get one.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   871
                    self._decodedvalues.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   872
                        bytestringchunk(b'', first=True, last=True)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   873
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   874
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   875
                    # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   876
                    # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   877
                    assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   878
                    self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   879
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   880
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   881
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   882
                        b'unexpected special value when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   883
                        b'expecting bytestring chunk: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   884
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   885
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   886
            # This value represents the non-initial chunk in an indefinite
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   887
            # length bytestring.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   888
            elif self._state == self._STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENT:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   889
                # We received a full chunk.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   890
                if special == SPECIAL_NONE:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   891
                    self._decodedvalues.append(bytestringchunk(value))
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   892
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   893
                # The end of stream marker.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   894
                elif special == SPECIAL_INDEFINITE_BREAK:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   895
                    self._decodedvalues.append(bytestringchunk(b'', last=True))
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   896
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   897
                    # Since indefinite length bytestrings can't be used in
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   898
                    # collections, we must be at the root level.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   899
                    assert not self._collectionstack
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   900
                    self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   901
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   902
                else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   903
                    raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   904
                        b'unexpected special value when '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   905
                        b'expecting bytestring chunk: %d' % special
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   906
                    )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   907
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   908
            else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   909
                raise CBORDecodeError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   910
                    b'unhandled decoder state: %d' % self._state
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   911
                )
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   912
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   913
            # We could have just added the final value in a collection. End
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   914
            # all complete collections at the top of the stack.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   915
            while True:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   916
                # Bail if we're not waiting on a new collection item.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   917
                if self._state not in (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   918
                    self._STATE_WANT_ARRAY_VALUE,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   919
                    self._STATE_WANT_MAP_KEY,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   920
                    self._STATE_WANT_SET_VALUE,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   921
                ):
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   922
                    break
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   923
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   924
                # Or we are expecting more items for this collection.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   925
                lastc = self._collectionstack[-1]
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   926
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   927
                if lastc[b'remaining']:
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   928
                    break
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   929
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   930
                # The collection at the top of the stack is complete.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   931
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   932
                # Discard it, as it isn't needed for future items.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   933
                self._collectionstack.pop()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   934
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   935
                # If this is a nested collection, we don't emit it, since it
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   936
                # will be emitted by its parent collection. But we do need to
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   937
                # update state to reflect what the new top-most collection
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   938
                # on the stack is.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   939
                if self._collectionstack:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   940
                    self._state = {
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   941
                        list: self._STATE_WANT_ARRAY_VALUE,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   942
                        dict: self._STATE_WANT_MAP_KEY,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   943
                        set: self._STATE_WANT_SET_VALUE,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   944
                    }[type(self._collectionstack[-1][b'v'])]
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   945
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   946
                # If this is the root collection, emit it.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   947
                else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   948
                    self._decodedvalues.append(lastc[b'v'])
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   949
                    self._state = self._STATE_NONE
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   950
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   951
        return (
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   952
            bool(self._decodedvalues),
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   953
            offset - initialoffset,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   954
            0,
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   955
        )
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   956
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   957
    def getavailable(self):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   958
        """Returns an iterator over fully decoded values.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   959
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   960
        Once values are retrieved, they won't be available on the next call.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   961
        """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   962
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   963
        l = list(self._decodedvalues)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   964
        self._decodedvalues = []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   965
        return l
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
   966
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   967
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   968
class bufferingdecoder(object):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   969
    """A CBOR decoder that buffers undecoded input.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   970
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   971
    This is a glorified wrapper around ``sansiodecoder`` that adds a buffering
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   972
    layer. All input that isn't consumed by ``sansiodecoder`` will be buffered
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   973
    and concatenated with any new input that arrives later.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   974
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   975
    TODO consider adding limits as to the maximum amount of data that can
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   976
    be buffered.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   977
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
   978
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   979
    def __init__(self):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   980
        self._decoder = sansiodecoder()
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
   981
        self._chunks = []
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
   982
        self._wanted = 0
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   983
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   984
    def decode(self, b):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   985
        """Attempt to decode bytes to CBOR values.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   986
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   987
        Returns a tuple with the following fields:
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   988
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   989
        * Bool indicating whether new values are available for retrieval.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   990
        * Integer number of bytes decoded from the new input.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   991
        * Integer number of bytes wanted to decode the next value.
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
   992
        """
40124
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
   993
        # We /might/ be able to support passing a bytearray all the
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
   994
        # way through. For now, let's cheat.
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
   995
        if isinstance(b, bytearray):
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
   996
            b = bytes(b)
b638219a23c3 cborutil: cast bytearray to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40030
diff changeset
   997
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
   998
        # Our strategy for buffering is to aggregate the incoming chunks in a
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
   999
        # list until we've received enough data to decode the next item.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1000
        # This is slightly more complicated than using an ``io.BytesIO``
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1001
        # or continuously concatenating incoming data. However, because it
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1002
        # isn't constantly reallocating backing memory for a growing buffer,
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1003
        # it prevents excessive memory thrashing and is significantly faster,
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1004
        # especially in cases where the percentage of input chunks that don't
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1005
        # decode into a full item is high.
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1006
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1007
        if self._chunks:
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1008
            # A previous call said we needed N bytes to decode the next item.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1009
            # But this call doesn't provide enough data. We buffer the incoming
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1010
            # chunk without attempting to decode.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1011
            if len(b) < self._wanted:
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1012
                self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1013
                self._wanted -= len(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1014
                return False, 0, self._wanted
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1015
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1016
            # Else we may have enough data to decode the next item. Aggregate
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1017
            # old data with new and reset the buffer.
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1018
            newlen = len(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1019
            self._chunks.append(b)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1020
            b = b''.join(self._chunks)
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1021
            self._chunks = []
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1022
            oldlen = len(b) - newlen
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1023
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1024
        else:
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1025
            oldlen = 0
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1026
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1027
        available, readcount, wanted = self._decoder.decode(b)
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1028
        self._wanted = wanted
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1029
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1030
        if readcount < len(b):
40030
62160d3077cd cborutil: change buffering strategy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39456
diff changeset
  1031
            self._chunks.append(b[readcount:])
39413
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1032
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1033
        return available, readcount - oldlen, wanted
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1034
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1035
    def getavailable(self):
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1036
        return self._decoder.getavailable()
babad5ebaf0a cborutil: add a buffering decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39412
diff changeset
  1037
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42480
diff changeset
  1038
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1039
def decodeall(b):
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1040
    """Decode all CBOR items present in an iterable of bytes.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1041
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1042
    In addition to regular decode errors, raises CBORDecodeError if the
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1043
    entirety of the passed buffer does not fully decode to complete CBOR
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1044
    values. This includes failure to decode any value, incomplete collection
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1045
    types, incomplete indefinite length items, and extra data at the end of
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1046
    the buffer.
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1047
    """
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1048
    if not b:
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1049
        return []
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1050
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1051
    decoder = sansiodecoder()
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1052
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1053
    havevalues, readcount, wantbytes = decoder.decode(b)
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1054
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1055
    if readcount != len(b):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1056
        raise CBORDecodeError(b'input data not fully consumed')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1057
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1058
    if decoder.inprogress:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
  1059
        raise CBORDecodeError(b'input data not complete')
39411
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1060
aeb551a3bb8a cborutil: implement sans I/O decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37898
diff changeset
  1061
    return decoder.getavailable()