wireprotov2: allow multiple fields to follow revision maps
The *data wire protocol commands emit a series of CBOR values.
Because revision/delta data may be large, their data is emitted
outside the map as a top-level bytestring value.
Before this commit, we'd emit a single optional bytestring
value after the revision descriptor map. This got the job done.
But it was limiting in that we could only send a single field.
And, it required the consumer to know that the presence of a
key in the map implied the existence of a following bytestring
value.
This commit changes the encoding strategy so top-level bytestring
values in the stream are explicitly denoted in a "fieldsfollowing"
key. This key contains an array defining what fields that follow
and the expected size of each field.
By defining things this way, we can easily send N bytestring
values without any ambiguity about their order. In addition,
clients only need to know how to parse ``fieldsfollowing`` to
know if extra values are present.
Because this breaks backwards compatibility, we've bumped the version
number of the wire protocol version 2 API endpoint.
Differential Revision: https://phab.mercurial-scm.org/D4620
$ . $TESTDIR/wireprotohelpers.sh
$ hg init server
$ enablehttpv2 server
$ cd server
$ hg debugdrawdag << EOF
> C D
> |/
> B
> |
> A
> EOF
$ hg phase --public -r C
$ hg book -r C @
$ hg log -T '{rev}:{node} {desc}\n'
3:be0ef73c17ade3fc89dc41701eb9fc3a91b58282 D
2:26805aba1e600a82e93661149f2313866a221a7b C
1:112478962961147124edd43549aedd1a335e44bf B
0:426bada5c67598ca65036d57d9e4b64b0c1ce7a0 A
$ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
$ cat hg.pid > $DAEMON_PIDS
Request for namespaces works
$ sendhttpv2peer << EOF
> command listkeys
> namespace namespaces
> EOF
creating http peer for wire protocol version 2
sending listkeys command
s> POST /api/exp-http-v2-0002/ro/listkeys HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 50\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> *\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceJnamespacesDnameHlistkeys
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 28\r\n
s> \x00\x00\x01\x00\x02\x001
s> \xa3Ibookmarks@Jnamespaces@Fphases@
s> \r\n
received frame(size=32; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: {
b'bookmarks': b'',
b'namespaces': b'',
b'phases': b''
}
Request for phases works
$ sendhttpv2peer << EOF
> command listkeys
> namespace phases
> EOF
creating http peer for wire protocol version 2
sending listkeys command
s> POST /api/exp-http-v2-0002/ro/listkeys HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 46\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> &\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceFphasesDnameHlistkeys
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 45\r\n
s> =\x00\x00\x01\x00\x02\x001
s> \xa2X(be0ef73c17ade3fc89dc41701eb9fc3a91b58282A1JpublishingDTrue
s> \r\n
received frame(size=61; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: {
b'be0ef73c17ade3fc89dc41701eb9fc3a91b58282': b'1',
b'publishing': b'True'
}
Request for bookmarks works
$ sendhttpv2peer << EOF
> command listkeys
> namespace bookmarks
> EOF
creating http peer for wire protocol version 2
sending listkeys command
s> POST /api/exp-http-v2-0002/ro/listkeys HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 49\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> )\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceIbookmarksDnameHlistkeys
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 35\r\n
s> -\x00\x00\x01\x00\x02\x001
s> \xa1A@X(26805aba1e600a82e93661149f2313866a221a7b
s> \r\n
received frame(size=45; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: {
b'@': b'26805aba1e600a82e93661149f2313866a221a7b'
}
$ cat error.log