wireproto: correctly escape batched args and responses (issue4739) This issue appears to be as old as wireproto batching itself: I can reproduce the failure as far back as 08ef6b5f3715 trivially by rebasing the test changes in this patch, which was back in the 1.9 era. I didn't test before that change, because prior to that the testfile has a different name and I'm lazy. Note that the test thought it was checking this case, but it actually wasn't: it put a literal ; in the arg and response for its greet command, but the mangle/unmangle step defined in the test meant that instead of "Fo, =;o" going over the wire, "Gp-!><p" went instead, which doesn't contain any special characters (those being [.=;]) and thus not exercising the escaping. The test has been updated to use pre-unmangled special characters, so the request is now "Fo+<:o", which mangles to "Gp,=;p". I have confirmed that the test fails without the adjustment to the escaping rules in wireproto.py. No existing clients of RPC batching were depending on the old behavior in any way. The only *actual* users of batchable RPCs in core were: 1) largefiles, wherein it batches up many statlfile calls. It sends hexlified hashes over the wire and gets a 0, 1, or 2 back as a response. No risk of special characters. 2) setdiscovery, which was using heads() and known(), both of which communicate via hexlified nodes. Again, no risk of special characters. Since the escaping functionality has been completely broken since it was introduced, we know that it has no users. As such, we can change the escaping mechanism without having to worry about backwards compatibility issues. For the curious, this was detected by chance: it happens that the lz4-compressed text of a test file for remotefilelog compressed to something containing a ;, which then caused the failure when I moved remotefilelog to using batching for file content fetching.

     1

  $ cat <<EOF > merge

     2

  > import sys, os

     3

  > print "merging for", os.path.basename(sys.argv[1])

     4

  > EOF

     5

  $ HGMERGE="python ../merge"; export HGMERGE

     6

     7

  $ hg init A1

     8

  $ cd A1

     9

  $ echo This is file foo1 > foo

    10

  $ echo This is file bar1 > bar

    11

  $ hg add foo bar

    12

  $ hg commit -m "commit text"

    13

    14

  $ cd ..

    15

  $ hg clone A1 B1

    16

  updating to branch default

    17

  2 files updated, 0 files merged, 0 files removed, 0 files unresolved

    18

    19

  $ cd A1

    20

  $ rm bar

    21

  $ hg remove bar

    22

  $ hg commit -m "commit test"

    23

    24

  $ cd ../B1

    25

  $ echo This is file foo22 > foo

    26

  $ hg commit -m "commit test"

    27

    28

  $ cd ..

    29

  $ hg clone A1 A2

    30

  updating to branch default

    31

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

    32

  $ hg clone B1 B2

    33

  updating to branch default

    34

  2 files updated, 0 files merged, 0 files removed, 0 files unresolved

    35

    36

  $ cd A1

    37

  $ hg pull ../B1

    38

  pulling from ../B1

    39

  searching for changes

    40

  adding changesets

    41

  adding manifests

    42

  adding file changes

    43

  added 1 changesets with 1 changes to 1 files (+1 heads)

    44

  (run 'hg heads' to see heads, 'hg merge' to merge)

    45

  $ hg merge

    46

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

    47

  (branch merge, don't forget to commit)

    48

  $ hg commit -m "commit test"

    49

bar should remain deleted.

    50

  $ hg manifest --debug

    51

  f9b0e817f6a48de3564c6b2957687c5e7297c5a0 644   foo

    52

    53

  $ cd ../B2

    54

  $ hg pull ../A2

    55

  pulling from ../A2

    56

  searching for changes

    57

  adding changesets

    58

  adding manifests

    59

  adding file changes

    60

  added 1 changesets with 0 changes to 0 files (+1 heads)

    61

  (run 'hg heads' to see heads, 'hg merge' to merge)

    62

  $ hg merge

    63

  0 files updated, 0 files merged, 1 files removed, 0 files unresolved

    64

  (branch merge, don't forget to commit)

    65

  $ hg commit -m "commit test"

    66

bar should remain deleted.

    67

  $ hg manifest --debug

    68

  f9b0e817f6a48de3564c6b2957687c5e7297c5a0 644   foo

    69

    70

  $ cd ..