wireproto: correctly escape batched args and responses (issue4739) This issue appears to be as old as wireproto batching itself: I can reproduce the failure as far back as 08ef6b5f3715 trivially by rebasing the test changes in this patch, which was back in the 1.9 era. I didn't test before that change, because prior to that the testfile has a different name and I'm lazy. Note that the test thought it was checking this case, but it actually wasn't: it put a literal ; in the arg and response for its greet command, but the mangle/unmangle step defined in the test meant that instead of "Fo, =;o" going over the wire, "Gp-!><p" went instead, which doesn't contain any special characters (those being [.=;]) and thus not exercising the escaping. The test has been updated to use pre-unmangled special characters, so the request is now "Fo+<:o", which mangles to "Gp,=;p". I have confirmed that the test fails without the adjustment to the escaping rules in wireproto.py. No existing clients of RPC batching were depending on the old behavior in any way. The only *actual* users of batchable RPCs in core were: 1) largefiles, wherein it batches up many statlfile calls. It sends hexlified hashes over the wire and gets a 0, 1, or 2 back as a response. No risk of special characters. 2) setdiscovery, which was using heads() and known(), both of which communicate via hexlified nodes. Again, no risk of special characters. Since the escaping functionality has been completely broken since it was introduced, we know that it has no users. As such, we can change the escaping mechanism without having to worry about backwards compatibility issues. For the curious, this was detected by chance: it happens that the lz4-compressed text of a test file for remotefilelog compressed to something containing a ;, which then caused the failure when I moved remotefilelog to using batching for file content fetching.

     1

  $ hg init test

     2

  $ cd test

     3

     4

  $ cat > .hg/hgrc <<EOF

     5

  > [server]

     6

  > validate=1

     7

  > EOF

     8

     9

  $ echo alpha > alpha

    10

  $ echo beta > beta

    11

  $ hg addr

    12

  adding alpha

    13

  adding beta

    14

  $ hg ci -m 1

    15

    16

  $ cd ..

    17

  $ hg clone test test-clone

    18

  updating to branch default

    19

  2 files updated, 0 files merged, 0 files removed, 0 files unresolved

    20

    21

Test spurious filelog entries:

    22

    23

  $ cd test-clone

    24

  $ echo blah >> beta

    25

  $ cp .hg/store/data/beta.i tmp1

    26

  $ hg ci -m 2

    27

  $ cp .hg/store/data/beta.i tmp2

    28

  $ hg -q rollback

    29

  $ mv tmp2 .hg/store/data/beta.i

    30

  $ echo blah >> beta

    31

  $ hg ci -m '2 (corrupt)'

    32

    33

Expected to fail:

    34

    35

  $ hg verify

    36

  checking changesets

    37

  checking manifests

    38

  crosschecking files in changesets and manifests

    39

  checking files

    40

   beta@1: dddc47b3ba30 not in manifests

    41

  2 files, 2 changesets, 4 total revisions

    42

  1 integrity errors encountered!

    43

  (first damaged changeset appears to be 1)

    44

  [1]

    45

    46

  $ hg push

    47

  pushing to $TESTTMP/test (glob)

    48

  searching for changes

    49

  adding changesets

    50

  adding manifests

    51

  adding file changes

    52

  transaction abort!

    53

  rollback completed

    54

  abort: received spurious file revlog entry

    55

  [255]

    56

    57

  $ hg -q rollback

    58

  $ mv tmp1 .hg/store/data/beta.i

    59

  $ echo beta > beta

    60

    61

Test missing filelog entries:

    62

    63

  $ cp .hg/store/data/beta.i tmp

    64

  $ echo blah >> beta

    65

  $ hg ci -m '2 (corrupt)'

    66

  $ mv tmp .hg/store/data/beta.i

    67

    68

Expected to fail:

    69

    70

  $ hg verify

    71

  checking changesets

    72

  checking manifests

    73

  crosschecking files in changesets and manifests

    74

  checking files

    75

   beta@1: dddc47b3ba30 in manifests not found

    76

  2 files, 2 changesets, 2 total revisions

    77

  1 integrity errors encountered!

    78

  (first damaged changeset appears to be 1)

    79

  [1]

    80

    81

  $ hg push

    82

  pushing to $TESTTMP/test (glob)

    83

  searching for changes

    84

  adding changesets

    85

  adding manifests

    86

  adding file changes

    87

  transaction abort!

    88

  rollback completed

    89

  abort: missing file data for beta:dddc47b3ba30e54484720ce0f4f768a0f4b6efb9 - run hg verify

    90

  [255]

    91

    92

  $ cd ..