keepalive: be more careful about self._rbuf when calling super impls In Python 3, HTTPResponse implements read() in terms of readinto(), which was calling back into our readinto(), which duplicates self._rbuf if it's not empty. Before calling into super's read(), ensure self._rbuf is empty. Inheritance is bad, and undocumented self-use of your public API is one of many reasons. Differential Revision: https://phab.mercurial-scm.org/D4728

wireprotov2: teach changesetdata to fetch ancestors until depth For shallow clone, it is useful to specify a starting node and tell the server to send up to N ancestors from that starting point. This enables the server to perform the DAG walk without the client having to discover the base/stop node(s) first. This commit implements support for said queries on the changesetdata command. Differential Revision: https://phab.mercurial-scm.org/D4621

wireprotov2: allow multiple fields to follow revision maps The *data wire protocol commands emit a series of CBOR values. Because revision/delta data may be large, their data is emitted outside the map as a top-level bytestring value. Before this commit, we'd emit a single optional bytestring value after the revision descriptor map. This got the job done. But it was limiting in that we could only send a single field. And, it required the consumer to know that the presence of a key in the map implied the existence of a following bytestring value. This commit changes the encoding strategy so top-level bytestring values in the stream are explicitly denoted in a "fieldsfollowing" key. This key contains an array defining what fields that follow and the expected size of each field. By defining things this way, we can easily send N bytestring values without any ambiguity about their order. In addition, clients only need to know how to parse ``fieldsfollowing`` to know if extra values are present. Because this breaks backwards compatibility, we've bumped the version number of the wire protocol version 2 API endpoint. Differential Revision: https://phab.mercurial-scm.org/D4620

wireprotov2: advertise set of valid values for requestable fields changesetdata, manifestdata, and filedata all allow the caller to specify what data fields to request. Data fields are extensible and may evolve over time. In order to prevent clients from making requests for fields that are not available, the client needs to know what fields are available. This commit teaches the server to declare a set of "valid values" for wire protocol command arguments. That set of values is exposed in the command's capabilities descriptor. The changesetdata, manifestdata, and filedata commands all declare their set of available "fields." Differential Revision: https://phab.mercurial-scm.org/D4619

wireprotov2: expose rich arguments metadata Now that we internally store rich metadata about arguments, it makes sense to make that metadata available to the client. This will allow clients to validate outgoing command requests before they are sent over the wire. Strictly speaking, we should bump the wire protocol version for this change since it is backwards incompatible. But no client-side code touches the arguments map and I don't want to incur the work. Differential Revision: https://phab.mercurial-scm.org/D4618

wireprotov2: advertise recognized path filter prefixes While the wire protocol doesn't yet support it, we'll eventually have commands that accept narrow patterns to specify the set of files relevant to a command. For security and performance reasons, only specific filter types are allowed. This commit teaches the server to advertise the set of allowed filter types. By doing so, clients can e.g. validate user-specified patterns against the server's abilities without having to send a command to retrieve data. Having the data in the capabilities data structure will also serve as a check against unwanted BC. Differential Revision: https://phab.mercurial-scm.org/D4616

wireprotov2: declare command arguments richly Previously, we declared command arguments with an example of their value. After this commit, we declare command arguments as a dict of metadata. This allows us to define the value type, whether the argument is required, and provide default values. This in turn allows us to have nice things, such as less boilerplate code in individual commands for validating input and assigning default values. It should also make command behavior more consistent as a result. Test output changed slightly because I realized that the "fields" argument wasn't being consistently defined as a set. Oops! Other test output changed because of slight differences in code performing type validation. Differential Revision: https://phab.mercurial-scm.org/D4615

chgserver: do not send system() back to client if stdio redirected (issue5992) As the chg client doesn't know server-side stdio redirection, the server shouldn't upcall on "runsystem" request if the stdio streams are redirected. This patch teaches ui to remember the redirection flag, which is updated by the caller right now. Future patches (for default) will add ui methods to manage this flag internally.

chgserver: update comment describing when to fall back to core _runsystem() The condition "output needs to be captured" was moved to the core ui, and removed at fbce78c58f1e "chg: refactor ui.system() to be partly overridden." The next patch will add one more "if" to address the issue 5992.

procutil: compare fd number to see if stdio protection is needed (issue5992) When I wrote this function for commandserver at 69f86b937035, testing object identity was suffice, and I was sloppy enough not to compare fileno() values. However, it doesn't work in chg session because chgserver reopens stdio to apply new buffering mode. This patch partially fixes the issue 5992. Still we have another problem in chgui._runsystem().