sshpeer: initial definition and implementation of new SSH protocol The existing SSH protocol has several design flaws. Future commits will elaborate on these flaws as new features are introduced to combat these flaws. For now, hopefully you can take me for my word that a ground up rewrite of the SSH protocol is needed. This commit lays the foundation for a new SSH protocol by defining a mechanism to upgrade the SSH transport channel away from the default (version 1) protocol to something modern (which we'll call "version 2" for now). This upgrade process is detailed in the internals documentation for the wire protocol. The gist of it is the client sends a request line preceding the "hello" command/line which basically says "I'm requesting an upgrade: here's what I support." If the server recognizes that line, it processes the upgrade request and the transport channel is switched to use the new version of the protocol. If not, it sends an empty response, which is how all Mercurial SSH servers from the beginning of time reacted to unknown commands. The upgrade request is effectively ignored and the client continues to use the existing version of the protocol as if nothing happened. The new version of the SSH protocol is completely identical to version 1 aside from the upgrade dance and the bytes that follow. The immediate bytes that follow the protocol switch are defined to be a length framed "capabilities: " line containing the remote's advertised capabilities. In reality, this looks very similar to what the "hello" response would look like. But it will evolve quickly. The methodology by which the protocol will evolve is important. I'm not going to introduce the new protocol all at once. That would likely lead to endless bike shedding and forward progress would stall. Instead, I intend to tricle out new features and diversions from the existing protocol in small, incremental changes. To support the gradual evolution of the protocol, the on-the-wire advertised protocol name contains an "exp" to denote "experimental" and a 4 digit field to capture the sub-version of the protocol. Whenever we make a BC change to the wire protocol, we can increment this version and lock out all older clients because it will appear as a completely different protocol version. This means we can incur as many breaking changes as we want. We don't have to commit to supporting any one feature or idea for a long period of time. We can even evolve the handshake mechanism, because that is defined as being an implementation detail of the negotiated protocol version! Hopefully this lowers the barrier to accepting changes to the protocol and for experimenting with "radical" ideas during its development. In core, sshpeer received most of the attention. We haven't even implemented the server bits for the new protocol in core yet. Instead, we add very primitive support to our test server, mainly just to exercise the added code paths in sshpeer. Differential Revision: https://phab.mercurial-scm.org/D2061 # no-check-commit because of required foo_bar naming

internals: refactor wire protocol documentation Upcoming work will introduce a new version of the HTTP and SSH transports. The differences will be significant enough to consider them new transports. So, we now attach a version number to each transport. In addition, having the handshake documented after the transport and in a single shared section made it harder to follow the flow of the connection. The handshake documentation is now moved to the protocol section it describes. We now have a generic section about the purpose of the handshake, which was rewritten significantly. Differential Revision: https://phab.mercurial-scm.org/D2060

revlog: rename 'self.checkinlinesize' into '_enforceinlinesize' The name change has two motivations: 1) The function has no external caller, so we move it to protected space. 2) the function does more than checking it also split the data if we have more data than 'inline' supports.

revlog: add a _datareadfp context manager for data access needs The helper handles: 1) is there a file handle already open that we shall just reuse, 2) is the revlog inlined or not. Using a context manager for all read access will help setting up file pointer caching in later changesets.

revlog: use context manager for data file lifetime in checksize This is clearer, safer and more modern.

revlog: use context manager for index file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for data file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for index file life time in __init__ This is clearer, safer and more modern.

revlog: move index file opening in a method Having file operation centralized into a single spot help to factor common logic out (eg: special flag handling according to the mode). It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

revlog: move datafile opening in a method Having file operation centralized into a single spot help to factor common logic out. It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

parseindex: implement context manager method on the wrapper This is needed for incoming cleanups.

parseindex: also forward keyword argument in a debug wrapper Otherwise, it gets in the way of a coming refactoring.

dumprevlog: handle being passed a mode parameter This makes it closer to an actual file opener. Otherwise, it gets in the way of a coming refactoring.

changelog: add the missing 'closed' property on 'appender' object This mimic file object further.

changelog: implement context manager method for 'appender' object This object mimics a file and was missing the context manager API.

diff: improve ui.write performance when not coloring on Windows Differential Revision: https://phab.mercurial-scm.org/D2022

hgsh: enable clang-format Nothing looks awful, so we can just turn it on. Differential Revision: https://phab.mercurial-scm.org/D2059

chg: enable clang-format on all .c and .h files Nothing in here looks awful, so I think we may as well just do it. # skip-blame because it's just reformatting with no functionality change Differential Revision: https://phab.mercurial-scm.org/D2058

python3: whitelist an additional 11 tests I think these are mostly the result of Pulkit's recent work. Thanks! Differential Revision: https://phab.mercurial-scm.org/D2078

ui: add explicit path to write prompt text bypassing buffers The prompt= flag was added at e35d7f131483, when colorui had its own write() function. Since we've merged colorui to ui, we can simply call the unbuffered write() function.

ui: write prompt text in ui.prompt(), not in ui._readline() self.label() is replaced by label= option, which should make it clearer why we can't pass the text to raw_input(prompt).

logcmdutil: mark changesetprinter.showpatch() as private

logcmdutil: drop default arguments from changesetdisplayer/templater() calls

logcmdutil: make default parameters of changesetprinters consistent

mdiff: use str.startswith/endswith() instead of slicing

patch: unify check_binary and binary flags Follows up 079b27b5a869. If opts.text=True, check_binary is ignored, so we can just pass the binary flag to unidiff(). perfunidiff now takes any inputs as text files, which I think is a desired behavior.

fileset: don't abort when running copied() on a revision with a removed file It looks like AND with any status-y fileset would trigger this, as added() and removed() also failed. The 4.5-rc revision is a convenient test case, but the merge isn't necessary.

py3: make sure we open the file in bytes mode # skip-blame because we are just adding b Differential Revision: https://phab.mercurial-scm.org/D2077

py3: use bytes instead of str Differential Revision: https://phab.mercurial-scm.org/D2076

py3: add b'' prefixes to string literals in test files # skip-blame because we are just adding b'' Differential Revision: https://phab.mercurial-scm.org/D2075