Wed, 20 May 2015 11:31:38 -0500 sshpeer: run the ssh command unbuffered
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 20 May 2015 11:31:38 -0500] rev 25408
sshpeer: run the ssh command unbuffered This is necessary to use non-blocking IO base on polling. Such polling is needed to restore real time output with ssh peer. Changeset fce065538bcf is talking about 5x regression on Mac OS X when playing with this value. So we introduced our own buffering layer in previous changesets. This seems to keep the regression away (we are even issuing much less read).
Sun, 31 May 2015 00:00:36 -0700 sshpeer: use a 'bufferedinputpipe' for standard output of the ssh process
Pierre-Yves David <pierre-yves.david@fb.com> [Sun, 31 May 2015 00:00:36 -0700] rev 25407
sshpeer: use a 'bufferedinputpipe' for standard output of the ssh process We need this pipe to still be buffered when will switch to unbuffered pipe. (switch motivated by the need of using polling to restore real time output from ssh server). This is the only pipe that needs to be wrapped because this is the one who do extensive usage of 'readline'. The stderr pipe of the process is alway read in non blocking raw chunk, so it won't benefit from the buffering.
Sat, 30 May 2015 23:55:24 -0700 util: introduce a bufferedinputpipe utility
Pierre-Yves David <pierre-yves.david@fb.com> [Sat, 30 May 2015 23:55:24 -0700] rev 25406
util: introduce a bufferedinputpipe utility To restore real time server output through ssh, we need to using polling feature (like select) on the pipes used to communicate with the ssh client. However we cannot use select alongside python level buffering of these pipe (because we need to know if the buffer is non-empty before calling select). However, unbuffered performance are terrible, presumably because the 'readline' call is issuing 'read(1)' call until it find a '\n'. To work around that we introduces our own overlay that do buffering by hand, exposing the state of the buffer to the outside world. The usage of polling IO will be introduced later in the 'sshpeer' module. All its logic will be very specific to the way mercurial communicate over ssh and does not belong to the generic 'util' module.
Wed, 27 May 2015 22:11:37 -0700 test: drop all the forced use of bundle2
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 27 May 2015 22:11:37 -0700] rev 25405
test: drop all the forced use of bundle2 Using bundle2 during exchange is now the default, we do not need all this explicit enabling of bundle2 exchange in test anymore.
Fri, 06 Feb 2015 17:41:24 +0000 bundle2: use bundle2 by default
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 06 Feb 2015 17:41:24 +0000] rev 25404
bundle2: use bundle2 by default All the test change have been isolated and validated. We have free to turn on bundle2 as the default exchange protocol. "To reach a port we must set sail – Sail, not tie at anchor Sail, not drift."
Mon, 01 Jun 2015 10:28:40 -0700 wireprotocol: distinguish list and set in getbundle argument
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 01 Jun 2015 10:28:40 -0700] rev 25403
wireprotocol: distinguish list and set in getbundle argument The 'bundlecaps' argument is expected to be a set, but 'listkeys' is expected to be a list where ordering matters. We introduce a new 'scsv' argument type for the 'set' version and move 'csv' to the 'list' version. 'test-ssh.t' is changed because this introduced an instability in the order we were producing listkeys parts.
Mon, 25 May 2015 17:14:11 -0700 exchange: support transferring .hgtags fnodes mapping
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 25 May 2015 17:14:11 -0700] rev 25402
exchange: support transferring .hgtags fnodes mapping On Mozilla's mozilla-beta repository .hgtags fnodes resolution takes ~18s from a clean cache on my machine. This means that the first time a user runs `hg tags`, `hg log`, or any other command that displays or accesses tags data, a ~18s pause will occur. There is no output during this pause. This results in a poor user experience and perception that Mercurial is slow. The .hgtags changeset to filenode mapping is deterministic. This patch takes advantage of that property by implementing support for transferring .hgtags filenodes mappings in a dedicated bundle2 part. When a client advertising support for the "hgtagsfnodes" capability requests a bundle, a mapping of changesets to .hgtags filenodes will be sent to the client. Only mappings of head changesets included in the bundle will be sent. The transfer of this mapping effectively eliminates one time tags cache related pauses after initial clone. The mappings are sent as binary data. So, 40 bytes per pair of SHA-1s. On the aforementioned mozilla-beta repository, 659 * 40 = 26,360 raw bytes of mappings are sent over the wire (in addition to the bundle part headers). Assuming 18s to populate the cache, we only need to transfer this extra data faster than 1.5 KB/s for overall clone + tags cache population time to be shorter. Put into perspective, the mozilla-beta repository is ~1 GB in size. So, this additional data constitutes <0.01% of the cloned data. The marginal overhead for a multi-second performance win on clones in my opinion justifies an on-by-default behavior.
Mon, 01 Jun 2015 20:23:22 -0700 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 01 Jun 2015 20:23:22 -0700] rev 25401
bundle2: part handler for processing .hgtags fnodes mappings .hgtags fnodes cache entries can be expensive to compute, especially if there are hundreds of even thousands of them. This patch implements support for receiving a bundle2 part that contains a mapping of changeset to .hgtags fnodes. An upcoming patch will teach the server to send this part, allowing clients to bypass having to redundantly compute these values. A number of tests changed due to the client advertising the "hgtagsfnodes" capability.
Tue, 02 Jun 2015 19:58:06 -0700 changegroup: rename _computeoutgoing to computeoutgoing
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 02 Jun 2015 19:58:06 -0700] rev 25400
changegroup: rename _computeoutgoing to computeoutgoing We're going to use this function from another module in an upcoming patch. Drop the _ prefix to mark it as non-private.
Tue, 02 Jun 2015 13:24:39 -0500 merge with stable
Matt Mackall <mpm@selenic.com> [Tue, 02 Jun 2015 13:24:39 -0500] rev 25399
merge with stable
(0) -10000 -3000 -1000 -300 -100 -10 +10 +100 +300 +1000 +3000 +10000 tip