test-push-http: do not clear pid file It's okay now, but we'll end up leaking daemon processes if we add some more.

debugcolor: fix crash by empty styles (issue5856)

tests: explicitly define compression engines for tests The zstd compression engine requires C extensions and isn't present in pure Python builds. The compression engine list leaks into the server capabilities string. Unless we're testing functionality specific to a compression format, the set of compression formats supported by a server doesn't matter much. So this commit explicitly defines the server's compression engines for some tests so behavior is consistent between pure and non-pure builds. Differential Revision: https://phab.mercurial-scm.org/D3431

tests: update no-zstd branch of test-treediscovery.t as in 330ada7e8ea5 This side of the test got overlooked. We should probably consider having a way to run some of our tests through a "no-zstd" case just like we run some things through a "no-obsmarkers" case, but that's not an appropriate thing for stable. Differential Revision: https://phab.mercurial-scm.org/D3430

tests: glob away content-length changes relating to missing zstd bindings This doesn't fix everything in these two tests around missing zstd: we still get some changes in the CBOR payload in ways that I think we probably shouldn't bother to glob around. Maybe we should just disable zstd support in some of these lower-level wireproto tests? Differential Revision: https://phab.mercurial-scm.org/D3429

revlog: make pure version of _partialmatch() support 40-byte hex nodeids Without this patch, test-histedit-arguments.t would fail when run with --pure. It turned out to be because the pure version of _partialmatch() does not support full 40-byte hex nodeids. When histedit's instructions include things like "pick tip", it resolves the "tip" revision early to a full nodeid (but plain hex nodeid prefixes are not resolved to full nodeids). Then the nodeid (full or not) is looked up using to a full nodeid later. This step is what fails in pure mode. It has been failing since my c4131138eadb (histedit: look up partial nodeid as partial nodeid, 2018-04-06). I haven't verified, but I suspect histedit instructions like "pick <full hex nodeid>" would have been failing before my commit too, though. The fix is trivial: change a "< 40" to "<= 40". Differential Revision: https://phab.mercurial-scm.org/D3428

hgweb: reuse body file object when hgwebdir calls hgweb (issue5851) An unintended side-effect of f0a851542a05 was that the request body file object (which uses a util.cappedreader) was constructed twice when hgwebdir called into hgweb. Since we attempt to read all remaining data from this file object when Content-Length is defined and since there were two instances of this object and the client supplied no additional data to read, this resulted in deadlock. The fix implemented in this commit is to reuse the request body file object when it is passed from hgwebdir to hgweb. A test demonstrating `hg clone` and `hg push` via hgwebdir has been added. Without this patch, the test hangs when doing `hg clone`. Surprisingly, this must mean that we have effectively no test coverage of the wire protocol when run via hgwebdir. Differential Revision: https://phab.mercurial-scm.org/D3427

remotenames: mark the extension as EXPERIMENTAL I still don't feel confident about locking the behavior of all the things in the remotenames extension. Moreover the extension was introduced in this cycle only. Let's mark this extension EXPERIMENTAL for now so that we can change things especially the storage layer if required in next cycle. I will like to use cbor at storage layer too. Differential Revision: https://phab.mercurial-scm.org/D3426

tests: fix test-check-commit.t when all commits are public I'm 99% sure this is a portable use of /bin/[, and it seems to fix the issue I noticed on the buildbot on my machine.

import: fix crash on --exact check of empty commit (issue5702)

tests: mark test-check-interfaces.py as requiring a repo This was failing our 4.6rc1 build like this: mercurial.error.RepoError: repository /tmp/build-debs.zMTRhC/src-4.6rc1 not found Differential Revision: https://phab.mercurial-scm.org/D3425

sshpeer: reflect actual command activity one handshake The output from devel-peer-request is expected to give data about request and roundtrip done to the server. Changeset a9cffd14aa04 changed some of that by grouping hello and between commands call. However, the old sequence of command was "emulated" in sshpeer. Update the sshpeer to reflect this grouping of commands and update the tests that use it.

tests: drop a useless glob in test-infinite-bundlestore.t With the previous breakage tamed, the lack of test output difference was causing the test runner to report "no result code from test" because of this glob.

infinitepush: ensure fileindex bookmarks use '/' separators (issue5840) After loading up with status messages, I noticed that the subsequent matcher was rejecting 'scratch\mybranch' on Windows. No bookmarks were reported back, and the tests subsequently failed. I did a search for 'match', and nothing else looks like it needs to be fixed up, but someone who understands this code should also take a look. I also tried setting `infinitepush.branchpattern=re:scratch\\.*` in library-infinitepush.sh without this change, but that didn't work. Still, should we ban '\' in these bookmarks to avoid confusion? I thought I saw code that sandwiches a pattern between 're:^' and '.*', so perhaps regex characters will need special care? I also noticed comments in externalbundlestore.{read,write} that it won't work on Windows because of opening an open file. But I don't see a test failure, so this may lack test coverage.

interfaceutil: module to stub out zope.interface The startup time of `hg` increased during the 4.6 development cycle. A cause of that was importing more modules and doing more work at module import time. The import of zope.interface and the declaring of various interfaces is partially responsible for the startup time regression. Our current usage of zope.interface doesn't do much at run time: we are merely declaring interfaces and stating that certain types implement various interfaces. Core Mercurial is not (yet) using of any of zope.interface features that actually require that interface plumbing be defined. The only place we actually need the interface metadata is in test-check-interfaces.py. This commit establishes a new interfaceutil module. It exposes the subset of the zope.interface API that we currently use. By default, the APIs no-op. But if an environment variable is set, we export the real zope.interface APIs. Existing importers of zope.interface have been converted to use the new module. test-check-interfaces.py has been updated to define the environment variable so the real zope.interface is used. The net effect of this change is we stop importing 9 zope.interface.* modules and we no longer perform interface bookkeeping when registering interfaces. On my i7-6700K on Linux, a shell loop that runs `hg log -r .` 300 times on a repo with 1 commit shows a significant CPU time improvement (average of 4 runs): 4.5: 14.814s before: 19.028s after: 16.945s And with `run-tests.py -j10` (single run): 4.5: ~3100s (~51.7m) before: ~4450s (~74.2m) after: ~3980s (~66.3m) So this claws back about half of the regressions in 4.6. Differential Revision: https://phab.mercurial-scm.org/D3419

test-fix: normalize precision of mtime copied by 'cp -p' Appears that MSYS cp only copies mtime in seconds.

merge stable heads

Added signature for changeset 1ec874717d8a

Added tag 4.6rc1 for changeset 1ec874717d8a

internals: correct capitalization of 'compression' stream level parameter

test-check-code: prevent from adding Python modules shadowed by ancient C

diffhelper: rename module to avoid conflicts with ancient C module (issue5846) Historically we had had C extensions in mercurial/, which shadows the pure Python modules of the same name forever unless we do clean build/install. I'm sloppy to think about new name, so just dropped the "s".

diffhelpers: backport 9e40bc4c1bde from C implementation 9e40bc4c1bde just says "harden testhunk." I don't think this would be the case, but it makes some sense to avoid negative index.

version: make parser more robust for rc variants and ill-formed strings

phabricator: specify some metadata compatibly with arc Differential Revision: https://phab.mercurial-scm.org/D3414

diff: restore original color scheme for worddiff I'm not young. "red dim" is nearly invisible, and "red" vs "red dim" is too subtle to see difference. And, "underline" is necessary to highlight whitespace changes.

test-fix: fix use of 'f --newer' to check that foo.whole is not updated Here we want to assert not 'mtime(foo.whole.orig) < mtime(foo.whole)'. The condition has to be inverted since 'f --newer' says "newer" if the mtimes match. Alternatively, we could insert 'sleep 2' before 'cp' to ensure that foo.whole.orig is newer than foo.whole, but a fewer sleeps should be better.

context: translate FilteredIndex/LookupError at repo[changeid] (API) This partially backs out ecd3f6909184. It seems layering violation for repo[changeid] to raise storage-level exceptions transparently. Otherwise, we would have to rewrite callers to catch all of them. try: repo[rev_or_node] except (error.RepoLookupError, error.FilteredIndexError, error.FilteredLookupError): pass This would also fix filectx._changectx(), which catches FilteredRepoLookupError to fall back to the unfiltered path.

inifinitepush: fix filebundlestore to close file

notify: only notify for non-filtered revision This should fix issue5821 for hook targeting individual revision.

notify: access the initial revision on an unfiltered repository (issue5821) This should avoid crash when the first revision pushed end up being hidden.

notify: use changelog API to iterate over revision number (issue5821) This will avoid iterating over filtered revision.

infinitepush: open files in binary mode This fixes the scary looking abort in test-infinitepush-ci.t when unbundling: --- tests/test-infinitepush-ci.t +++ tests/test-infinitepush-ci.t.err @@ -84,15 +84,12 @@ $ hg unbundle .hg/scratchbranches/filebundlestore/a4/c2/a4c202c147a9c4bb91bbadb56321fc5f3950f7f2 adding changesets - adding manifests - adding file changes - added 1 changesets with 1 changes to 1 files - new changesets 6cb0989601f1 - (run 'hg update' to get a working copy) - - $ hg glog - o 1:6cb0989601f1 added a - | public + transaction abort! + rollback completed + abort: stream ended unexpectedly (got 68 bytes, expected 218759168) + [255] + + $ hg glog @ 0:67145f466344 initialcommit public This was found by grepping for '"r', "'r", '"w' and "'w" after manually creating a bundle from the same revision, diffing against the corrupt one, and seeing CRs sprinkled around. Sadly, the missing bookmarks are still a problem in the two remaining test failures.

tests: stabilize test-fix.t for Windows

Added signature for changeset ed5448edcbfa

Added tag 4.6rc0 for changeset ed5448edcbfa

merge with default to begin 4.6 freeze # no-check-commit because of many vendored packages

rebase: introduce support for automatically rebasing orphan changes _destautorebase(SRC) is based on the _destrestack(SRC) revset from fbamend. The supporting _possibledestination function is extracted from evolve, with minor cleanups. We've considered some alternatives here: * This change, but with --auto as the flag name. We're hedging our bets on this a little in this change so that if this ends up being the wrong direction we haven't burned the valauble --auto name on rebase. * --destination auto: I've got reservations about the discoverability of this, and we don't currently have a good story for a revset alias of sorts that changes behavior depending on the context in which it's used. * A "rebase presets" feature, where we could use the currently-an-error positional argument space for the rebase command to define presets, so that users could define a 'linearize' preset that specifies --revision='orphan()-obsolete()' and --dest=_destautoorphanrebase(SRC). Personally, I find the third option somewhat appealing, but am hesitant to "spend" the functionality space of positional arguments to the rebase command. We should revisit the way we expose this functionality sometime in the 4.7 cycle once we've had a chance to vet the implementation of the functionality. Differential Revision: https://phab.mercurial-scm.org/D2668

py3: whitelist six new passing tests Differential Revision: https://phab.mercurial-scm.org/D3406

wireproto: rename wireproto to wireprotov1server (API) We have wireprotov2server, wireprotov1peer, and wireprotov2peer. wireproto only contains server functionality. So it makes sense to rename it to wireprotov1server so the naming aligns with everything else. Differential Revision: https://phab.mercurial-scm.org/D3400

wireproto: move version 2 commands dict to wireprotov2server This was the final piece of version 2 referenced in wireproto. The break between server implementations is now much cleaner. Differential Revision: https://phab.mercurial-scm.org/D3399

wireproto: move supportedcompengines out of wireproto This function is used by both version 1 and version 2. It belongs in a common module. "wireprototypes" may not be the best module name. I may rename it... Differential Revision: https://phab.mercurial-scm.org/D3398

wireproto: reimplement dispatch() for version 2 server The code is minimal. I'm trying to create a cleaner break between version 1 and version 2 server code. Differential Revision: https://phab.mercurial-scm.org/D3397

wireproto: move command registration types to wireprototypes These are shared across wire protocol implementations. wireprototypes is our module for common code. Differential Revision: https://phab.mercurial-scm.org/D3396

wireproto: make version 2 @wireprotocommand an independent function Previously, the code for this decorator was shared between version 1 and version 2 commands. Very few parts of the function were identical. So I don't think sharing is justified. wireprotov2server now has its own @wireprotocommand decorator function. Because the decorator is no longer shared, code for configuring the transport policy has been removed. i.e. commands must have separate implementations for each wire protocol version. Differential Revision: https://phab.mercurial-scm.org/D3395

wireproto: don't pass transportpolicy argument The default is version 1 only. So we don't need to pass this argument when declaring commands. Differential Revision: https://phab.mercurial-scm.org/D3394

forget: rename --confirm to --interactive Differential Revision: https://phab.mercurial-scm.org/D3405

notify: add maxdiffstat option to truncate long file lists Large scale changes like a new GCC version can easily result in 1MB+ emails due to diffstat alone. The new maxdiffstat option truncates the list similar to what maxdiff already provides for the diffs. Differential Revision: https://phab.mercurial-scm.org/D3402

log: fix crash on empty revision with --copies switch If a revset is empty, .max() raises ValueError. I don't see any reason to recompute the revs, so I made it reuse the one returned by logcmdutil.getrevs(). If no revs specified by command line, the endrev will be smartset.spanset(repo) + 1, which is basically the same as len(repo), the default of getrenamedfn(). If --follow specified, revs.max() points to the working parent, which seems more correct.

revsetlang: do not pass in non-bytes to parse() Since parse() isn't a simple function, we shouldn't expect it would raise TypeError or ValueError for invalid inputs. Before, TypeError was raised at 'if pos != len(spec)', which was quite late to report an error. This patch also makes tokenize() detect invalid object before converting it to a py3-safe bytes. Spotted while adding the 'revset(...)' hack to _parsewith().

fix: use templater to substitute values in command string bytes.format() isn't supported on Python 3. Luckily, our template syntax is similar so we can reuse it. We need a hack to disable \-escapes as '\' is a directory separator on Windows.

logcmdutil: pass formatter to jsonchangeset as argument And rename the class. Now we can reuse it for 'log -Tcbor' (and '-Tpickle' if we want.)

logcmdutil: rewrite jsonchangeset printer to be backed by jsonformatter This is a bit slower than the original implementation, but I don't think that would actually matter. It's still faster than full templating. $ hg log -Tjson -r0:5000 --time > /dev/null (orig) time: real 1.550 secs (user 1.500+0.000 sys 0.040+0.000) (new) time: real 1.810 secs (user 1.740+0.000 sys 0.070+0.000) cf. $ hg log -Tdefault -r0:5000 --time > /dev/null time: real 4.980 secs (user 4.850+0.000 sys 0.130+0.000) $ hg log -r0:5000 --time > /dev/null time: real 2.340 secs (user 2.220+0.000 sys 0.100+0.000) $ hg log -r0:5000 -q --time > /dev/null time: real 0.750 secs (user 0.670+0.000 sys 0.070+0.000) The test output changes because keys are sorted alphabetically.

logcmdutil: remove unused attribute 'cache' from jsonchangeset printer Perhaps it's a copy-pasta of changeset_templater at bd15932846a4.

formatter: convert timestamp to int Spotted while porting jsonchangeset printer to formatter. A timestamp may be float type, but its meaningful part is just int.

bundle: introduce per-engine compression level If experimental.bundlecomplevel.$engine is set, prefer it over the generic experimental.bundlecomplevel. Given that compression levels have widely different meanings across engines, this allows much saner configuration. Differential Revision: https://phab.mercurial-scm.org/D3393

bundlespec: drop externalnames flag Always provide the human readable version of compression and version. Add the translated wire format name in the new wirecompression and wireversion fields. Differential Revision: https://phab.mercurial-scm.org/D3392

revlog: make shortest() take a full binary nodeid (API) Follow-up to 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14). Differential Revision: https://phab.mercurial-scm.org/D3403

lfs: add the 'Authorization' property to the Batch API response, if present The client copies all of these properties under 'header' to the HTTP Headers of the subsequent GET or PUT request that it performs. That allows the Basic HTTP authentication used to authorize the Batch API request to also authorize the upload/download action. There's likely further work to do here. There's an 'authenticated' boolean key in the Batch API response that can be set, and there is an 'LFS-Authenticate' header that is used instead of 'WWW-Authenticate'[1]. (We likely need to support both, since some hosting solutions are likely to only respond with the latter.) In any event, this works with SCM Manager, so there is real world benefit. I'm limiting the headers returned to 'Basic', because that's all the lfs spec calls out. In practice, I've seen gitbucket emit custom header content[2]. [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md#response-errors [2] https://github.com/gitbucket/gitbucket/blob/35655f33c7713f08515ed640ece0948acd6d6168/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala#L119

lfs: enable the final download count status message At this point, I think all of the core commands are prefetching, except grep and verify. Verify will need some special handling, in case the revlogs are corrupt. Grep has an issue that still needs to be debugged, but we probably need to give the behavior some thought too- it would be a shame to have to download everything in order to search. I think the benefit of having this info for all commands outweighs extra printing in a command that is arguably not well behaved in this context anyway.