context: add convenience method for returning a memfilectx from a patch This is mostly a copy of what makememctx does but refactored to make it behave more like our other convenience methods.

memctx: refactor inline getfilectx into convenience method No actual logic is changing, just moving code so __init__ is easier to read.

hgweb: refresh styling of gitweb's search form gitweb was missing the hint hover box. So that was added. Also, the positioning of the form was absolute and it didn't vertically align on all pages. The element has been moved inline with the navigation links (which now are contained in a div) and flexbox is used to obtain sane alignment of the navigation links and search form. For those new to flexbox, "justify-content: space-between" basically says to maximize space elements. You can use it to easily get left and right justified containers without having to worry about width, floating, etc. "align-items: center" centers all items in a cross-axis. I've literally wasted hours trying to figure out both these problems before flexbox. Flexbox is amazing. Flexbox has been supported by Chrome and Firefox for a few years. But it is only supported by IE 11. I'm willing to wager that people using this either won't be using IE or will be using IE 11. So I'm willing to be a bit aggressive in adopting flexbox because it makes CSS alignment so much easier.

hgweb: consistently add search form to all gitweb pages Paper has it on all pages. Not sure why gitweb doesn't. I think it should be everywhere because it is a useful feature. Also, we weren't consistently adding the HTML in the same place. This was OK since the element is absolutely positioned. But this bothered me a bit, so I went ahead and fixed it.

hgweb: consolidate search form for gitweb

hgweb: consolidate search form for monoblue Same deal as for paper.

hgweb: consolidate search form for paper AFAICT this was mostly a bunch of copy pasta. The only variation is some pages defined a "value" attribute. The "query" variable will just be empty on pages that don't accept it. So let's consolidate the template and remove the redundancy.

test-extension: fix load path for Windows The previous code was trying to load the extension from /tmp/hgtests.xxx/..., but the actual path for tests is C:\Users\...\Temp\hgtests.xxx\... I assume that the former is an MSYS path that maps somewhere under C:\MinGW.

test-setdiscovery: stabilize for Windows Windows wants double quotes here.

convert: correct the documentation about whitespace in branchmap branches Might as well let the users know they can get rid of branch names with spaces.

py3: use pycompat.bytestr() instead of str()

py3: convert bool variables to bytes

context: avoid writing outdated dirstate out (issue5584) Before this patch, workingctx.status() may cause writing outdated dirstate out, if: - .hg/dirstate is changed simultaneously after last loading it, - there is any file, which should be dirstate.normal()-ed Typical issue case is: - the working directory is updated by "hg update" - .hg/dirstate is updated in background (e.g. fsmonitor) This patch compares identities of dirstate before and after acquisition of wlock, and avoids writing outdated dirstate out, if change of .hg/dirstate is detected.

tests: factor external procedures out for portability Fortunately, "&&" is treated as "execute next, if previous doesn't fail" both on POSIX and Windows. But keeping portability of "dirstaterace.command" manually is troublesome. This patch factors external procedures out as a shell script for portability. "sh SCRIPT" always allows scripting in POSIX style. This change is also for convenience. Fixed script name can reduce command line arguments. "r" prefix is needed for "sh '$TESTTMP/dirstaterace.sh'", because $TESTTMP contains backslash on Windows.

dirstate: add identity information to detect simultaneous changing in storage This identity is used to examine whether dirstate is simultaneously changed in storage after previous caching (see issue5584 for detail). util.cachestat can't be used for this purpose, because it has no valuable information on Windows. On the other hand, util.filestat can detect changing dirstate in storage certainly, regardless of platforms. https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan Strictly speaking, if underlying filesystem doesn't support ctime/mtime, util.filestat can't detect simultaneous changing in storage as expected. But simultaneous changing on such (very rare) platform can't be detected regardless of this patch series. Therefore, util.filestat should be reasonable identity for almost all usecases.

util: make filestat.__eq__ return True if both of self and old have None stat For convenience to compare two filestat objects regardless of None-ness of stat field.

vfs: create copy at renaming to avoid file stat ambiguity if needed In order to fix issue5418, bff5ccbe5ead made vfs.rename(checkambig=True) omit advancing mtime of renamed file, if renamed file is owned by another (EPERM is raised in this case). But this omission causes rewinding mtime at restoration in such situation, and makes avoiding file stat ambiguity difficult, because ExactCacheValidationPlan assumes that mtime should be advanced, if a file is changed in same ctime. https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan Ambiguity of file stat also requires issue5584 to be fixed with other than file stat, but "hash of file", "generation ID" and so on were already rejected ideas (please see original RFC linked from "Outline of issue" in ExactCacheValidationPlan page). This omission occurs: - only for non append-only files (dirstate, bookmarks, and phaseroots), and - only if previous transaction is rollbacked by another user The latter means "sharing a repository clone via group permission". This is reasonable usecase, but not ordinary for many users, IMHO. "hg rollback" itself has been deprecated since Mercurial 2.7, too. Therefore, increasing the cost at rollbacking previous transaction executed by another a little seems reasonable, for avoidance of file stat ambiguity. This patch does: - create copy of (already renamed) source file, if advancing mtime fails for EPERM - rename from copied file to destination file, and - advance mtime of renamed file, which is now owned by current user This patch also factors "self.join(src)" out to reduce redundancy.

vfs: factor out "rename and avoid ambiguity" to reuse This makes subsequent patch simple.

util: make filestat.avoidambig() return whether ambiguity is avoided or not

debugcommands: issue warning when repo has secret changesets (issue5589) This seems like a prudent thing to do. As the inline comment says, we may want to make this abort once the functionality is stabilized as part of `hg bundle`. Let's save that debate for another day.

streamclone: consider secret changesets (BC) (issue5589) Previously, a repo containing secret changesets would be served via stream clone, transferring those secret changesets. While secret changesets aren't meant to imply strong security (if you really want to keep them secret, others shouldn't have read access to the repo), we should at least make an effort to protect secret changesets when possible. After this commit, we no longer serve stream clones for repos containing secret changesets by default. This is backwards incompatible behavior. In case anyone is relying on the behavior, we provide a config option to opt into the old behavior. Note that this defense is only beneficial for remote repos accessed via the wire protocol: if a client has access to the files backing a repo, they can get to the raw data and see secret revisions.

json: pass formatting options recursively This bug was introduced in 654e9a1c8a6c. It's okay to escape <>, but is unnecessary for command output.

json: avoid extra string manipulation of dict keys A key must be string per JSON spec, and that's also true for template dicts.

test-obsolete: include <> in user field to check JSON escapes I found json() filter doesn't pass formatting options recursively. That's why <> are escaped.

templatefilers: correct filename in header comment

repoview: remove special casing of "requirements" At the time this code was introduced (3a6ddacb7198), the inline comment was true. This changed in e3a928bd1cd4. The proxy is no longer needed.

bookmarks: move variable initialization earlier Since we no longer set '_clean = False' during the initialization loop, we can move the attribute assignment earlier in the function for clarity. (no speed improvement expected or measured ;-) )

bookmarks: directly use base dict 'setitem' The bmstore '__setitem__' method is setting an extra flag that is not needed during initialization. Skipping the method will allow further cleanup and yield some speedup as a side effect. Before: ! wall 0.009120 comb 0.010000 user 0.010000 sys 0.000000 (best of 312) After: ! wall 0.007874 comb 0.010000 user 0.010000 sys 0.000000 (best of 360)

bookmarks: rely on exception for malformed lines Since we already have an exception context open, for other thing, we can simplify the code a bit and rely on exception handling for invalid lines. Speed is not the main motivation for this changes. However as I'm in the middle of benchmarking things we can see a small positive impact. Before: ! wall 0.009358 comb 0.000000 user 0.000000 sys 0.000000 (best of 303) After: ! wall 0.009173 comb 0.010000 user 0.010000 sys 0.000000 (best of 310)

bookmarks: explicitly convert to 'node' during initialization We know the content of the file is supposed to be full hex. So we can do the translation ourselves and directly check if the node is known. As nice side effect we now have proper error handling for invalid node value. Before: ! wall 0.021580 comb 0.020000 user 0.020000 sys 0.000000 (best of 134) After: ! wall 0.009342 comb 0.010000 user 0.010000 sys 0.000000 (best of 302)

bookmarks: prefetch 'lookup' outside of the loop Skipping the attribute lookup up raise a significant speedup. Example on a repository with about 4000 bookmarks. Before: ! wall 0.026027 comb 0.020000 user 0.020000 sys 0.000000 (best of 112) After: ! wall 0.021580 comb 0.020000 user 0.020000 sys 0.000000 (best of 134) (This is also in its own changeset to clarify the perf win from another coming changesets)

perf: add a perfbookmarks command A new command dedicated to benchmark of bookmark initialization.

perfphases: add a flag to also include file access time The flag purges all phases data so we'll have to read the file from disk again.

perf: have a generic "clearstorecache" function There are multiple places where we will want to purge some store cache. So we promote the existing _clearobsstore function to a something reusable.

localrepo: move filtername to __init__ This is obviously an instance attribute, not a type attribute. The modern Python style is to use __init__ for defining these. This exposes statichttprepo as inheriting from localrepository without calling its __init__. As a result, its __init__ defines a lot of variables that methods on localrepository's methods need. But factoring the common bits into a separate class is for another day.

obsolete: move obsstore creation logic from localrepo This code has more to do with obsolete.py than localrepo.py. Let's move it there.

match: allow pats to be None match.match already interprets "!bool(patterns)" as matching everything (but includes and excludes still apply). We might as well allow None, which lets us simplify some callers a bit. I originally wrote this patch while trying to change match.match(patterns=[]) to mean to match no patterns. This patch is one step towards that goal. I'm not sure it'll be worth the effort to go all the way there, but I think this patch still makes sense on its own.

tests: add a test for installing hg with pip in a virtualenv Since we're doing so much clever junk in our setup.py, let's have a test that exercises it. Thanks to Matt Harbison for testing this on Windows and verifying that installenv/*/hg would work as a way to work around bin being called Scripts on Windows.

hghave: add check for virtualenv

setup: introduce dummy copies of setuptools flags Since we're filtering out some egg gunk, we need to emulate these flags which disable eggs so that pip still works.

extensions: catch uisetup and extsetup failures and don't let them break hg Otherwise users of the patience diff extension will be unable to run anything at all in hg 4.3 until they figure out what's broken.

tests: add test demonstrating how broken third-party extensions can get I intend to fix this, but will do the fix as a separate change to make the behavior change obvious. This was inspired by some users having the patience diff extension, which broke when we moved bdiff.so so thoroughly the users can't even run 'hg debuginstall'.

extensions: move wrapfilecache function from fsmonitor It makes more sense to put this in core, so other extensions can trivially get access to it without having to rely on importing fsmonitor.

chmod: create a new file when flags are set on a hardlinked file For performance reasons we have several repositories where the files in the working directory of 1 repo are hardlinks to the files of the other repo When an update in one repo results in a chmod of a such a file, the hardlink has to be deleted and replaced by a regular file to make sure that the change does not happen in the other repo

run-tests: make --restart work with output dir

run-tests: output coverage to output dir There do not appear to be any tests for this, and I've never used either of these options before, but this works.

run-tests: write JSON reports to output dir

run-tests: write test times to output dir

run-tests: allow specifying an output dir to write .errs to I'm trying to use run-tests.py on a read-only file system. This series allows that to happen.

run-tests: add information about skipped tests to XUnit output The XUnit spec supports skipped tests.

run-tests: wrap failures in an XUnit 'failure' element This is closer to what most XUnit consumers can understand.

discovery: log discovery result in non-trivial cases We log the discovery summary, the number of roundtrips and the elapsed time. This is useful to understand where slow push might come from when lloking at the blackbox.

discovery: include timing in the debug output Having such date easily available is useful. It also prepare the inclusion of some discovery related data in blackbox.

tests: remove sys.executable from "required tools" In practice this doesn't appear to have been true for some time - we reference Python using the $PYTHON variable in all the tests now (which we have to for PyPy and Python 3), and I've been using ~/.../python.exe to test with tip of the cpython 3.6 release branch while working on manifest tests in Python 3 and everything seems to be just fine. The only real observable difference from this change is that I stop getting a warning about python.exe not being a thing on $PATH, which seems like an improvement.

perfbranchmap: add an option to purge the revbranch cache The perf extension needs to be able to reflect this reality too. (eg: 4s vs 60s on a Million-ish revisions repository).

push: add a way to allow concurrent pushes on unrelated heads Client has a mechanism for the server to check that nothing changed server side since the client prepared a push. That check is wide and any head changed on the server will lead to an aborted push. We introduce a way for the client to send a less strict checking. That logic will check that no heads impacted by the push have been affected. If other unrelated heads (including named branches heads) have been affected, the push will proceed. This is very helpful for repositories with high developers traffic on different heads, a common setup. That behavior is currently controlled by an experimental option. The config should live in the "server" section but bike-shedding of the name will happen in the next changesets. Servers advertise this capability through a new bundle2 capability 'checkeads', using the value 'related'. The 'test-push-race.t' is updated to check that new capabilities on the documented cases.

headsummary: expose the 'discardedheads' set in the headssummary That information will be useful to detect push race on related part of the history. See next changeset for details.

checkheads: perform obsolescence post processing directly in _headssummary The goal is to have the function directly return something meaningful and useful for the whole pull. Note: we skip adding post-processing in '_oldheadssummary' because if a client is too old for branchmap it will be too old for obsolescence too.

headssummary: directly feed the function with the 'pushop' object Our goal is to be able to perform the post processing directly into the '_headssummary' function. However before this patch the '_headsummary' function only had access to repo, remote, outgoing while the '_postprocessobsolete' function takes a 'pushop' object. Experience shows that having the 'pushop' object helps extensions so we update '_headssummary' to take a pushop object as argument.

checkheads: gather the postprocessing with other obsolescence specific code We extract this function from the loop and gather it with the rest of the obsolescence specific code. That will help to clarify the move of the whole logic inside the "heads summary" computation.