filectx: extract helper method to obtain filectx pointing to its introrev

dagop: copy basefilectx.ancestors() to free function The primary goal of this series is to make follow() support multiple start revisions. dagop.filectxancestors() will be extended to take multiple filectxs. basefilectx.ancestors() is not forwarded to this function because doing that would resurrect the performance issue fixed by 24b57c3899f8.

test-log: test that fctx.ancestors() can't index parents only by linkrev This covers a possible bug that could be caused by the following change: --- a/mercurial/context.py +++ b/mercurial/context.py @@ -1047,7 +1047,7 @@ class basefilectx(object): while True: for parent in c.parents()[:cut]: - visit[(parent.linkrev(), parent.filenode())] = parent + visit[parent.linkrev()] = parent if not visit: break c = visit.pop(max(visit))

pull: retrieve bookmarks through the binary part when possible This makes pull consistent with the part used by push and provide us with a more compact representation of bookmarks. In addition, this opens the way for smarter bookmark exchanges (e.g. filtering by names or only sending the bookmark relevant to the pulled set, etc).

getbundle: add support for 'bookmarks' boolean argument This new argument requests a 'bookmarks' part from the server. It is meant to be used instead of the "listkeys" request.

bundle2: support a 'records' mode for the 'bookmarks' part In this mode, the bookmarks changes are record in the 'bundleoperation' records instead of inflicted to the repository. This is necessary to use the part when pulling.

bundle2: add a 'modes' dictionary to the bundle operation This new attribute allows the codes requesting an unbundling to pass important information to individual part handlers. The current target use case is to allow for receiving 'bookmarks' part without directly updating local repository, but just recording the received data instead. This is necessary for pull where the remote bookmarks are processed locally. I expect the concept to be beneficial to other parts in the future. To clarify the bookmark behavior on pull, the remote bookmark value are not just taken -as-is- into the local repository. There is an extra step to detect bookmark divergence. The remote bookmarks data are stored until this processing happens.

bookmark: use the 'bookmarks' bundle2 part to push bookmark update (issue5165) We use the new binary parts we introduced earlier to exchange bookmark. The payload is a bit more compact since we use binary and the length of bookmarks is no longer constrained to 255. .. fix:: Issue 5165 Bookmark, whose name is longer than 255, can again be exchanged again between 4.4+ client and servers.

bookmark: introduce in advance a variant of the exchange test We are about to introduce a new way to push bookmark to server. We introduce the test variant before actually updating the exchange to help the output changes to stand out when it happens.

push: move bundle2-pushkey based bookmarks exchange in its own function We are about to introduce an alternative way to push bookmark over bundle2.

bookmark: add pushkey hook compatiblity to the bundle2 part Currently, pushing a bookmark update triggers a pushkey hooks. It is likely that users in the wild use such hooks to control bookmark movement. Using a non push-key mechanism to exchange bookmark means these hooks are no longer called, possibly breaking existing users setup. So we add explicit call to the pushkey hooks in the handling of the bundle2 part. This behavior can be disabled with a new config knob: 'server.bookmarks-pushkey-compat'.

bookmark: introduce a 'bookmarks' part This part can carry and apply bookmarks information. We start with adding the core behavior of the part. In its current form, the part is only suitable for push since it plain update the bookmark without consideration for the local state. Support of the behavior needed for pulling will be added in later changesets.

push: include a 'check:bookmarks' part when possible Before updating the actual bookmark update, we can start with updating the way we check for push race. Checking bookmarks state earlier is useful even if we still use pushkey. Aborting before the changegroup is added can save a lot of time.

bookmark: add a 'check:bookmarks' bundle2 part This part checks that bookmarks are still at the node they are expected to be. This allows a pushing client to detect push race where the repository was updated between the time it discovered the server state and the time it managed to finish its push. Such checking already exists when pushing bookmark through pushkey. This new part can be inserted at the beginning of the bundle, triggering abort earlier. In addition, we would like to move away from pushey to push bookmark. A step useful to solve issue5165.

bookmark: add methods to binary encode and decode bookmark values Coming new bundle2 parts related to bookmark will use a binary encoding. It encodes a series of '(bookmark, node)' pairs. Bookmark name has a high enough size limit to not be affected by issue5165. (64K length, we are well covered)

tests: remove {a..h} bashism from remotenames I'm not bothering with a check-code test because this is a weird construct that I didn't even know existed before it was breaking the BSD build, and it also appears to fail if /bin/sh is dash like it is on our Linux builder. Differential Revision: https://phab.mercurial-scm.org/D1605

hgweb: move common vertex code to Graph.prototype Just to give some context to the return values: vertex() needs to return two HTML elements as strings, <li> to be used as a background and a <li> to be shown in foreground. The latter was made obsolete recently when changesets started to be rendered server-side, but background elements are still useful for now.

hgweb: create Graph methods using a prototype This way it's possible to call the original methods even if they were overridden.

hgweb: remove unused Graph.cur property It was introduced in 0dba955c2636, but was already unused. I missed it in e46f0b653002.

tests: remove shell function helper from test-largefiles-misc Now that all the complexity is in a Python script, we can just directly invoke the tool. Differential Revision: https://phab.mercurial-scm.org/D1599

contrib: ban find(1)'s -printf operator, as it is a GNU-ism Differential Revision: https://phab.mercurial-scm.org/D1598

merge with stable

repoview: include filter name in repr for debugging

repoview: extract a factory function of proxy class This makes sure that dynamically-created class objects are isolated from local binding of repo instances. The type cache is moved to module level as it isn't tied to each instance.

repoview: do not include filter name in name of proxy class The type object is shared across all filters. I'll add __repr__() instead.

setup: convert version strings to unicode on Python 3 Fixes the following error: stderr from 'hg log -T x -r only(.,'b'4.4.2'')': b' hg: parse error at 10: unexpected token: symbol'

thirdparty: move selectors2 module to where it should be

rewriteutil: use precheck() in uncommit and amend commands Differential Revision: https://phab.mercurial-scm.org/D1526

rewriteutil: add a precheck function to check if revs can be rewritten The precheck function is intended to be used before we start rewritting changesets. Differential Revision: https://phab.mercurial-scm.org/D1503

rewriteutil: add utility function to check if we can create new unstable cset This patch adds a new file which will contain utility functions related to rewritting changesets. It also adds a utility function to check if the rewritting operation creates new unstable changesets and are we allowed to create them. This rewriteutil.py introduced in this patch and the utility functions added in the upcoming patches exists in the evolve extension are being ported from there. Differential Revision: https://phab.mercurial-scm.org/D1502

test-run-tests: do not rebuild hg in the test d600bda4 and fc0f3ed0 added code to call `$PYTHON run-tests.py ...`. That will rebuild hg, is slow and could have other crashes from setup.py, like: Unable to find a working hg binary to extract the version from the repository tags Therefore use `run-tests.py -l` instead. Differential Revision: https://phab.mercurial-scm.org/D1595

remotenames: consider existing data while storing newer data Previously reviewed as D1357. Differential Revision: https://phab.mercurial-scm.org/D1551

remotenames: add functions to read remotenames data from .hg/remotenames/ This patch functions which can be used to read remotenames data from .hg/remotenames/. The logic for the function which reads the remotenames file is taken from the remotenames extension. Previously reviewed as D940. Differential Revision: https://phab.mercurial-scm.org/D1550

remotenames: add test showing overwriting on remotenames data The current storage logic every time overwrites the existing data with the new data. This patch adds test to demonstrate that. To fix this, we need to add logic to read existing remotenames data and merge with existing data which will be added in upcoming changesets. Previously reviewed as D1356. Differential Revision: https://phab.mercurial-scm.org/D1549

remotenames: add functionality to store remotenames under .hg/hgremotenames/ This patch moves the functionality from remotenames extension to store remotenames to core. Storage format used by remotenames extension: A single file `.hg/remotenames` with an entry in each line where each line is of format: `node nametype remotepath/name` where nametype is either 'bookmarks' or 'branches'. This was not the best way to store data, so while moving to core the storage format was changed but yet not the final format. The storage format used by core after this patch will be: * A file for each type of name i.e. bookmarks and branches in .hg/remotenames/ directory * A version number on the top of the file. The version for current format is 0. * An entry in each line where each line is of the format `node\0remotepath\0name` The logic to sync with existing remotenames file and saving journals and other related things will be moved to core in next patches incrementally. Thanks to Ryan, Augie and Durham for suggestions on storage format. Previously reviewed as D939. Differential Revision: https://phab.mercurial-scm.org/D1548

remotenames: move function to pull remotenames from the remoterepo to core This patch is the first patch of the series moving functionality from hgremotenames extension to core. There are lot of functionality in the extension which in the end enables us to store branch heads and bookmarks location on a server from which we are pulling or cloning from. This will help us in creating a better bookmark workflow where we can show user that a certain server has this bookmarks at this node. It will also introduce namespaces related to remote bookmarks and remote branches. This patch moves the functionality to pull branches and bookmarks from a server from which we are pulling to core behind config option `experimental.remotenames`. This patch adds a test which helps us to analyse whether things are working or not. We are currently writing things to ui, we will write information to files in upcoming patches. Previously reviewed as D937. Differential Revision: https://phab.mercurial-scm.org/D1547

test: fix bad replace for fixing pure-only build When we replaced the patterns, glob was removed on the fixed line, it was a mistake and caused the pure-only build to fails. Differential Revision: https://phab.mercurial-scm.org/D1592

test: fix common-pattern for pure variant The $USUAL_COMPRESSIONS$ value that was taken was not compatible with the pure variant systems as zlib seems to not be available in these case. Differential Revision: https://phab.mercurial-scm.org/D1562

tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages I suspect some more of these are globbed out, so this is a bit of future proofing.

tests: add a substitution for ECONNRESET/WSAECONNRESET messages

tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages

tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages Automatic replacement seems better than trying to figure out a check-code rule. I didn't bother looking to see why the error message and file name is reversed in the annotate and histedit tests, based on Windows or not. I originally had this as a list of tuples, conditional on the platform. But there are a couple of 'No such file or directory' messages emitted by Mercurial itself, so unconditional is required for stability. There are also several variants of what I assume is 'connection refused' and 'unknown host' in test-clone.t and test-clonebundles.t for Docker, FreeBSD jails, etc. Yes, these are handled by (re) tags, but maybe it would be better to capture those strings in order to avoid whack-a-mole in future tests. All of this points to using a dictionary containing one or more strings-to-be-replaced values.

setup: only write some autogenerated files if they change Without this change, setup.py always writes some files on every invocation. This prevents some builds from being a no-op when they should. And, since times can sneak into generated .pyc files, this prevents file content from being deterministic between builds. As part of the refactor, we treat file content as bytes. The only potential regression from this would be if some tool is looking at mtimes of the changed files to determine if further action should be taken. But I don't think anything critically important is keyed off the mtimes of these specific files. Differential Revision: https://phab.mercurial-scm.org/D1580

python3: whitelist many more passing tests Differential Revision: https://phab.mercurial-scm.org/D1584

fancyopts: fix handling of "--" value in earlygetopt()

fancyopts: use getopt.gnu_getopt() The issue described in the docstring has been fixed since Python 20ab2260dc93, which is in 2.7. https://hg.python.org/cpython/rev/20ab2260dc93 https://bugs.python.org/issue4458 This fixes the handling of '--' value.

dispatch: replace _earlygetopt(strip=True) with new parser The execution order in cmdalias.__init__() is adjusted to set stripped args to self.givenargs, which is no longer updated in place.

dispatch: replace _earlyreq*() with new fancyopts-based parser

dispatch: alias --repo to --repository while parsing early options This prepares for replacing old _early*opt() functions. My initial attempt was to extend options table to support 'repository|repo' syntax. It worked, but seemed too invasive. So I decided to add an optional argument to fancyopts() instead. This also changes the nevernegate dict to be keyed by a canonical_name, not by an option-name for clarity.

spartan: render changesets server-side on /graph page

monoblue: render changesets server-side on /graph page

gitweb: render changesets server-side on /graph page

paper: render changesets server-side on /graph page

hgweb: only include graph-related data in jsdata variable on /graph pages (BC) Historically, client-side graph code was not only rendering the graph itself, but it was also adding all of the changeset information to the page as well. It meant that JavaScript code needed to construct valid HTML as a string (although proper escaping was done server-side). It wasn't too clunky, even though it meant that a lot of server-side things were duplicated client-side for no good reason, but the worst thing about it was the data format it used. It was somewhat future-proof, but not human-friendly, because it was just a tuple: it was possible to append things to it (as was done in e.g. 270f57d35525), but you'd then have to remember the indices and reading the resulting JS code wasn't easy, because cur[8] is not descriptive at all. So what would need to happen for graph to have more features, such as more changeset information or a different vertex style (branch-closing, obsolete)? First you'd need to take some property, process it (e.g. escape and pass through templatefilters function, and mind the encoding too), append it to jsdata and remember its index, then go add nearly identical JavaScript code to 4 different hgweb themes that use jsdata to render HTML, and finally try and forget how brittle it all felt. Oh yeah, and the indices go to double digits if we add 2 more items, say phase and obsolescence, and there are more to come. Rendering vertex in a different style would need another property (say, character "o", "_", or "x"), except if you want to be backwards-compatible, it would need to go after tags and bookmarks, and that just doesn't feel right. So here I'm trying to fix both the duplication of code and the data format: - changesets will be rendered by hgweb templates the same way as changelog and other such pages, so jsdata won't need any information that's not needed for rendering the graph itself - jsdata will be a dict, or an Object in JS, which is a lot nicer to humans and is a lot more future-proof in the long run, because it doesn't use numeric indices What about hgweb themes? Obviously, this will break all hgweb themes that render graph in JavaScript, including 3rd-party custom ones. But this will also reduce the size of client-side code and make it more uniform, so that it can be shared across hgweb themes, further reducing its size. The next few patches demonstrate that it's not hard to adapt a theme to these changes. And in a later series, I'm planning to move duplicate JS code from */graph.tmpl to mercurial.js and leave only 4 lines of code embedded in those <script> elements, and even that would be just to allow redefining graph.vertex function. So adapting a custom 3rd-party theme to these changes would mean: - creating or copying graphnode.tmpl and adding it to the map file (if a theme doesn't already use __base__) - modifying one line in graph.tmpl and simply removing the bigger part of JavaScript code from there Making these changes in this patch and not updating every hgweb theme that uses jsdata at the same time is a bit of a cheat to make this series more manageable: /graph pages that use jsdata are broken by this patch, but since there are no tests that would detect this, bisect works fine; and themes are updated separately, in the next 4 patches of this series to ease reviewing.

tests: write and use a custom helper script to avoid find's -printf -printf on find is a GNU-ism and will be banned in an upcoming check-code change. Differential Revision: https://phab.mercurial-scm.org/D1597

hgweb: adopt child nodes in ajaxScrollInit on /graph pages too ajaxScrollInit is a function that loads more elements (e.g. changelog entries) when browser window is scrolled down to the bottom of the page. It basically fetches the next page from the server as HTML, finds container element in that document and "adopts" (essentially, moves) all its child nodes to the container in the current document. Currently, hgweb doesn't render any changesets on /graph page (everything is done in JavaScript), so there are no children to adopt. But there will be soon, so let's create a reusable function that does it. Hardcoding #graphnodes selector is suboptimal, but graph code already does this in two other places.

lfs: enable the extension locally after converting to an 'lfs' repo This is consistent with clone and share in the previous commits.

lfs: enable the extension locally after sharing a repo with 'lfs' requirement This is consistent with clone in the previous commit.

lfs: enable the extension locally after cloning a repo with 'lfs' requirement We do the same thing on clone for the largefiles extension, as a convenience. Similar to largefiles, it's probably safer to only enable this extension on a per repo basis because it is trivial to add an lfs file. And that gives the repository some centralized VCS characteristics.

log: translate column labels at once (issue5750) This makes sure that all columns are aligned. getlogcolumns() is hosted by templatekw so the namespaces module can see it. i18n/de.po is updated so test-log.t passes with no error. "obsolete:" and "instability:" are kept untranslated.

log: remove temporary variable 'date' used only once

merge with stable

lock: allow to configure when the lock messages are displayed We add a new 'ui.timeout.warn' config to set a grace period before we display lock related warning: waiting for lock on PATH held by PROCESS The config is based on 'ui.timeout' and expresses a number of seconds before the warning is displayed. Negative values disable the warning altogether. The messages go to the debug output to help people trouble-shooting deadlocks.

lock: add a trylock method handling the timeout and messaging logic We are about to make the messages around lock more flexible. We move all the currently logic into a function in the lock module. We'll update the message scheme in the next changeset.

lock: use configint for 'ui.timeout' config The ui object can do the conversion itself.

scmutil: improve format pattern used in nodesummaries As spotted by Yuya Nishihara, that value is an integer.

tests: trivial fixes for Windows

py3: use encoding.strtolocal() to convert string to bytes Differential Revision: https://phab.mercurial-scm.org/D1557

py3: use pycompat.bytestr() or '%d' in place of str() Differential Revision: https://phab.mercurial-scm.org/D1556

archive: pass thru mtime for directory archives, like other archive types do Without this files in the output archive directory have their mtimes set to whatever time they were written their. This is in this inconsistent with the other archivers, eg. zip, which use exactly the same time for all files. Works on my machine (Windows), but I don't have a *nix box available to run anything more thorough, unfortunately.

hgweb: remove negative top from .info line in graph "top: -Xpx" shifts a block up by X pixels, which can be used to visually compress two lines of text to have less space between them, in this case it's used for the changesets on /graph page. But not only it's not needed there (both lines fit fine into their allowed vertical space), but it would also look better (not as crammed, more vertically centered) without these negative values. "position: relative" is needed solely for the "top" property to have effect on the element, no children of the .info element rely on it, so let's remove it as well.

unamend: drop unused vars, query after taking lock, use ctx.hex() for extras This is the followup of review on D821. Differential Revision: https://phab.mercurial-scm.org/D1579

test-lfs: drop a hack for ignoring convert devel-warnings This was fixed on stable in 281214150561.

convert: restore the ability to use bzr < 2.6.0 (issue5733) This effectively conditionalizes a234b32b744a. Some Linux distributions (like CentOS 7) use really old versions, and the change referenced was causing exceptions to be thrown. Even though the deprecation warning says 'since 2.5.0', it wasn't marked as such in 2.5.1, but is by 2.6.0. This was tested with 2.4.2 and 2.6.0 with PYTHONWARNINGS=::DeprecationWarning, and both paths were exercized.

py3: use bytes in place of basestring All strings in mercurial must be bytes so we can easily replace basestring with bytes. Differential Revision: https://phab.mercurial-scm.org/D1555

amend: make a copy of "extra" to avoid mutating an input I don't know of any problems this has caused, it just seems less surprising. Differential Revision: https://phab.mercurial-scm.org/D1575

tests: removes bashism from test-unamend.t FreeBSD builders fail because of bashism. Differential Revision: https://phab.mercurial-scm.org/D1578

run-tests: avoid calculating _testdir again Differential Revision: https://phab.mercurial-scm.org/D1574

run-tests: simplify by using dict.pop() with default Differential Revision: https://phab.mercurial-scm.org/D1573

py3: use byteskwargs in sparse.py This removes several dozen failures in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1482

py3: define __next__ in patch.py This needed to appease Python 3's iterator protocol. This is crasher #5 in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1480

run-tests: mechanism to report exceptions during test execution Sometimes when running tests you introduce a ton of exceptions. The most extreme example of this is running Mercurial with Python 3, which currently spews thousands of exceptions when running the test harness. This commit adds an opt-in feature to run-tests.py to aggregate exceptions encountered by `hg` when running tests. When --exceptions is used, the test harness enables the "logexceptions" extension in the test environment. This extension wraps the Mercurial function to handle exceptions and writes information about the exception to a random filename in a directory defined by the test harness via an environment variable. At the end of the test harness, these files are parsed, aggregated, and a list of all unique Mercurial frames triggering exceptions is printed in order of frequency. This feature is intended to aid Python 3 development. I've only really tested it on Python 3. There is no shortage of improvements that could be made. e.g. we could write a separate file containing the exception report - maybe even an HTML report. We also don't capture which tests demonstrate the exceptions, so there's no turnkey way to test whether a code change made an exception disappear. Perfect is the enemy of good. I think the current patch is useful enough to land. Whoever uses it can send patches to imprve its usefulness. Differential Revision: https://phab.mercurial-scm.org/D1477

run-tests: make --extra-config-opt work with Python 3 And add test coverage to ensure it works. Differential Revision: https://phab.mercurial-scm.org/D1476

run-tests: organize options into argument groups And sort arguments so help output is more legible. There are probably a ton of ways to group things. I tried to picture the test harness as a pipeline and attempted to draw boundaries around stages in that pipeline to create the groupings. Differential Revision: https://phab.mercurial-scm.org/D1475

run-tests: convert to argparse optparse has been deprecated since Python 3.2. Best to get on the new boat before the old one sinks. It looks like argparse formats its usage string differently than optparse. Meh. Differential Revision: https://phab.mercurial-scm.org/D1474

merge with stable

server: introduce a 'experimental.single-head-per-branch' option When the option is set, the repository will reject any transaction adding multiple heads to the same named branch. For now we reject all scenario with multiple heads. One could imagine handling closed branches differently. We prefer to keep things simple for now. The feature might get extended later. Branch closing is not the best experience Mercurial has to offer anyway.

scmutil: extra utility to display a reasonable amount of nodes Push have some logic to display a reasonable amount nodes. We extract it to an utility function to make it reusable.

uncommit: unify functions _uncommitdirstate and _unamenddirstate to one The _unamenddirstate() function was inspired by _uncommitdirstate() function as the logic was same but we were unable to use the latter function directly. So previous patch introduced the _unamenddirstate() function and now this patch unifies both the function and we have a _fixdirstate() function. Adding function in previous patch and unifying in a later patch makes the reasoning easier and also leaves the last patch dedicated to what it is meant to be. Differential Revision: https://phab.mercurial-scm.org/D971

unamend: move fb extension unamend to core unamend extension adds an unamend command which undoes the effect of the amend command. This patch moves the unamend command from that extension to uncommit extension and this one does not completely undoes the effect of amend command as it creates a new commit, rather than reviving the old one back. This also adds tests for the same. .. feature:: A new unamend command in uncommit extension which undoes the effect of the amend command by creating a new changeset which was there before amend and moving the changes that were amended to the working directory. Differential Revision: https://phab.mercurial-scm.org/D821

color: respect HGPLAINEXCEPT=color to allow colors while scripting (issue5749) I'd also like --color=always on the command-line to override HGPLAIN=1 et al, but that's more work, and this seems like a better fix. We've got a fair number of programs that actually want to automate hg and get colored output to users, so they should set HGPLAINEXCEPT=alias (what we usually recommend), but this has been breaking them because they then lose color. .. feature:: The ``HGPLAINEXCEPT`` environment variable can now include ``color`` to allow automatic output colorization in otherwise automated environments. Differential Revision: https://phab.mercurial-scm.org/D1532

test-ssh: do not actually look up "brokenrepository" by DNS

largefiles: explicitly set the source and sink types to 'hg' for lfconvert I stumbled into this prior to adding the type indicator on the source and sink, but there's no reason to try to infer the types for this conversion.

lfs: add a repo requirement for this extension when converting to lfs This covers both the vanilla repo -> lfs repo and largefiles -> lfs conversions. The largefiles extension adds the requirement directly, because it has a dedicated command to convert. Using the convert extension is better, because it supports more features. I'd like ideas about how to ensure that converting away from lfs works on all files. (See comments in test-lfs.t)

convert: allow the sink object to be wrapped when the extension isn't loaded The next patch will wrap the conversion code, in order to write out a requirement for 'lfs' when appropriate. Wrapping convcmd.convertsink() in an afterloaded callback works fine when the convert extension is enabled by the user. The problem here is that lfconvert uses the convert extension, whether or not it was formally enabled by the user. My first attempt was to have lfs install an afterloaded callback that would wrap the convert sink if convert was loaded, or wrap lfconvert if it wasn't. Then the lfconvert override could install an afterloaded callback to try wrapping the convert sink again, before calling the original lfconvert. But that breaks down if largefiles can't load the convert extension on the fly. [1] Further, some tests were failing with an error indicating that the size of the afterloaded list changed while iterating it. Yuya mentioned that maybe some bits of convert could be moved into core, but I'm not sure where to draw that line. The convertsink() method depends on the list of sinks, which in turn depends on the sink classes. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/108038.html

convert: save an indicator of the repo type for sources and sinks This seems like basic info to have, and will be used shortly when deciding whether or not to wrap the class for lfs conversions. The other option is to just add a function to each class. But this seems better in that the strings aren't duplicated, and the constructor for most of these will run even if the VCS isn't installed, so it's easier to catch errors.

lfs: add a repo requirement for this extension once an lfs file is committed Largefiles does the same thing (also delayed until the first largefile commit), to prevent access to the repo without the extension. In the case of this extension, not having the extension loaded while accessing an lfs file results in cryptic errors about "missing processor for flag '0x2000'". If enabled locally but not remotely, the cryptic error message is about no common changegroup version. (It wants '03', which is currently experimental.) The largefiles extension looks for any tracked file that starts with '.hglf/'. Unfortunately, that doesn't work here. I didn't see any way to get the files that were just committed, without doing a full status. But since there's no secondary check on adding an lfs file once the extension is loaded and a threshold set, the best practice is to only enable this locally on a repo that needs it. That should minimize the unnecessary overhead for repos without an lfs file.

Added signature for changeset a92b9f8e11ba

Added tag 4.4.2 for changeset a92b9f8e11ba

merge: check created file dirs for path conflicts only once (issue5716) In large repositories, updates involving the creation of many files check the same directories repeatedly in the wctx manifest. Move these checks out to a separate loop to avoid repeated checks hitting the manifest. Differential Revision: https://phab.mercurial-scm.org/D1226

merge: cache unknown dir checks (issue5716) As mentioned in D1222, the recent pathconflicts change regresses update performance in large repositories when many files are being updated. To mitigate this, we introduce two caches of directories that have already found to be either: - unknown directories, but which are not aliased by files and so don't need to be checked if they are files again; and - missing directores, which cannot cause path conflicts, and cannot contain a file that causes a path conflict. When checking the paths of a file, testing against this caches means we can skip tests that involve touching the filesystem. Differential Revision: https://phab.mercurial-scm.org/D1224

dispatch: add HGPLAIN=+strictflags to restrict early parsing of global options If this feature is enabled, early options are parsed using the global options table. As the parser stops processing options when non/unknown option is encountered, it won't mistakenly take an option value as a new early option. Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I think it's unlikely to lead to an RCE. To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled only when +strictflags is set. Also the strict parser doesn't support '--repo', a short for '--repository' yet. This limitation will be removed later. As this feature is backward incompatible, I decided to add a new opt-in mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively, we could add a new environment variable. Any bikeshedding is welcome. Note that HGPLAIN=+strictflags doesn't work correctly in chg session since command arguments are pre-processed in C. This wouldn't be easily fixed.

fancyopts: add early-options parser compatible with getopt() The next patch will add a flag for strict parsing of early options, where we'll have to parse all early options at once instead of processing them one-by-one by dispatch._earlygetopt(). That's why I decided to hook fancyopts(). All dispatch._early*opt() functions is planned to be replaced with this function. But in this stable series, only the strict mode will be handled by fancyopts.earlygetopt().

largefiles: pay attention to dropped standin files when updating largefiles Previously, the largefile for a dropped standin would be deleted here, and then restored from the cache. This had the effect of clobbering uncommitted changes if a revert caused the file to be forgotten, which is not what happens with a normal file. Now the removal and update is skipped for dropped largefiles, and the corresponding standin is deleted from disk. This was noticed when working on issue5738 because the forgotten standin files were left behind, and that changes the behavior of the next rename to that directory. My first attempt was to cleanup the standins before calling this. That failed, because this function deletes the largefile if the corresponding standin is missing. This function is called by the revert command, merge (and therefore update), and patch, via the scmutil.marktouched() override. So it should be pretty narrow in scope. I didn't mark issue5738 as fixed because the move related issues can still happen if the main tree and the .hglf subtree get out of sync somehow. I don't see an easy fix for that, but that should be an edge case. If whoever queues this thinks it is good enough to close out the bug and can cram it into the summary, go for it.

test-largefiles: demonstrate problems with renaming and reverting a directory These things were uncovered looking at issue5738. First, if the destination directory exists under .hglf, the source is moved under the destination instead of renaming the last component for `hg mv srcdir dstdir`. This is extra confusing, because it occurs even if the user visible destination (i.e. the path _not_ under .hglf) does not exist. Additionally, when a largefile is forgotten via revert, any modifications end up getting clobbered. For normal files, the forgotten file is left unchanged, as shown by test-import.t. The forget command on a largefile will correctly leave the file unmodified.

tests: add globs for Windows

cat: record the current behavior of wildcard matches in subrepos Mercurial subrepos support any match patterns.

match: remove doc about undefined behavior of visitdir() This was added by 8545bd381504, but core matchers support visitdir() of arbitrary locations since 2773540c3650, and verifier._verifymanifest() doesn't seem to strictly obey the restriction. I have no idea how important this API contract is for third-party extensions. That's why this patch is RFC.

merge with stable

hgweb: add .jshintrc with some basic rules This file is picked up automatically by jshint, so no extra changes required in test-check-jshint.t.

hgweb: look up "URLSearchParams" in "window" to work around jshint issues Unfortunately, current version of jshint (2.9.5) doesn't know such a global variable and complains that it's undefined. Since this line tries to look up URLSearchParams in a global scope (i.e. window), let's simply preface it with "window." to work around jshint.

hgweb: define locally used variables as actually local in mercurial.js Variables that are used or assigned without any declaration using var (or let, or const) are considered global. In many cases this is inadvertent and actually causes a variable leaking to a broader scope, such as a temporary variable used inside a loop suddenly being accessible in global scope. (This corresponds to "undef" option of jshint). So this patch limits the scope of variables that don't need to be global. There are a lot of helper variables in Graph.render() used in a loop, I've declared them all on one line to reduce patch size. "radius" is special because it wasn't passed to graph.vertex, but was used there (it worked because this variable leaked to global scope). "window.graph" is created by an inline script in graph.tmpl so that it can be used in ajaxScrollInit() function, this patch makes this fact explicit by assigning window.graph to a local variable.

hgweb: rename an instance of XMLHttpRequest to xhr in mercurial.js "xhr" is a really widespread name for this kind of things, no idea where did "xfr" come from (the original 2228bd109706 doesn't explain that). Let's just change one letter so the name makes more sense.

hgweb: properly iterate over arrays and objects in mercurial.js In JavaScript, using for-in loops to access every property of an object can have unexpected results when inheritance is involved. For example, if some piece of code adds a property (it may be a method too) to Object.prototype, then all for-in loops that iterate over keys of any object (also anything that inherits Object) will get that property on one of the iterations. To filter out such unexpected properties for-in loops have to use Object.hasOwnProperty() method. (This corresponds to "forin" option of jshint). In the two first cases "data" and "edges" are arrays, to it's simpler to just switch to using a regular for-with-a-counter loop.

hgweb: use strict equals in mercurial.js This patch changes "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Some of the changes in this patch are straightforward, e.g. when comparing results of typeof (they could only be strings). The same goes for 'none' and similar strings that can't be sensibly coerced to some other type. Two changes that compare values to "1" and "0" can be clarified: getAttribute() returns either a string or null, but comparing null to a string is always false, so no logic is lost.

hgweb: use strict equals, remove non-breaking space in followlines.js The first hunk had a non-breaking space character just before "{", it's not an error or anything, but let's fix it while we're at it. (This corresponds to "nonbsp" option of jshint). Hunks 2 and 3 change "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Most of this file already uses strict equals operator, and in the code affected type coercion is not needed, because tagName and selectableTag are both strings and endId and startId are both numbers.

run-tests: make "| foo (re)" not match everything We make "foo (re)" match the entire line by adding a \Z to the regular expression before matching. However, that doesn't help when the regular expression is something like "| foo", because that gets translated to "| foo\Z", where the "|" has lower precedence and it thus matches the empty string, which is of course a prefix of every string. Fix by wrapping expression in a group before adding the \Z to the end. Differential Revision: https://phab.mercurial-scm.org/D1546

tests: fix regex in test-subrepo-git.t to match entire string Due to a bug in the test runner (fixed by the next commit), the regex used for matching lines like " foobar | 2 +-" stoppped at the "|" and the test passed even though the rest of the line did not match. The test seems to have been supposed to match "|" and "+" literally on those lines, so this changes the regex to escape those characters. It also changes a "\s*" to "\s+" since I think we'll always include a space after the "|" in the diffstat output. Differential Revision: https://phab.mercurial-scm.org/D1545

contrib: improve check-code ban on $LOCALIP in output without (glob) Differential Revision: https://phab.mercurial-scm.org/D1553

tests: re-add (glob) for $LOCALIP matches This should fix most of the failing tests on the FreeBSD builder, since it has no 127/8 series IP as a side effect of being trapped in a jail. Differential Revision: https://phab.mercurial-scm.org/D1552