revsetlang: use iterator to track current argument in formatspec()

revsetlang: unnest "if True" in formatrevspec()

revsetlang: use str.find() to scan expr in formatspec() There should be no need to walk character one by one in Python.

revsetlang: avoid string concatenation in formatspec()

test-glog: hook cmdutil.getlogrevs() so -frREV is rewritten accordingly Before, these tests didn't match the real behavior of "log -frREV".

test-glog: dump computed set It's more important than the revset expression built from command options.

test-glog: drop uninteresting nodes from AST output This makes future test changes more readable.

lfs: remove the verification option when writing to the local store This partially reverts 417e8e040102 and bb6a80fc969a. But since there's now a dedicated download function, there's no functional change. The last sentence in the commit message of the latter is wrong- write() didn't need the one time hash check if verification wasn't requested. I suspect I missed 'read()' in there ("... but _read()_ also needs to do a one time check..."), because that did fail without the hash check before linking to the usercache. The write() method simply took the same check for consistency. While here, clarify that the write() method is *only* for storing content directly from filelog, which has already checked the hash. If someone can come up with a way to bridge the differences between writing to a file and sending a urlreq.request across the wire, we can create an upload() function and cleanup read() in a similar way. About the only common thread I see is an open() that verifies the content before returning a file descriptor.

lfs: use the localstore download method to transfer from remote stores Both gitlfsremote and file based remotes benefit from not requiring the whole file in memory (though the whole file is still loaded when passing through the revlog interface). With a method specific to downloading from a remote store, the misleading 'use hg verify' hint is removed. The behavior is otherwise unchanged, in that a download from both remote store types will yield a copy of the blob via util.atomictempfile. There's no response payload defined for the non 'download' actions, but the previous code attempted to read the payload in this case anyway. This refactored code made that more obvious, so any payload is printed as a debug message, just in case.

lfs: introduce a localstore method for downloading from remote stores The current local.write() method requires the full data, which means concatenating file chunks in memory when downloading from a git server. The dedicated method downloads in chunks, verifies the content on the fly, and creates the usercache hardlink if successful. It can also be used for the file system based remotestore. An explicit division of labor between downloading from a remote store (which should be verified) and writing to the store because of a commit or similar (which doesn't need verification), seems clearer. I can't figure out how to make a similar function for upload, because for a file remote store, it's a simple open/read/write operation. For a gitremote store, it's open the file and a urlreq.request(), and process that.

win32: do not call GetVolumePathName() with the minimum buffer length It fails on Windows XP even though the doc says "a safer but slower way to set the size of the return buffer is to call the GetFullPathName function, and then make sure that the buffer size is at least the same size as the full path that GetFullPathName returns." https://msdn.microsoft.com/en-us/library/windows/desktop/aa364996(v=vs.85).aspx Well, more "safe" way would be to simply rely on MAX_PATH for common scenarios.

win32: allocate buffer of maximum length for GetVolumeInformation() It's documented that "the maximum buffer size is MAX_PATH+1", which is slightly larger than 256. https://msdn.microsoft.com/en-us/library/windows/desktop/aa364993(v=vs.85).aspx

log: drop unused expr from return value of getlogrevs() Future patches will move some processing of the --follow option out of _makelogrevset(), where the returned 'expr' value will be less consistent with the 'revs'. So let's remove it from the public interface.

log: don't expand aliases in revset built from command options Only -rREV should be rewritten with user aliases.

graphlog: remove redundant check for empty revs

log: remove redundant handling of --limit A returned 'revs' set is pre-filtered by cmdutil.getlogrevs(). The result of displayer.flush() isn't used anymore, so removed.

log: use smartset.slice() to limit number of revisions to be displayed

log: merge getlogrevs() and getgraphlogrevs() cmdutil.graphlog() is updated to receive (revs, filematcher) as arguments to make sure that opts['graph'] is set when getlogrevs() is invoked.

revset: parse x^:: as (x^):: (issue5764) We have to make '::' a valid primary expression to parse 'x^::' as '(x)^(::)' first, but that doesn't change the language because a prefix operator '::y' precedes a primary '::'. I can't think of an intuitive meaning of '::', so it's just rejected. Given 'x::y' can be considered to default to {x = roots(), y = heads()}, '::' could be 'roots()::heads()', which seems not any useful.

test-run-tests: stabilize the test (issue5735) Previously there is a race condition because things happen in this order: 1. Check shouldStop 2. If shouldStop is false, print the diff 3. Call fail() -> set shouldStop The check and set should really happen in a same critical section. This patch adds a lock to address the issue. Test Plan: Run `run-tests.py -l test-run-tests.t` 20 times on gcc112 and the race condition does not reproduce. Differential Revision: https://phab.mercurial-scm.org/D1830

run-tests: avoid set PYTHONUSERBASE environment variable to None According to python documentation, get_config_var can return None and it was during running mercurial tests under a virtual environment with the local built mercurial installed as a python package with `pip install -e .`. On the other hand, if mercurial is installed in the user library folders with `pip install --local -e .` the issue doesn't reproduce. The fallback is that it will just set an empty string and everything should work as before, according to https://www.python.org/dev/peps/pep-0370/: The path to the user base directory can be overwritten with the environment variable PYTHONUSERBASE. The default location is used when PYTHONUSERBASE is not set or empty.

templatefilters: fix doc of basename() os.path.basename() does NOT strip trailing slashes. > Note that the result of this function is different from the Unix basename > program; where basename for '/foo/bar/' returns 'bar', the basename() > function returns an empty string (''). https://docs.python.org/2.7/library/os.path.html#os.path.basename

identify: document -r. explicitly how to disable wdir scanning (issue5622)

lfs: add a comment to describe subtle local blobstore open() behavior

tests: make #testcase available as env var in test It can be useful for e.g. helper functions to behave differently depending on which test case is being executed. This patch helps with that by making the #testcase case available as a TESTCASE environment variable. Differential Revision: https://phab.mercurial-scm.org/D1816

archive: add support to specify hidden revs if directaccess config is set Differential Revision: https://phab.mercurial-scm.org/D1812

annotate: add support to specify hidden revs if directaccess config is set Differential Revision: https://phab.mercurial-scm.org/D1811

hgweb: stop adding strings to innerHTML of #graphnodes and #nodebgs (BC) There's nothing that uses this functionality anymore, since all HTML is generated server-side now. Changeset entries are still accessible in JS via DOM (that's how we set left padding), so it's still possible to do something unusual by overriding Graph.prototype.vertex().

spartan: make actual changeset entries have backgrounds on /graph This patch removes a separate container for backgrounds only and puts backgrounds on changeset elements themselves. This makes it so that JS code doesn't need to create background elements separately anymore. There's a bit of manipulating positions of elements: every changeset entry has a "fg" element that gets a higher z-index than <canvas> element. This prioritizes text information, so that even if it somehow gets close to the graph drawn on <canvas> (which has z-index of 5), it'll still be on top of it. This can happen if commit message has a long first line and browser window is narrow, for example.

gitweb: make actual changeset entries have backgrounds on /graph This patch removes a separate container for backgrounds only and puts backgrounds on changeset elements themselves. This makes it so that JS code doesn't need to create background elements separately anymore. There's a bit of manipulating positions of elements: every changeset entry has a "fg" element that gets a higher z-index than <canvas> element. This prioritizes text information, so that even if it somehow gets close to the graph drawn on <canvas> (which has z-index of 5), it'll still be on top of it. This can happen if commit message has a long first line and browser window is narrow, for example.

monoblue: make actual changeset entries have backgrounds on /graph This patch removes a separate container for backgrounds only and puts backgrounds on changeset elements themselves. This makes it so that JS code doesn't need to create background elements separately anymore. There's a bit of manipulating positions of elements: every changeset entry has a "fg" element that gets a higher z-index than <canvas> element. This prioritizes text information, so that even if it somehow gets close to the graph drawn on <canvas> (which has z-index of 5), it'll still be on top of it. This can happen if commit message has a long first line and browser window is narrow, for example. 'parity' is added to each entry because there's no stripes2 class in monoblue, only in paper.

paper: make actual changeset entries have backgrounds on /graph This patch removes a separate container for backgrounds only and puts backgrounds on changeset elements themselves. This makes it so that JS code doesn't need to create background elements separately anymore. There's a bit of manipulating positions of elements: every changeset entry has a "fg" element that gets a higher z-index than <canvas> element. This prioritizes text information, so that even if it somehow gets close to the graph drawn on <canvas> (which has z-index of 5), it'll still be on top of it. This can happen if commit message has a long first line and browser window is narrow, for example.

crecord: fallback to color = no when curses.use_default_colors() fails Even when python was setup/compiled with curses, curses.use_default_colors() may raise a curses exception when the TERM environment variable specifies a value where no propper color configuration is possible. This patch falls back to non-color mode to let the user continue instead of failing with an unhandled exception.

crecord: honor "ui.color = no" config option The current implementation of crecord ignores the ui.color setting. This patch checks the ui.color config option and does the curses setup without colors when the option is set to a falsy value. For other (or missing) setting of ui.color, curses setup is done as before and uses colors.

lfs: use the local store method for opening a blob I noticed that when I cloned without updating and then turned around and pushed that clone to an lfs server, it was only trying to find the blob in the local store. Writes to the dummyremote (file based store) use local.read(), which looks at both the usercache and local store.

lfs: add a local store method for opening a blob The has() and read() methods already dynamically switch between the usercache and local store. This should generally be preferred to directly accessing the vfs instances outside of the store. The file is now explicitly opened in binary mode for clarity. (It was also being opened in binary mode before, but only because vfs.__call__() appends 'b' if needed when not opening with 'text=True'.)

hgdemandimport: use correct hyperlink to python-bug in comments (issue5765) Differential Revision: https://phab.mercurial-scm.org/D1810

monoblue: adjust font size of graph entries /shortlog uses 1.1em font for changesets, so it makes sense to use the same size on /graph too. In practice, 1.2em is 14.784px, and 1.1em is 13.552px.

hgweb: make .info a block element by default Using a <div> instead of a <span> with "display: block" makes more sense. While at it, let's adjust top margin in monoblue so that text in .info doesn't overlap changeset "tags" elements (they are quite large in monoblue).

revlog: don't use slicing to return parents This is the only place we use a slice on index entries, which are currently tuples. In preparation for moving away from tuples, let's stop using slices so we don't have to implement that support on the new type. We also tweak the logic slightly so the exception only catches the IndexError on the index lookup, not on the index entry lookup. The old code should never have been buggy. But it was semantically wrong. Differential Revision: https://phab.mercurial-scm.org/D1764

tests: switch test from '--config ui.editor=~/foo' to HGEDITOR=~/foo The test is currently not testing anything, the '--config ui.editor=~/path' argument is apparently of lower priority than the HGEDITOR that run-tests.py is setting. Differential Revision: https://phab.mercurial-scm.org/D1809

revert: support reverting to hidden cset if directaccess config is set This patch adds support for reverting files to a hidden changeset if `experimental.directacess=True` is set. Differential Revision: https://phab.mercurial-scm.org/D1763

update: support updating to hidden cset if directaccess config is set This patch adds support for updating to a hidden changeset without using --hidden if `experimental.directacces=True` is set. The update command will print out a warning when updating to a hidden changeset saying: `updating to a hidden changeset <hash>` The warning is also printed when directaccess is not used and --hidden is passed which I think is good behaviour. Tests are added for the directaccess part and updated output of tests which have case of updating to hidden cset using `--hidden`. Differential Revision: https://phab.mercurial-scm.org/D1762

commands: check for empty rev before passing to scmutil.unhidehashlikerevs Differential Revision: https://phab.mercurial-scm.org/D1760

test-convert-cvs: change TZ=US/Hawaii to TZ=Pacific/Johnston The former was limited to be known on Linux and the test failed on FreeBSD and Solaris platforms. The newer is known on Linux, FreeBSD and Solaris.

osutil: implement getfsmountpoint() on BSD systems I don't have a BSD system handy to test this, but it looks simple enough from the man page.

debugfs: display the tested path and mount point of the filesystem, if known While implementing win32.getfstype(), I noticed that MSYS path mangling is getting in the way. Given a path \\host\share\dir: - If strong quoted, hg receives it unchanged, and it works as expected - If double quoted, it converts to \host\share\dir - If unquoted, it converts to \hostsharedir The second and third cases are problematic because those are valid paths relative to the current drive letter, so os.path.realpath() will expand it as such. The net effect is to silently turn a network path test into (typically) a "C:\" test. Additionally, the command hangs after printing out 'symlink: no' for the third case (but is interruptable with Ctrl + C). This path mangling only comes into play because of the command line arguments- it won't affect internally obtained paths. Therefore, the simplest thing to do is to provide feedback on what the command is acting on. I also added the mount point, because Windows supports nesting [1] volumes (see the examples in "Junction Points and Mounted Folders"), and it was a useful diagnostic for figuring out why the wrong filesystem was printed out in the cases above. I opted not to call os.path.realpath() on the path argument, to make it clearer that the mangling isn't being done by Mercurial. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa364996(v=vs.85).aspx

util: add a function to show the mount point of the filesystem For now, this is Windows only, since Linux doesn't have the value in its statfs structure, and I don't have a BSD system to test with.

win32: split a utility function to obtain the volume out of getfstype() This is only done on Windows because it's simple enough to call statfs() on Unix. The goal is to display this in `hg debugfs`.

util: whitelist NTFS for hardlink creation (issue4580)

win32: implement util.getfstype() This will allow NTFS to be added to the hardlink whitelist, and resume creating hardlinks in transactions (which was disabled globally in 07a92bbd02e5; see also e5ce49a30146). I opted to report "cifs" for remote volumes because this shows in `hg debugfs`, which also reports that hardlinks are supported for these volumes. So being able to distinguish it from "unknown" seems useful. The documentation [1] seems to indicate that SMB isn't supported by these functions, but experimenting shows that mapped drives are reported as "NTFS" on Windows 7. I don't have a second Windows machine, but instead shared a temp directory on C:\. In this setup, both of the following were detected as 'cifs' with the explicit GetDriveType() check: Z:\repo>hg ci -A C:\>hg -R \\hostname\temp\repo ci -A # (without Z:\ being mapped) It looks like this is called 6 times to add and commit a single new file, so I'm a little surprised this isn't cached. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa364993(v=vs.85).aspx

util: move getfstype() to the platform modules This makes room for implementing on Windows using ctypes.

graphlog: add another graph node type, unstable, using character "*" (BC)

largefiles: modernize how capabilities are added to the wire protocol See 982f13bef503, which came well after this code was originally written.

lfs: show a friendly message when pushing lfs to a server without lfs enabled Upfront disclaimer: I don't know anything about the wire protocol, and this was pretty much cargo-culted from largefiles, and then clonebundles, since it seems more modern. I was surprised that exchange.push() will ensure all of the proper requirements when exchanging between two local repos, but doesn't care when one is remote. All this new capability marker does is inform the client that the extension is enabled remotely. It may or may not contain commits with external blobs. Open issues: - largefiles uses 'largefiles=serve' for its capability. Someday I hope to be able to push lfs blobs to an `hg serve` instance. That will probably require a distinct capability. Should it change to '=serve' then? Or just add an 'lfs-serve' capability then? - The flip side of this is more complicated. It looks like largefiles adds an 'lheads' command for the client to signal to the server that the extension is loaded. That is then converted to 'heads' and sent through the normal wire protocol plumbing. A client using the 'heads' command directly is kicked out with a message indicating that the largefiles extension must be loaded. We could do similar with 'lfsheads', but then a repo with both largefiles and lfs blobs can't be pushed over the wire. Hopefully somebody with more wire protocol experience can think of something else. I see 'x-hgarg-1' on some commands in the tests, but not on heads, and didn't dig any further.

lfs: allow non-lfs exchanges when the extension is only enabled on one side Once the 'lfs' requirement is added, the extension must be loaded on both sides, and changegroup3 used. But there's no reason that I can see for bailing with cryptic errors if lfs is not required, but randomly enabled somewhere.

lfs: add the 'lfs' requirement in the changegroup transaction introducing lfs A hook like this is how largefiles manages to do the same. Largefiles uses a changegroup hook, but this uses pretxnchangegroup because that actually causes the transaction to rollback in the unlikely event that writing the requirements out fails. Sadly, the requires file itself isn't rolled back if a subsequent hook fails, but that seems trivial. Now that commit, changegroup and convert are covered, I don't think there's any way to get an lfs repo without the requirement. The grep exit code is blotted out of some test-lfs-serve.t tests now showing the requirement, because run-tests.py doesn't support conditionalizing the exit code.

test-lfs: add tests covering http exchanges This tries to test every combination of having the extension enabled/disabled on each side, and then push/pull/clone/identify lfs and non-lfs content. SSH is ignored here, because there's enough going on as it is. The root issue here is again that requirements are not exchanged and preserved on push/pull/clone. Doing so should eliminate the cryptic error messages when using `hg serve`. The 500 server error is triggered by "ValueError: no common changegroup version", because the extension forces changegroup3. Or, if changegroup3 is enabled manually, it is triggered by "abort: missing processor for flag '0x2000'!". Sadly, run-tests.py doesn't support conditionalizing the exit code like it does lines of output. Therefore, a couple of tests blot out the exit code by appending "|| true", since these failures will go away shortly anyway.

test-lfs: add tests covering local exchanges The root issue here is that requirements are not exchanged and preserved on push/pull. This can be handled with a changegroup hook. Testing for remote exchanges is much more extensive (it's possible for one process or the other to not have the extension loaded at all), so it is added separately.

smartset: split generatorset classes to avoid cycle I uncovered a cycle manifesting in a memory leak by running `hgperfrevset '::tip'`. The cycle was due to generatorset.__init__ assigning a bound method to self.__contains__. Internet sleuthing revealed that assigning a bound method to an instance attribute always creates a cycle. This commit creates two new variants of generatorset for the special cases of ascending and descending generators. The special implementations of __contains__ have been extracted to these classes where they are defined as __contains__. generatorset now implements __new__ and changes the spawned type to one of the new classes if needed. Differential Revision: https://phab.mercurial-scm.org/D1780