shelve: use full hash in tests Using revision number is fragile. Moving to full hash to help with further development. Differential Revision: https://phab.mercurial-scm.org/D3685

phabricator: preserve the phase when amending in the Differential fields I have no idea if it's better to change scmutil.cleanupnodes() so that it has the option to either apply a specific phase (e.g. for various --secret switches) or carry over the phase of the old node. The benefit would be that the caller doesn't have to remember to do this. The con is maybe inefficiency? I wrote this up as issue5918. I'm leaving that open since Yuya flagged it as an API bug. Since most other callers already do this, it's the simplest fix. (It's not obvious that `split`, `fix` and `rebase` are doing this, but there is test coverage for `fix` and `rebase`, and experimenting with `split` shows it does the right thing.)

manifest: fix possible SEGV caused by uninitialized lazymanifest fields Before, uninitialized self->pydata would be passed to lazymanifest_dealloc() on OOM, and Py_DECREF(self->pydata) would crash if we were unlucky. It's still wrong to do malloc() thingy in tp_init because __init__() may be called more than once [1], but I don't want to go a step further in stable branch. [1]: https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new "The tp_new function should ... do only as much further initialization as is absolutely necessary. Initialization that can safely be ignored or repeated should be placed in the tp_init handler."

tests: replace `echo -n` with `printf` per check-code Differential Revision: https://phab.mercurial-scm.org/D3749

crecord: fix line number in hunk header (issue5917) `@@ -1,1 +-1,0 @@` is not a valid patch hunk header. Change it to `@@ -1,1 +0,0 @@`. Differential Revision: https://phab.mercurial-scm.org/D3737

py3: ditch email.parser.BytesParser which appears to be plain crap As I said before, BytesParser is a thin wrapper over the unicode Parser, and it's too thin to return bytes back. Today, I found it does normalize newline characters to '\n's thanks to the careless use of TextIOWrapper. So, this patch replaces BytesParser with Parser + TextIOWrapper, and fix newline handling. Since I don't know what's the least bad encoding strategy here, I just copied it from BytesParser. I've moved new parse() function from pycompat, as it is no longer a trivial wrapper.

py3: remove b'' from error message of disallowed filename

py3: remove b'' from output of test-eol.t

py3: replace s[-1] with s.endswith() in eol handling

py3: fix loop over byte string in wireprotov1peer Before, it would always return [True]s on Python 3 because list(b"0") == [48].

py3: glob out some error messages in test-fileset.t Python3 provides more detailed messages, which is fine.

fileset: raise ProgrammingError for bad existing() calls And glob out Py2/3 difference.

py3: cast bytes encoding name to str in fileset.py

import: use context manager for wlock Differential Revision: https://phab.mercurial-scm.org/D3747

tag: use context manager for locks Differential Revision: https://phab.mercurial-scm.org/D3746

commit: use context manager for locks Differential Revision: https://phab.mercurial-scm.org/D3745

backout: use context manager for locks Differential Revision: https://phab.mercurial-scm.org/D3744

backout: use context manager for dirstateguard Differential Revision: https://phab.mercurial-scm.org/D3743

backout: use context manager for config override Differential Revision: https://phab.mercurial-scm.org/D3742

update: use context manager for config override (API) Note that update wasn't resetting the value before, so any extensions that called commands.update() and relied on ui.forcemerge being set after it returned would now have to set it themselves. (There technically a small API change in all of the patches in this series, I believe: If extensions relied on the methods to *clear* ui.forcemerge, then they would have to do that themselves now, because ui.configoverride() actually restores the previous config, it doesn't just clear it like these functions did before.) Differential Revision: https://phab.mercurial-scm.org/D3741

resolve: use context manager for config override Differential Revision: https://phab.mercurial-scm.org/D3740

merge: use context manager for config override Differential Revision: https://phab.mercurial-scm.org/D3739

graft: use context manager for config override Differential Revision: https://phab.mercurial-scm.org/D3738

tests: suppress read(-1) -> '' calls in fileobjectobserver This stabilizes the test output of the low-level wireproto tests between Python 2 and 3. I don't feel great about this change, but otherwise we get a ton of extra read(-1) output on Python 3, and this feels like a more sustainable solution. Bonus: test-ssh-proto-unbundle.t now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3733

debugcommands: work around logiofd being a pipe and unseekable This was breaking in our Python 3 build, but not Python 2. I don't know how it ever worked in Python 2, but this passes on both. Differential Revision: https://phab.mercurial-scm.org/D3732

py3: replace `unicode` with pycompat.unicode unicode() is not available on Python 3 and throws a NameError because unicodes are now default str() on py3. Differential Revision: https://phab.mercurial-scm.org/D3708

cmdutil: say that `graft --stop` stops the graft instead of aborting I am going to add a --abort flag to graft soon, so we make sure we differentiate between the what --stop flag does and what --abort does and don't use the words interchangeably. Differential Revision: https://phab.mercurial-scm.org/D3735

py3: two more passing tests from the ratchet in buildbot Differential Revision: https://phab.mercurial-scm.org/D3736

bdiff: one more safe call of bdiff_freehunks(NULL)

bdiff: document that bdiff_freehunks() accepts NULL blocks() of cext/bdiff.c may pass NULL on OOM.

hgweb: propagate http headers from ErrorResponse for web interface commands This makes it possible for e.g. authorization hooks to provide appropriate headers to make the web browser ask for credentials. It's done in the same way as the existing code in wireprotoserver.py.

py3: fix map() use in templatekw.showpredecessors() Now it matters since hybrid.tobool() no longer consumes the generator.

merge with stable

templatefilters: rename commonprefix to commondir Two reasons: * It makes it clearer that it's not a generic common string prefix (e.g. commonprefix(["bar", "baz"]) is not "ba", but "") * If we ever want a filter for generic common string prefix, then the name is now available for that. "commondir" does not describe the prefix-ness, however. I'm happy to rename it "commondirprefix" or "commonprefixdir" if others prefer. Differential Revision: https://phab.mercurial-scm.org/D3731

bitmanipulation: fix undefined behavior in bit shift in getbe32 OSS-Fuzz caught this in its ubsan mode[0]. I'm not worried about a security issue here because in practice this should work out the way we naively expected, we're just making things explicit to the compiler with the casts. 0: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8876 Differential Revision: https://phab.mercurial-scm.org/D3729

lazymanifest: don't crash when out of memory (issue5916) self->lines can be NULL if we failed to allocate memory for it.

cext: stop worrying and love the free(NULL) There is no need to check for a NULL pointer before calling free since free(NULL) is defined by C standards as a no-op. Lots of software relies on this behavior so it is completely safe to call even on the most obscure of systems.

templatefilters: undeprecate hgdate See the previous patch for why. Backed out changeset 0fe65bb7e160

templater: restore the original string format of {date} Unfortunately, python-hglib relies on that. I could fix python-hglib, but there would be other tools that take a decimal separator as the separator of unixtime and tzoffset. The showfmt is set per instance since new code uses '%d %d' format by default.

hgweb: insist http_status value is a sysstr My previous change was extremely confusing to figure out, because I thought I was looking at a client-side problem. For the low cost of an isinstance assert, we can make those errors not happen in the future. Differential Revision: https://phab.mercurial-scm.org/D3727

hgweb: pass a sysstr to low-level _start_response method This fixes a regression in Python 3 support introduced in 7de7bd407251 on the stable branch. We're so early in do_hgweb that I don't see any especially better choices than this. Differential Revision: https://phab.mercurial-scm.org/D3726

tests: fix printenv script on Python 3 Differential Revision: https://phab.mercurial-scm.org/D3725

cvsps: port changeset __repr__ to py3 This appears to be unused except as a debugging aid, as it didn't break until I added a debug print() in service of the previous fix. Sigh. Differential Revision: https://phab.mercurial-scm.org/D3724

cvsps: avoid comparison between None and a tuple in date sorting Avoids badness on Python 3. I had to figure out which entries in this object *could* be None experimentally, but I think I've got them all now. Differential Revision: https://phab.mercurial-scm.org/D3723

py3: buildbot informs me we have two new passing tests Differential Revision: https://phab.mercurial-scm.org/D3722

tests: fix test-patch.t on pickier /bin/sh implementations This is a graft of 0b39edeff033 and f44306940c94 from default because I'm tired of seeing the FreeBSD build be red on stable. See those revisions for details on what's going on here.

commandserver: close server's fds explicitly from a worker The forked worker does not need to accept connections from the server's socket fd. So let's just close them explicitly to avoid surprises. Differential Revision: https://phab.mercurial-scm.org/D3720

run-tests: restrict the test cases allowed characters Most test cases in core and in extension only use `A-Za-z0-9-` but a few tests in extensions also have a `.` in their test cases names. Also add a `_` to allow all kind of case naming format. Differential Revision: https://phab.mercurial-scm.org/D3721

templater: abstract truth testing to fix {if(list_of_empty_strings)} Non-empty list should always be True even if it's stringified to ''. Spotted by Martin von Zweigbergk.

show: stringify filtered list of tags before testing emptiness `names % "{ifeq(name, 'tip', '', name)}"}"` may return [''], which shouldn't be falsy, but it is thanks to a templater bug. This patch converts [''] to '' before testing truthness.

templatefilters: deprecate hgdate as {date|hgdate} is the default format

templater: make date wrapper support dot/map operations No idea if it will be useful, but it just works.

templater: introduce a wrapper for date tuple (BC) Strictly speaking, this is BC, but I believe the original string format (str(float(unixtime)) + str(int(tzoffset))) was just plain wrong.

templater: promote tomap() to an interface type I originally considered merging tomap() with itermaps()/getmember(), but decided to not. We might want to add support for chained map operations (e.g. {foo % func() % ...}), where func() will return a mappable object, and 'foo % func()' will be a mappedgenerator of mappable objects.

templater: rename mappable to hybriditem as it is the primary use case This frees up the name 'mappable' for new interface type.

serve: do not daemonize by default by --print-url Per discussion in D3649, -d/--daemon should be explicitly specified to avoid surprising result. Differential Revision: https://phab.mercurial-scm.org/D3703

mercurial: add .t files to .editorconfig The notable entry here prevents the stripping of trailing whitespace, which is important in .t tests. Differential Revision: https://phab.mercurial-scm.org/D3702

graft: add a new `--stop` flag to stop interrupted graft This patch adds a new flag `--stop` to `hg graft` command which stops the interrupted graft. The `--stop` flag takes back you to the last successful step i.e. it will keep your grafted commits, it will just clear the mergestate and interrupted graft state. The `--stop` is different from `--abort` flag as the latter also undoes all the work done till now which is sometimes not what the user wants. Suppose you grafted a lot of changesets, you encountered conflicts, you resolved them, did `hg graft --continue`, again encountered conflicts, continue, again encountered conflicts. Now you are tired of solving merge conflicts and want to resume this sometimes later. If you use the `--abort` functionality, it will strip your already grafted changesets, making you loose the work you have done resolving merge conflicts. A general goal related to this flag is to add this flag to `rebase` and `histedit` too. The evolve command already has this --stop flag. Tests are added for the new flag. .. feature:: `hg graft` now has a `--stop` flag to stop interrupted graft. Differential Revision: https://phab.mercurial-scm.org/D3668

graft: fix the help text to say `graft reapplies previous options` The options which graft accepts are: * rev * continue * edit * log * force * currentdate * currentuser * date * user * tool * dryrun Out of these following should be reapplied in case conflicts occur and user does `hg graft --continue`: * rev * log * force * currentuser * currentdate * date * user rev and force were already reapplied by --continue. Some recent refactors made graft to store the values of other options in graftstate and reapplies them at `hg graft --continue`. Now all the options which should be reapplied are taken care of, we can easily remove the help note which says that `--continue does not reapply earlier options` and help text saying that we reapplies earlier options. .. feature:: `hg graft --continue` now reapplies all the earlier options `hg graft` now preserves the value of all the flags and reapplies them in case of `hg graft --continue`. Differential Revision: https://phab.mercurial-scm.org/D3667

files: drop now-unnecessary filtering of removed dirstate files workingctx.matches() now filters out removed files. Differential Revision: https://phab.mercurial-scm.org/D3712

context: make workingctx.matches() filter our removed files (API) It seems surprising that workingctx.matches() includes files that had been removed in the working copy. The callers don't want that either (besides the `hg locate` that was changed in the previous patch). The only observable difference that I'm aware of is that `hg log -T 'wdir()' -r '{files(...)}'` will no longer include removed files (an improvement, IMO). That matches `hg files` (but does not match the deprecated `hg locate`). Differential Revision: https://phab.mercurial-scm.org/D3711

locate: explicitly use dirstate.matches() for working copy `hg locate` is odd in that it includes files that have been removed from the working copy. It relies on workingctx.matches() for that. I want to make workingctx.matches() not include removed files, so this patch makes `hg locate` handle the working copy differently instead. Differential Revision: https://phab.mercurial-scm.org/D3710

revsets: define a none() revset This can be useful for default values in corp-wide configuration (e.g. for commits to hide, which defaults to none()), and perhaps in scripts. I named it none() rather than empty() to match all(). Differential Revision: https://phab.mercurial-scm.org/D3713

statprof: fix save and load Fix these functions even if they don't are used at the moment. Thanks to Yuya Nishihara for spotting that.

perftemplating: abort on incompatible version This is what the other debug commands do. Thanks to Yuya Nishihara for pointing this out.

templater: fix string representation of wrapped None flatten() and stringify() skip None, which means wrappedvalue(None).show() must return '' instead of 'None'. This isn't a problem right now, but we'll encounter it once we start using wrapped types extensively.

templater: show slightly better hint on map operation error It was super hard to find where the templater went wrong if a keyword was evaluated to '' or None.

templater: inline unwraphybrid() flatten() is the solo user of this function.

templater: drop hybrid-ness on unwrapvalue() Proxy methods are no longer necessary as list/dict functions go through the wrapped interface.

templater: mark .keytype as a private attribute

templater: abstract ifcontains() over wrapped types This allows us to make .keytype private. There's a minor BC that a hybrid dict/list of keytype=None now strictly checks the type of the needle. For example, {ifcontains(rev, files)} no longer matches a file named "1" at the rev=1. I made this change for consistency with the get(dict, key) function. We can restore the old behavior by making keytype=bytes the default if desired.

templater: inline wraphybridvalue() All wraphybridvalue() calls are now handled in the hybrid class.

templater: abstract min/max away I'm not certain how many get*() functions I'll add to the wrapped types, but getmin() and getmax() will allow us to optimize a revset wrapper.

stringutil: fix prettyrepr() to not orphan foo=<...> line

debugwalk: pretty-print nested matcher While porting filesets to composition of matchers, I found the original one-liner was hard to read.

debugwalk: show matcher output only if -v/--verbose And drop "egrep -v". This matches the behavior of "hg debugrevspec".

stringutil: promote smartset.prettyformat() to utility function It will be used by debugwalk.

profiling: introduce a "profiling.time-track" option This option allows to switch the time used by the statistical profiler from cpu to real time. Our goal is to profile IO intensive operations using the mercurial profiler.

statprof: also gather wall time Gathering wall time information is useful for operations that are IO (files, network) intensive. For now we only expose it in the final summary.

perftemplating: allow to specify the template to test This is useful to test the evolution of rendering time for a given template.

perftemplating: stop going through the log command Only benchmark the rendering phase by moving steps outside of the timed function: * revisions resolution, * template parsing

perftemplating: move revision argument to flag only The revision we want to render are less important than the templates. We move revision specification behind the usual `--rev` flag.

perftemplating: drop usage of buffer The buffer can consume a lot of memory and change various internal behaviors. Writing to dev/null seems more appropriate when it comes to benchmark.

perftemplating: move template formating into its own function The lambda is not practical when time will come to alter the benchmark code.

perftemplating: fix the revisions argument Before this change, the argument passed to log where a string instead of a list of string. This meant only single character rev were supported... We now properly accepts argument of any length. We also make it possible to specify multiple REV arguments in the same go.

py3: make sure util.username() always returns bytes Differential Revision: https://phab.mercurial-scm.org/D3706

py3: add b'' prefix to make the regex bytes # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3705

phases: use "published" in the phase movement message Using "published" seems smoother than "became public" and more in line with the "phase.publishing" configuration.

phase: clarify the message about movement on command changeset The current message is a bit generic. Since we only print it for phase movement on changeset already common before the pull, we add "local" to the message in and attempt to clarify what changeset the phase movement affected.

fuzz: try and generate an interesting mpatch seed from a repo Differential Revision: https://phab.mercurial-scm.org/D3697

fuzz: structured helpers for creating mpatch seed corpus entries Maybe this is better. I'm not sold, honestly. Differential Revision: https://phab.mercurial-scm.org/D3696

fuzz: new fuzzer for the mpatch code Differential Revision: https://phab.mercurial-scm.org/D3695

run-tests: follow-up on the test-case format It turns out the original regex doesn't support real test cases names like the one Mercurial is using. Update the regex to being able to precisely select them on the command line. Differential Revision: https://phab.mercurial-scm.org/D3699

templater: resolve type of dict key in getmember() This seems more correct and is consistent with the future wrapped.contains() function, where a key type has to be resolved depending on a container type.

templater: promote getmember() to an interface of wrapped types

templater: move getdictitem() to hybrid class Since a raw dict will never be returned by evalwrapped(), we don't need to support d.get(key).

templater: add try-except stub to runmember() New wrapped interface will raise ParseError if the underlying object doesn't support dict-like lookup operation.

templater: do dict lookup over a wrapped object Dict/list lookup operations will be moved to a wrapped interface so that a returned element can inherit hybrid-ness automatically. wraphybridvalue() will be inlined.

merge with stable

graft: reuse the --log value passed initially in `hg graft --continue` (BC) We now stores the value of --log flag passed initially in the graftstate and reuse that value when doing `hg graft --continue` which is a nice behavior. The test updates demonstrate the fix. Since we now preserve the value by default, drop the mention of `--log` flag from the hint which we see after conflicts. Differential Revision: https://phab.mercurial-scm.org/D3662

graft: add test showing --continue not preserving --log passed earlier This patch shows that when we do `hg graft --continue`, the value of --log flag passed when initial graft was run was not used. Next patch will fix this and tests will help us in realising that. Differential Revision: https://phab.mercurial-scm.org/D3661

test-fuzz-targets: look for clang-6.0 binary as well Debian sid is still shipped with clang 4.0 by default. This allows me to run the test without replacing the system clang to clang-6.0.

test-fuzz-targets: rebuild executable silently This makes sure the build is not broken.

fuzz: fix "make clean" to pass even if no binaries built yet

fuzz: compile xdiff.cc with -std=c++17 Otherwise the build would fail as follows: In file included from xdiff.cc:13: ./fuzzutil.h:23:10: fatal error: 'third_party/absl/types/optional.h' file not found #include "third_party/absl/types/optional.h"

fuzz: fix use of undeclared function memcpy()

fuzz: fix the default make target

fuzz: expand variables by make Otherwise it wouldn't work unless CC and CXX were exported from the environment.

templater: always map over a wrapped object _checkeditermaps() is no longer necessary as the hgweb issue was resolved.

templater: consistently join() string-like object per character (BC) The old behavior was copied from join() of a lazy generator string, which was unified to the behavior of join() of a byte string by the previous patch. This patch fixes the mappable type to do the same.

templater: always join() over a wrapped object (BC) This is a behavior change in a sense that join() of a byte string is no longer "implementation dependent." Before, if a byte string was backed by a lazy generator, join() would concatenate each chunk with the specified separator, which seems wrong. The new behavior is always join() each byte. TypeError on join() over uniterable is also fixed.

templater: add wrapped types for pure non-list/dict values These wrapper types will allow us to get rid of some isinstance() business. A bytes object needs to support sequence-like operations (e.g. join(), ifcontains(), etc.) That's why we have two wrapper classes. Tests will be added later.

templater: unify unwrapvalue() with _unwrapvalue() All weird generators got removed from the hgweb codebase. We still have inconsistent behavior regarding join() of a byte string, which will be addressed later.

templater: inline unwrapvalue() The current unwrapvalue() will be superseded by _unwrapvalue(). Note that _unwrapvalue() can simply return thing.tovalue() if thing is a wrapped object. That's because tovalue() is guaranteed to not return a generator of strings.

run-tests: update the test case name format Manually typing parenthesis and spaces will be tedious when trying to launch a specific test case. I'm proposing a simpler format that is less hard to remember and type right. There was other possibilities envisaged like `::` or `!`, I think `#` is slight easier to type but I'm open to any suggestion on the new format. Differential Revision: https://phab.mercurial-scm.org/D3556

run-tests: add support for running specific test cases Differential Revision: https://phab.mercurial-scm.org/D3555

configitems: drop the deprecated alias for commands.update.check experimental.updatecheck was deprecated and should have been removed once 4.4 was released. Let's drop it now. Differential Revision: https://phab.mercurial-scm.org/D3679

graft: correct documentation about options can be reapplied Now options --user and --date can be reapplied after we hit a conflict. Differential Revision: https://phab.mercurial-scm.org/D3678

cmdutil: use internal separators when building the terse list Status uses internal separators, so this is more correct. See c974320d20b9 and 362096cfdb1f. A utility method is still needed, so that the paths starting with '/' aren't created when `self.path` is empty.

advanceboundary: add dryrun parameter Added logic to find those csets whose phase will be changed (when running without --dryrun) while advancing boundary and return those csets. Differential Revision: https://phab.mercurial-scm.org/D3671