Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Mar 2018 15:08:33 -0800] rev 36799
wireprotoserver: check permissions in main dispatch function
The permissions checking code merged from stable is out of place
in the refactored hgweb_mod module.
This commit moves the main call to wireprotoserver. We still have
some lingering code in hgweb_mod. This will get addressed later.
Differential Revision: https://phab.mercurial-scm.org/D2717
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Mar 2018 15:02:53 -0800] rev 36798
wireprotoserver: check if command available before calling it
The previous behavior was just plain wrong. I have no clue how it
landed. My guess is a merge conflict resolution gone wrong on my
end a few weeks ago.
Differential Revision: https://phab.mercurial-scm.org/D2716
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:43:17 -0600] rev 36797
py3: drop encoding.strio()
Its buffered nature makes TextIOWrapper unsuitable for temporarily wrapping
bytes I/O.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:42:37 -0600] rev 36796
ui: adjust Windows workaround to new _readline() code
It's only needed when rawinput() is called. Also made it Py3 compatible.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:38:53 -0600] rev 36795
ui: do not use rawinput() when we have to replace sys.stdin/stdout
See the inline comment for why. The current Python3 hack doesn't work if
more than one user inputs are expected because TextIOWrapper fills its
internal buffer at the first read() request.
Maybe we could write an unbuffered TextIOWrapper, but I don't want to make
things more complicated. Instead, this patch reinvents raw_input(' ') of
no readline support.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:32:26 -0600] rev 36794
ui: do not try readline support if fin/fout aren't standard streams
It's unlikely for a non-stdio stream to be a tty. Minimizing readline support
makes it much simpler to work around the unicode input() function of Python 3.
This also works on chg which duplicates client's tty to stdio fds.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:28:59 -0600] rev 36793
util: add public isstdin/isstdout() functions
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 03:05:49 -0600] rev 36792
ui: add debug commands to test interactive prompt
Interactive operations aren't easily covered by tests. So let's add commands
to test them manually.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:14:11 -0600] rev 36791
ui: inline util.bytesinput() into ui._readline()
Prepares for rework of Python 3 support, which is currently broken due to
read-ahead buffer of TextIOWrapper.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Mar 2018 02:05:25 -0600] rev 36790
hgk: stop using util.bytesinput() to read a single line from stdin
Appears that the stdio here is an IPC channel between hg and hgk (tk)
processes, which shouldn't need a fancy readline support.
Augie Fackler <augie@google.com> [Mon, 29 Aug 2016 10:42:58 -0400] rev 36789
bookmarks: test for exchanging long bookmark names (issue5165)
As far as I can tell a test for a long bookmark name never actually
got added when this was fixed. Let's document that a 300 byte bookmark
is something we're supporting (this patch started out life over a year
ago as a way for me to validate the problem, and I recently found it.)
I think the nonzero exits from the push operations are only because no
new changesets get exchanged. Please correct me if I'm wrong on that. :)
Differential Revision: https://phab.mercurial-scm.org/D2727
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 11:46:03 -0500] rev 36788
phabricator: follow-up phab auth improvements with backwards compat mode
We'll rip this out before we ship 4.6, but this gives people a window
to migrate.
Differential Revision: https://phab.mercurial-scm.org/D2703
Tom Prince <mozilla@hocat.ca> [Sat, 20 Jan 2018 02:41:10 -0700] rev 36787
phabricator: specify API tokens per host, rather than per repo
Differential Revision: https://phab.mercurial-scm.org/D1919
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 18:47:07 -0500] rev 36786
py3: drop b'' from generate-working-copy-states.py output
I could make everything bytes, but decoding filename seemed easier and we
don't have to worry about non-ascii filenames here.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 18:41:09 -0500] rev 36785
py3: make test-commit-multiple.t byte-safe
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 18:34:46 -0500] rev 36784
py3: fix type of default username
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 18:21:16 -0500] rev 36783
py3: read/write plain lock file in binary mode
A lock file shouldn't contain '\n', so this isn't a BC.
Augie Fackler <augie@google.com> [Mon, 05 Mar 2018 12:31:08 -0500] rev 36782
util: stop calling os.stat_float_times()
It had Python-wide side effects, and it disappears in 3.7.0.
As of this change, we're mostly working on 3.7.0b2. There are a few
worrying failures, mostly around regular expressions, but we'll have
to tackle those separately.
Differential Revision: https://phab.mercurial-scm.org/D2697
Augie Fackler <augie@google.com> [Mon, 05 Mar 2018 12:30:20 -0500] rev 36781
cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime
The latter is floating point by default, and we've been doing
os.stat_float_times(False). Unfortunately, os.stat_float_times was
removed between Python 3.7.0a1 and 3.7.0b2, so we have to stop using
it.
Differential Revision: https://phab.mercurial-scm.org/D2696
Augie Fackler <augie@google.com> [Mon, 05 Mar 2018 15:07:32 -0500] rev 36780
osutil: implement minimal __getitem__ compatibility on our custom listdir type
We previously declined to do this, but the removal of the deprecated
os.stat_float_times() method in Python 3.7 forces our hand.
Differential Revision: https://phab.mercurial-scm.org/D2695
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 21:14:24 -0500] rev 36779
hgweb: adapt to socket._fileobject changes in Python 3
Differential Revision: https://phab.mercurial-scm.org/D2688
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:20:24 -0500] rev 36778
debugcommands: fix some %r output with bytestr() wrappers
Almost fixes test-merge-tools.t. I think the remaining failure there
is due to some overspecified tempfile names.
Differential Revision: https://phab.mercurial-scm.org/D2675
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Mar 2018 11:00:17 -0800] rev 36777
tests: add test for issue 5494 but with --collapse
This was not fixed, so the test case currently demonstrates the
breakage.
Differential Revision: https://phab.mercurial-scm.org/D2714
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Mar 2018 10:55:57 -0800] rev 36776
tests: .hg/merge is a directory, so use `test -d`
This part of test-rebase-interrupts.t would have passed before the fix
in a580b2d65ded (rebase: make sure merge state is cleaned up for no-op
rebases (issue5494), 2017-05-18).
Differential Revision: https://phab.mercurial-scm.org/D2713
Martin von Zweigbergk <martinvonz@google.com> [Tue, 06 Mar 2018 14:29:20 -0800] rev 36775
rebase: only store collapse message once
The message is determined by the user passing --message or --log when
the rebase is started. There's no need to write it to a file for each
rebased commit; writing it once at the start of the rebase is enough.
Differential Revision: https://phab.mercurial-scm.org/D2712
Martin von Zweigbergk <martinvonz@google.com> [Tue, 06 Mar 2018 09:39:24 -0800] rev 36774
rebase: collapse two nested if-conditions
Also change the order since it feel to me like it's more about
--collapse than it is about --keep.
Differential Revision: https://phab.mercurial-scm.org/D2711
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Mar 2018 20:12:25 -0800] rev 36773
rebase: reduce scope of "dsguard" variables a bit
Differential Revision: https://phab.mercurial-scm.org/D2710
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Mar 2018 09:46:53 -0800] rev 36772
rebase: remove unused argument "state" from rebasenode()
Differential Revision: https://phab.mercurial-scm.org/D2709
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Mar 2018 10:31:01 -0800] rev 36771
rebase: delete obsolete internal "keepopen" option
The option was apparently introduced for use by the "pbranch"
extension, see f2558a8228be (rebase: add option to not commit after a
collapsing, 2010-02-07). However, it doesn't seem like it was ever
used by that extension (according to `hg grep` in a clone of
https://bitbucket.org/parren/hg-pbranch/), so let's delete it.
Differential Revision: https://phab.mercurial-scm.org/D2708
Rishabh Madan <rishabhmadan96@gmail.com> [Sun, 04 Mar 2018 00:25:58 +0530] rev 36770
releasenotes: allow notes for multiple directives in a single changeset
This problem was caught in da91e7309daf8ffc51bf3e6f4b2d8a16ef5af95a. This patch just makes sure there is no warning when we
encounter such a case.
Differential Revision: https://phab.mercurial-scm.org/D2254
Rishabh Madan <rishabhmadan96@gmail.com> [Sun, 04 Mar 2018 00:15:35 +0530] rev 36769
releasenotes: mention changeset with warning and abort
Output the changeset hash with the warning/abort message just to know where things messed up.
Differential Revision: https://phab.mercurial-scm.org/D2253
Rishabh Madan <rishabhmadan96@gmail.com> [Sat, 03 Mar 2018 23:47:22 +0530] rev 36768
releasenotes: replace abort with warning while parsing (issue5775)
During the 4.5 development cycle, the extension broke on two different changesets.
This fixes the issue by ensuring that it just throws a warning when it encounters
unexpected behaviour, instead of aborting.
Differential Revision: https://phab.mercurial-scm.org/D2255
Vincent Parrett <vincent@finalbuilder.com> [Wed, 07 Mar 2018 09:07:34 +1100] rev 36767
archival: fileit should not use atomictemp, causes performance regression
Differential Revision: https://phab.mercurial-scm.org/D2704
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 03 Mar 2018 18:55:43 -0500] rev 36766
perf: teach perfbdiff to call blocks() and to use xdiff
Differential Revision: https://phab.mercurial-scm.org/D2624
Jun Wu <quark@fb.com> [Tue, 06 Mar 2018 19:31:17 -0800] rev 36765
fuzz: fix xdiff build
Recent xdiff code cleanups removed some files and changed some structures.
Update fuzz code.
Differential Revision: https://phab.mercurial-scm.org/D2707
Jun Wu <quark@fb.com> [Tue, 06 Mar 2018 18:51:11 -0800] rev 36764
xdiff: remove xmerge related logic
hg has its own merge algorithm with flexible config options.
Differential Revision: https://phab.mercurial-scm.org/D2706
Jun Wu <quark@fb.com> [Tue, 06 Mar 2018 18:41:08 -0800] rev 36763
xdiff: remove xemit related logic
xemit handles "diff formatting and output" with options like context lines,
whether show function names, etc. That is handled more cleanly at a higher
level in hg.
Removing context line parameters would also make the trimming logic (D2686)
cleaner and more confident. See [1].
[1]: https://github.com/git/git/commit/d2f82950a9226ae1102a7a97f03440a4bf8c6c09
Differential Revision: https://phab.mercurial-scm.org/D2705
Jun Wu <quark@fb.com> [Sun, 04 Mar 2018 00:17:49 -0800] rev 36762
xdiff: remove unused structure, functions, and constants
`bdiffparam_t` is unused. `xdl_fall_back_diff` is no longer used after
D2573. `XDL_MMB_READONLY`, `XDL_MMF_ATOMIC` are unused. `XDL_BDOP*` are
unused since there is no xdiff binary diff algorithm. `anchors` feature is
not used. It's also relatively new in git.
Differential Revision: https://phab.mercurial-scm.org/D2684
Jun Wu <quark@fb.com> [Sun, 04 Mar 2018 00:07:04 -0800] rev 36761
xdiff: remove whitespace related feature
In Mercurial, whitespace related handling are done at a higher level than
the low-level diff algorithm so "ignore spaces". So it's not used by mdiff.
Some of the upcoming optimizations would be more difficult with whitespace
related features kept. So let's remove them.
Differential Revision: https://phab.mercurial-scm.org/D2683
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Mar 2018 14:32:14 -0800] rev 36760
merge with stable
There were a handful of merge conflicts in the wire protocol code due
to significant refactoring in default. When resolving the conflicts,
I tried to produce the minimal number of changes to make the incoming
security patches work with the new code.
I will send some follow-up commits to get the security patches better
integrated into default.
Kevin Bullock <kbullock@ringworld.org> [Tue, 06 Mar 2018 13:19:54 -0600] rev 36759
Added signature for changeset 8bba684efde7
Kevin Bullock <kbullock@ringworld.org> [Tue, 06 Mar 2018 13:19:52 -0600] rev 36758
Added tag 4.5.2 for changeset 8bba684efde7
Kevin Bullock <kbullock+mercurial@ringworld.org> [Tue, 06 Mar 2018 13:17:07 -0600] rev 36757
merge with security patches
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 18 Feb 2018 17:20:38 -0800] rev 36756
hgweb: always perform permissions checks on protocol commands (BC) (SEC)
Previously, the HTTP request handling code would only perform
permissions checking on a wire protocol command if that wire protocol
command defined its permissions / operation type. This meant that
commands (possibly provided by extensions) not defining their
operation type would bypass permissions check. This could lead
to exfiltration of data from servers and mutating repositories that
were supposed to be read-only.
This security issue has been present since the permissions table
was introduced by d3147b4e3e8a in 2008.
This commit changes the behavior of the HTTP server to always
perform permissions checking for protocol requests. If an
explicit permission for a wire protocol command is not defined,
the server assumes the command can be used for writing and
governs access accordingly.
.. bc::
Wire protocol commands not defining their operation type in
``wireproto.PERMISSIONS`` are now assumed to be used for
"push" operations and access control to run those commands
is now enforced accordingly.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 20 Feb 2018 18:55:58 -0800] rev 36755
wireproto: check permissions when executing "batch" command (BC) (SEC)
For as long as the "batch" command has existed (introduced by
bd88561afb4b and first released as part of Mercurial 1.9), that command
(like most wire commands introduced after 2008) lacked an entry in
the hgweb permissions table. And since we don't verify permissions if
an entry is missing from the permissions table, this meant that
executing a command via "batch" would bypass all permissions
checks.
The security implications are significant: a Mercurial HTTP server
would allow writes via "batch" wire protocol commands as long as
the HTTP request were processed by Mercurial and the process running
the Mercurial HTTP server had write access to the repository. The
Mercurial defaults of servers being read-only and the various web.*
config options to define access control were bypassed.
In addition, "batch" could be used to exfiltrate data from servers
that were configured to not allow read access.
Both forms of permissions bypass could be mitigated to some extent
by using HTTP authentication. This would prevent HTTP requests from
hitting Mercurial's server logic. However, any authenticated request
would still be able to bypass permissions checks via "batch" commands.
The easiest exploit was to send "pushkey" commands via "batch" and
modify the state of bookmarks, phases, and obsolescence markers.
However, I suspect a well-crafted HTTP request could trick the server
into running the "unbundle" wire protocol command, effectively
performing a full `hg push` to create new changesets on the remote.
This commit plugs this gaping security hole by having the "batch"
command perform permissions checking on each sub-command that is
being batched. We do this by threading a permissions checking
callable all the way to the protocol handler. The threading is a
bit hacky from a code perspective. But it preserves API compatibility,
which is the proper thing to do on the stable branch.
One of the subtle things we do is assume that a command with an
undefined permission is a "push" command. This is the safest thing to
do from a security perspective: we don't want to take chances that
a command could perform a write even though the server is configured
to not allow writes.
As the test changes demonstrate, it is no longer possible to bypass
permissions via the "batch" wire protocol command.
.. bc::
The "batch" wire protocol command now enforces permissions of
each invoked sub-command. Wire protocol commands must define
their operation type or the "batch" command will assume they
can write data and will prevent their execution on HTTP servers
unless the HTTP request method is POST, the server is configured
to allow pushes, and the (possibly authenticated) HTTP user is
authorized to perform a push.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 20 Feb 2018 18:54:27 -0800] rev 36754
wireproto: declare operation type for most commands (BC) (SEC)
The permissions model of hgweb relies on a dictionary to declare
the operation associated with each command - either "pull" or
"push." This dictionary was established by d3147b4e3e8a in 2008.
Unfortunately, we neglected to update this dictionary as new
wire protocol commands were introduced.
This commit defines the operations of most wire protocol commands
in the permissions dictionary. The "batch" command is omitted because
it is special and requires a more complex solution.
Since permissions checking is skipped unless a command has an entry in
this dictionary (this security issue will be addressed in a subsequent
commit), the practical effect of this change is that various wire
protocol commands now HTTP 401 if web.deny_read or web.allow-pull,
etc are set to deny access. This is reflected by test changes. Note
how various `hg pull` and `hg push` operations now fail before
discovery. (They fail during the initial "capabilities" request.)
This change fixes a security issue where built-in wire protocol
commands would return repository data even if the web config were
configured to deny access to that data.
I'm on the fence as to whether we should HTTP 401 the capabilities
request. On one hand, it can expose repository metadata and can tell
callers things like what version of Mercurial the server is running.
On the other hand, a client may need to know the capabilities in order
to authenticate in a follow-up request. It appears that Mercurial
clients handle the HTTP 401 on *any* protocol request, so we should
be OK sending a 401 for "capabilities." But if this causes problems,
it should be possible to allow "capabilities" to always work.
.. bc::
Various read-only wire protocol commands now return HTTP 401
Unauthorized if the hgweb configuration denies read/pull access to
the repository.
Previously, various wire protocol commands would still work and
return data if read access was disabled.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 20 Feb 2018 18:53:39 -0800] rev 36753
wireproto: move command permissions dict out of hgweb_mod
The operation type associated with wire protocol commands is supposed
to be defined in a dictionary so it can be used for permissions
checking.
Since this metadata is closely associated with wire protocol commands
themselves, it makes sense to define it in the same module where
wire protocol commands are defined.
This commit moves hgweb_mod.perms to wireproto.PERMISSIONS and
updates most references in the code to use the new home. The old
symbol remains an alias for the new symbol. Tests pass with the
code pointing at the old symbol. So this should be API compatible
for extensions.
As part of the code move, we split up the assignment to the dict
so it is next to the @wireprotocommand. This reinforces that a
@wireprotocommand should have an entry in this dict.
In the future, we'll want to declare permissions as part of the
@wireprotocommand decorator. But this isn't appropriate for the
stable branch.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 20 Feb 2018 19:09:01 -0800] rev 36752
tests: comprehensively test HTTP server permissions checking
We didn't have test coverage for numerous web.* config options. We
add that test coverage.
Included in the tests are tests for custom commands. We have commands
that are supposedly read-only and perform writes and a variation of
each that does and does not define its operation type in
hgweb_mod.perms.
The tests reveal a handful of security bugs related to permissions
checking. Subsequent commits will address these security bugs.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 18 Feb 2018 10:40:49 -0800] rev 36751
tests: extract HTTP permissions tests to own test file
We're about to implement a lot more coverage of the permissions
mechanism. In preparation for that, establish a new test file
to hold permissions checks.
As part of this, we inline the important parts of the "req" helper
function.
Kevin Bullock <kbullock@ringworld.org> [Tue, 06 Mar 2018 13:08:00 -0600] rev 36750
Added signature for changeset 369aadf7a326
Kevin Bullock <kbullock@ringworld.org> [Tue, 06 Mar 2018 13:07:58 -0600] rev 36749
Added tag 4.5.1 for changeset 369aadf7a326
Jun Wu <quark@fb.com> [Tue, 13 Feb 2018 11:35:32 -0800] rev 36748
revlog: resolve lfs rawtext to vanilla rawtext before applying delta
This happens when a LFS delta base gets a non-LFS delta from another client.
In that case, the LFS delta base needs to be converted to non-LFS version
before applying the delta.
Differential Revision: https://phab.mercurial-scm.org/D2069
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 21:16:36 -0500] rev 36747
sslutil: some more forcebytes() on some exception messages
At this point, test-https.t no longer dumps tracebacks
everywhere. Instead, we get some results that look like we're not
adequately finding things in hg's configuration, which should be
manageable (if somewhat annoying to find and fix.)
Differential Revision: https://phab.mercurial-scm.org/D2690
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 21:15:37 -0500] rev 36746
sslutil: sslcontext needs the cipher name as a sysstr
Differential Revision: https://phab.mercurial-scm.org/D2689
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 18:03:55 -0500] rev 36745
sslutil: lots of unicode/bytes cleanup
In general, we handle hostnames as bytes, except where Python forces
them to be unicodes.
This fixes all the tracebacks I was seeing in test-https.t, but
there's still some ECONNRESET weirdness that I can't hunt down...
Differential Revision: https://phab.mercurial-scm.org/D2687
Jun Wu <quark@fb.com> [Tue, 13 Feb 2018 11:35:32 -0800] rev 36744
revlog: do not use delta for lfs revisions
This is similar to what we have done for changegroups. It is needed to make
sure the delta application code path can assume deltas are always against
vanilla (ex. non-LFS) rawtext so the next fix becomes possible.
Differential Revision: https://phab.mercurial-scm.org/D2068
Jun Wu <quark@fb.com> [Tue, 06 Feb 2018 19:08:25 -0800] rev 36743
changegroup: do not delta lfs revisions
There is no way to distinguish whether a delta base is LFS or non-LFS.
If the delta is against LFS rawtext, and the client trying to apply it has
the base revision stored as fulltext, the delta (aka. bundle) will fail to
apply.
This patch forbids using delta for LFS revisions in changegroup so bad
deltas won't be transmitted.
Note: this does not solve the problem entirely. It solves LFS delta applying
to non-LFS base. But the other direction: non-LFS delta applying to LFS base
is not solved yet.
Differential Revision: https://phab.mercurial-scm.org/D2067
Jun Wu <quark@fb.com> [Tue, 06 Feb 2018 16:08:57 -0800] rev 36742
lfs: add a test showing bundle application could be broken
When a bundle containing LFS delta uses non-LFS delta-base, or vice-versa,
the bundle will fail to apply.
Differential Revision: https://phab.mercurial-scm.org/D2066
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Mar 2018 20:22:34 -0500] rev 36741
debugwireproto: handle unimplemented util.poll() for Windows
This is the same logic used in sshpeer.doublepipe. It doesn't completely fix
test-ssh-proto{,-unbundle}.t ("read(-1) -> X" is changed to "read(X) -> X", the
order of some lines are changed, and abort messages seem to be missing), but it
cuts down a ton on the failure spew.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:55:51 -0500] rev 36740
py3: byte-stringify test-blackbox.t
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:54:14 -0500] rev 36739
py3: byte-stringify blackbox-readonly-dispatch.py
# skip-blame because just adding some b''
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:50:35 -0500] rev 36738
py3: make blackbox-readonly-dispatch.py use ui instead of print()
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:48:17 -0500] rev 36737
py3: fix int formatting of "incoming changes" log
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:48:01 -0500] rev 36736
largefiles: use %d instead of %s to process ints
Differential Revision: https://phab.mercurial-scm.org/D2677
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:11:15 -0500] rev 36735
transaction: fix an error string with bytestr() on a repr()d value
Fixes test-rollback.t on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D2674
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:23:10 -0500] rev 36734
py3: work around comparison between int and None in tagmerge
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:13:46 -0500] rev 36733
py3: do not mutate dict while iterating in tagmerge
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 16:01:18 -0500] rev 36732
py3: fix type of ui.configitems(ignoresub=True) result
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 15:53:10 -0500] rev 36731
py3: don't use str() to stringify pushloc
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 15:26:26 -0500] rev 36730
py3: byte-stringify test-config.t and test-config-env.py
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 15:24:45 -0500] rev 36729
py3: use startswith() instead of slicing to detect leading whitespace
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:06:47 -0500] rev 36728
archival: use py3 friendly replacements for chr() and long()
Differential Revision: https://phab.mercurial-scm.org/D2673
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:06:27 -0500] rev 36727
archival: ensure file mode for gzipfile is sysstr
Differential Revision: https://phab.mercurial-scm.org/D2672
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:06:10 -0500] rev 36726
archival: fix a missing r'' on a kwargs check
# skip-blame just an r prefix
Differential Revision: https://phab.mercurial-scm.org/D2671
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 16:05:44 -0500] rev 36725
py3: more passing tests (ten this time)
Differential Revision: https://phab.mercurial-scm.org/D2670
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 15:55:55 -0500] rev 36724
util: fix unsafe url abort with bytestr() on url
Differential Revision: https://phab.mercurial-scm.org/D2669
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Mar 2018 15:16:42 -0500] rev 36723
xdiff: fix builds on Windows
This works on my ancient Fedora system too, without warnings. There are,
however, warnings about various 64 to 32 bit conversions on Windows that need to
be examined.
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Mar 2018 01:08:40 +0530] rev 36722
py3: use bytes() instead of str()
Differential Revision: https://phab.mercurial-scm.org/D2665
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Mar 2018 01:05:54 +0530] rev 36721
py3: use b"%d" instead of str() to convert integers to bytes
Differential Revision: https://phab.mercurial-scm.org/D2664
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Mar 2018 01:03:23 +0530] rev 36720
py3: use util.forcebytestr to convert testedwith value to bytes
Bad extensions can put anything in testedwith so we should use util.forcebytestr
here.
Differential Revision: https://phab.mercurial-scm.org/D2663
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 22:33:59 +0530] rev 36719
py3: use pycompat.bytestr instead of str
Differential Revision: https://phab.mercurial-scm.org/D2648
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 07:40:21 -0800] rev 36718
setdiscovery: avoid a Yoda condition
Differential Revision: https://phab.mercurial-scm.org/D2646
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 07:40:11 -0800] rev 36717
setdiscovery: remove unnecessary sample size limiting
Both _takequicksample() and _takefullsample() already limit their
result to the request size, so there's no need to let the caller do
that again.
Differential Revision: https://phab.mercurial-scm.org/D2645
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 07:39:46 -0800] rev 36716
setdiscovery: remove initialsamplesize from a condition
It seems more direct to compare the actual sample size. That way we
can change the sample taken earlier in the code without breaking the
condition.
Differential Revision: https://phab.mercurial-scm.org/D2644
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 07:37:08 -0800] rev 36715
setdiscovery: back out changeset 5cfdf6137af8 (issue5809)
As explained in the bug report, this commit caused a performance
regression. The problem occurs when the local repo has very many
heads. Before 5cfdf6137af8, we used to get the remote's list of heads
and if these heads mostly overlapped with the local repo's heads, we
would mark these common heads as common, which would greatly reduce
the size of the set of undecided nodes.
Note that a similar problem existed before 5cfdf6137af8: If the local
repo had very many heads and the server just had a few (or many heads
from a disjoint set), we would do the same kind of slow discovery as
we would with 5cfdf6137af8 in the case where local and remote repos
share a large set of common nodes.
For now, we just back out 5cfdf6137af8. We should improve the
discovery in the "local has many heads, remote has few heads" case,
but let's do that after backing this out.
Differential Revision: https://phab.mercurial-scm.org/D2643
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 13:04:12 -0500] rev 36714
hgweb: fix up trailing slash detection on Python 3
Fixes a couple of hgweb tests.
Differential Revision: https://phab.mercurial-scm.org/D2661
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 13:03:22 -0500] rev 36713
hgweb: convert req.form to bytes for all keys and values
This is just going to be a lot cleaner for our internals.
Differential Revision: https://phab.mercurial-scm.org/D2660
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:33:15 -0500] rev 36712
templater: show repr of string we're rejecting
I feel like this should make it a little easier to hunt down problems.
Differential Revision: https://phab.mercurial-scm.org/D2659
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:21:01 -0500] rev 36711
webutil: some %d instead of %s love on ints
Differential Revision: https://phab.mercurial-scm.org/D2658
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:17:02 -0500] rev 36710
py3: whitelist three more cases
Differential Revision: https://phab.mercurial-scm.org/D2657
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:08:53 -0500] rev 36709
archival: our filenames are bytes, not strs
Differential Revision: https://phab.mercurial-scm.org/D2656
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:08:37 -0500] rev 36708
archival: tar file modes need to be sysstrs
Differential Revision: https://phab.mercurial-scm.org/D2655
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 12:08:19 -0500] rev 36707
archival: fsdecode paths before passing to tar or zip objects
Both of these traffic in unicodes for filenames on Python 3, and
inspection of the tarfile module shows that it uses the filesystem
encoding, so fsdecode is the right choice.
Differential Revision: https://phab.mercurial-scm.org/D2654
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 05:15:24 +0530] rev 36706
py3: add b'' prefixes in tests/test-minirst.py
# skip-blame because just b'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D2653
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 22:40:33 +0530] rev 36705
py3: make sure __repr__ returns a str
# skip-blame because just r'' prefix
Differential Revision: https://phab.mercurial-scm.org/D2652
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 22:40:08 +0530] rev 36704
py3: make sure regular expressions are bytes
# skip-blame because just b'' prefix
Differential Revision: https://phab.mercurial-scm.org/D2651
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 05:53:59 +0530] rev 36703
py3: use bytes instead of str to make sure we use bytes internally
Differential Revision: https://phab.mercurial-scm.org/D2650
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Mar 2018 22:37:41 +0530] rev 36702
py3: use util.forcebytestr instead of str to convert error messages
Differential Revision: https://phab.mercurial-scm.org/D2649
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Mar 2018 23:49:39 -0500] rev 36701
lock: block signal interrupt while making a lock file
On Windows where symlink isn't supported, util.makelock() could leave an empty
file if interrupted immediately after os.open(). This empty lock never dies
as it has no process id recorded.
ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
# an interrupt may occur here
os.write(ld, info)
os.close(ld)
This was a long-standing bug of TortoiseHg which runs a command-server and
kills it by CTRL_C_EVENT, reported by random Windows users.
https://bitbucket.org/tortoisehg/thg/issues/4873/#comment-43591129
At first, I tried to fix makelock() to clean up a stale lock file, which
turned out to be hard because any instructions may be interrupted by a
signal.
ld = None
try:
# CALL_FUNCTION # os.open(...)
# an interrupt may occur here
# STORE_FAST # ld = ...
ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
os.write(ld, info)
...
return True
except:
if ld:
...
os.unlink(pathname)
return False
So I decided to block signals by temporarily replacing the signal handlers
so makelcok() and held = 1 will never be interrupted.
Many thanks to Fernando Najera for investigating the issue.
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 09:40:12 -0500] rev 36700
fuzz: add some more docs about building/running fuzzers
Differential Revision: https://phab.mercurial-scm.org/D2635
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 11:49:33 -0500] rev 36699
util: also silence py3 warnings from codec module
Fixes warnings like this:
+ mercurial/util.py:2446: DeprecationWarning: invalid escape sequence '\d'
+ return codecs.escape_decode(s)[0]
Differential Revision: https://phab.mercurial-scm.org/D2642
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 14:53:57 -0500] rev 36698
test-annotate: set stdin and stdout to binary to get CR unmodified
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 13:19:05 -0500] rev 36697
test-annotate: rewrite sed with some python
I hope this will fix the test failure seen on FreeBSD and Windows.
Kyle Lippincott <spectral@google.com> [Thu, 15 Feb 2018 18:05:58 -0800] rev 36696
docs: small fixes for profiling.nested and the overall description
- profiling.nested defaults to 0, not 5
- profiling is not always done with lsprof
Differential Revision: https://phab.mercurial-scm.org/D2641
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 10:20:41 -0500] rev 36695
scmutil: fix oversight in b76248e51605c6 where I forgot to use msg
Thanks to Yuya for spotting my mistake.
Differential Revision: https://phab.mercurial-scm.org/D2636
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 10:23:07 -0500] rev 36694
hghave: remove unused "as ex" in exception block
I overlooked this when removing a debug print in another change.
Differential Revision: https://phab.mercurial-scm.org/D2637
Augie Fackler <augie@google.com> [Sat, 03 Mar 2018 18:33:10 -0500] rev 36693
tests: port test-log to Python 3
Required some porting to >>> inline Python instead of using heredocs
into $PYTHON.
Differential Revision: https://phab.mercurial-scm.org/D2621
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Mar 2018 19:12:47 -0500] rev 36692
py3: make gettext domain a system string
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Mar 2018 07:03:50 -0500] rev 36691
templater: fix position of terminator character in error message
Since a template expression starts after the '{' character, the expression
should be considered to end immediately before the terminator '{'.
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 10:42:51 -0500] rev 36690
merge with stable
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Mar 2018 22:29:24 -0500] rev 36689
test-subrepo: glob away an unstable hash
This is the instability mentioned at the beginning of the series. I don't like
hiding it, but I don't want to sit on a fix for a user reported problem while
trying to figure this out.
The instability seems related to the cset with a .hgsub with a remote URL.
(There's very little existing remote URL subrepo testing.)
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Mar 2018 11:37:00 -0500] rev 36688
subrepo: activate clone pooling to enable sharing with remote URLs
This is the easiest way to ensure that repositories with remote subrepo
references can share the subrepos, consistent with how local subrepos can be
shared.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Mar 2018 11:13:00 -0500] rev 36687
subrepo: don't attempt to share remote sources (issue5793)
Untangling _abssource() to resolve the new subrepo relative to the shared
parent's share path, and then either sharing from there (if it exists), or
cloning to that location and then sharing, is probably more than should be
attempted on stable. Absolute subrepo references are discouraged, so for now,
this resumes the behavior prior to 68e0bcb90357 of cloning the absolute subrepo
locally.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Feb 2018 00:29:27 -0500] rev 36686
test-subrepo: demonstrate problems with subrepo sharing and absolute paths
This affects remote paths in .hgsub, as well as clone pooling from a remote
source.
For reasons unknown, there are stability issues with the relative-path.t tests.
If run as a single test, it is stable. If run with --loop, or with -jX for X>1,
the hash of the parent repo changes. I'm seeing this on both Windows and Fedora
26. I added an `hg log --debug`, and the manifest hash changes, but I have no
idea why.
Ryan McElroy <rmcelroy@fb.com> [Sat, 03 Mar 2018 15:31:37 -0800] rev 36685
revsetlang: add a hint for more useful parse errors
This logic is largely based on the similar logic added to template error
messages in D2608 and D2609, but with a few tweaks based on how revsets
actually work.
Differential Revision: https://phab.mercurial-scm.org/D2619
Ryan McElroy <rmcelroy@fb.com> [Sat, 03 Mar 2018 11:07:46 -0800] rev 36684
setup: ignore extension load failures when finding working hg
Previously, `make local` would fail if any extension was not properly loading.
Differential Revision: https://phab.mercurial-scm.org/D2589
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Mar 2018 00:35:59 -0500] rev 36683
profile: colorize output on Windows
Previously, the ANSI codes were printed to the screen, throwing off the
alignment. We could probably do this unconditionally, but it's kind of a hack,
so I figured I'd limit the scope.
Kevin Bullock <kbullock+mercurial@ringworld.org> [Sat, 03 Mar 2018 19:02:50 -0500] rev 36682
dispatch: don't clamp the range of the exit code twice
We already limit the range to (0, 255) in the call to sys.exit(). The
duplicated operation can't possibly be hurting us, but let's clean it up
to avoid confusion.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Mar 2018 23:29:40 -0500] rev 36681
bdiff: avoid pointer arithmetic on void*
MSVC 2008 complains:
mercurial/cext/bdiff.c(106) : error C2036: 'void *' : unknown size
mercurial/cext/bdiff.c(107) : error C2036: 'void *' : unknown size
Maybe it's a gcc extension?
https://stackoverflow.com/questions/37460579/error-c2036-void-unknown-size
Augie Fackler <augie@google.com> [Sat, 03 Mar 2018 19:26:30 -0500] rev 36680
fuzz: add a quick README to try and document how to test new fuzzers
Differential Revision: https://phab.mercurial-scm.org/D2633