Gregory Szorc <gregory.szorc@gmail.com> [Mon, 09 Apr 2018 12:19:37 -0700] rev 37624
futures: get rid of extend_path
This is used so mutliple directories can provide a package. We don't
need it when vendoring.
Differential Revision: https://phab.mercurial-scm.org/D3262
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 14:48:24 -0700] rev 37623
thirdparty: vendor futures 3.2.0
Python 3 has a concurrent.futures package in the standard library
for representing futures. The "futures" package on PyPI is a backport
of this package to work with Python 2.
The wire protocol code today has its own future concept for handling
of "batch" requests. The frame-based protocol will also want to
use futures.
I've heavily used the "futures" package on Python 2 in other projects
and it is pretty nice. It even has a built-in thread and process pool
for running functions in parallel. I've used this heavily for concurrent
I/O and other GIL-less activities.
The existing futures API in the wire protocol code is not as nice as
concurrent.futures. Since concurrent.futures is in the Python standard
library and will presumably be the long-term future for futures in our
code base, let's vendor the backport so we can use proper futures today.
# no-check-commit because of style violations
Differential Revision: https://phab.mercurial-scm.org/D3261
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 15:05:49 +0530] rev 37622
py3: make sure decode() first argument is str
Uses pycompat.sysstr() to make sure we uses bytes on Python 2 and unicodes on
Python 3.
Differential Revision: https://phab.mercurial-scm.org/D3279
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 12 Apr 2018 23:14:38 -0700] rev 37621
patch: make extract() a context manager (API)
Previously, this function was creating a temporary file and relying
on callers to unlink it. Yuck.
We convert the function to a context manager and tie the lifetime of
the temporary file to that of the context manager. This changed
indentation not only from the context manager, but also from the
elination of try blocks. It was just easier to split the heart of
extract() into its own function.
The single consumer of this function has been refactored to use it as
a context manager. Code for cleaning up the file in tryimportone()
has also been removed.
.. api::
``patch.extract()`` is now a context manager. Callers no longer have
to worry about deleting the temporary file it creates, as the file is
tied to the lifetime of the context manager.
Differential Revision: https://phab.mercurial-scm.org/D3306
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 12 Apr 2018 23:06:27 -0700] rev 37620
cmdutil: pass in parsed patch to tryimportone() (API)
Previously, we parsed the patch in tryimportone(). This assumes the
input is in a patch format that needs to be parsed. We want to support
feeding in data from other formats. So let's let the caller handle the
parsing.
One wonky thing about patch parsing is that patch.extract() creates
a temp file to hold the diffs and it is up to tryimportone() to
unlink that temp file. I'll improve this in a subsequent commit.
Differential Revision: https://phab.mercurial-scm.org/D3305
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 12 Apr 2018 20:42:42 -0700] rev 37619
stringutil: support more types with pprint()
bytearray wasn't working. Integers and floats were not being
formatted.
I /think/ %f is portable across both Python 2 and 3, as it should
default to 6 decimal points on each.
Differential Revision: https://phab.mercurial-scm.org/D3302
Augie Fackler <augie@google.com> [Thu, 12 Apr 2018 14:27:13 -0400] rev 37618
fix: port most of the way to python 3
Only most of the way because we now have to decide: if we want to keep
the current .format() interface for the config in hgrc, we have to use
unicodes to do formatting in Python 3, rather than bytes. I'm
basically fine with that, so a follow-up patch will do so.
Differential Revision: https://phab.mercurial-scm.org/D3300
Matt Harbison <matt_harbison@yahoo.com> [Thu, 12 Apr 2018 17:24:55 -0700] rev 37617
lfs: teach the blob server to handle --prefix
Matt Harbison <matt_harbison@yahoo.com> [Thu, 05 Apr 2018 15:42:40 -0400] rev 37616
hgweb: fallback to checking wsgireq.env for REPO_NAME for 3rd party hosting
Starting with d7fd203e36cc, SCM Manager began to 404 any repository access.
What's happening is that it is generating a python script that creates an hgweb
application (not hgwebdir), and launches hgweb via wsgicgi. It must be setting
REPO_NAME in the process environment before launching this script, which gets
picked up and put into wsgireq.env when wsgicgi launches the hgweb application.
>From there, other variables (notably 'apppath' and 'dispatchpath') are
constructed differently.
d7fd203e36cc^ (working):
apppath: /hg/eng/devsetup
dispatchpath:
pathinfo: /eng/devsetup
reponame: eng/devsetup
d7fd203e36cc:
apppath: /hg
dispatchpath: eng/devsetup
pathinfo: /eng/devsetup
reponame: None
REPO_NAME: eng/devsetup
Rather than having an existing installation break when Mercurial is upgraded,
just resume checking the environment. I have no idea how many other hosting
solutions would break without restoring this.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 12:51:09 -0700] rev 37615
peer: scatter module to the wind (API)
peer.py hardly contained any code. The code it did contain was
generic to the version 1 peer interface or specific to the
local repository peer.
So code has been moved to wireprotov1peer and localrepo, as
appropriate.
Differential Revision: https://phab.mercurial-scm.org/D3260
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 12:49:08 -0700] rev 37614
wireproto: move version 1 peer functionality to standalone module (API)
wireproto.py contains code for both the client and the server. There
*should* be a somewhat strong separation between the two.
This commit extracts the client-side code from wireproto.py into a new
module - wireprotov1peer.
Differential Revision: https://phab.mercurial-scm.org/D3259
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 10:51:38 -0700] rev 37613
wireproto: move gboptsmap to wireprototypes and rename (API)
This is also shared between client and server and will need to
exist in a shared module when that code is split into different
modules.
Differential Revision: https://phab.mercurial-scm.org/D3258
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 10:50:58 -0700] rev 37612
wireproto: move value encoding functions to wireprototypes (API)
These functions should live in the same place. I plan to separate
client from server code in upcoming commits. wireprototypes is
where we are putting shared code like this.
Differential Revision: https://phab.mercurial-scm.org/D3257
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 10 Apr 2018 19:09:35 -0700] rev 37611
httppeer: basic implementation of capabilities interface
This is a bit crude. The capabilities mechanism for version 2 of
the wire protocol is a bit different from version 1. And code
in core is relying on strings passed to capable() matching strings
advertised by the "capabilities" wire protocol command. I may
refactor the internal checking mechanism to be a bit more
abstract or based on interfaces. Time will tell...
Differential Revision: https://phab.mercurial-scm.org/D3256
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 10 Apr 2018 18:47:09 -0700] rev 37610
repository: split capabilities methods into separate interface
So we can implement them without having to implement support for
every command.
Differential Revision: https://phab.mercurial-scm.org/D3255
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 11 Apr 2018 11:03:45 -0700] rev 37609
httppeer: implement ipeerconnection
This is low hanging fruit. We might as well start somewhere.
Differential Revision: https://phab.mercurial-scm.org/D3254
Augie Fackler <augie@google.com> [Thu, 12 Apr 2018 13:25:54 -0400] rev 37608
py3: whitelist another six passing tests
Differential Revision: https://phab.mercurial-scm.org/D3286
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 17:43:00 -0400] rev 37607
py3: whitelist another nine passing tests
Differential Revision: https://phab.mercurial-scm.org/D3253
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 14:01:37 -0400] rev 37606
hgweb: use our forked wsgiheaders module instead of stdlib one
Now we use bytes for headers, rather than native strings.
Differential Revision: https://phab.mercurial-scm.org/D2854
Augie Fackler <augie@google.com> [Thu, 12 Apr 2018 10:00:09 -0700] rev 37605
wsgiheaders: import a bytes-ified fork of wsgiref.headers from cpython@46f5072
This will let us restore Python 3 compatibility for tests that do http things.
Differential Revision: https://phab.mercurial-scm.org/D3245
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 23:13:55 +0900] rev 37604
export: enable formatter support (API)
This change is basically the same as "hg cat". A formatter object is created
by caller.
.. api::
``cmdutil.export()`` takes a formatter as an argument.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 22:59:49 +0900] rev 37603
export: extract function to write patch to file object (API)
This is common use case of cmdutil.export(), and we wouldn't want to handle
formatter thingy everywhere.
.. api::
The ``fp`` argument is removed from ``cmdutil.export()``. Use
``cmdutil.exportfile()`` instead.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 22:39:43 +0900] rev 37602
export: port _exportsingle() to formatter
Pass 'fm' instead of 'write', and use fm.plain(), fm.write(), etc. instead.
The callers will be updated later.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 22:08:55 +0900] rev 37601
export: serialize revisions to be exported per destination file
Prepares for porting to the formatter API, where we can't simply append
to existing files because JSON can't be streamed for example.
The modemap hack is removed since cmdutil.export() was the only user.
I also made the destination filename printed only once.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 21:56:02 +0900] rev 37600
export: split cmdutil.export() to single-file and maybe-multiple-files cases
Porting "hg export" to formatter is a bit hard because cmdutil.export() may
append to files if the fntemplate is specified. This patch splits the hard
part from the trivial case.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 21:47:17 +0900] rev 37599
export: remove unused argument 'rev' from _exportsingle()
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 21:33:47 +0900] rev 37598
export: do not start pager if output will be written to file
A copy of 3b569745af6c.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 23:25:58 +0900] rev 37597
formatter: carry opts to file-based formatters by basefm
This makes it slightly easier to port "hg export" to formatter.
Yuya Nishihara <yuya@tcha.org> [Thu, 12 Apr 2018 23:24:33 +0900] rev 37596
formatter: remove unused private attributes from baseformatter
No idea what they were for.
Danny Hooper <hooper@google.com> [Fri, 30 Mar 2018 16:40:25 -0700] rev 37595
fix: add --all flag to fix non-public non-obsolete revisions
Differential Revision: https://phab.mercurial-scm.org/D3213
Augie Fackler <augie@google.com> [Sat, 24 Mar 2018 14:28:24 -0400] rev 37594
fsmonitor: layer on another hack in bser.c for os.stat() compat (issue5811)
It's unclear to me how these `bserobj_tuple` objects are used, other
than as stat objects. This should fix fsmonitor in the wake of
ffa3026d4196 and similar changes. I regret the hack here, but the code
already has plenty of hg-specific hacks. :(
It feels like we should be able to use int(result.st_mtime) globally,
but that doesn't work. See issue4836 for a bug that was hard to track
down relating to rounding behavior causing very subtle dirstate
problems.
Differential Revision: https://phab.mercurial-scm.org/D2939
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 17:24:38 -0400] rev 37593
tests: use `f --newer` instead of `stat -c` in test-fix.t
Also increase sleep to two seconds so this test will likely pass on FAT32.
Differential Revision: https://phab.mercurial-scm.org/D3252
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 17:07:07 -0400] rev 37592
tests: glob away fqdn wherever we print it
These localhost instances are actually from a getfqdn call, which
means on some of my test systems it comes out as localhost.localdomain
or
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa. I'm
tired of this, so let's glob it away.
Differential Revision: https://phab.mercurial-scm.org/D3251
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 16:26:40 -0400] rev 37591
hgweb: use native strings when interfacing with stdlib headers
We're still parsing the stdlib-provided headers here, so we need to
tread carefully and use native strings. Yuck.
Differential Revision: https://phab.mercurial-scm.org/D3250
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 16:10:02 -0400] rev 37590
wireprotoserver: headers are bytes for us internally, use bytes
This re-fixes test-pull-http.t on Python 3. Probably many others as well.
Differential Revision: https://phab.mercurial-scm.org/D3249
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 16:09:23 -0400] rev 37589
hgweb: put response headers back into str for Python 3
This fixes a lot of hanging tests on Python 3, because
"Content-Length" was getting sent as "b'Content-Length'" (yes, really)
and then clients would expect a close-is-end body instead of counting
off a certain number of bytes.
Differential Revision: https://phab.mercurial-scm.org/D3248
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 14:57:11 -0400] rev 37588
tests: load showstack in test-pull-http.t so network hangs are easier to find
This also gives us some minimal "it loads" coverage on showstack,
which I rather like. showstack doesn't work on Windows per mbarbison,
so it's disabled there.
I added this in service of debugging a hang introduced on Python 3 by
revision a88d68dc3ee8. I'm still not sure what the problem there is,
but this at least gives us a little bit of a chance to figure out
what's going on.
Differential Revision: https://phab.mercurial-scm.org/D3247
Augie Fackler <augie@google.com> [Wed, 11 Apr 2018 14:39:49 -0400] rev 37587
keepalive: rewrite readinto() to not use read()
It turns out http.client on Python 3 sometimes uses readinto() in the
implementation of read(). Unfortunately, Python 2 doesn't have a
readinto() in httplib's client, so we have to support both codepaths.
Subclassing is bad, folks.
Differential Revision: https://phab.mercurial-scm.org/D3246
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 17:22:59 +0530] rev 37586
py3: use bytes() instead of str() on util.url()
We internally deal with bytes and anything as string breaks things.
Differential Revision: https://phab.mercurial-scm.org/D3285
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 17:21:56 +0530] rev 37585
py3: use stringutil.forcebytestr() to convert error messages to bytes
Differential Revision: https://phab.mercurial-scm.org/D3284
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 08:06:39 -0700] rev 37584
py3: suppress the return value of write() in tests/test-subrepo-missing.t
write() on Python 3 returns a value whereas does not return anything on Python 2.
So we need to supress the value.
Differential Revision: https://phab.mercurial-scm.org/D3283
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 16:52:32 +0530] rev 37583
py3: add b'' prefixes in tests/test-linerange.py
This makes the test pass on Python 3.
# skip-blame because just b'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D3282
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 15:23:36 +0530] rev 37582
py3: add b'' prefix to make the regex bytes
# skip-blame because just b'' prefix
Differential Revision: https://phab.mercurial-scm.org/D3281
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 15:22:56 +0530] rev 37581
py3: use b'%d' to convert int to bytes instead of str()
Differential Revision: https://phab.mercurial-scm.org/D3280
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 15:03:15 +0530] rev 37580
py3: use print as a function in test-convert-git.t
Differential Revision: https://phab.mercurial-scm.org/D3278
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 14:30:37 +0530] rev 37579
py3: use '%d' for integers instead of '%s'
Differential Revision: https://phab.mercurial-scm.org/D3277
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 14:28:08 +0530] rev 37578
py3: prevent transformer from adding b'' by adding r'' prefix
These are cases where we need to use str, therefore we add r'' prefix.
# skip-blame because just r'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D3276
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 12 Apr 2018 14:26:31 +0530] rev 37577
py3: use pycompat.{strkwargs|byteskwargs} in infinitepush
Differential Revision: https://phab.mercurial-scm.org/D3275
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 11 Apr 2018 22:36:16 +0530] rev 37576
py3: make sure we open file in bytes mode
Differential Revision: https://phab.mercurial-scm.org/D3274
Yuya Nishihara <yuya@tcha.org> [Sat, 07 Apr 2018 01:37:25 +0900] rev 37575
diffhelpers: be more tolerant for stripped empty lines of CRLF ending
Exchange. It appears to trim lines containing only whitespace as well as
converting LF to CRLF.
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 21:08:52 +0900] rev 37574
diffhelpers: make return value of testhunk() more Pythonic
It's no longer C.
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 21:06:46 +0900] rev 37573
patch: error out if reached to EOF while reading hunk
This was where out-of-bounds read occurred in old C extension.
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:55:05 +0900] rev 37572
diffhelpers: remove unused return value from fixnewline() and addlines()
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:54:00 +0900] rev 37571
diffhelpers: move out of pure package
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:52:54 +0900] rev 37570
diffhelpers: naming and whitespace cleanup
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:51:23 +0900] rev 37569
diffhelpers: remove C implementation in favor of pure Python version
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:49:39 +0900] rev 37568
patch: stop using cext.diffhelpers
The C implementation has a couple of memory bugs, and lacks error handling
which could lead to SEGV. I could fix them one by one (and I mostly finished
that), but the performance gain provided by cext.diffhelper is quite low.
Besides, diffhelpers.addlines() calls back Python, linereader.readline(),
from the innermost loop.
$ hg export -R mozilla-central 0:100 > patch
$ ls -lh patch
-rw-r--r-- 184M patch
$ hg init repo && hg -R repo import patch --time --bypass
(cext) time: real 34.970 secs (user 32.720+0.000 sys 2.230+0.000)
(pure) time: real 35.950 secs (user 33.600+0.000 sys 2.330+0.000)
So, let's simply use the pure Python implementation.
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:47:43 +0900] rev 37567
diffhelpers: port docstrings from cext to pure
I'll remove the C implementation.
Yuya Nishihara <yuya@tcha.org> [Mon, 09 Apr 2018 20:44:41 +0900] rev 37566
py3: get rid of character access from pure.diffhelpers
's' is a result of readline(), so 'c == "\n"' means 's == "\n"'.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 11 Apr 2018 18:23:29 -0400] rev 37565
lfs: handle paths that don't end with '/' when inferring the blob store
While here, I also checked the lfs.url config directly instead of testing the
scheme, as requested by Yuya.