extdiff: avoid unexpected quoting arguments for external tools (issue4463)
Before this patch, all command line arguments for external tools are
quoted by the combination of "shlex.split" and "util.shellquote". But
this causes some problems.
- some problematic commands can't work correctly with quoted arguments
For example, 'WinMerge /r ....' is OK, but 'WinMerge "/r" ....' is
NG. See also below for detail about this problem.
https://bitbucket.org/tortoisehg/thg/issue/3978/
- quoting itself may change semantics of arguments
For example, when the environment variable CONCAT="foo bar baz':
- mydiff $CONCAT => mydiff foo bar baz (taking 3 arguments)
- mydiff "$CONCAT" => mydiff "foo bar baz" (taking only 1 argument)
For another example, single quoting (= "util.shellquote") on POSIX
environment prevents shells from expanding environment variables,
tilde, and so on:
- mydiff "$HOME" => mydiff /home/foobar
- mydiff '$HOME' => mydiff $HOME
- "shlex.split" can't handle some special characters correctly
It just splits specified command line by whitespaces.
For example, "echo foo;echo bar" is split into ["echo",
"foo;echo", "bar"].
On the other hand, if quoting itself is omitted, users can't specify
options including space characters with "--option" at runtime.
The root cause of this issue is that "shlex.split + util.shellquote"
combination loses whether users really want to quote each command line
elements or not, even though these can be quoted arbitrarily in
configurations.
To resolve this problem, this patch does:
- prevent configurations from being processed by "shlex.split" and
"util.shellquote"
only (possibly) "findexe"-ed or "findexternaltool"-ed command path
is "util.shellquote", because it may contain whitespaces.
- quote options specified by "--option" via command line at runtime
This patch also makes "dodiff()" take only one "args" argument instead
of "diffcmd" and "diffopts. It also omits applying "util.shellquote"
on "args", because "args" should be already stringified in "extdiff()"
and "mydiff()".
The last hunk for "test-extdiff.t" replaces two whitespaces by single
whitespace, because change of "' '.join()" logic causes omitting
redundant whitespaces.
$ echo "[extensions]" >> $HGRCPATH
$ echo "extdiff=" >> $HGRCPATH
$ hg init a
$ cd a
$ echo a > a
$ echo b > b
$ hg add
adding a
adding b
Should diff cloned directories:
$ hg extdiff -o -r $opt
Only in a: a
Only in a: b
[1]
$ cat <<EOF >> $HGRCPATH
> [extdiff]
> cmd.falabala = echo
> opts.falabala = diffing
> cmd.edspace = echo
> opts.edspace = "name <user@example.com>"
> EOF
$ hg falabala
diffing a.000000000000 a
[1]
$ hg help falabala
hg falabala [OPTION]... [FILE]...
use 'echo' to diff repository (or selected files)
Show differences between revisions for the specified files, using the
'echo' program.
When two revision arguments are given, then changes are shown between
those revisions. If only one revision is specified then that revision is
compared to the working directory, and, when no revisions are specified,
the working directory files are compared to its parent.
options ([+] can be repeated):
-o --option OPT [+] pass option to comparison program
-r --rev REV [+] revision
-c --change REV change made by revision
-I --include PATTERN [+] include names matching the given patterns
-X --exclude PATTERN [+] exclude names matching the given patterns
(some details hidden, use --verbose to show complete help)
$ hg ci -d '0 0' -mtest1
$ echo b >> a
$ hg ci -d '1 0' -mtest2
Should diff cloned files directly:
$ hg falabala -r 0:1
diffing */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
Test diff during merge:
$ hg update -C 0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo c >> c
$ hg add c
$ hg ci -m "new branch" -d '1 0'
created new head
$ hg merge 1
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
Should diff cloned file against wc file:
$ hg falabala
diffing */extdiff.*/a.2a13a4d2da36/a */a/a (glob)
[1]
Test --change option:
$ hg ci -d '2 0' -mtest3
$ hg falabala -c 1
diffing */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
Check diff are made from the first parent:
$ hg falabala -c 3 || echo "diff-like tools yield a non-zero exit code"
diffing */extdiff.*/a.2a13a4d2da36/a a.46c0e4daeb72/a (glob)
diff-like tools yield a non-zero exit code
issue4463: usage of command line configuration without additional quoting
$ cat <<EOF >> $HGRCPATH
> [extdiff]
> cmd.4463a = echo
> opts.4463a = a-naked 'single quoted' "double quoted"
> 4463b = echo b-naked 'single quoted' "double quoted"
> echo =
> EOF
$ hg update -q -C 0
$ echo a >> a
#if windows
$ hg --debug 4463a | grep '^running'
running '"echo" a-naked \'single quoted\' "double quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug 4463b | grep '^running'
running 'echo b-naked \'single quoted\' "double quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug echo | grep '^running'
running '"*echo*" "*\\a" "*\\a"' in */extdiff.* (glob)
#else
$ hg --debug 4463a | grep '^running'
running '\'echo\' a-naked \'single quoted\' "double quoted" \'*/a\' \'$TESTTMP/a/a\'' in */extdiff.* (glob)
$ hg --debug 4463b | grep '^running'
running 'echo b-naked \'single quoted\' "double quoted" \'*/a\' \'$TESTTMP/a/a\'' in */extdiff.* (glob)
$ hg --debug echo | grep '^running'
running "'*echo*' '*/a' '$TESTTMP/a/a'" in */extdiff.* (glob)
#endif
(getting options from other than extdiff section)
$ cat <<EOF >> $HGRCPATH
> [extdiff]
> # using diff-tools diffargs
> 4463b2 = echo
> # using merge-tools diffargs
> 4463b3 = echo
> # no diffargs
> 4463b4 = echo
> [diff-tools]
> 4463b2.diffargs = b2-naked 'single quoted' "double quoted"
> [merge-tools]
> 4463b3.diffargs = b3-naked 'single quoted' "double quoted"
> EOF
#if windows
$ hg --debug 4463b2 | grep '^running'
running 'echo b2-naked \'single quoted\' "double quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug 4463b3 | grep '^running'
running 'echo b3-naked \'single quoted\' "double quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug 4463b4 | grep '^running'
running 'echo "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug 4463b4 --option 'being quoted' | grep '^running'
running 'echo "being quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
$ hg --debug extdiff -p echo --option 'being quoted' | grep '^running'
running '"echo" "being quoted" "*\\a" "*\\a"' in */extdiff.* (glob)
#else
$ hg --debug 4463b2 | grep '^running'
running 'echo b2-naked \'single quoted\' "double quoted" \'*/a\' \'$TESTTMP/a/a\'' in */extdiff.* (glob)
$ hg --debug 4463b3 | grep '^running'
running 'echo b3-naked \'single quoted\' "double quoted" \'*/a\' \'$TESTTMP/a/a\'' in */extdiff.* (glob)
$ hg --debug 4463b4 | grep '^running'
running "echo '*/a' '$TESTTMP/a/a'" in */extdiff.* (glob)
$ hg --debug 4463b4 --option 'being quoted' | grep '^running'
running "echo 'being quoted' '*/a' '$TESTTMP/a/a'" in */extdiff.* (glob)
$ hg --debug extdiff -p echo --option 'being quoted' | grep '^running'
running "'echo' 'being quoted' '*/a' '$TESTTMP/a/a'" in */extdiff.* (glob)
#endif
#if execbit
Test extdiff of multiple files in tmp dir:
$ hg update -C 0 > /dev/null
$ echo changed > a
$ echo changed > b
$ chmod +x b
Diff in working directory, before:
$ hg diff --git
diff --git a/a b/a
--- a/a
+++ b/a
@@ -1,1 +1,1 @@
-a
+changed
diff --git a/b b/b
old mode 100644
new mode 100755
--- a/b
+++ b/b
@@ -1,1 +1,1 @@
-b
+changed
Edit with extdiff -p:
Prepare custom diff/edit tool:
$ cat > 'diff tool.py' << EOT
> #!/usr/bin/env python
> import time
> time.sleep(1) # avoid unchanged-timestamp problems
> file('a/a', 'ab').write('edited\n')
> file('a/b', 'ab').write('edited\n')
> EOT
$ chmod +x 'diff tool.py'
will change to /tmp/extdiff.TMP and populate directories a.TMP and a
and start tool
$ hg extdiff -p "`pwd`/diff tool.py"
[1]
Diff in working directory, after:
$ hg diff --git
diff --git a/a b/a
--- a/a
+++ b/a
@@ -1,1 +1,2 @@
-a
+changed
+edited
diff --git a/b b/b
old mode 100644
new mode 100755
--- a/b
+++ b/b
@@ -1,1 +1,2 @@
-b
+changed
+edited
Test extdiff with --option:
$ hg extdiff -p echo -o this -c 1
this */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
$ hg falabala -o this -c 1
diffing this */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
Test extdiff's handling of options with spaces in them:
$ hg edspace -c 1
name <user@example.com> */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
$ hg extdiff -p echo -o "name <user@example.com>" -c 1
name <user@example.com> */extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
Test with revsets:
$ hg extdif -p echo -c "rev(1)"
*/extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
$ hg extdif -p echo -r "0::1"
*/extdiff.*/a.8a5febb7f867/a a.34eed99112ab/a (glob)
[1]
Fallback to merge-tools.tool.executable|regkey
$ mkdir dir
$ cat > 'dir/tool.sh' << EOF
> #!/bin/sh
> echo "** custom diff **"
> EOF
$ chmod +x dir/tool.sh
$ tool=`pwd`/dir/tool.sh
$ hg --debug tl --config extdiff.tl= --config merge-tools.tl.executable=$tool
making snapshot of 2 files from rev * (glob)
a
b
making snapshot of 2 files from working directory
a
b
running "'$TESTTMP/a/dir/tool.sh' 'a.*' 'a'" in */extdiff.* (glob)
** custom diff **
cleaning up temp directory
[1]
$ cd ..
#endif
#if symlink
Test symlinks handling (issue1909)
$ hg init testsymlinks
$ cd testsymlinks
$ echo a > a
$ hg ci -Am adda
adding a
$ echo a >> a
$ ln -s missing linka
$ hg add linka
$ hg falabala -r 0 --traceback
diffing testsymlinks.07f494440405 testsymlinks
[1]
$ cd ..
#endif