Mercurial Mercurial > mercurial / changeset
summary | shortlog | changelog | graph | tags | bookmarks | branches | files | changeset | raw | help
Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
encoding: default ambiguous character to narrow stable
authorMatt Mackall <mpm@selenic.com>
Wed, 27 Oct 2010 15:35:21 -0500
branchstable
changeset 12866 eddc20306ab6
parent 12865 4c50552fc9bc
child 12867 eda77c3e246a
encoding: default ambiguous character to narrow The current implementation of colwidth was treating 'A'mbiguous characters as wide, which was incorrect in a non-East Asian context. As per http://unicode.org/reports/tr11/#Recommendations, we should instead default to 'narrow' if we don't know better. As character width is dependent on the particular font used and we have no idea what fonts are in use, this recommendation applies. This introduces HGENCODINGAMBIGUOUS to get the old behavior back.
mercurial/encoding.py file | annotate | diff | comparison | revisions
mercurial/help/environment.txt file | annotate | diff | comparison | revisions
tests/test-encoding.t file | annotate | diff | comparison | revisions 
--- a/mercurial/encoding.py	Tue Oct 26 12:18:39 2010 +0200
+++ b/mercurial/encoding.py	Wed Oct 27 15:35:21 2010 -0500
@@ -87,11 +87,17 @@
     except LookupError, k:
         raise error.Abort("%s, please check your locale settings" % k)
 
+# How to treat ambiguous-width characters. Set to 'wide' to treat as wide.
+ambiguous = os.environ.get("HGENCODINGAMBIGUOUS", "narrow")
+
 def colwidth(s):
     "Find the column width of a UTF-8 string for display"
     d = s.decode(encoding, 'replace')
     if hasattr(unicodedata, 'east_asian_width'):
+        wide = "WF"
+        if ambiguous == "wide":
+            wide = "WFA"
         w = unicodedata.east_asian_width
-        return sum([w(c) in 'WFA' and 2 or 1 for c in d])
+        return sum([w(c) in wide and 2 or 1 for c in d])
     return len(d)
 
--- a/mercurial/help/environment.txt	Tue Oct 26 12:18:39 2010 +0200
+++ b/mercurial/help/environment.txt	Wed Oct 27 15:35:21 2010 -0500
@@ -24,6 +24,13 @@
     "ignore", which drops them. This setting can be overridden with
     the --encodingmode command-line option.
 
+HGENCODINGAMBIGUOUS
+    This sets Mercurial's behavior for handling characters with
+    "ambiguous" widths like accented Latin characters with East Asian
+    fonts. By default, Mercurial assumes ambiguous characters are
+    narrow, set this variable to "wide" if such characters cause
+    formatting problems.
+
 HGMERGE
     An executable to use for resolving merge conflicts. The program
     will be executed with three arguments: local file, remote file,
--- a/tests/test-encoding.t	Tue Oct 26 12:18:39 2010 +0200
+++ b/tests/test-encoding.t	Wed Oct 27 15:35:21 2010 -0500
@@ -166,13 +166,13 @@
 
   $ HGENCODING=latin-1 hg tags
   tip                                5:093c6077d1c8
-  é                                 3:ca661e7520de
+  é                                  3:ca661e7520de
 
 hg tags (utf-8)
 
   $ HGENCODING=utf-8 hg tags
   tip                                5:093c6077d1c8
-  é                                 3:ca661e7520de
+  é                                  3:ca661e7520de
 
 hg branches (ascii)
 
@@ -183,13 +183,13 @@
 hg branches (latin-1)
 
   $ HGENCODING=latin-1 hg branches
-  é                             5:093c6077d1c8
+  é                              5:093c6077d1c8
   default                        4:94db611b4196 (inactive)
 
 hg branches (utf-8)
 
   $ HGENCODING=utf-8 hg branches
-  é                             5:093c6077d1c8
+  é                              5:093c6077d1c8
   default                        4:94db611b4196 (inactive)
   $ echo '[ui]' >> .hg/hgrc
   $ echo 'fallbackencoding = koi8-r' >> .hg/hgrc
mercurial