convert: transcode CVS log messages by specified encoding (issue5597)
Converting from CVS to Mercurial assumes that CVS log messages in "cvs
rlog" output are encoded in UTF-8 (or basic Latin-1). But cvs itself
is usually unaware of encoding of log messages, in practice.
Therefore, if there are commits, of which log message is encoded in
other than UTF-8, log message of corresponded revisions in the
converted repository will be broken.
To avoid such broken log messages, this patch transcodes CVS log
messages by encoding specified via "convert.cvsps.logencoding"
configuration.
This patch accepts multiple encoding for convenience, because
"multiple encoding mixed in a repository" easily occurs. For example,
UTF-8 (recent POSIX), cp932 (Windows), and EUC-JP (legacy POSIX) are
well known encoding for Japanese.
#!/usr/bin/env python
#
# Dumps output generated by Mercurial's command server in a formatted style to a
# given file or stderr if '-' is specified. Output is also written in its raw
# format to stdout.
#
# $ ./hg serve --cmds pipe | ./contrib/debugcmdserver.py -
# o, 52 -> 'capabilities: getencoding runcommand\nencoding: UTF-8'
from __future__ import absolute_import, print_function
import struct
import sys
if len(sys.argv) != 2:
print('usage: debugcmdserver.py FILE')
sys.exit(1)
outputfmt = '>cI'
outputfmtsize = struct.calcsize(outputfmt)
if sys.argv[1] == '-':
log = sys.stderr
else:
log = open(sys.argv[1], 'a')
def read(size):
data = sys.stdin.read(size)
if not data:
raise EOFError
sys.stdout.write(data)
sys.stdout.flush()
return data
try:
while True:
header = read(outputfmtsize)
channel, length = struct.unpack(outputfmt, header)
log.write('%s, %-4d' % (channel, length))
if channel in 'IL':
log.write(' -> waiting for input\n')
else:
data = read(length)
log.write(' -> %r\n' % data)
log.flush()
except EOFError:
pass
finally:
if log != sys.stderr:
log.close()