highlight: add option to prevent content-only based fallback
When Mozilla enabled Pygments on hg.mozilla.org, we got a lot of weirdly
colorized files. Upon further investigation, the hightlight extension
is first attempting a filename+content based match then falling back to a
purely content-driven detection mode in Pygments. Sounds good in theory.
Unfortunately, Pygments' content-driven detection establishes no minimum
threshold for returning a lexer. Furthermore, the detection code for
a number of languages is very liberal. For example, ActionScript 3 will
return a confidence of 0.3 (out of 1.0) if the first 1k of the file
we pass in matches the regex "\w+\s*:\s*\w"! Python matches on
"import ". It's no coincidence that a number of our extension-less files
were getting highlighted improperly.
This patch adds an option to have the highlighter not fall back to
purely content-based detection when filename+content detection failed.
This can be enabled to render unlighted text instead of taking the risk
that unknown file types are highlighted incorrectly. The old behavior is
still the default.
#!/bin/sh
#
# tcsh_completion_build.sh - script to generate tcsh completion
#
#
# Copyright (C) 2005 TK Soh.
#
# This is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your
# option) any later version.
#
#
# Description
# -----------
# This script generates a tcsh source file to support completion
# of Mercurial commands and options.
#
# Instruction:
# -----------
# Run this script to generate the tcsh source file, and source
# the file to add command completion support for Mercurial.
#
# tcsh% tcsh_completion.sh FILE
# tcsh% source FILE
#
# If FILE is not specified, tcsh_completion will be generated.
#
# Bugs:
# ----
# 1. command specific options are not supported
# 2. hg commands must be specified immediately after 'hg'.
#
tcsh_file=${1-tcsh_completion}
hg_commands=`hg --debug help | \
sed -e '1,/^list of commands:/d' \
-e '/^enabled extensions:/,$d' \
-e '/^additional help topics:/,$d' \
-e '/^ [^ ]/!d; s/[,:]//g;' | \
xargs -n5 | \
sed -e '$!s/$/ \\\\/g; 2,$s/^ */ /g'`
hg_global_options=`hg -v help | \
sed -e '1,/global/d;/^ *-/!d; s/ [^- ].*//' | \
sed -e 's/ *$//; $!s/$/ \\\\/g; 2,$s/^ */ /g'`
hg_version=`hg version | sed -e '1q'`
script_name=`basename $0`
cat > $tcsh_file <<END
#
# tcsh completion for Mercurial
#
# This file has been auto-generated by $script_name for
# $hg_version
#
# Copyright (C) 2005 TK Soh.
#
# This is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your
# option) any later version.
#
complete hg \\
'n/--cwd/d/' 'n/-R/d/' 'n/--repository/d/' \\
'C/-/($hg_global_options)/' \\
'p/1/($hg_commands)/'
END