highlight: add option to prevent content-only based fallback When Mozilla enabled Pygments on hg.mozilla.org, we got a lot of weirdly colorized files. Upon further investigation, the hightlight extension is first attempting a filename+content based match then falling back to a purely content-driven detection mode in Pygments. Sounds good in theory. Unfortunately, Pygments' content-driven detection establishes no minimum threshold for returning a lexer. Furthermore, the detection code for a number of languages is very liberal. For example, ActionScript 3 will return a confidence of 0.3 (out of 1.0) if the first 1k of the file we pass in matches the regex "\w+\s*:\s*\w"! Python matches on "import ". It's no coincidence that a number of our extension-less files were getting highlighted improperly. This patch adds an option to have the highlighter not fall back to purely content-based detection when filename+content detection failed. This can be enabled to render unlighted text instead of taking the risk that unknown file types are highlighted incorrectly. The old behavior is still the default.

     1

#!/usr/bin/env python

     2

# Dump revlogs as raw data stream

     3

# $ find .hg/store/ -name "*.i" | xargs dumprevlog > repo.dump

     4

     5

import sys

     6

from mercurial import revlog, node, util

     7

     8

for fp in (sys.stdin, sys.stdout, sys.stderr):

     9

    util.setbinary(fp)

    10

    11

for f in sys.argv[1:]:

    12

    binopen = lambda fn: open(fn, 'rb')

    13

    r = revlog.revlog(binopen, f)

    14

    print "file:", f

    15

    for i in r:

    16

        n = r.node(i)

    17

        p = r.parents(n)

    18

        d = r.revision(n)

    19

        print "node:", node.hex(n)

    20

        print "linkrev:", r.linkrev(i)

    21

        print "parents:", node.hex(p[0]), node.hex(p[1])

    22

        print "length:", len(d)

    23

        print "-start-"

    24

        print d

    25

        print "-end-"