# bugzilla.py - bugzilla integration for mercurial

#

# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>

# Copyright 2011 Jim Hague <jim.hague@acm.org>

#

# This software may be used and distributed according to the terms of the

# GNU General Public License version 2 or any later version.

'''hooks for integrating with the Bugzilla bug tracker

This hook extension adds comments on bugs in Bugzilla when changesets

that refer to bugs by Bugzilla ID are seen. The comment is formatted using

the Mercurial template mechanism.

The hook does not change bug status.

Three basic modes of access to Bugzilla are provided:

1. Access via the Bugzilla XMLRPC interface. Requires Bugzilla 3.4 or later.

2. Check data via the Bugzilla XMLRPC interface and submit bug change

   via email to Bugzilla email interface. Requires Bugzilla 3.4 or later.

3. Writing directly to the Bugzilla database. Only Bugzilla installations

   using MySQL are supported. Requires Python MySQLdb.

Writing directly to the database is susceptible to schema changes, and

relies on a Bugzilla contrib script to send out bug change

notification emails. This script runs as the user running Mercurial,

must be run on the host with the Bugzilla install, and requires

permission to read Bugzilla configuration details and the necessary

MySQL user and password to have full access rights to the Bugzilla

database. For these reasons this access mode is now considered

deprecated, and will not be updated for new Bugzilla versions going

forward.

Access via XMLRPC needs a Bugzilla username and password to be specified

in the configuration. Comments are added under that username. Since the

configuration must be readable by all Mercurial users, it is recommended

that the rights of that user are restricted in Bugzilla to the minimum

necessary to add comments.

Access via XMLRPC/email uses XMLRPC to query Bugzilla, but sends

email to the Bugzilla email interface to submit comments to bugs.

The From: address in the email is set to the email address of the Mercurial

user, so the comment appears to come from the Mercurial user. In the event

that the Mercurial user email is not recognised by Bugzilla as a Bugzilla

user, the email associated with the Bugzilla username used to log into

Bugzilla is used instead as the source of the comment.

Configuration items common to all access modes:

bugzilla.version

  This access type to use. Values recognised are:

  :``xmlrpc``:       Bugzilla XMLRPC interface.

  :``xmlrpc+email``: Bugzilla XMLRPC and email interfaces.

  :``3.0``:          MySQL access, Bugzilla 3.0 and later.

  :``2.18``:         MySQL access, Bugzilla 2.18 and up to but not

                     including 3.0.

  :``2.16``:         MySQL access, Bugzilla 2.16 and up to but not

                     including 2.18.

bugzilla.regexp

  Regular expression to match bug IDs in changeset commit message.

  Must contain one "()" group. The default expression matches ``Bug

  1234``, ``Bug no. 1234``, ``Bug number 1234``, ``Bugs 1234,5678``,

  ``Bug 1234 and 5678`` and variations thereof. Matching is case

  insensitive.

bugzilla.style

  The style file to use when formatting comments.

bugzilla.template

  Template to use when formatting comments. Overrides style if

  specified. In addition to the usual Mercurial keywords, the

  extension specifies:

  :``{bug}``:     The Bugzilla bug ID.

  :``{root}``:    The full pathname of the Mercurial repository.

  :``{webroot}``: Stripped pathname of the Mercurial repository.

  :``{hgweb}``:   Base URL for browsing Mercurial repositories.

  Default ``changeset {node|short} in repo {root} refers to bug

  {bug}.\\ndetails:\\n\\t{desc|tabindent}``

bugzilla.strip

  The number of path separator characters to strip from the front of

  the Mercurial repository path (``{root}`` in templates) to produce

  ``{webroot}``. For example, a repository with ``{root}``

  ``/var/local/my-project`` with a strip of 2 gives a value for

  ``{webroot}`` of ``my-project``. Default 0.

web.baseurl

  Base URL for browsing Mercurial repositories. Referenced from

  templates as ``{hgweb}``.

Configuration items common to XMLRPC+email and MySQL access modes:

bugzilla.usermap

  Path of file containing Mercurial committer email to Bugzilla user email

  mappings. If specified, the file should contain one mapping per

  line::

    committer = Bugzilla user

  See also the ``[usermap]`` section.

The ``[usermap]`` section is used to specify mappings of Mercurial

committer email to Bugzilla user email. See also ``bugzilla.usermap``.

Contains entries of the form ``committer = Bugzilla user``.

XMLRPC access mode configuration:

bugzilla.bzurl

  The base URL for the Bugzilla installation.

  Default ``http://localhost/bugzilla``.

bugzilla.user

  The username to use to log into Bugzilla via XMLRPC. Default

  ``bugs``.

bugzilla.password

  The password for Bugzilla login.

XMLRPC+email access mode uses the XMLRPC access mode configuration items,

and also:

bugzilla.bzemail

  The Bugzilla email address.

In addition, the Mercurial email settings must be configured. See the

documentation in hgrc(5), sections ``[email]`` and ``[smtp]``.

MySQL access mode configuration:

bugzilla.host

  Hostname of the MySQL server holding the Bugzilla database.

  Default ``localhost``.

bugzilla.db

  Name of the Bugzilla database in MySQL. Default ``bugs``.

bugzilla.user

  Username to use to access MySQL server. Default ``bugs``.

bugzilla.password

  Password to use to access MySQL server.

bugzilla.timeout

  Database connection timeout (seconds). Default 5.

bugzilla.bzuser

  Fallback Bugzilla user name to record comments with, if changeset

  committer cannot be found as a Bugzilla user.

bugzilla.bzdir

   Bugzilla install directory. Used by default notify. Default

   ``/var/www/html/bugzilla``.

bugzilla.notify

  The command to run to get Bugzilla to send bug change notification

  emails. Substitutes from a map with 3 keys, ``bzdir``, ``id`` (bug

  id) and ``user`` (committer bugzilla email). Default depends on

  version; from 2.18 it is "cd %(bzdir)s && perl -T

  contrib/sendbugmail.pl %(id)s %(user)s".

Activating the extension::

    [extensions]

    bugzilla =

    [hooks]

    # run bugzilla hook on every change pulled or pushed in here

    incoming.bugzilla = python:hgext.bugzilla.hook

Example configurations:

XMLRPC example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

XMLRPC+email example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. Bug comments

are sent to the Bugzilla email address

``bugzilla@my-project.org``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc

    bzemail=bugzilla@my-project.org

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

MySQL example configuration. This has a local Bugzilla 3.2 installation

in ``/opt/bugzilla-3.2``. The MySQL database is on ``localhost``,

the Bugzilla database name is ``bugs`` and MySQL is

accessed with MySQL username ``bugs`` password ``XYZZY``. It is used

with a collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    host=localhost

    password=XYZZY

    version=3.0

    bzuser=unknown@domain.com

    bzdir=/opt/bugzilla-3.2

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

All the above add a comment to the Bugzilla bug record of the form::

    Changeset 3b16791d6642 in repository-name.

    http://my-project.org/hg/repository-name/rev/3b16791d6642

    Changeset commit comment. Bug 1234.

'''

from mercurial.i18n import _

from mercurial.node import short

from mercurial import cmdutil, mail, templater, util

import re, time, xmlrpclib

class bzaccess(object):

    '''Base class for access to Bugzilla.'''

    def __init__(self, ui):

        self.ui = ui

        usermap = self.ui.config('bugzilla', 'usermap')

        if usermap:

            self.ui.readconfig(usermap, sections=['usermap'])

    def map_committer(self, user):

        '''map name of committer to Bugzilla user name.'''

        for committer, bzuser in self.ui.configitems('usermap'):

            if committer.lower() == user.lower():

                return bzuser

        return user

    # Methods to be implemented by access classes.

    def filter_real_bug_ids(self, ids):

        '''remove bug IDs that do not exist in Bugzilla from set.'''

        pass

    def filter_cset_known_bug_ids(self, node, ids):

        '''remove bug IDs where node occurs in comment text from set.'''

        pass

    def add_comment(self, bugid, text, committer):

        '''add comment to bug.

        If possible add the comment as being from the committer of

        the changeset. Otherwise use the default Bugzilla user.

        '''

        pass

    def notify(self, ids, committer):

        '''Force sending of Bugzilla notification emails.'''

        pass

# Bugzilla via direct access to MySQL database.

class bzmysql(bzaccess):

    '''Support for direct MySQL access to Bugzilla.

    The earliest Bugzilla version this is tested with is version 2.16.

    If your Bugzilla is version 3.2 or above, you are strongly

    recommended to use the XMLRPC access method instead.

    '''

    @staticmethod

    def sql_buglist(ids):

        '''return SQL-friendly list of bug ids'''

        return '(' + ','.join(map(str, ids)) + ')'

    _MySQLdb = None

    def __init__(self, ui):

        try:

            import MySQLdb as mysql

            bzmysql._MySQLdb = mysql

        except ImportError, err:

            raise util.Abort(_('python mysql support not available: %s') % err)

        bzaccess.__init__(self, ui)

        host = self.ui.config('bugzilla', 'host', 'localhost')

        user = self.ui.config('bugzilla', 'user', 'bugs')

        passwd = self.ui.config('bugzilla', 'password')

        db = self.ui.config('bugzilla', 'db', 'bugs')

        timeout = int(self.ui.config('bugzilla', 'timeout', 5))

        self.ui.note(_('connecting to %s:%s as %s, password %s\n') %

                     (host, db, user, '*' * len(passwd)))

        self.conn = bzmysql._MySQLdb.connect(host=host,

                                                   user=user, passwd=passwd,

                                                   db=db,

                                                   connect_timeout=timeout)

        self.cursor = self.conn.cursor()

        self.longdesc_id = self.get_longdesc_id()

        self.user_ids = {}

        self.default_notify = "cd %(bzdir)s && ./processmail %(id)s %(user)s"

    def run(self, *args, **kwargs):

        '''run a query.'''

        self.ui.note(_('query: %s %s\n') % (args, kwargs))

        try:

            self.cursor.execute(*args, **kwargs)

        except bzmysql._MySQLdb.MySQLError:

            self.ui.note(_('failed query: %s %s\n') % (args, kwargs))

            raise

    def get_longdesc_id(self):

        '''get identity of longdesc field'''

        self.run('select fieldid from fielddefs where name = "longdesc"')

        ids = self.cursor.fetchall()

        if len(ids) != 1:

            raise util.Abort(_('unknown database schema'))

        return ids[0][0]

    def filter_real_bug_ids(self, ids):

        '''filter not-existing bug ids from set.'''

        self.run('select bug_id from bugs where bug_id in %s' %

                 bzmysql.sql_buglist(ids))

        return set([c[0] for c in self.cursor.fetchall()])

    def filter_cset_known_bug_ids(self, node, ids):

        '''filter bug ids that already refer to this changeset from set.'''

        self.run('''select bug_id from longdescs where

                    bug_id in %s and thetext like "%%%s%%"''' %

                 (bzmysql.sql_buglist(ids), short(node)))

        for (id,) in self.cursor.fetchall():

            self.ui.status(_('bug %d already knows about changeset %s\n') %

                           (id, short(node)))

            ids.discard(id)

        return ids

    def notify(self, ids, committer):

        '''tell bugzilla to send mail.'''

        self.ui.status(_('telling bugzilla to send mail:\n'))

        (user, userid) = self.get_bugzilla_user(committer)

        for id in ids:

            self.ui.status(_('  bug %s\n') % id)

            cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)

            bzdir = self.ui.config('bugzilla', 'bzdir', '/var/www/html/bugzilla')

            try:

                # Backwards-compatible with old notify string, which

                # took one string. This will throw with a new format

                # string.

                cmd = cmdfmt % id

            except TypeError:

                cmd = cmdfmt % {'bzdir': bzdir, 'id': id, 'user': user}

            self.ui.note(_('running notify command %s\n') % cmd)

            fp = util.popen('(%s) 2>&1' % cmd)

            out = fp.read()

            ret = fp.close()

            if ret:

                self.ui.warn(out)

                raise util.Abort(_('bugzilla notify command %s') %

                                 util.explainexit(ret)[0])

        self.ui.status(_('done\n'))

    def get_user_id(self, user):

        '''look up numeric bugzilla user id.'''

        try:

            return self.user_ids[user]

        except KeyError:

            try:

                userid = int(user)

            except ValueError:

                self.ui.note(_('looking up user %s\n') % user)

                self.run('''select userid from profiles

                            where login_name like %s''', user)

                all = self.cursor.fetchall()

                if len(all) != 1:

                    raise KeyError(user)

                userid = int(all[0][0])

            self.user_ids[user] = userid

            return userid

    def get_bugzilla_user(self, committer):