# bugzilla.py - bugzilla integration for mercurial

#

# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>

# Copyright 2011-4 Jim Hague <jim.hague@acm.org>

#

# This software may be used and distributed according to the terms of the

# GNU General Public License version 2 or any later version.

'''hooks for integrating with the Bugzilla bug tracker

This hook extension adds comments on bugs in Bugzilla when changesets

that refer to bugs by Bugzilla ID are seen. The comment is formatted using

the Mercurial template mechanism.

The bug references can optionally include an update for Bugzilla of the

hours spent working on the bug. Bugs can also be marked fixed.

Three basic modes of access to Bugzilla are provided:

1. Access via the Bugzilla XMLRPC interface. Requires Bugzilla 3.4 or later.

2. Check data via the Bugzilla XMLRPC interface and submit bug change

   via email to Bugzilla email interface. Requires Bugzilla 3.4 or later.

3. Writing directly to the Bugzilla database. Only Bugzilla installations

   using MySQL are supported. Requires Python MySQLdb.

Writing directly to the database is susceptible to schema changes, and

relies on a Bugzilla contrib script to send out bug change

notification emails. This script runs as the user running Mercurial,

must be run on the host with the Bugzilla install, and requires

permission to read Bugzilla configuration details and the necessary

MySQL user and password to have full access rights to the Bugzilla

database. For these reasons this access mode is now considered

deprecated, and will not be updated for new Bugzilla versions going

forward. Only adding comments is supported in this access mode.

Access via XMLRPC needs a Bugzilla username and password to be specified

in the configuration. Comments are added under that username. Since the

configuration must be readable by all Mercurial users, it is recommended

that the rights of that user are restricted in Bugzilla to the minimum

necessary to add comments. Marking bugs fixed requires Bugzilla 4.0 and later.

Access via XMLRPC/email uses XMLRPC to query Bugzilla, but sends

email to the Bugzilla email interface to submit comments to bugs.

The From: address in the email is set to the email address of the Mercurial

user, so the comment appears to come from the Mercurial user. In the event

that the Mercurial user email is not recognized by Bugzilla as a Bugzilla

user, the email associated with the Bugzilla username used to log into

Bugzilla is used instead as the source of the comment. Marking bugs fixed

works on all supported Bugzilla versions.

Configuration items common to all access modes:

bugzilla.version

  The access type to use. Values recognized are:

  :``xmlrpc``:       Bugzilla XMLRPC interface.

  :``xmlrpc+email``: Bugzilla XMLRPC and email interfaces.

  :``3.0``:          MySQL access, Bugzilla 3.0 and later.

  :``2.18``:         MySQL access, Bugzilla 2.18 and up to but not

                     including 3.0.

  :``2.16``:         MySQL access, Bugzilla 2.16 and up to but not

                     including 2.18.

bugzilla.regexp

  Regular expression to match bug IDs for update in changeset commit message.

  It must contain one "()" named group ``<ids>`` containing the bug

  IDs separated by non-digit characters. It may also contain

  a named group ``<hours>`` with a floating-point number giving the

  hours worked on the bug. If no named groups are present, the first

  "()" group is assumed to contain the bug IDs, and work time is not

  updated. The default expression matches ``Bug 1234``, ``Bug no. 1234``,

  ``Bug number 1234``, ``Bugs 1234,5678``, ``Bug 1234 and 5678`` and

  variations thereof, followed by an hours number prefixed by ``h`` or

  ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.

bugzilla.fixregexp

  Regular expression to match bug IDs for marking fixed in changeset

  commit message. This must contain a "()" named group ``<ids>` containing

  the bug IDs separated by non-digit characters. It may also contain

  a named group ``<hours>`` with a floating-point number giving the

  hours worked on the bug. If no named groups are present, the first

  "()" group is assumed to contain the bug IDs, and work time is not

  updated. The default expression matches ``Fixes 1234``, ``Fixes bug 1234``,

  ``Fixes bugs 1234,5678``, ``Fixes 1234 and 5678`` and

  variations thereof, followed by an hours number prefixed by ``h`` or

  ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.

bugzilla.fixstatus

  The status to set a bug to when marking fixed. Default ``RESOLVED``.

bugzilla.fixresolution

  The resolution to set a bug to when marking fixed. Default ``FIXED``.

bugzilla.style

  The style file to use when formatting comments.

bugzilla.template

  Template to use when formatting comments. Overrides style if

  specified. In addition to the usual Mercurial keywords, the

  extension specifies:

  :``{bug}``:     The Bugzilla bug ID.

  :``{root}``:    The full pathname of the Mercurial repository.

  :``{webroot}``: Stripped pathname of the Mercurial repository.

  :``{hgweb}``:   Base URL for browsing Mercurial repositories.

  Default ``changeset {node|short} in repo {root} refers to bug

  {bug}.\\ndetails:\\n\\t{desc|tabindent}``

bugzilla.strip

  The number of path separator characters to strip from the front of

  the Mercurial repository path (``{root}`` in templates) to produce

  ``{webroot}``. For example, a repository with ``{root}``

  ``/var/local/my-project`` with a strip of 2 gives a value for

  ``{webroot}`` of ``my-project``. Default 0.

web.baseurl

  Base URL for browsing Mercurial repositories. Referenced from

  templates as ``{hgweb}``.

Configuration items common to XMLRPC+email and MySQL access modes:

bugzilla.usermap

  Path of file containing Mercurial committer email to Bugzilla user email

  mappings. If specified, the file should contain one mapping per

  line::

    committer = Bugzilla user

  See also the ``[usermap]`` section.

The ``[usermap]`` section is used to specify mappings of Mercurial

committer email to Bugzilla user email. See also ``bugzilla.usermap``.

Contains entries of the form ``committer = Bugzilla user``.

XMLRPC access mode configuration:

bugzilla.bzurl

  The base URL for the Bugzilla installation.

  Default ``http://localhost/bugzilla``.

bugzilla.user

  The username to use to log into Bugzilla via XMLRPC. Default

  ``bugs``.

bugzilla.password

  The password for Bugzilla login.

XMLRPC+email access mode uses the XMLRPC access mode configuration items,

and also:

bugzilla.bzemail

  The Bugzilla email address.

In addition, the Mercurial email settings must be configured. See the

documentation in hgrc(5), sections ``[email]`` and ``[smtp]``.

MySQL access mode configuration:

bugzilla.host

  Hostname of the MySQL server holding the Bugzilla database.

  Default ``localhost``.

bugzilla.db

  Name of the Bugzilla database in MySQL. Default ``bugs``.

bugzilla.user

  Username to use to access MySQL server. Default ``bugs``.

bugzilla.password

  Password to use to access MySQL server.

bugzilla.timeout

  Database connection timeout (seconds). Default 5.

bugzilla.bzuser

  Fallback Bugzilla user name to record comments with, if changeset

  committer cannot be found as a Bugzilla user.

bugzilla.bzdir

   Bugzilla install directory. Used by default notify. Default

   ``/var/www/html/bugzilla``.

bugzilla.notify

  The command to run to get Bugzilla to send bug change notification

  emails. Substitutes from a map with 3 keys, ``bzdir``, ``id`` (bug

  id) and ``user`` (committer bugzilla email). Default depends on

  version; from 2.18 it is "cd %(bzdir)s && perl -T

  contrib/sendbugmail.pl %(id)s %(user)s".

Activating the extension::

    [extensions]

    bugzilla =

    [hooks]

    # run bugzilla hook on every change pulled or pushed in here

    incoming.bugzilla = python:hgext.bugzilla.hook

Example configurations:

XMLRPC example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

XMLRPC+email example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. Bug comments

are sent to the Bugzilla email address

``bugzilla@my-project.org``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc+email

    bzemail=bugzilla@my-project.org

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

MySQL example configuration. This has a local Bugzilla 3.2 installation

in ``/opt/bugzilla-3.2``. The MySQL database is on ``localhost``,

the Bugzilla database name is ``bugs`` and MySQL is

accessed with MySQL username ``bugs`` password ``XYZZY``. It is used

with a collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    host=localhost

    password=XYZZY

    version=3.0

    bzuser=unknown@domain.com

    bzdir=/opt/bugzilla-3.2

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

All the above add a comment to the Bugzilla bug record of the form::

    Changeset 3b16791d6642 in repository-name.

    http://my-project.org/hg/repository-name/rev/3b16791d6642

    Changeset commit comment. Bug 1234.

'''

from mercurial.i18n import _

from mercurial.node import short

from mercurial import cmdutil, mail, util

import re, time, urlparse, xmlrpclib

# Note for extension authors: ONLY specify testedwith = 'internal' for

# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should

# be specifying the version(s) of Mercurial they are tested with, or

# leave the attribute unspecified.

testedwith = 'internal'

class bzaccess(object):

    '''Base class for access to Bugzilla.'''

    def __init__(self, ui):

        self.ui = ui

        usermap = self.ui.config('bugzilla', 'usermap')

        if usermap:

            self.ui.readconfig(usermap, sections=['usermap'])

    def map_committer(self, user):

        '''map name of committer to Bugzilla user name.'''

        for committer, bzuser in self.ui.configitems('usermap'):

            if committer.lower() == user.lower():

                return bzuser

        return user

    # Methods to be implemented by access classes.

    #

    # 'bugs' is a dict keyed on bug id, where values are a dict holding

    # updates to bug state. Recognized dict keys are:

    #

    # 'hours': Value, float containing work hours to be updated.

    # 'fix':   If key present, bug is to be marked fixed. Value ignored.

    def filter_real_bug_ids(self, bugs):

        '''remove bug IDs that do not exist in Bugzilla from bugs.'''

        pass

    def filter_cset_known_bug_ids(self, node, bugs):

        '''remove bug IDs where node occurs in comment text from bugs.'''

        pass

    def updatebug(self, bugid, newstate, text, committer):

        '''update the specified bug. Add comment text and set new states.

        If possible add the comment as being from the committer of

        the changeset. Otherwise use the default Bugzilla user.

        '''

        pass

    def notify(self, bugs, committer):

        '''Force sending of Bugzilla notification emails.

        Only required if the access method does not trigger notification

        emails automatically.

        '''

        pass

# Bugzilla via direct access to MySQL database.

class bzmysql(bzaccess):

    '''Support for direct MySQL access to Bugzilla.

    The earliest Bugzilla version this is tested with is version 2.16.

    If your Bugzilla is version 3.4 or above, you are strongly

    recommended to use the XMLRPC access method instead.

    '''

    @staticmethod

    def sql_buglist(ids):

        '''return SQL-friendly list of bug ids'''

        return '(' + ','.join(map(str, ids)) + ')'

    _MySQLdb = None

    def __init__(self, ui):

        try:

            import MySQLdb as mysql

            bzmysql._MySQLdb = mysql

        except ImportError as err:

            raise util.Abort(_('python mysql support not available: %s') % err)

        bzaccess.__init__(self, ui)

        host = self.ui.config('bugzilla', 'host', 'localhost')

        user = self.ui.config('bugzilla', 'user', 'bugs')

        passwd = self.ui.config('bugzilla', 'password')

        db = self.ui.config('bugzilla', 'db', 'bugs')

        timeout = int(self.ui.config('bugzilla', 'timeout', 5))

        self.ui.note(_('connecting to %s:%s as %s, password %s\n') %

                     (host, db, user, '*' * len(passwd)))

        self.conn = bzmysql._MySQLdb.connect(host=host,

                                                   user=user, passwd=passwd,

                                                   db=db,

                                                   connect_timeout=timeout)

        self.cursor = self.conn.cursor()

        self.longdesc_id = self.get_longdesc_id()

        self.user_ids = {}

        self.default_notify = "cd %(bzdir)s && ./processmail %(id)s %(user)s"

    def run(self, *args, **kwargs):

        '''run a query.'''

        self.ui.note(_('query: %s %s\n') % (args, kwargs))

        try:

            self.cursor.execute(*args, **kwargs)

        except bzmysql._MySQLdb.MySQLError:

            self.ui.note(_('failed query: %s %s\n') % (args, kwargs))

            raise

    def get_longdesc_id(self):

        '''get identity of longdesc field'''

        self.run('select fieldid from fielddefs where name = "longdesc"')

        ids = self.cursor.fetchall()

        if len(ids) != 1:

            raise util.Abort(_('unknown database schema'))

        return ids[0][0]

    def filter_real_bug_ids(self, bugs):

        '''filter not-existing bugs from set.'''

        self.run('select bug_id from bugs where bug_id in %s' %

                 bzmysql.sql_buglist(bugs.keys()))

        existing = [id for (id,) in self.cursor.fetchall()]

        for id in bugs.keys():

            if id not in existing:

                self.ui.status(_('bug %d does not exist\n') % id)

                del bugs[id]

    def filter_cset_known_bug_ids(self, node, bugs):

        '''filter bug ids that already refer to this changeset from set.'''

        self.run('''select bug_id from longdescs where

                    bug_id in %s and thetext like "%%%s%%"''' %

                 (bzmysql.sql_buglist(bugs.keys()), short(node)))

        for (id,) in self.cursor.fetchall():

            self.ui.status(_('bug %d already knows about changeset %s\n') %

                           (id, short(node)))

            del bugs[id]

    def notify(self, bugs, committer):

        '''tell bugzilla to send mail.'''

        self.ui.status(_('telling bugzilla to send mail:\n'))

        (user, userid) = self.get_bugzilla_user(committer)

        for id in bugs.keys():

            self.ui.status(_('  bug %s\n') % id)

            cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)