# bugzilla.py - bugzilla integration for mercurial

#

# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>

# Copyright 2011-4 Jim Hague <jim.hague@acm.org>

#

# This software may be used and distributed according to the terms of the

# GNU General Public License version 2 or any later version.

'''hooks for integrating with the Bugzilla bug tracker

This hook extension adds comments on bugs in Bugzilla when changesets

that refer to bugs by Bugzilla ID are seen. The comment is formatted using

the Mercurial template mechanism.

The bug references can optionally include an update for Bugzilla of the

hours spent working on the bug. Bugs can also be marked fixed.

Four basic modes of access to Bugzilla are provided:

1. Access via the Bugzilla REST-API. Requires bugzilla 5.0 or later.

2. Access via the Bugzilla XMLRPC interface. Requires Bugzilla 3.4 or later.

3. Check data via the Bugzilla XMLRPC interface and submit bug change

   via email to Bugzilla email interface. Requires Bugzilla 3.4 or later.

4. Writing directly to the Bugzilla database. Only Bugzilla installations

   using MySQL are supported. Requires Python MySQLdb.

Writing directly to the database is susceptible to schema changes, and

relies on a Bugzilla contrib script to send out bug change

notification emails. This script runs as the user running Mercurial,

must be run on the host with the Bugzilla install, and requires

permission to read Bugzilla configuration details and the necessary

MySQL user and password to have full access rights to the Bugzilla

database. For these reasons this access mode is now considered

deprecated, and will not be updated for new Bugzilla versions going

forward. Only adding comments is supported in this access mode.

Access via XMLRPC needs a Bugzilla username and password to be specified

in the configuration. Comments are added under that username. Since the

configuration must be readable by all Mercurial users, it is recommended

that the rights of that user are restricted in Bugzilla to the minimum

necessary to add comments. Marking bugs fixed requires Bugzilla 4.0 and later.

Access via XMLRPC/email uses XMLRPC to query Bugzilla, but sends

email to the Bugzilla email interface to submit comments to bugs.

The From: address in the email is set to the email address of the Mercurial

user, so the comment appears to come from the Mercurial user. In the event

that the Mercurial user email is not recognized by Bugzilla as a Bugzilla

user, the email associated with the Bugzilla username used to log into

Bugzilla is used instead as the source of the comment. Marking bugs fixed

works on all supported Bugzilla versions.

Access via the REST-API needs either a Bugzilla username and password

or an apikey specified in the configuration. Comments are made under

the given username or the user associated with the apikey in Bugzilla.

Configuration items common to all access modes:

bugzilla.version

  The access type to use. Values recognized are:

  :``restapi``:      Bugzilla REST-API, Bugzilla 5.0 and later.

  :``xmlrpc``:       Bugzilla XMLRPC interface.

  :``xmlrpc+email``: Bugzilla XMLRPC and email interfaces.

  :``3.0``:          MySQL access, Bugzilla 3.0 and later.

  :``2.18``:         MySQL access, Bugzilla 2.18 and up to but not

                     including 3.0.

  :``2.16``:         MySQL access, Bugzilla 2.16 and up to but not

                     including 2.18.

bugzilla.regexp

  Regular expression to match bug IDs for update in changeset commit message.

  It must contain one "()" named group ``<ids>`` containing the bug

  IDs separated by non-digit characters. It may also contain

  a named group ``<hours>`` with a floating-point number giving the

  hours worked on the bug. If no named groups are present, the first

  "()" group is assumed to contain the bug IDs, and work time is not

  updated. The default expression matches ``Bug 1234``, ``Bug no. 1234``,

  ``Bug number 1234``, ``Bugs 1234,5678``, ``Bug 1234 and 5678`` and

  variations thereof, followed by an hours number prefixed by ``h`` or

  ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.

bugzilla.fixregexp

  Regular expression to match bug IDs for marking fixed in changeset

  commit message. This must contain a "()" named group ``<ids>` containing

  the bug IDs separated by non-digit characters. It may also contain

  a named group ``<hours>`` with a floating-point number giving the

  hours worked on the bug. If no named groups are present, the first

  "()" group is assumed to contain the bug IDs, and work time is not

  updated. The default expression matches ``Fixes 1234``, ``Fixes bug 1234``,

  ``Fixes bugs 1234,5678``, ``Fixes 1234 and 5678`` and

  variations thereof, followed by an hours number prefixed by ``h`` or

  ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.

bugzilla.fixstatus

  The status to set a bug to when marking fixed. Default ``RESOLVED``.

bugzilla.fixresolution

  The resolution to set a bug to when marking fixed. Default ``FIXED``.

bugzilla.style

  The style file to use when formatting comments.

bugzilla.template

  Template to use when formatting comments. Overrides style if

  specified. In addition to the usual Mercurial keywords, the

  extension specifies:

  :``{bug}``:     The Bugzilla bug ID.

  :``{root}``:    The full pathname of the Mercurial repository.

  :``{webroot}``: Stripped pathname of the Mercurial repository.

  :``{hgweb}``:   Base URL for browsing Mercurial repositories.

  Default ``changeset {node|short} in repo {root} refers to bug

  {bug}.\\ndetails:\\n\\t{desc|tabindent}``

bugzilla.strip

  The number of path separator characters to strip from the front of

  the Mercurial repository path (``{root}`` in templates) to produce

  ``{webroot}``. For example, a repository with ``{root}``

  ``/var/local/my-project`` with a strip of 2 gives a value for

  ``{webroot}`` of ``my-project``. Default 0.

web.baseurl

  Base URL for browsing Mercurial repositories. Referenced from

  templates as ``{hgweb}``.

Configuration items common to XMLRPC+email and MySQL access modes:

bugzilla.usermap

  Path of file containing Mercurial committer email to Bugzilla user email

  mappings. If specified, the file should contain one mapping per

  line::

    committer = Bugzilla user

  See also the ``[usermap]`` section.

The ``[usermap]`` section is used to specify mappings of Mercurial

committer email to Bugzilla user email. See also ``bugzilla.usermap``.

Contains entries of the form ``committer = Bugzilla user``.

XMLRPC and REST-API access mode configuration:

bugzilla.bzurl

  The base URL for the Bugzilla installation.

  Default ``http://localhost/bugzilla``.

bugzilla.user

  The username to use to log into Bugzilla via XMLRPC. Default

  ``bugs``.

bugzilla.password

  The password for Bugzilla login.

REST-API access mode uses the options listed above as well as:

bugzilla.apikey

  An apikey generated on the Bugzilla instance for api access.

  Using an apikey removes the need to store the user and password

  options.

XMLRPC+email access mode uses the XMLRPC access mode configuration items,

and also:

bugzilla.bzemail

  The Bugzilla email address.

In addition, the Mercurial email settings must be configured. See the

documentation in hgrc(5), sections ``[email]`` and ``[smtp]``.

MySQL access mode configuration:

bugzilla.host

  Hostname of the MySQL server holding the Bugzilla database.

  Default ``localhost``.

bugzilla.db

  Name of the Bugzilla database in MySQL. Default ``bugs``.

bugzilla.user

  Username to use to access MySQL server. Default ``bugs``.

bugzilla.password

  Password to use to access MySQL server.

bugzilla.timeout

  Database connection timeout (seconds). Default 5.

bugzilla.bzuser

  Fallback Bugzilla user name to record comments with, if changeset

  committer cannot be found as a Bugzilla user.

bugzilla.bzdir

   Bugzilla install directory. Used by default notify. Default

   ``/var/www/html/bugzilla``.

bugzilla.notify

  The command to run to get Bugzilla to send bug change notification

  emails. Substitutes from a map with 3 keys, ``bzdir``, ``id`` (bug

  id) and ``user`` (committer bugzilla email). Default depends on

  version; from 2.18 it is "cd %(bzdir)s && perl -T

  contrib/sendbugmail.pl %(id)s %(user)s".

Activating the extension::

    [extensions]

    bugzilla =

    [hooks]

    # run bugzilla hook on every change pulled or pushed in here

    incoming.bugzilla = python:hgext.bugzilla.hook

Example configurations:

XMLRPC example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

XMLRPC+email example configuration. This uses the Bugzilla at

``http://my-project.org/bugzilla``, logging in as user

``bugmail@my-project.org`` with password ``plugh``. It is used with a

collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. Bug comments

are sent to the Bugzilla email address

``bugzilla@my-project.org``. ::

    [bugzilla]

    bzurl=http://my-project.org/bugzilla

    user=bugmail@my-project.org

    password=plugh

    version=xmlrpc+email

    bzemail=bugzilla@my-project.org

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

MySQL example configuration. This has a local Bugzilla 3.2 installation

in ``/opt/bugzilla-3.2``. The MySQL database is on ``localhost``,

the Bugzilla database name is ``bugs`` and MySQL is

accessed with MySQL username ``bugs`` password ``XYZZY``. It is used

with a collection of Mercurial repositories in ``/var/local/hg/repos/``,

with a web interface at ``http://my-project.org/hg``. ::

    [bugzilla]

    host=localhost

    password=XYZZY

    version=3.0

    bzuser=unknown@domain.com

    bzdir=/opt/bugzilla-3.2

    template=Changeset {node|short} in {root|basename}.

             {hgweb}/{webroot}/rev/{node|short}\\n

             {desc}\\n

    strip=5

    [web]

    baseurl=http://my-project.org/hg

    [usermap]

    user@emaildomain.com=user.name@bugzilladomain.com

All the above add a comment to the Bugzilla bug record of the form::

    Changeset 3b16791d6642 in repository-name.

    http://my-project.org/hg/repository-name/rev/3b16791d6642

    Changeset commit comment. Bug 1234.

'''

from __future__ import absolute_import

import json

import re

import time

from mercurial.i18n import _

from mercurial.node import short

from mercurial import (

    error,

    logcmdutil,

    mail,

    pycompat,

    registrar,

    url,

    util,

)

from mercurial.utils import (

    procutil,

    stringutil,

)

xmlrpclib = util.xmlrpclib

# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for

# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should

# be specifying the version(s) of Mercurial they are tested with, or

# leave the attribute unspecified.

testedwith = 'ships-with-hg-core'

configtable = {}

configitem = registrar.configitem(configtable)

configitem('bugzilla', 'apikey',

    default='',

)

configitem('bugzilla', 'bzdir',

    default='/var/www/html/bugzilla',

)

configitem('bugzilla', 'bzemail',

    default=None,

)

configitem('bugzilla', 'bzurl',

    default='http://localhost/bugzilla/',

)

configitem('bugzilla', 'bzuser',

    default=None,

)

configitem('bugzilla', 'db',

    default='bugs',

)

configitem('bugzilla', 'fixregexp',

    default=(br'fix(?:es)?\s*(?:bugs?\s*)?,?\s*'

             br'(?:nos?\.?|num(?:ber)?s?)?\s*'

             br'(?P<ids>(?:#?\d+\s*(?:,?\s*(?:and)?)?\s*)+)'

             br'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')

)

configitem('bugzilla', 'fixresolution',

    default='FIXED',

)

configitem('bugzilla', 'fixstatus',

    default='RESOLVED',

)

configitem('bugzilla', 'host',

    default='localhost',

)

configitem('bugzilla', 'notify',

    default=configitem.dynamicdefault,

)

configitem('bugzilla', 'password',

    default=None,

)

configitem('bugzilla', 'regexp',

    default=(br'bugs?\s*,?\s*(?:#|nos?\.?|num(?:ber)?s?)?\s*'

             br'(?P<ids>(?:\d+\s*(?:,?\s*(?:and)?)?\s*)+)'

             br'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')

)

configitem('bugzilla', 'strip',

    default=0,

)

configitem('bugzilla', 'style',

    default=None,

)

configitem('bugzilla', 'template',

    default=None,

)

configitem('bugzilla', 'timeout',

    default=5,

)

configitem('bugzilla', 'user',

    default='bugs',

)

configitem('bugzilla', 'usermap',

    default=None,

)

configitem('bugzilla', 'version',

    default=None,

)

class bzaccess(object):

    '''Base class for access to Bugzilla.'''

    def __init__(self, ui):

        self.ui = ui

        usermap = self.ui.config('bugzilla', 'usermap')

        if usermap:

            self.ui.readconfig(usermap, sections=['usermap'])

    def map_committer(self, user):

        '''map name of committer to Bugzilla user name.'''

        for committer, bzuser in self.ui.configitems('usermap'):

            if committer.lower() == user.lower():

                return bzuser

        return user

    # Methods to be implemented by access classes.

    #

    # 'bugs' is a dict keyed on bug id, where values are a dict holding

    # updates to bug state. Recognized dict keys are:

    #

    # 'hours': Value, float containing work hours to be updated.

    # 'fix':   If key present, bug is to be marked fixed. Value ignored.

    def filter_real_bug_ids(self, bugs):

        '''remove bug IDs that do not exist in Bugzilla from bugs.'''

    def filter_cset_known_bug_ids(self, node, bugs):

        '''remove bug IDs where node occurs in comment text from bugs.'''

    def updatebug(self, bugid, newstate, text, committer):

        '''update the specified bug. Add comment text and set new states.

        If possible add the comment as being from the committer of

        the changeset. Otherwise use the default Bugzilla user.

        '''

    def notify(self, bugs, committer):

        '''Force sending of Bugzilla notification emails.

        Only required if the access method does not trigger notification

        emails automatically.

        '''

# Bugzilla via direct access to MySQL database.

class bzmysql(bzaccess):

    '''Support for direct MySQL access to Bugzilla.