Amethyst Reese

I’ve been spending time migrating my server from Ubuntu 10.04 to Arch Linux, and in the process I thought very deeply about every PHP application installed on the old server. Five out of the six PHP apps were only there to support the one that really stood on its own: MantisBT, my long-standing bug tracker of choice.

Now, I’ve been a core developer, and de facto release manager, for the project for many years — since I was still in university and getting paid by my then-employer to contribute features and plugins that they wanted to use for their engineering team. Those plugins, like Source Integration, wouldn’t be free without me fighting to license and release them for the community. And I wouldn’t be where I am today without the experience and help I received in turn from the very same community.

I know I haven’t been as involved in the project these days as I would like to be; there is an endless list of features and improvements to make to both the core system and the array of plugins I’ve created for it. Some great community members have stepped up and filled my place at times, while I have at least tried to stay active on the mailing lists and in the IRC channels. I’ve still guided and cut the last couple releases, but I haven’t played a part in shaping the future of the project.

There are multiple competing visions charting new paths for the aging project, with conflicting goals and revision histories, and it really needs a stronger leader to take the reins and guide the project to its next milestones. I’m unfortunately not the person to fill this role, for many reasons. Maybe a few years ago it would have been better timing.

Lately, I’ve come to the realization that I can no longer bring myself to work with PHP for personal projects. I don’t like the syntax, I don’t like where the language is heading, I don’t like how much memory and CPU it requires to run on a web server, and I just spend the whole time wishing I was writing Python code instead.

This blog hasn’t run on PHP or a database for just over a year now, and with my MantisBT install being used mainly for projects I don’t have the time or will to work on, it just seems to be dead weight. Github can serve my needs well enough for the few remaining projects I work on, and without needing a complicated setup on my end. Turning off MantisBT means I no longer need MySQL, PHPMyAdmin, APC, or even mod_php at all. The remaining apps can easily be replaced with external services.

So basically, this is me announcing what I’ve already been practicing for many months now: I will no longer be a developer for Mantis Bug Tracker, but I will remain involved as a mentor for other core developers, or for those seeking some advice on my plugins or creating their own. I won’t be maintaining any of my plugins, but I will look at and accept pull requests until someone else wants to step up as maintainer. I will be removing MantisBT from my site, but will keep a database dump in case I ever need to reference it in the future.

This is not me withdrawing from open source; I have many other projects that I’ve been working on, most of which are written in Python or C++. I find them more enjoyable to deal with, and most importantly, they allow me to break out of the realm of writing web applications. IRC, as old as it is, has been my point of intrigue lately, and is at the core of my current “pet” project.

Regardless of language, you can still find me on Github, where all my toys are available for the public to point and laugh at. And as always, I will answer questions on Freenode as “amyreese”, or via email, although there may sometimes be a long delay before I can reply.

Thank you to everyone who’s contributed to MantisBT or its plugins, and thank you to everyone who helped me on the way to where I’m going.

When users are having troubles with setting up the source integration plugin with their repositories, I’ve found that there is a common set of pain points. I’m going to list them here, along with their respective solutions, in hopes that users can more easily find the fixes in the future. This post will be updated as new tips or advice arrive.

General

• Linking changesets to issues requires commit messages to match the regular expressions specified in the source integration configuration page. By default, phrases like “fixes #123” or “resolved #123, #123” are supported, in present, past, singular, and plural forms.

• Remote checkin must be enabled for use by post-commit hooks that don’t use an API key to authenticate, and must list the IP addresses or blocks for any repository server that will push commit data to Mantis.

• Remote imports must be enabled for use by cronjob-based scripts that will trigger Mantis to pull in new changeset data for a given repository.

Subversion

• Your web server needs to have the SVN client binaries installed, and they must be accessible by the account that your web server uses. Similarly, if the binaries are in a directory that’s not included in your web server’s default path, you will need to specify the directory path in your source integration configuration. On Linux, this is usually /usr/bin or /usr/local/bin — on Windows, something like c:/path/to/subversion/bin is likely required.

• For Windows servers, you may also need to enable the “Windows start” source integration option.

• Make sure that the individual repository URL that you use for Mantis is the same URL you would use to checkout the repository when using an SVN client.

• When using repositories that require authentication, make sure the username and password you enter in Mantis has read access to the entire repository.

• If your repositories are hosted using HTTPS/SSL, and your server is using self-signed certificates, you will need to have SVN version 1.6 or newer installed, and you will need to enable the “Trust all SSL certs” option from source integration.

• If your commit messages contain UTF-8 or other non-ASCII characters, or your system’s locale is not set to “en_US”, then you may need to look at issue #93 and #130 on my bug tracker for help modifying your server or SourceSVN plugin to use an appropriate encoding or locale.

Git

• If using Gitweb, versions newer than 1.6 may not work correctly, as the Gitweb plugin is scraping HTML from the viewer, and changes to the HTML structure will break the Gitweb plugin. The version of Gitweb in use on MantisForge is known to work with the plugin.

• Using Github with private organization repositories is not yet supported. Private repositories for normal user accounts, however, is supported. Enter the username and API key from any Github account with access to the repository, and it should import correctly.

• Using branches other than “master” will require you to modify the individual repository configuration in Mantis to list any branches that you want to be imported.

Today marks an incremental release to the Source Integration plugin framework for MantisBT, now at version 0.16.3. This marks the introduction of support for using an API key to authorize inbound changeset data from repository data sources. It is initially supported for integration with Github, where large array of servers made it all but impossible to whitelist allowed IPs for remote commit data.

Future plans for the feature include support for configuring multiple API keys simultaneously, as well as adding support for using the API keys from other data sources, such as SVN or Gitweb repositories. Additionally, the old options for listing allowed IP addresses is now considered deprecated, and will likely be removed entirely at the next major release.

To set up an API key in your MantisBT instance, visit the Repository Configuration page, where you will find a new option labelled, strangely enough, “API Key”. You will need to generate your own key; the best way to do this is on a machine with OpenSSL by running the following command to create a secure, random string of hexadecimal digits, and then copying the resulting string to MantisBT:

$ openssl rand -hex 12

Once this is done, you can enable this on your Github projects by visiting the Service Hooks admin page for your repositories, activating the MantisBT hook, and copying the same key string into the “Api Key” field there. Any future pushes to your Github repo should send data to your MantisBT install using the API key to authorize the data submission.

I really like PHP as a technology, both for its extensibility and its deployment style. I think it is the quickest and most straightforward platform to create web applications with, and frameworks like CodeIgniter make it even better.

I’ve long been on the fence regarding PHP’s type coercion and comparison issues, but a recent bug in Mantis Bug Tracker has made me /facepalm for the first time in my long history of working with PHP:

When I click on “Edit” next to 1.2, mantis shows me the 1.20 properties. When I click 1.1 it shows me 1.10!

The offending snippet of code:

foreach( $g_cache_versions as $t_version ) {
   if ( ( $t_version['version'] == $p_version )
      && ( $t_version['project_id'] == $c_project_id ) ) {
      return $t_version['id'];
   }
}

At first glance, it seems perfectly normal… and then you read the commit log, emphasis mine:

This is due to an incorrect version name comparison in version_get_id whereby the following check between strings was occurring:

if( "1.1" == "1.10" ) { ... }

PHP evaluates this expression to true because 1.1 and 1.10 are treated as floats. We however need to preserve the string type during this comparison, thus we need to use the === comparison operator instead.

I thought I’d seen it all…

Chris Dornfeld at Unitz has an excellent, detailed overview of the options available when importing Subversion repositories into MantisBT using my Source Integration framework. My previous article on the topic fails to go into particulars with SVN repositories, mainly because many of the options weren’t even dreamed of at the time, let alone implemented.

Anyways, the information and explanations presented in his article are pretty accurate. Until I can get the time to fully document the SVN and Git plugins, or the framework as a whole, consider Chris’s post to be canonical information.

Cheers

Since I’m about to present the topic, I’m making the presentation slides available for all. There will indeed be video of the talk, which I will post as soon I can afterwards.

The slides are hosted directly on Google Docs: http://docs.google.com/Presentation?id=dffhxmhs_1273ct3kc8

Context: BarCamp Rochester — Anyone and everyone is invited to attend, and everyone is highly encouraged to present something of their own, no matter what it is. It’s happening this weekend at RIT, where I’m studying (and graduating in about a month!) for Software Engineering.

I’ll be attending it to give a presentation on MantisBT and the Source Integration framework. Specifically, I’ll be covering the myriad of new features that have made it into the project over the past week and a half. I plan to walk through setting up a project in Mantis, creating a new repository on GitHub, linking the Source Integration framework to that, and showing how the branch mapping and auto-resolving features work. Should be interesting.

After the presentation, I plan to post the slides, along with my presentation notes, up here. If anyone decides to video the talk, I’ll also make sure I can get a copy of that as well, but no promises.

Cheers!

Update: The information in this post is unfortunately out of date. While much of the code samples are similar, the new method of creating new plugins has changed in subtle, but very significant ways. Until a new guide can be written, please refer to the existing plugins for code examples.

Considering that my last post on Integrating Git and SVN has garnered a fair amount of attention, I thought that it would be useful to discuss my Source Integration framework in more detail. Specifically, I’ll be covering topics such as the design and implementation of the framework and, more importantly, how developers can go about implementing support for other version control tools.

The point of this is to show that it’s quite possible to integrate just about any type of version control tool with the Source Integration system; indeed I planned from the beginning to create a generalized framework that would support many different types and paradigms for version control. This should at least be evident in that I have already created extension plugins for Git and Subversion - it should be quite possible to extend the concepts further to Mercurial, Bazaar, CVS, or any other tool.

For the point of brevity, I’ll make the assumption that the developer at least has a fair understanding of PHP, their version control tool, and how events and plugins work in MantisBT. If you are not yet familiar with the plugin system, there is currently a basic introduction in the MantisBT Developer’s Guide, which I’ll hopefully be adding more information to in the near future.

With the ongoing work towards a 1.2 release, the Mantis Bug Tracker features a brand new plugin and event system, which will allow users to implement entirely new features for MantisBT without ever needing to touch any of the core codebase. It’s a very extensible system, and allows plugin authors to implement only what they need, while still allowing advanced plugins as much flexibility as possible. Plugins can be as small as a single file with 20 lines of code, or as large as entire hierarchies of files, pages, with their own API’s.

As the core developer of the new plugin system, I have been working on a variety of plugins. In particular, I have created a vastly improved method of integrating source control repositories within MantisBT. The plugin package is named, aptly enough, Source Integration, and implements a generic framework that will allow integration with multiple repositories, each potentially using any source control systems available, simply by creating an extension plugin for each new tool. Currently, I have implemented integration packages for both Git and Subversion, my two source control tools of choice.

The Source Integration package tracks repository information based on a series of changesets, each of which may have a list of affected files. The data representation is generic enough to cover version control concepts used by all types of tools, from the ubiquitous CVS and Subversion, to modern distributed tools like Git and Hg. However, the system takes the stance of implementing as few details as possible, so it relies on existing repository-viewing tools for tasks such as viewing commit diffs, file contents, tree browsing, etc. Extension plugins handle translating tool-specific information, like history logs or checkin data, into the generalized data objects used by the framework. Extensions also generate URL’s for viewing files and diffs, but everything else is handled automatically by the core framework.

The true benefit of the Source Integration package lies in the amount of repository integration that it implements within MantisBT. When importing changesets from your repository, Source looks at the commit message of each changeset for references to bug numbers in your tracker, and sets up links in the database for any bugs mentioned. When viewing bugs mentioned in commit messages, a new section is displayed after the bugnotes called “Related Changesets”, giving a list of linked changes, including information about the changeset, such as the branch, author, timestamp, and a list of changed files.