Amethyst Reese

Along a similar vein as my previous post on Why I Prefer Android, Ralf Rottmann, a self-described “Apple fanboy”, has described why I prefer Android better than I could in my own words:

The latest version of Android outshines the latest version of iOS in almost every single aspect. I find it to be better in terms of the performance, smoothness of the rendering engine, cross-app and OS level integration, innovation across the board, look & feel customizability and variety of the available apps.

On the topic of app and system integration:

Another great example: Sharing stuff on social networks. On iOS, I have to rely on the developers again. Flipboard, as one of the better examples, gives me the ability to directly share with Google+, Twitter and Facebook. On my Nexus 4, I have 20+ options. That is, because every app I install can register as a sharing provider. It’s a core feature of the Android operating system.

…

All of this is entirely impossible on iOS today. I’ve stopped counting how often I felt annoyed because I clicked a link to a location in Mobile Safari and would have loved the Google Maps app to launch. Instead, Apple’s own Maps app is hardcoded into the system. And there’s no way for me to change it.

Regarding possibilities for app developers:

On iOS, many things I always wished to see being developed, simply cannot be done because of the strict sandbox Apple enforces around apps.

…

I also have apps [on Android] that give me great insight into the use of mobile data across the device and all apps. Or the battery consumption. Or which apps talk home and how frequently.

None of it is available for iOS. And possibly won’t be at any time in the near future.

And summing up the way I’ve felt for a long time when using iOS devices:

… whenever I grab my iPhone for testing purposes, iOS feels pretty old, outdated and less user friendly. For me, there currently is no way of going back. Once you get used to all of these capabilities, it’s hard to live without them.

There are many things Apple got right with iOS, like making a consistent user experience, and encouraging users to spend money in the market for quality apps. But when they place so many restrictions and limitations on how you can use your phone, and what your software is allowed to do on your own device, I gladly give up those things that make iOS so great for the freedom to run apps that can do what I want — and expect — from a modern computing device.

For the last couple years, this site has been managed and updated using the open source static site generator called Poole. It is an excellent and simple system, comprised of two Python files — the Poole source code and a site-specific macros.py — a simple page template, input documents and static content. This is more than enough for a small static website, but more complicated sites, like blogs, require quite a few macros in order to generate things like archives, tag pages, or even RSS feeds. Over the 18 months that I worked with Poole, my macro file had gotten a bit disorganization, and the limitation of working from only a single template file was starting to strain on what I could do with it.

Now keep in mind, I still like many of the features that drew me to Poole in the first place, such as the use of a document-centric build process, with YAML headers for defining page metadata rather than an inline, proprietary format used by projects like Jekyll. I also enjoy the simplicity of the content representations, but the lack of an extensible content pipeline is my biggest complaint. Supporting formats other than Markdown requires adding yet another hook to the macro file; generating archives and tag pages required hacking in a multi-phase build using os.exec(); and inlining another page’s content post-render was just not possible, resulting in Markdown named-link collisions when rendering multiple posts to a single page.

I wanted something new. So, for the past few months, I’ve been working on Nib.

The resulting design is based heavily on the concepts of Poole, but built around a primary goal of producing a proper content pipeline that is simultaneously aware of the differentiation between resources and documents, and defines multiple stages where plugins can hook into the process and alter or generate page contents at build time. Indeed, most of the actual functionality of Nib is contained within a handful of plugins, while the main module merely defines a framework for the content pipeline.

In effect, adding support for more content or resource formats should be as simple as adding a new plugin attached to the appropriate file extensions. Advanced content manipulation, generating “virtual pages”, and aggregating pages or documents into multiple locations are all possible as well.

The Markdown plugin is 14 lines of code; the LessCSS plugin is 13; even the blog plugin — which generates the archive pages, tag listings, and Atom feed — takes only 86 lines to do a better job than the old Poole macros that required double the effort.

Today, I’ve thrown the switch. While it looks nearly identical to the old site generated by Poole, the page you are now reading has officially been generated by Nib. The news feed sees the biggest makeover, having changed formats from RSS to Atom, and now offers full content posts. Link-style posts, a format that I’ve been wanting to experiment with, are also a new option. Both features would have required more effort on Poole than I was interested in expending, but the new architecture has allowed me to indulge myself.

Nib is still in an extremely early, and unstable, phase though. It works for the needs of my site, and does come with some basic documentation and a sample site to start from, but it’s far from complete. Near term goals include adding support for an intelligent menu, as well as support for more content and resource formats, like reStructuredText or SASS. Contributions are always welcome though, even at this early stage. Nib is liberally licensed, and I would love to hear feedback from anyone trying to use it. Hopefully it will be useful for someone other than myself.

Asked of me on Hacker News:

What do you prefer about Android?

I prefer Android because it allows apps to do more things for the user, and allows them to better integrate with the system as a whole.

I can replace the on screen keyboard with one that has a full five-row keyboard for times when I SSH into a machine. On a similar vein, when I SSH into a machine, I can actually leave the SSH session running in the background while I switch to another app, without fearing that the OS is about to kill my SSH app while I’m looking something up or responding to a text message. I can also leave my IRC client running in the background without it constantly needing to reconnect when I switch back to it.

Intents in Android, especially in combination with the global Share mechanism, allow any app to receive arbitrary data from any other app, meaning apps don’t need to know about specific apps or services in order to integrate with them. Clicking on a URL allows you to choose which browser (or set a default) to open the link in, allowing you to use alternate browsers (or alternate email clients, SMS apps, dialers, etc); tapping Share in the browser allows you to send the current URL to any application that can receive a URL, making apps like Instapaper, Pinboard, and 3rd party Twitter clients have the same capabilities as first party applications.

Sideloading apps means I’m not limited to installing programs from the Android Market/Play Store, and can do things like buy apps directly from the Humble Indie Bundle and install them on my own.

Proper background service support, and allowing apps to affect things outside their sandbox, lets me run programs like Locale that can monitor the phone’s status, location, etc, and modify the phone’s settings automatically based on a set of conditions that I’ve pre-arranged. My phone automatically silences itself at night time and while I’m physically at work, turns on my Wifi when I’m at home or work while defaulting it off when I’m out and about, and more.

That’s just some of the reasons I like Android better than iOS.

The past two months have been intense. At the beginning of May, I was let go from BioWare and Electronic Arts in a round of budget cuts for my studio. Since then, I’ve played far too many video games (and beaten multiple titles), slogged through innumerable interviews, redesigned my website from the ground up, and celebrated my seventh wedding anniversary in some of the best summer weather I’ve ever experienced.

Oh, and I got a job at Facebook.

On May 3rd, BioWare informed me that my position in the studio was being eliminated. My terrible luck had struck again. But this time was different; within a few days, a recruiter from Facebook found this website and my open source work, reached out to me, and encouraged me to apply for an engineering role. Four weeks later, after phone screens and an on-site interview, I was offered a position as a Production Engineer. My luck had not only reversed, it had gone off the scale in the opposite direction.

Alongside Facebook, all my other leads paled in comparison. I wanted to stay in gaming, but here was the opportunity of a lifetime, to join one of the biggest and best engineering companies in the world, and the chance to learn from some of the brightest minds in the industry. The position is similar to the role I had at BioWare: writing and maintaining software to manage and automate the vast array of infrastructure that supports the front-end applications and engineers. It was an offer I couldn’t refuse.

Monday was my orientation, and I was immediately impressed by the level of passion, vision and dedication from the entire team. Boz discussed the culture of “hack”, and told a story about building a loft in a war room when it couldn’t fit everyone in just two dimensions. Chris Cox gave an inspiring talk about where Facebook is heading, and I got a surreal sense of wonder when he mentioned our long term goals or the level of impact we have on global society and the way people communicate. It feels amazing to be part of a company that truly wants to change the world and improve the lives of hundreds of millions of people.

Even the training process embodies those same ideals. The first few weeks, every engineer goes through “bootcamp”, attending learning and development sessions, sitting among other bootcamp engineers, and working with veteran teammates on real tasks to get familiar with the infrastructure, tools and codebase. Everyone is given the chance to progress at their own pace, with as much or as little help as needed. I’ve already learned so much in a few days, and I’ve also had the chance to share my own knowledge and experience with my fellow newcomers. And yet there’s still so much left to cover.

Wednesday was my first day at a desk. Taped to the monitor was a sheet of paper printed with bold, red lettering.

What would you do if you weren’t afraid?

I don’t have an answer for that yet, but I will soon.

I’ve been spending time migrating my server from Ubuntu 10.04 to Arch Linux, and in the process I thought very deeply about every PHP application installed on the old server. Five out of the six PHP apps were only there to support the one that really stood on its own: MantisBT, my long-standing bug tracker of choice.

Now, I’ve been a core developer, and de facto release manager, for the project for many years — since I was still in university and getting paid by my then-employer to contribute features and plugins that they wanted to use for their engineering team. Those plugins, like Source Integration, wouldn’t be free without me fighting to license and release them for the community. And I wouldn’t be where I am today without the experience and help I received in turn from the very same community.

I know I haven’t been as involved in the project these days as I would like to be; there is an endless list of features and improvements to make to both the core system and the array of plugins I’ve created for it. Some great community members have stepped up and filled my place at times, while I have at least tried to stay active on the mailing lists and in the IRC channels. I’ve still guided and cut the last couple releases, but I haven’t played a part in shaping the future of the project.

There are multiple competing visions charting new paths for the aging project, with conflicting goals and revision histories, and it really needs a stronger leader to take the reins and guide the project to its next milestones. I’m unfortunately not the person to fill this role, for many reasons. Maybe a few years ago it would have been better timing.

Lately, I’ve come to the realization that I can no longer bring myself to work with PHP for personal projects. I don’t like the syntax, I don’t like where the language is heading, I don’t like how much memory and CPU it requires to run on a web server, and I just spend the whole time wishing I was writing Python code instead.

This blog hasn’t run on PHP or a database for just over a year now, and with my MantisBT install being used mainly for projects I don’t have the time or will to work on, it just seems to be dead weight. Github can serve my needs well enough for the few remaining projects I work on, and without needing a complicated setup on my end. Turning off MantisBT means I no longer need MySQL, PHPMyAdmin, APC, or even mod_php at all. The remaining apps can easily be replaced with external services.

So basically, this is me announcing what I’ve already been practicing for many months now: I will no longer be a developer for Mantis Bug Tracker, but I will remain involved as a mentor for other core developers, or for those seeking some advice on my plugins or creating their own. I won’t be maintaining any of my plugins, but I will look at and accept pull requests until someone else wants to step up as maintainer. I will be removing MantisBT from my site, but will keep a database dump in case I ever need to reference it in the future.

This is not me withdrawing from open source; I have many other projects that I’ve been working on, most of which are written in Python or C++. I find them more enjoyable to deal with, and most importantly, they allow me to break out of the realm of writing web applications. IRC, as old as it is, has been my point of intrigue lately, and is at the core of my current “pet” project.

Regardless of language, you can still find me on Github, where all my toys are available for the public to point and laugh at. And as always, I will answer questions on Freenode as “amyreese”, or via email, although there may sometimes be a long delay before I can reply.

Thank you to everyone who’s contributed to MantisBT or its plugins, and thank you to everyone who helped me on the way to where I’m going.

Hey man,

Just wanted to say thanks for the ZNC Push Module you made. I’m using it with Notify My Android and it works great! This should be a standard module in ZNC imo. Stellar work my friend.

Best,
~Lucien

It makes me smile every time I hear from someone using my software, knowing that I can make an impact on peoples daily lives, no matter how small it may be. Thank you to everyone who has taken the time to thank me for my work.

Last week, I received an offer from EA BioWare: they want me to join their San Francisco operations team as a Platform Systems Engineer! I still can’t believe that this is happening, but I’m going to be moving to California to work for a game studio. This is practically my dreams come true, and I’m excited and overwhelmed beyond my imagination.

I won’t be working on games directly, but I’ll be working in a devops role to automate and manage servers for games and internal projects. It’s a good fit with my experience in web applications, system administration, and tool development. It also can be a potential stepping stone to a game development position later on.

The next two months are likely to be hectic, as I move to temporary housing in California, and then find an apartment and move into it. Ongoing projects may have to take a back seat until I can get the free time again. But the weather and job will be worth it!

Based around the core conditions and functionality from my original project, ZNC Push is a module for ZNC that will send push notifications to multiple services for any private message or channel highlight that matches a configurable set of conditions, including the user’s /away status, time since the last notification, number of clients connected to ZNC, and more. Currently supported push services include Boxcar, NMA, Notifo, Pushover, Prowl, and Supertoasty.

The module is released under the MIT license, and the source code and full documentation can be found on the project’s Github repository page.

This is a post I’ve been meaning to write for a long time. I have a rather complicated setup involving multiple layers, but the end result is amazing. I maintain a 24/7 presence on the internet — on multiple IRC servers and instant messaging services — and I can send and receive messages from any computer or device I happen to be using at the time.

From any SSH client, I can seamlessly pick up my IRC session where I last left it, regardless of where I started that session. Similarly, I can connect with my phone’s IRC client, get a short backlog of recent conversations, or answer pending private messages. When I’m not already engaged in a conversation, I get near-instant notification on my phone and desktop, allowing me to respond at my leisure and from any location. I never miss a private message because I was connected from the wrong place, other users always see a single nick, and I get a central, searchable history of every channel and private message.

With this sort of setup, I gain a lot of freedom — to deal with conversations on my terms — and convenience. It’s served me well for a couple years, and I’ve enjoyed IRC much more since putting it all together. For each layer, I’ll detail the tasks it covers, the software I’ve chosen, and give a copy of any configuration files or options needed to replicate my environment.

Yesterday, my wife and I visited the National Museum of the United States Air Force at Wright-Patterson AFB in Dayton, Ohio. We walked through the three main hangars, mostly in chronological order. I snapped a lot of pictures along the way with my Nexus S; I picked out the blurry duplicates where possible, and captioned the photos as best I could. They actually turned out better than expected, considering the low light levels and how unsteady my arms got.

I didn’t get to take pictures of the XB-70 like I wanted to though; it was in their Research and Development hangar, which requires an ID check and bus ride to visit. Hopefully we’ll get to return soon and rectify that.

The full gallery is on Picasa, and contains 130 photos.

B-29 “Bockscar”

F-86 Saber

F-117 Nighthawk

F-22 Raptor

Spotify-Gnome is a program that provides Gnome media key support for the Spotify Linux client. It supports the play/pause, stop, next, and previous signals, and is compatible with both Gnome 2 and Gnome 3.

The Spotify client supports DBus for controlling the player, using the MPRIS Specification, but does not listen for basic media key signals provided by Gnome. This program acts as a “wrapper” around Spotify to translate media key signals from Gnome and send them to the Spotify client.

The program is released under the MIT license, and the source code and documentation can be found on the project’s Github repository page.

Many thanks to Mike Houston at kothar.net and Fran Dieguez at Mabishu for their blog postings that pointed me in the right directions to get this implemented.

When users are having troubles with setting up the source integration plugin with their repositories, I’ve found that there is a common set of pain points. I’m going to list them here, along with their respective solutions, in hopes that users can more easily find the fixes in the future. This post will be updated as new tips or advice arrive.

General

• Linking changesets to issues requires commit messages to match the regular expressions specified in the source integration configuration page. By default, phrases like “fixes #123” or “resolved #123, #123” are supported, in present, past, singular, and plural forms.

• Remote checkin must be enabled for use by post-commit hooks that don’t use an API key to authenticate, and must list the IP addresses or blocks for any repository server that will push commit data to Mantis.

• Remote imports must be enabled for use by cronjob-based scripts that will trigger Mantis to pull in new changeset data for a given repository.

Subversion

• Your web server needs to have the SVN client binaries installed, and they must be accessible by the account that your web server uses. Similarly, if the binaries are in a directory that’s not included in your web server’s default path, you will need to specify the directory path in your source integration configuration. On Linux, this is usually /usr/bin or /usr/local/bin — on Windows, something like c:/path/to/subversion/bin is likely required.

• For Windows servers, you may also need to enable the “Windows start” source integration option.

• Make sure that the individual repository URL that you use for Mantis is the same URL you would use to checkout the repository when using an SVN client.

• When using repositories that require authentication, make sure the username and password you enter in Mantis has read access to the entire repository.

• If your repositories are hosted using HTTPS/SSL, and your server is using self-signed certificates, you will need to have SVN version 1.6 or newer installed, and you will need to enable the “Trust all SSL certs” option from source integration.

• If your commit messages contain UTF-8 or other non-ASCII characters, or your system’s locale is not set to “en_US”, then you may need to look at issue #93 and #130 on my bug tracker for help modifying your server or SourceSVN plugin to use an appropriate encoding or locale.

Git

• If using Gitweb, versions newer than 1.6 may not work correctly, as the Gitweb plugin is scraping HTML from the viewer, and changes to the HTML structure will break the Gitweb plugin. The version of Gitweb in use on MantisForge is known to work with the plugin.

• Using Github with private organization repositories is not yet supported. Private repositories for normal user accounts, however, is supported. Enter the username and API key from any Github account with access to the repository, and it should import correctly.

• Using branches other than “master” will require you to modify the individual repository configuration in Mantis to list any branches that you want to be imported.

This past Friday, I bought the new Samsung Galaxy Tab 10.1, and I absolutely love it. The new Honeycomb interface looks amazing, and is quite an improvement over what’s available on my Nexus S with Gingerbread. I love that the back, home, and task switching buttons are now rendered on the screen instead of having hardware buttons, and combining those buttons into the notification and status bar means you don’t even lose screen space compared to previous versions. The new application switcher looks really nice, but does seem to have some odd behaviors, like not showing the browser that I just switched away from, or seemingly choosing at random whether to display the list from the top instead of from the bottom where you would expect.

There are a lot of applications that don’t yet take advantage of what’s offered by the Honeycomb APIs, but still tend to work really well. It mainly depends on how well the author designed the application to scale with the user’s screen size and density. An example of doing it wrong is the official Facebook app; it’s still usable mind you, but it certainly looks dumb in process, showing the main menu as a large grid of tiny icons with massive amounts of whitespace between them. It would have gone a long way if they had simply scaled the images to fill the screen.

I’ve been meaning to write about some of my most cherished apps, and seeing them in new form has given me an even better reason to get to it. Some of them really only work well on a phone, and others have only gotten better than ever when given a tablet form factor to call their own. So in no particular order, here are my favorite apps for Android. All prices are rounded up from Market estimates at time of writing, and all screenshots are taken from my devices:

Today marks an incremental release to the Source Integration plugin framework for MantisBT, now at version 0.16.3. This marks the introduction of support for using an API key to authorize inbound changeset data from repository data sources. It is initially supported for integration with Github, where large array of servers made it all but impossible to whitelist allowed IPs for remote commit data.

Future plans for the feature include support for configuring multiple API keys simultaneously, as well as adding support for using the API keys from other data sources, such as SVN or Gitweb repositories. Additionally, the old options for listing allowed IP addresses is now considered deprecated, and will likely be removed entirely at the next major release.

To set up an API key in your MantisBT instance, visit the Repository Configuration page, where you will find a new option labelled, strangely enough, “API Key”. You will need to generate your own key; the best way to do this is on a machine with OpenSSL by running the following command to create a secure, random string of hexadecimal digits, and then copying the resulting string to MantisBT:

$ openssl rand -hex 12

Once this is done, you can enable this on your Github projects by visiting the Service Hooks admin page for your repositories, activating the MantisBT hook, and copying the same key string into the “Api Key” field there. Any future pushes to your Github repo should send data to your MantisBT install using the API key to authorize the data submission.

I’ve recently been working on a parallel processing task in Python, using the multiprocessing module’s Pool class to manage multiple worker processes. Work comes in large batches, so there are frequent periods (especially right after startup) where all of the workers are idle. Unfortunately, when workers are idle, Python’s KeyboardInterrupt is not handled correctly by the multiprocessing module, which results in not only a lot of stacktraces spewed to the console, but also means the parent process will hang indefinitely.

There is quite a lot of suggestions for mitigating this issue, such as given in this question on Stack Overflow. Many places point to Bryce Boe’s article, where he advocates rolling your own replacement for the multiprocessing module’s Pool class, but that seems to not only invite bugs and added maintenance overhead, but also doesn’t address the root cause.

I have figured out (what I think is) a better solution to the problem, and have not found anyone else mentioning it online, so I have decided to share that here. It not only solves the problem of handling the interrupt for both idle and busy worker processes, but also precludes the need for worker processes to even care about KeyboardInterrupt in the first place.

The solution is to prevent the child processes from ever receiving KeyboardInterrupt in the first place, and leaving it completely up to the parent process to catch the interrupt and clean up the process pool as it sees fit. In my opinion this is the most optimal solution, because it reduces the amount of error handling code in the child process, and prevents needless error spew from idle workers.

The following example shows how to do this, and how it works with both idle and busy workers:

#!/usr/bin/env python

# Copyright (c) Amethyst Reese
# Licensed under the MIT License

import multiprocessing
import os
import signal
import time

def init_worker():
    signal.signal(signal.SIGINT, signal.SIG_IGN)

def run_worker():
    time.sleep(15)

def main():
    print "Initializng 5 workers"
    pool = multiprocessing.Pool(5, init_worker)

    print "Starting 3 jobs of 15 seconds each"
    for i in range(3):
        pool.apply_async(run_worker)

    try:
        print "Waiting 10 seconds"
        time.sleep(10)

    except KeyboardInterrupt:
        print "Caught KeyboardInterrupt, terminating workers"
        pool.terminate()
        pool.join()

    else:
        print "Quitting normally"
        pool.close()
        pool.join()

if __name__ == "__main__":
    main()

This code is also available on Github as amyreese/multiprocessing-keyboardinterrupt. If you think there’s a better way to accomplish this, please feel free to fork it and submit a pull request. Otherwise, hopefully this helps settle this issue for good.