Talk Python #304 - asyncio all the things with Omnilib

I joined Michael Kennedy on the excellent Talk Python podcast, episode #304, last week to chat about the Omnilib Project, an organization of open source packages I started, and how they fit into the modern world of Python and AsyncIO. We also discuss how I got started in programming and Python, with a rare glimpse into just how nerdy I was as a child. 😅

Listen in Overcast, any podcast player, or right here:

Talk @ North Bay Python 2019: What is a Coroutine Anyway?

Back in 2019–remember the before times?–I gave a talk at North Bay Python 2019 on the topics of coroutines, AsyncIO, and how they work. It’s a crazy journey from bytecode and runtime instructions to building our own terrible event loop in pure Python using nothing but generators and tears. Enjoy!

And if that was too slow, don’t miss the shorter rendition of this talk that I gave at PyCascades 2020, right before COVID ruined everything.

I Am a Gender Fluid Person

I am a gender fluid person. My preferred pronouns are they/them.

This is difficult for me to say publicly. I’ve been trying to find the right time and the right words for over a year now.

Most of my life, I’ve known deep down that I didn’t fit into the traditional male gender role or expectations, even if I didn’t understand what that meant. I was often insulted, ridiculed, or harassed on a regular basis for being myself. I learned through trial by error how to “blend in”, and suppressed other emotions or behaviors for fear of how I would be perceived by everyone else. There wasn’t a lot of introspection involved–just reactions to external judgement. Without the vast array of media that’s now become available to everyone these days, I don’t know that I could have fully understood who I was even if I’d tried. I just didn’t have the conceptual framework or cognitive tools necessary to unpack notions of gender roles, let alone to analyze how they affected me or my own position within them.

But now I do, and I’ve become more aware of who I’ve been all along. The past few years have been a slow awakening and acknowledgement of how I feel and what truly makes me feel happy. It’s complicated, and I don’t claim to have a full definition of who I am, but I like to believe it includes some combination of courage, femininity, intelligence, empathy, and emotional strength. I’ve started to embrace this in what I wear and how I express myself as a person. I’ve bought and worn more “women’s” clothes, shoes, and accessories in the past two years than traditional “masculine” items, and it’s been absolutely liberating. Some of my closest coworkers, friends, and family have seen a small number of these over the past couple years, but I suspect most have not realized what they were seeing, because they only see what I’ve been willing–or felt safe enough–to share with the rest of the world.

I live in daily fear that expressing my true self in public may result in uncomfortable, dangerous, or even deadly consequences. Transgender, non-binary, and queer people face incredible risk in our society, just trying to live as who we are. As we become more visible in society, the number of hate crimes being committed against us is increasing. Even in progressive areas, I’ve witnessed name calling and harassment of queer people by bigots who thrive on intimidation and violence. This is absolutely unacceptable in our society, but it “works”; fear is an effective tactic. I’ve refused to wear overtly feminine clothes in public, because no matter how many allies I have in this world, it only takes one asshole to ruin–or end–my life, and I can’t know who they are until it’s too late.

Coming out here won’t change that. I already expect to see harassment–or at least be treated differently–by those who read this. I’m prepared to delete comments, and block people online or in real life, that refuse to accept me for who I am. Being visibly queer will most likely carry a stigma and risk for the rest of my life, but maybe it will also let me be more comfortable around those that do accept me, and that alone will be worth it to me. I don’t know if I’ll ever feel safe being myself in public, but I’m fortunate to now have a full-time remote position at one of the most diverse and accepting companies I’ve ever seen, increasing the time I can spend with my best friend, being the person I’ve always been. Over time, I’m sure that how I express myself will change and adapt as I learn more about myself and what makes me happy. I hope that some day I can feel safe enough to share more of my authentic self in public as well.

I’m truly thankful to have an amazing best friend, wife, and soul mate in Angela, who truly accepts and embraces me for who I am. She is, and always will be, the love of my life. At every step of this journey, she has been my rock and my guiding star, and I will never be able to express just how much it means to have her support.

I would also like to thank my friends, coworkers, and every other visible queer person I know that took these steps before me, so that I can more easily understand who I am and who I want to be.

Talk @ PyCon AU 2018: Refactoring Code with the Standard Library

A week ago today, I gave a talk at PyCon Australia in Sydney. I discussed refactoring in Python, and how to build refactoring tools using nothing but the standard library, building up from concepts to syntax trees to how the lib2to3 module works. The talk finished with the announcement of the open source refactoring framework I built at Facebook, Bowler, which I built using those same concepts. I really appreciated the questions from the audience and in the halls afterwards, and we really enjoyed our time in Sydney. Thanks to everyone that made PyCon so great!

Talk @ PyCon US 2018: Thinking Outside the GIL

This past Friday, I presented a talk at PyCon US 2018 – the first time I’ve been fortunate enough to attend. The talk was focused on achieving high performance from modern Python services through the use of AsyncIO and the multiprocessing modules. The turnout was better than I could have ever expected, and I was really happy to hear from everyone that stopped by the Facebook booth to ask questions, discuss Facebook engineering practices, or even just say “hello”. Thank you to everyone who made my first PyCon amazing!

Interview: PyDev of the Week

This week, I was interviewed by Mike Driscoll of Mouse vs Python for their “PyDev of the Week” series, focusing on developers in the Python community.

A short quote, and timely announcement:

I’m currently preparing a talk for PyCon 2018 on using asyncio with multiprocessing for highly-parallel monitoring and/or scraping workloads. To go with this talk, I’m working on some simple example code that I hope to publish on Github. This will be my first major conference talk, so I’m both excited and absolutely terrified! 😅

I’m looking forward to giving that talk, and will post a video here afterwards!

Elite Dangerous

It’s no secret that I’m a huge fan of space, video games, or video games set in space. I was a very early backer of Star Citizen, and am still eagerly awaiting the release of the persistent universe. But I also backed Elite: Dangerous as well, and although the initial scope was far more limited than Star Citizen’s, Elite has already seen a “1.0” release, and I’ve been playing it an awful lot, all the way up from the first beta.

And damn, it’s good!

Voyager 1, found in the Sol system, ~2M Ls out
Voyager 1, found in the Sol system, ~2M Ls out

The discipline and focus of the team at Frontier Development has produced one of the most polished space sim games I’ve ever played, and while there are plenty of gaps in features compared to the competition, everything that’s there is done so well that I continue to be impressed every time I get in that cockpit and launch into the void.

After putting dozens of hours into the final release, I’ve gathered a few notes here that will hopefully help others in their journies to the stars. I will attempt to keep this up to date for as long as I’m playing the game.

Continue reading »


When I was in elementary school, I discovered a series of books in the library that detailed, in glorious full page color, photos of the outer solar system as captured by Voyager 1 and Voyager 2 on their epic journeys to places never before explored by humanity. I was so stunned and enthralled at the thoughts of visiting those distant worlds; it indeed captured my imagination, and continues to inspire me even now, two decades after I first learned of their amazing quests.

Jupiter's Great Red Spot, as photographed by Voyager 1
Jupiter’s Great Red Spot, as photographed by Voyager 1.

I’ve spent countless hours dreaming of visiting the planets and stars of our universe: in games like Wing Commander, Freespace, Freelancer, Mass Effect, and X3, exploring fictional galaxies, and in Orbiter Sim, recreating my own journeys to the planets and moons of our own solar system. I can’t wait for Star Citizen and X Rebirth to give me further ways to escape the confines of Earth and experience worlds that I will never be able to visit otherwise.

Volcanic eruption on Io, as photographed by Voyager 1
Volcanic eruption on Io, as photographed by Voyager 1

And I have NASA to thank for inspiring so many of these wonderful outlets, from the Mercury, Gemini, and Apollo projects that took us to the Moon, the Voyager missions for showing us just how beautiful our own back yard really is, to the Mars rovers for exploring our closest neighbor, and to future NASA missions like New Horizons that are leading the way towards understanding our tiny place in this vast universe.

The Voyager probe.
The Voyager probe.

Today, it was finally confirmed that Voyager 1 has indeed crossed the boundary of the heliosphere and forever left our solar system behind.


I simply find it incredible that not only is Voyager still working after four decades of exceptional service, but that it’s still capable of aiming its antennas at Earth, sending a coherent signal using only 23W of power, and actually sending useful data back to us. That alone is a hugely amazing achievement, and I think everyone at NASA should be eternally proud of their accomplishment. The fact that we now have even more data points regarding the boundaries of our solar system is just pure icing on top of that cake, and I congratulate everyone involved in the project.

If you asked me what my favorite spacecraft is, fictional or not, it wouldn’t be the Enterprise, the Millenium Falcon, the Normandy, or even the Apollo rockets; it would be Voyager. And it will forever hold a special place in my heart.

So thank you, NASA. Thank you.

Android is Better

Paul Stamatiou, on falling in love with Android:

I set out to write an article about how I feel Android provides unique affordances that create a unique cohesive mobile experience (more on that below) rather than talking openness, features and apps. However, the more time I spent living with Android it became obvious that being able to do anything and suit a variety of needs is a pillar of the Android experience.

I was trying to find out why Android felt so remarkably different to use, beyond aesthetics. I found there are a few pieces that help contribute to this magical user experience: the global back button, intents and Google Now.

Delight. It’s what designers strive to produce in the experiences they craft. Google Now has hit the nail on the head. I can’t even count the number of times friends have pulled out their phone to show me the smart things Google Now did for them.

Using Android feels like one fluid experience.

And it just keeps getting better and better!

Say "Yes" to JavaScript

Armin Ronacher, on Firefox’s removal of the option to disable Javascript:

My immediate response to change of removing the switch was: “Thank god, that should have happened ages ago”.

You don’t get extra privacy by disabling JavaScript. I can fully track you even without JavaScript. At the same time I can enhance your browser experience through better written JavaScript code that allows me to do things with your browser that plain HTML does not allow.

If you’re really paranoid about your privacy, use Firefox, enable the Do Not Track flag in your options, enable click to play for plugins like Flash or Silverlight, and install the Ghostery and Adblock extensions. Anything else is just going to break the web at your own expense.

What Microsoft Should Have Done

Soren Johnson, lead designer of Civilization 4:

The answer is to make digital games so attractive that players will abandon physical discs on their own. (One might call this the Steam strategy.) Microsoft could have avoided this whole fiasco by maintaining the old disc-based ecosystem while softly undermining it with three moves that create an alternate digital future.

Combined, these three changes would destroy the traditional retail market. The $40 price would make digital games cheaper at release; the ongoing heavy sales would undercut the used games market; and persistence would make digital games easier to maintain across multiple devices. Microsoft needs to make buying games digitally a better deal for the consumer than buying them physically.

I was extremely disappointed to hear Microsoft cave in to rabid demands to maintain the status quo. I was really looking forward to their plans for combined physical and digital ownership, where I could get all the benefits of buying physical copies, including special and collectors’ editions of my favorite titles, while simultaneously retaining all the benefits of a digital copy, like the ability to forego disc-swapping.

How long will we have to wait for consoles to catch up with Steam?

Push Push Push

One of my favorite projects that I’ve created has to be ZNC Push, a plugin for ZNC that generates push notifications for IRC highlights and private messages. It supports a wide array of push services for all of the major mobile platforms, and gives the user deep flexibility in choosing how and when to trigger these notifications.

However, I have very little insight into usage statistics for the plugin. Pushover is the only service that gives me gross usage data, but considering ZNC Push supports six other networks as well as custom push URLs, this only represents an unknown portion of the total usage. But I will continue to draw baseless conclusions from it regardless! Do note that I can’t see anything more than raw volume of notifications; I do not have access to who is sending/receiving these, nor the contents of those messages.

So without further ado, some pretty graphs straight from the Pushover dashboard:

Daily notification volume on the order of hundreds per day

ZNC Push reached a milestone of sending 15,000 push notifications in May through the Pushover service!

Push volume has been growing steadily since Pushover support was added, with a small dip in December and January that is likely attributeable to the holidays for most users. May marked the second occasion that ZNC Push has exceeded the volume “cap”, and Pushover has again graciously increased that in support of this open source project. At this sort of grawth rate, I would expect the next volume cap (25,000/mo) will be reached by October at the latest. I’m excited to see such growth for a small side project with such a niche use case.

Sometimes, it’s amazing to stop and realize just how far this project has come. What started in January of 2011 as a basic module for the Notifo push service, has blossomed into a mature system supporting multiple push services and far more customization options than I had ever envisioned from the beginning. Knowing that it’s being used by so many people, and helping them stay more connected with their online communities, makes me a very happy engineer.

Many thanks to Pushover for granting such a large volume of free notifications to ZNC Push. If you’re not already using them for your push service of choice, I highly recommend them. They have apps for both Android and iPhone, and they continue to improve them and add new features over time.

And as always, my eternal appreciation to everyone who has contributed time and energy to helping me improve the module over the past two and a half years! The project wouldn’t be where it is today without your support.

Favorite Android Apps of 2012

Eighteen months ago, I wrote an article about my Favorite Android Apps, but a lot has changed with Android and its app ecosystem since then. Developers have started putting more focus on design and usability when creating apps, and of course newer apps have come along that I have come to rely upon. I myself have upgraded to newer devices, with a Nexus 7 replacing my Galaxy Tab 10.1, and a Galaxy Nexus from work has sadly replaced my beloved but aging Nexus S. So what follows are some of my favorite new apps from 2012, and some follow-up on what my previous favorite apps look like in modern attire. All screenshots are taken from devices running Android 4.2.1 (CM 10.1 nightlies).

Continue reading »

Why Ralf Prefers Android

Along a similar vein as my previous post on Why I Prefer Android, Ralf Rottmann, a self-described “Apple fanboy”, has described why I prefer Android better than I could in my own words:

The latest version of Android outshines the latest version of iOS in almost every single aspect. I find it to be better in terms of the performance, smoothness of the rendering engine, cross-app and OS level integration, innovation across the board, look & feel customizability and variety of the available apps.

On the topic of app and system integration:

Another great example: Sharing stuff on social networks. On iOS, I have to rely on the developers again. Flipboard, as one of the better examples, gives me the ability to directly share with Google+, Twitter and Facebook. On my Nexus 4, I have 20+ options. That is, because every app I install can register as a sharing provider. It’s a core feature of the Android operating system.

All of this is entirely impossible on iOS today. I’ve stopped counting how often I felt annoyed because I clicked a link to a location in Mobile Safari and would have loved the Google Maps app to launch. Instead, Apple’s own Maps app is hardcoded into the system. And there’s no way for me to change it.

Regarding possibilities for app developers:

On iOS, many things I always wished to see being developed, simply cannot be done because of the strict sandbox Apple enforces around apps.

I also have apps [on Android] that give me great insight into the use of mobile data across the device and all apps. Or the battery consumption. Or which apps talk home and how frequently.

None of it is available for iOS. And possibly won’t be at any time in the near future.

And summing up the way I’ve felt for a long time when using iOS devices:

… whenever I grab my iPhone for testing purposes, iOS feels pretty old, outdated and less user friendly. For me, there currently is no way of going back. Once you get used to all of these capabilities, it’s hard to live without them.

There are many things Apple got right with iOS, like making a consistent user experience, and encouraging users to spend money in the market for quality apps. But when they place so many restrictions and limitations on how you can use your phone, and what your software is allowed to do on your own device, I gladly give up those things that make iOS so great for the freedom to run apps that can do what I want — and expect — from a modern computing device.

Powered by Nib

For the last couple years, this site has been managed and updated using the open source static site generator called Poole. It is an excellent and simple system, comprised of two Python files — the Poole source code and a site-specific — a simple page template, input documents and static content. This is more than enough for a small static website, but more complicated sites, like blogs, require quite a few macros in order to generate things like archives, tag pages, or even RSS feeds. Over the 18 months that I worked with Poole, my macro file had gotten a bit disorganization, and the limitation of working from only a single template file was starting to strain on what I could do with it.

Now keep in mind, I still like many of the features that drew me to Poole in the first place, such as the use of a document-centric build process, with YAML headers for defining page metadata rather than an inline, proprietary format used by projects like Jekyll. I also enjoy the simplicity of the content representations, but the lack of an extensible content pipeline is my biggest complaint. Supporting formats other than Markdown requires adding yet another hook to the macro file; generating archives and tag pages required hacking in a multi-phase build using os.exec(); and inlining another page’s content post-render was just not possible, resulting in Markdown named-link collisions when rendering multiple posts to a single page.

I wanted something new. So, for the past few months, I’ve been working on Nib.

The resulting design is based heavily on the concepts of Poole, but built around a primary goal of producing a proper content pipeline that is simultaneously aware of the differentiation between resources and documents, and defines multiple stages where plugins can hook into the process and alter or generate page contents at build time. Indeed, most of the actual functionality of Nib is contained within a handful of plugins, while the main module merely defines a framework for the content pipeline.

In effect, adding support for more content or resource formats should be as simple as adding a new plugin attached to the appropriate file extensions. Advanced content manipulation, generating “virtual pages”, and aggregating pages or documents into multiple locations are all possible as well.

The Markdown plugin is 14 lines of code; the LessCSS plugin is 13; even the blog plugin — which generates the archive pages, tag listings, and Atom feed — takes only 86 lines to do a better job than the old Poole macros that required double the effort.

Today, I’ve thrown the switch. While it looks nearly identical to the old site generated by Poole, the page you are now reading has officially been generated by Nib. The news feed sees the biggest makeover, having changed formats from RSS to Atom, and now offers full content posts. Link-style posts, a format that I’ve been wanting to experiment with, are also a new option. Both features would have required more effort on Poole than I was interested in expending, but the new architecture has allowed me to indulge myself.

Nib is still in an extremely early, and unstable, phase though. It works for the needs of my site, and does come with some basic documentation and a sample site to start from, but it’s far from complete. Near term goals include adding support for an intelligent menu, as well as support for more content and resource formats, like reStructuredText or SASS. Contributions are always welcome though, even at this early stage. Nib is liberally licensed, and I would love to hear feedback from anyone trying to use it. Hopefully it will be useful for someone other than myself.