Running a Time-Limited Subprocess In Python (concurrency caveats inside!)

By Tuesday, January 13, 2015 3 Software Engineering Permalink 1

I tried to write a “simple” Python function. All it had to do is to run a command line in a subprocess, and enforce a timeout on that subprocess.

Turns out that this “simple” goal is not so simple. I considered multiple approaches to solve this, and ran into interesting concurrency issues.

In this post I describe a solution based on Python subprocess and various threading constructs. I demonstrate a concurrency inconsistency I discovered when testing my solution on Linux and OS X.

The conclusion, if that’s all what you’re looking for, is that timer.is_alive() is not a safe way to test whether a timer had expired!

Note: My experience is based on Python 2.7 with the default subprocess module from the standard library. If you’re on Python 3.3+, a timeout argument was added to subprocess. You can also install python-subprocess32, which brings this joy to Python 2.4-2.7.

Continue Reading…

Weekly Review, January 10

By Saturday, January 10, 2015 0 Personal Permalink 0

I thought I’d skip this weekly review too, but if I can get away with a minimalistic one, it will save me from a single-post week!

The reason for the relative silence is a rare combination of approaching DayJob milestone, and being sick for the entire week… Be warned that these conditions are expected to exist during the next week as well!

So I will leave you with the Google Analytics stats of this site from September 1st through December 31st 2014, while I go tend to my Sinusitis. Maybe next week I’ll try to analyze it a little.

The Weekly Review is a recurring (sort-of-)weekly summary, reviewing highlights from the last week.

Continue Reading…

App Highlights: Yatse, the XBMC / Kodi Remote

By Monday, January 5, 2015 0 Mobile , , , Permalink 1

Yatse is my Kodi (formerly XBMC) remote Android app of choice.

Kodi is a free and open source media center application. It’s what I currently use on my dedicated HTPC computer in the living room (via OpenELEC).

The Yatse Remote app allows me to control the media center from my phone. It’s so good doing that, that ever since I installed it, I never needed to pull out the HTPC keyboard / mouse!

There are plenty of alternative Kodi remote apps, including an official one from the XBMC team. I briefly used the official app a while ago (maybe early 2013?), and got quickly frustrated.

The Yatse app is free, with a $6.99 unlocker for extra features (and developer support). I never had any need for the extra features, so I purchased the upgrade only to support the developer.

Yatse also has various plugins for even more features!

App Highlights is a recurring series. From time to time, I highlight one Android app that I found useful. Feel free to suggest apps for me to highlight, but be advised that I focus on apps that I actually use.

Continue Reading…

Weekly Review, January 3

By Saturday, January 3, 2015 0 Personal Permalink 1

For the first time since starting this weekly review, I skipped a week! Did you notice? 😉

Anyway, I just finished “audio-reading” The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution. It was a long one! At 17:30 hours net play time, it took me about 4 weeks to finish it at 1.25x rate, but it was worth it! I followed a recommendation by Jamie Todd Rubin, and enjoyed the excellent stories laying out the evolution of the digital age. I always thought that “historical subjects” can become interesting easily if told through stories of actual people involved, as opposed to dry facts and numbers. This book proved that point for me. As a common geek, I admit that it isn’t hard for me to find interest in stories about people ranging from Ada Lovelace through Sergey Brin, but I really think also non-geeks will find this book interesting.

The Weekly Review is a recurring (sort-of-)weekly summary, reviewing highlights from the last week(s).

Continue Reading…

Supporting the SCons Help Command and Quiet Flag

By Thursday, January 1, 2015 0 Software Engineering , , Permalink 1

This is the tenth post in my SCons series. The topic of this post is adding support for the silent / quiet flag, and the help command.

In previous episodes I’ve added various enhancements. Some of them print progress information (like processing modules, and two-pass processing). This makes my enhancements quite chatty, which isn’t a problem by itself. The problem is that these messages are printed also in SCons “silent mode” (scons -s), and this episode fixes that.

On the way, I also add a custom help message. Just because it’s fun.

The final result is available on my GitHub scons-series repository.

Continue Reading…

My Favorite Blog Feeds, 2014 Edition

By Tuesday, December 30, 2014 0 Personal , Permalink 2

I like being up-to-date on topics that interest me. The main tool I use to do that is RSS feeds. I use Feedly to subscribe to RSS feeds of information sources that I want to follow (mostly blogs).

While Feedly is available via web and mobile apps, I use the Android app almost exclusively. I might use the web version to organize my subscriptions from time to time, but I do the actual reading on my Nexus 5.

I made it a habit to catch up on my Feedly Must Read list as a filler, whenever I have “down time” (commute, waiting for something, coffee break, etc.). Much better than burning time on Facebook and such, in my opinion 🙂 . I religiously clear out my Must Read queue at least once a week (I estimate it’s currently around 1,200 items per week).

This post lists my favorite RSS feeds, as of December 2014.

Continue Reading…

Quick and Dirty Personal Social Analytics With Google App Engine

By Sunday, December 28, 2014 0 Software Engineering , , The Ostrich Website Permalink 1

I want to track basic metrics of my main social thingies – The Ostrich Facebook page, The Ostrich Twitter account, and The Ostrich Google+ page.

My short-term goal is to have simple graphs of some basic metrics over time:

  • Facebook page likes and shares, posts, post likes and shares.
  • Twitter followers (and following), tweets, favorites, retweets.
  • Google+ followers, views, +1’s, shares.

I’m not sure why I want this data, but if I do find a good reason in the future, I’d better start tracking it now!

In this post, I describe how I went from the idea to start collecting the data, to a deployed data collecting app using Google App Engine, in under 5 hours.

Continue Reading…

Automating Module Discovery in a SCons Project

By Thursday, December 25, 2014 0 Software Engineering , , Permalink 0

This is the ninth post in my SCons series. The topic of this post is automating module discovery.

Up until now, the module directories were hardcoded in site_scons/site_config.py. This was a natural choice, because you needed to specify the modules in the correct order. In the previous episode, I implemented a solution that supports arbitrary modules order. Now it’s much more natural to automate the process of module discovery, over manual specification.

As a developer in the project, you know that when you create a new module you need to create a SConscript to describe how to build it. It makes sense that the build system will be able to locate all modules to build, by looking for the SConscript files recursively from the project base directory.

In this episode, I describe how I implemented module auto-discovery by walking the project directory tree.

My implementation also correctly handles common caveats:

  • Avoid walking the build directory.
  • Skip “hidden” directories (like .git, and .whatever from your favorite IDE metadata).

In addition, my implementation provides several useful features:

  • Ability to limit recursion depth.
  • Support for “stop marker files”. If a stop marker file (e.g. .noscons) exists in a project directory, the directory (and all sub-directories) will be skipped.

The episode builds on top of the previous episode. The final result is available on my GitHub scons-series repository.

Continue Reading…

My Favorite Podcasts, 2014 Edition

By Monday, December 22, 2014 2 Personal , Permalink 0

Podcasts are a great way to consume information and entertainment. I use BeyondPod Pro to track, download, and listen to my favorite podcasts.

I listen to podcasts on my Nexus 5 during commutes and when I go running. I estimate this accumulates to around 12 hours per week. The total time is randomly divided between podcasts, audio books, and music. I estimate podcasts get about 5 hours per week. I usually listen to podcasts at 1.5x speed.

This post lists my favorite podcasts, as of December 2014. The list is taken from the BeyondPod Smart Playlist, which reflects the order and precedence of listening.

Continue Reading…

Weekly Review, December 20

By Saturday, December 20, 2014 0 Personal Permalink 0

Another week, another batch of assorted sick kids… I wonder, how do humans survive, as a specie??

Anyway, isn’t this Rubik’s cube solving robot super cool?! It’s trying to get funded now on Kickstarter. It’s not the final assembled robot that I found so cool, but the idea of building one! As far as I understand, this is what the Kickstarter project is about. Coming up with a kit that individuals (or schools) can buy and build and program themselves – that’s really cool!

You probably heard about the crazy Sony hack. I don’t intend to dig into the technicalities of this hack, or the political and business ramifications of it. Others are covering it more than enough. What I found interesting here is how this breach affects regular employees and their families. These employees didn’t expect all of their work and personal email (and more) to be released like that. I believe many of them, like lots of other “regular people” (myself included), “has nothing to hide”. When I use the Internet (in whatever form), I usually don’t worry too much about “privacy” – as I have nothing to hide. This breach makes me think – even though I don’t have anything to hide, how would I feel if my entire online activity is published like that? My personal information? Personal information of unsuspecting friends and family that happened to interact with me online? That’s unsettling…

Speaking of hacks, have you heard of the RevSlider vulnerability that made lots of WordPress sites exposed to attacks (AKA SoakSoak)? Luckily, I don’t use the vulnerable plugin on this site, so I’m good. Do you have a WordPress site? Are you vulnerable?

The Weekly Review is a recurring (so-far-)weekly summary, reviewing highlights from the last week.

Continue Reading…