insomnihack

When sleep eludes, the keyboard beckons

Python Unit Testing with Mock

January 24, 2010 Dale 4 Comments

In the past, while creating unit tests for C# classes, I’ve frequently used a mock object framework such as Moq to isolate classes under test. When I started working in Python, I initially didn’t think I’d need such a framework to write my tests. Creating a stub with a given set of attributes and methods is easy in language like Python.

However, I soon noticed I was writing a lot of these little stubs, and this was ‘dirtying up’ my nice, readable tests. The next problem I encountered was 3rd party libraries with more complex interfaces, like this Twitter library. It contains a client class that dynamically wraps Twitter’s RESTful API. So instead of calling http://twitter.com/statuses/friends.format, you call Twitter.statuses.friends(). The Twitter client doesn’t have a statuses.friends method defined, it creates the URL dynamically from the dot notation you enter.

Creating fakes to test code using such a class would be tedious. Additionally, I was looking at having to pass in instances of other 3rd party classes I was using. The whole thing was getting dangerously close to the DI/IoC mess we’d left behind when we switched from C#. As Jonathon commented a few weeks back:

In Python, hard is a smell.

Enter Mock. Mock is a mocking and testing library for Python by Michael Foord. Unlike many of the other mocking libraries for Python that use the record/replay pattern, Mock uses an action/assert pattern similar to Moq.

Below is a snippet of code that uses the previously mentioned Twitter RESTful client with the dynamic API methods. I’d like to mock up the actual API calls so that I’m not connecting to Twitter when I run my tests.

[python]
from twitter import Twitter, TwitterError

class TwitterClient(object):

def __init__(self, login, password):
self._twitter = Twitter(login, password)

def all_friends(self):
return self._get_statuses(
lambda crs: self._twitter.statuses.friends(cursor = crs))

def all_followers(self):
return self._get_statuses(
lambda crs: self._twitter.statuses.followers(cursor = crs))

def _get_statuses(self, call):
crs = -1
results = []
while crs != 0:
page = call(crs)
users = page[‘users’]
crs = page[‘next_cursor’]
results.extend(users)
return results
[/python]

Mock makes this easy.

[python]
class TwitterClientTests(unittest.TestCase):

def test_all_friends(self):
"""Verify TwitterClient.all_friends()"""
tc = TwitterClient(‘username’, ‘password’)
tc._twitter = Mock()
tc._twitter.statuses.friends.return_value = \
{‘users’:[{‘a’:1}], ‘next_cursor’:0}
result = tc.all_friends()
tc._twitter.statuses.friends.assert_called_with(cursor = -1)
self.assertEquals([{‘a’:1}], result)
[/python]

After creating my test class, I mock out the actual internal Twitter instance that is created. I then set the return value for the API method that all_friends will pass to _get_statuses. I don’t need to reproduce the full JSON returned from a real Twitter API call, I just need enough to test a single loop through the cursor-based call. Once I’m finished, in addition to validating the results returned using regular TestCase asserts, I can verify that my mock method was called with the correct parameters.

This just scratches the surface of what you can do with Mock. It contains facilities for patching, which is useful if you want to mock objects that get created during a method call (i.e. they aren’t easy-to-get-at class members like self.foo). Check out Mock’s documentation and introduction for more details.

Programming Python, unit tests

R&D Week One Wrap Up

January 9, 2010 Dale Leave a Comment

Its a new year, and at work, the emphasis is definitely on the ‘new’. I’ve joined a new team, social media, in a new department: R&D. I have a new desk (to be honest, its just a new spot in the cube farm), and new operating system: Linux. We’re working with what is for most of the team a new programming language, Python, and a new continuous integration server: Hudson.

Hudson

Hudson is pretty impressive. I’d never even heard about it before Ed found it last Tuesday. In addition to continuous integration, it can also monitor executions of externally-run jobs. We’re trying to favor simpler cron jobs running scripts over the complex service-oriented architectures we’ve worked with on previous projects, so this is a nice bonus. Hudson comes with a ton of plug-ins that enable it to integrate with SCMs such as git, execute scripts of various languages including Python, and run an assortment of tools including nose, coverage, and pylint (just to name a few we’re using). Hudson is written in Java, and it is pretty easy to set up.

VMWare

We’re running all of our Linux systems on VMWare: Workstation 7.0 on our desktops and ESXi on the server. Workstation’s easy install makes setting up Ubuntu 9.10 with an initial user and VMWare toolkit a snap. However, I did notice that for some reason it didn’t install the shared folders driver, so I had to run VMWare tools setup manually anyway to get shared folders working. I still have Windows on my desktop with Visual Studio, IIS, and SQL Server installed in case I have to go back and fix a Studio bug.

Why Python?

The reasons behind the switch from C# and .NET to Python were faster development and easier scalability. I’m not saying we couldn’t achieve scalability with .NET, IIS, and SQL Server. I’m saying for what we’ll be doing, its easier to accomplish with Python, Tornado, and nginx. Since we’re new team, any budget increases have to be earned by achieving specific milestones, so it doesn’t hurt that all of our software tools with the exception of VMWare Workstation are now free.

Small Team

Finally, its really nice to be back on a small development team. I only have to report to one person, so those CC’d email sh*t storms where you’re trying to keep several stake holders informed of your status is no longer an issue. The other two developers I work with are on either side of me and both highly skilled, so most design and implementation discussions are usually resolved pretty quickly. We even pair programmed a few days last week. It was occasionally frustrating as everyone gets up to speed on Python, but there was definitely value in it. We’re being pretty aggressive with our milestones, so I don’t think we’ll be pairing all the time.

Programming Linux, Python, tools

Tracking followers with python-twitter

December 25, 2009 Dale 1 Comment

There are a lot of applications out there to help manage your twitter account. Much of what these applications have to offer can be achieved with very little python code. For example, say you want to find out who is following you that you’re not following. Alternately, say you want to find out who you’re following, but who isn’t following you in return. You can find this information out easily with python-twitter and a few lines of code.

[python]
#!/usr/bin/env python

import twitter

if __name__ == ‘__main__’:
api = twitter.Api(username=’username’, password=’password’)
i_follow = api.GetFriends()
follow_me = api.GetFollowers()
i = [u.name for u in i_follow]
f = [u.name for u in follow_me]
i_should_follow = [n for n in f if n not in i]
i_should_drop = [n for n in i if n not in f]

print ‘I follow:’
print ‘\n’.join(i)
print
print ‘Following me:’
print ‘\n’.join(f)
print
print ‘I should consider following:’
print ‘\n’.join(i_should_follow)
print
print ‘I should consider not following:’
print ‘\n’.join(i_should_drop)
print
[/python]

Replace the dummy user name and password and give it a go.

Programming Python, twitter

« Previous Page
Next Page »

About Me

Hi, my name's Dale, and this is my tech blog. Read More…

Search

Tags

blogging c# database Django documentation Email furniture git hacks htpc IPython Linux mac nas packaging PyCharm Python Roleplaying tools twitter unit tests virtualenv Visual Studio vmware Windows

Copyright © 2025 · Equilibre Theme on Genesis Framework · WordPress · Log in