Debugging python reference counts

If you are wondering about the progress of inkface, it’s coming soon. I was planning to release version 0.1.2 couple weeks ago with a cool app demonstrating the improvements in the library (A twitter client), but after I saw the abysmal memory performance of the app on my n810, I decided to wait. Past two weeks I spent understanding the memory leaks in the library. In this post I would like to share my experiences while debugging reference count leaks in my python bindings.

The primary documentation on the subject from python.org is the first stepping stone. Considering how daunting the task of memory debugging is - some inspiration always helps. Read this post from Guido which explains how to approach the task.

In most of the cases we start debugging the memory leak after we have seen the program crawling or ever-increasing heap profile graph from valgrind massif. Whichever is the case, the theory of reference counts as explained in above docs, doesn’t make sense in this scenario. In such case, the following steps may help you get started.

  • Put debug statements in the dealloc functions of your Python Type objects. This will tell you when are they getting called, if at all. They will be called when Py_DECREF call on your python object decrements its refcount to zero.
  • Identify the python objects that you suspect to be leaking (i.e. the objects whose ref counts are not reaching zero when you expect them to) and write a simple routine that dumps their refcounts. You can write this routine inside your C module or in python - depending upon how your code is organized. In C you can find PyObject’s refcount by its member ob_refcnt. In python you can do the same by passing the object to sys.getrefcount() function. Call this refcount-dump routine from various places in your code and monitor how the reference count of these objects varies.
  • By calling the refcount-dump routine at strategic points you will soon narrow down to the area that is increasing the ref counts unexpectedly (or more correctly forgetting to decrease it when the job is done). Now look in the Python-C API docs for the behavior of the API calls you are making in this particular code segment. Understand the meanings of borrowed reference and new reference. Soon, you will find the places where you are supposed to call Py_DECREF, but you haven’t.
    While debugging my code I had following observations.

  • The ref counts on my objects were astronomical. After a while I figured that they were increasing with time. This observation clearly showed that the bug was in a loop structure. Soon I found the problematic code and put Py_DECREFs.
  • One common place of forgetting to release the ownership of an object is while iterating over a sequence using PyIter. Note that, PyIter_Next returns a new reference and you have to release it (roughly at the end of the iteration)
  • Sometimes Heisenberg’s law kicks in (“you can’t observe something without influencing it” as mentioned by Guido in above post). In my case, the python objects I was tracking were saved as values in a dictionary. So the only way to refer to them was

    sys.getrefcount(dict.values()[index])

This leads to a creation of list of values, thus incrementing the reference count of the object I am monitoring. There is no easy way that I know of to work around this, but just taking into consideration these additional refcounts helps.

  • In my code, I have a pure python class which encapsulates the objects created by C library. So the deallocation of C objects was dependent on the deallocation of pure python object. So I had to track its refcounts as well. One surprising thing I noticed was this object had very high refcount immediately after its instantiation. e.g.

    o = someclass()
    print sys.getrefcount(o) # 11

This was very unexpected. The reason behind it lied inside its constructor of course. This class has several methods. And inside the constructor I register these method objects as callback handlers. This understandably increases the refcount of method objects; however as it turns out it also increases the refcount of the parent object. So in my case, I got rid of these references by unregistering the callback handlers when I wanted to release the object.
After the whole exercise, I concluded that debugging memory leaks by means of debugging reference count leaks is a more useful way, compared to going through valgrind logs of a misbehaving C/C++ program.

These are my findings based on less than one day’s work. If you find any mistakes in my understanding, please do point out.

And yes… an update on Inkface is coming soon.


Home


Archive

2018

Why gamers like pixelated games

2017

Collecting Vs Hoarding Netflix

Game Design tip for Programmers - Less Automation

Review Ghost In The Shell - The blinds scene

Why Electron is the best development platform?

Project Dialog

2016

Don't Panic

The Entanglement Pattern

The flying cars analogy

Road Trip of Atlantic Canada

Having done attitude

2015

Before fixing performance issues

No Exams, Yes Video games

Podcasts Français

2014

Quotable Quotes

Conned?!

2013

Uruguay's President

New rules for Formula 1

Software Engineering Code of Ethics and Professional Practice

Something's missing

2012

Coffeescript: To switch or not to switch

Water from air

The single most reason that keeps me using Vim

Time travel fiction

When your git branch diverges from your remote

Something like Olivia - John Mayer

Online courses - New technology, Old rules

Kelsey Grammer is spectacular

Why Patriotism doesn't make sense

Why sea steading is important to pursue?

Prometheus

Rolling back latest git commit

Google Lunar X Prize

iTunes Download stopped (err =9006)

Good essay on user centric software design

Why we need to explore Seasteading?

2011

Master pieces from screen

How to write interactive CLI utility in node.js

The Gods Themselves by Isaac Asimov

Why OpenGL ES is so hard?

How to print stack trace anywhere in Javascript

It's not fair to compare Android with Internet Explorer

How to debug WebWorker threads?

Why can't I criticize Government if I don't vote? I sure can.

Elon Musk quote

Don't label yourself

Why the hi-tech minds are writing Social apps?

Truce with exercise

JavaScript is the next C

Ever growing todo list is a good sign for creative projects

Lesser women in tech... So what?

HTML5 offline cache - use swapCache carefully

How Wikipedia and Flattr can together change the World

2010

Discovering LastPass

Functional programming in Javascipt using Underscore.js

Youtube ETA - Chrome extension

Why secure airports when it's planes that fly?

Mac book red light audio jack problem on Linux

How to save HTML5 canvas image to Google App Engine

Why don't they publish eBooks for free but with embedded ads?

Silicon Valley and Hi-tech

Reality distortion field around

Indirect experiments

Music sharing

ReaderScope 2.0.6 - Find Comments

ReaderScope 2.0

predictably irrational

ReaderScope upcoming changes

Tuxmac

Why "Open" is not the first concern of user?

Mutt, Gmail, gdata, notify-send: A Perfect Email solution

Are you the Matt Damon of your game?

ReaderScope 1.9 - OAuth

New app "Find Comments"

@font-face-lift

ReaderScope 1.8.{2,3} - new authentication and global share

Portable apps for iPhone, Android, Pre, n900 - Can Processing.js help?

ReaderScope 1.8.1 - rethinking features

ReaderScope 1.8 - Random

Android fragmentation : How big a deal is it?

on Dropbox

iLikeIt

ReaderScope v1.7 - Widget, new Icons and .... Ads

on Git: tips for SVN users and Git in Dropbox cloud

How to do an online IDE for Android app development

Anti Asserts

Bono's top ten list

Make Android speak your Twits with "TwitSpeak"

2009

Reality of colors

Talking Twitter client

ReaderScope 1.5 - quicker, better, prettier

Preview android.R.drawable.* images

Is definition of "Evil" changing?

on Music: Battle Studies

ReaderScope 1.4 (with a Theme Pack)

On Chrome OS

Black on White

Android AsyncTask template

The AND-OR problem of laptop and mobile phone

ReaderScope v1.3.3 - offline podcasts

ReaderScope v1.3.2 - Social Channels

ReaderScope v1.3 - Beyond Google Reader

Frets on Fire for Maemo - Update

ReaderScope 1.2.7 - Podcasts, Expanded View

Sprint HTC Hero problems with ReaderScope

ReaderScope 1.2.3 - Search

ReaderScope 1.2.1 - Auto Login

ReaderScope v1.2

ReaderScope 1.1.3 - SDCard storage

Crash report for Android App

Wifi and Battery

ReaderScope 1.1.2

ReaderScope 1.1.1 quick-n-easy features

ReaderScope 1.1 - Social Beacons

ReaderScope 1.0.2 - making deep cache simple

ReaderScope 1.0.1

ReaderScope 1.0

Detecting HTC magic on Fedora

ReaderScope beta4 - offline fun

ReaderScope beta3 - Tweet the news

ReaderScope Beta2

ReaderScope beta1 is out

Intercepting Up/Down keys from bash script

Makeshift XML beautifier in python

HTC Magic (Indian version) - expectations and facts

using android AsyncTask

Android development with Vim + Eclim

Cutewit (preview)

Einstein's Dreams

Preprocessing the GUI

Evas + Inkface in v0.2.5

easy source

FoF - PyOpenGL = miniFoF (possibly for n810)

FoF update

Twitter client with inkface-pygame v0.2.2

Frets on Fire on Maemo 5 (Fremantle)

Static IP configuration on Fedora 10

Inkface+pygame v0.2.0

Smooth scrolling with Inkface+pygame

Inkface v0.2 update and automated GUI testing

Testing Android (Koolu Beta 3) on Openmoko Freerunner

Clutter animation paths with Inkscape+Inkface (v0.1.3)

Turning a page

Planning Inkface v0.2

Debian repository on Google App Engine

Inkface + Clutter

Inkface v0.1.2, Twitter-inkface client

2008

Inkface update

Debugging python reference counts

Paul Graham nails it again!

Fedora 10 on MacBook 2,1

A great essay

Thanksgiving at Crater Lake

Digital SLR buyer's guide

more fun with notify-send

(k)ubuntu 8.10 - not ready for my home yet

Migrating to Google App Engine

Android opened - first looks inside

Has it started?

Inkface v0.1

Musical discovery of the week

Die Berlin-Affäre

aus Berlin

Inkface - SVG based GUI design

Script fun

Visualizing source code

OpenMoko photo tour

New connection with the past

George Carlin - favorite clip

temperature

80-20 rule

Python fun

OSiM'08 Berlin, Germany

Of games

"Wine" tasting

Today's slashdot and open source

Cost of Open Source

Lions for Lambs

Ubuntu reviewed

Flickr uploads from n810

Cringely and Paul Graham

Accelerometer brainstorming

Clutter Webkit

OSiM USA 2008 - Day 2

OSiM USA 2008 - Day 1

Google code

Gr8 commercial

OSiM USA 2008, see you there

OpenMoko, Android and a great hacking post

two great videos

Blog from nokia tablets

2007

cool n770 hack

Picasso

Colbie Caillat

Firefox 3 is brilliant!!!

XML RPC server inside apache mod_python

best effort technologies

coding Monalisa

formula of a horror movie

next desktop jump

The "Screen"

Nice review article on virtualization technologies

Once again... a quote from 'Troy'

Story of a random number

quotes from George Carlin

"from time import sleep"

Sea surfing under the Golden Gate

Trip videos

2006

across two hours

What I read...

How do I backup to Amazon S3 storage service

Stranger than fiction

Gas prices... seems like big scam

Another nice quote

Hilarious nerd comment

Programming & Painting

Why do we like fiction?

Troy

If Harry Potter were a computer hacker ...

Gentoo, XGL, HGG

Henry Ford

VI - A capable IDE

New blog

Safari Bookshelf

2005

Back to the L4

Quotes

GCC 2.95 rpms

repeat (ps | grep) = filtered top

Halo fixed on ATI Mobility Radeon 9000

DirectX Installation Problem

Warning: const is a keyword in ANSI C

Not much on L4 front

Last two weeks

GRUBing

Stumbled on HURD

Installed HURD console

Make problem solved

2004

Handy way to create stack trace