Table of Contents

Python

Here we explain how you setup and install MemCachier with Python.

We recommend the python-binary-memcached memcache client as it is a pure Python library. However, if you prefer, you can try pylibmc which has a larger ecosystem.

This is a pure python client that supports the binary protocol and SASL authentication.

To install python-binary-memcached:

$ pip install python-binary-memcached

Be sure to update your requirements.txt file with these new requirements (note that your versions may differ than what’s below):

python-binary-memcached==0.28.0

Next, configure your memcached client in the following way:

import bmemcached
import os

servers = os.environ.get('MEMCACHIER_SERVERS', '').split(',')
user = os.environ.get('MEMCACHIER_USERNAME', '')
passw = os.environ.get('MEMCACHIER_PASSWORD', '')

mc = bmemcached.Client(servers, username=user, password=passw)

mc.enable_retry_delay(True)  # Enabled by default. Sets retry delay to 5s.

After this, you can start writing cache code in your Python app:

mc.set("foo", "bar")
print(mc.get("foo"))

Alternative client: pylibmc

This client relies on the C libmemcached library. This should be fairly straight-forward to install with your package manager on Linux or Windows. We also have a blog post for Ubuntu users on how to do this.

If you are caching sessions, also install libsasl2-modules with your OS package manager. libmemcached requires it for session caching support with SASL. Otherwise, you will likely see an error FAILED TO SEND AUTHENTICATION TO SERVER and potentially an associated message no mechanism available.

Once it’s installed, then install pylibmc:

$ pip install pylibmc

Be sure to update your requirements.txt file with these new requirements (note that your versions may differ than what’s below):

pylibmc==1.5.2

Heroku Users: The above pylibmc requirements must be added directly to your requirements.txt file. They shouldn’t be placed in an included pip requirement file. The Heroku Python buildpack checks the requirements.txt file and only that file for the presence of pylibmc to trigger bootstrapping libmemcached, which is prerequisite for installing pylibmc.

Next, configure your memcached client in the following way:

import pylibmc
import os

servers = os.environ.get('MEMCACHIER_SERVERS', '').split(',')
user = os.environ.get('MEMCACHIER_USERNAME', '')
passw = os.environ.get('MEMCACHIER_PASSWORD', '')

mc = pylibmc.Client(servers, binary=True,
                    username=user, password=passw,
                    behaviors={
                      # Faster IO
                      'tcp_nodelay': True,

                      # Keep connection alive
                      'tcp_keepalive': True,

                      # Timeout for set/get requests
                      'connect_timeout': 2000, # ms
                      'send_timeout': 750 * 1000, # us
                      'receive_timeout': 750 * 1000, # us
                      '_poll_timeout': 2000, # ms

                      # Better failover
                      'ketama': True,
                      'remove_failed': 1,
                      'retry_timeout': 2,
                      'dead_timeout': 30,
                    })

The values for MEMCACHIER_SERVERS, MEMCACHIER_USERNAME, and MEMCACHIER_PASSWORD are listed on your cache overview page. Make sure to add them to your environment.

After this, you can start writing cache code in your Python app:

mc.set("foo", "bar")
print(mc.get("foo"))

A confusing error message you may get from pylibmc is MemcachedError: error 37 from memcached_set: SYSTEM ERROR (Resource temporarily unavailable). This indicates that you are trying to store a value larger than 1MB. MemCachier has a hard limit of 1MB for the size of key-value pairs. To work around this, either consider sharding the data or using a different technology. The benefit of an in-memory key-value store diminishes at 1MB and higher.