“Caching” is a term you’ve probably heard mentioned before in various places (including this site). The idea behind caching is to store a copy of some piece of data so you can re-use it again later without jumping through whatever hoops you had to go through the first time to get it. There are different ways you can cache data (queries, objects, etc) and different medium in which you can store the cache (files, database, memory). Any way you do it, the main goal of caching is to increase the performance of your site or application. In many cases caching is used to lessen the amount of interaction with the database, which increases performance and decreases the load on your server.
I would like to talk about my personal favorite method of caching: memcached. I’ll show you how memcached works, how to install it, and how to use it to help your site/application run faster and scale better. According to the memcached site, “memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.” In plain English, this means memcached is an application that you can use to take advantage of spare free memory on any number of machines to cache pretty much anything you want (with a few exceptions) and retrieve it very quickly. Memcached was originally developed by Danga Interactive to help speed up LiveJournal. Some of memcached’s great features are that in runs on a number of platforms (Linux, BSD, Windows), is VERY fast, and has a number of client APIs already written so you’ll more than likely find libraries for any type of project you’re working on. We’ll focus on the PHP API in this article.
Before I get too far, I want to mention a couple of alternatives that may fit your particular situation.
Local Database Query Cache: Your database may have it’s own native query caching, which you don’t have to do much to use. The only drawback is that if a table is updated, its entire cache is thrown out.
The PHP APC extension: The APC extension is an opcode cache for your PHP scripts, but also provides a similar function to that of memcache. The biggest problem with APC is that you can only access the local APC cache. There are other distributed caching systems, such a MCache, but I have no personal experience with any of these, so I cannot opine on any advantages or disadvantages of using another tool.
Installation
First thing we need is an instance of memcached to store our data in. Unix/Linux folks can download the source from here and follow the instructions for installation on this page. Some distributions (like Ubuntu and CentOS) have memcached in their repositories, so you can use the native package installer like apt or yum to install memcache. Windows users can find binaries and installation instructions at http://jehiah.cz/projects/memcached-win32/. Any other questions you may have about memcached can possibly be answered on their excellent FAQ.
Now that we have memcached up and running, we need a way to talk to it. This is where the client APIs come in. We’ll be using the PHP API, so I’ll show you how to install the PHP memcached extenstion. The easiest way to do this is using the ‘pecl’ command: [pecl install memcache] this will download, compile, and install the extension. All you will need to do is add the line ‘extension=memcache.so’ to your php.ini file. You may also need to find and move the memcache.so file into the extensions/ directory (usually located at /usr/local/lib/php/extensions) by hand. Other options for installing PHP extensions and instructions for windows users can be found at http://us2.php.net/manual/en/install.pecl.php.
Implementation
Now that all the pieces are in place, let’s integrate memcached into our application. First thing we need to do is to connect to our memcached server:
- $memcache = new Memcache;
- $memcache->connect('localhost', 11211) or die ("Could not connect");
- ?>
This is assuming that memcached is running on the local machine and it’s using the default settings. You would usually do this connection when you open a database connection at the beginning of your application. If you want to connect to more than one memcached server, simply call $memcache->connect() again and pass in the name and port number of the additional server(s). Now that we’ve got a connection, let’s look at this section of code:
- $sql = "select * from pages where page_id=1";
- $qry = mysql_query($sql) or die(mysql_error()." : $sql");
- $result = mysql_fetch_object($qry);
- $content = $result->content;
- ?>
This fetches the ‘content’ field from our pages table. Now, if the data in the content field does not change very often, it is a good candidate for caching. Here’s one possible way how we would integrate memcached into our little section of code:
- //write query
- $sql = "select * from pages where page_id=1";
- //create an index key for memcache
- $key = md5('query'.$sql);
- //lookup value in memcache
- $result = $memcache->get($key);
- //check if we got something back
- if($result == null) {
- //fetch from database
- $qry = mysql_query($sql) or die(mysql_error()." : $sql");
- if(mysql_num_rows($qry) > 0) {
- $result = mysql_fetch_object($qry);
- //store in memcache
- $memcache->set($key,$result,0,3600);
- }
- }
- $content = $result->content;
- ?>
A bit more involved, but we are now using memcached! The above code first checks to see if we can find whatever it is we are looking for in memcache, and if we can’t find it, we fetch it from the database and use the result to populate the cache. In this example, I stored the entire $result object in cache and set its expiration to 3600 seconds (1 hour). The third flag in the set() function deals with whether to compress the data or not. Depending on your needs, you can store strings, numbers, objects, and arrays in memcache. Anything that is serializable in PHP can be cached, so database connections and file handles won’t work.
Now that we’re pulling data from memcache, what happens if the data in the database is updated? We can compensate for this in two ways. The easiest is to pass an expiration on the data that is fairly low, but you’ll have to deal with a little lag from the time you updated the database to when it will appear in the cache. The other way is to update the cache on the fly any time an update or delete occurs. This involves a bit more work as you may have to update many places in the cache depending on how many queries could possibly touch the data, but this is only necessary when doing query caching as in my example rather than just straight content caching.
Memcached affords us endless possibilities (query caching, content caching, session storage) and great flexibility. It’s an excellent option for increasing performance and scalability on any website without requiring a lot of additional resources.
July 26th, 2007 at 4:24 am
[…] Resources : http://www.danga.com/memcached/ http://www.search-this.com/2007/07/24/an-introduction-to-memcached/ http://en.wikipedia.org/wiki/Memcached http://www.socialtext.net/memcached/ […]
August 1st, 2007 at 12:39 am
Nice article, but I think you meant to put:
instead of:
$index is not a valid key because is undefined in the above script.
August 1st, 2007 at 8:46 am
You are 100% correct. I need to be more careful when copying and pasting code. 🙂
corrected.
December 20th, 2007 at 3:03 am
hi MrSpooky ,
thanks for this tutorial, i am using this and implemented memcache in mywebsite.
pls do for all pecl extension for this and also some mysql and mysqli difference and apc like.
thanks.
January 9th, 2008 at 3:32 am
How eviction policy work with memcahced.
I mean when LRU used with memcahced how removed data stored in the database.
How updated data in the cache stored in the database.
January 16th, 2008 at 7:23 am
how set and retrive mysql table values using memcached for window. please help me for set and retrive value of database table.
please write details.
January 16th, 2008 at 10:08 am
@manoj: If you take a closer look at the article, there is example code showing how to query then cache the results of a query to MySQL.
February 7th, 2008 at 4:09 am
how i update value of memcached when there should be some insertion and deletion take place in table.i m using memcache for autocomplete so please help me for making my autocomplete fast.
February 7th, 2008 at 9:01 am
@manoj: Without looking at your code, I couldn’t really recommend the best course of action, but in general implementing the above example would populate the cache after the first “select” on the new data, but if you absolutely must get it into memcache, then just perform a select immediately after inserting/updating and populate the cache with the results.
If ou are doing a delete, then just use $memcache->delete($key); to expire the entry in the cache.
February 7th, 2008 at 11:04 pm
thanks for reply.how i update value in memcache when new insertation should take place.suppose i store
all company that is started with ‘A’ in memcached. i want to know that when new company is added with ‘A’ then how i update memcached for that. i have set one key for all value of ‘A’.how i update that key value.
please help me.plese send me full description
of memcached using.
February 26th, 2008 at 4:41 am
Thanks for this tutorial, I was about 30 minutes searching for just this sort of thing to get me started with memcached.
February 27th, 2008 at 7:04 am
Hi,
Its a nice tutorial to get started. But I would like to know how can I store the entire results of a database query in memcache. Can I store the resultset object or I need to first put all the values in an array and then store that array in memcache?
March 14th, 2008 at 10:46 am
Great easy to follow & implement tutorial! I’ve recently been working with setting up memcached to store common result sets for categories & products. What happens, however, if you wish to retrieve & cache a very large result set such that your memory limit for process size is reached? This issue doesn’t come up when using “while($result = mysql_query($query))” as $result is set to the next record on each iteration. But if you store every row & column into an associative array before saving to cache, then you can see where large result sets might be problematic. An example of this would be showing ALL categories & products of an ecommerce store on one page for printing purposes.
Cheers!
March 31st, 2008 at 12:46 pm
Great piece of tutorial to help start anyone on memcached.
March 31st, 2008 at 7:51 pm
One tidbit that might apply to a couple of the above comments.
You can store any object, array or value that can be serialized into a string, so the only things you can’t store in memcache are file handles and database connections (although wouldn’t it be sweet to be able to pool DB connections in PHP with memcache?)
The max size of any value in memcached is 1MB, so large result sets from the DB won’t fit (trust me, I’ve tried).
May 15th, 2008 at 5:38 am
I didn’t think it was so easy to implement. Great tutorial.
May 23rd, 2008 at 6:12 am
Hi MrSpooky,
Indeed a very useful article with everything in one place and very concise, to show how to implement memcache.
thanks
November 5th, 2008 at 6:07 am
Thanx a lot bro… tat was just wat i wanted to start off… 🙂
November 18th, 2008 at 11:38 pm
Hi,
What do i do when place when am hosting does not have memcache
thanks in advance
November 22nd, 2008 at 12:52 am
how to store a file(text,word,pdf) within a Memcache and how to use the cached data without creating the file.
September 8th, 2009 at 8:20 pm
Thanks for clear explanation.
September 17th, 2009 at 10:36 pm
Hi MrSpooky Can you refer me some more elaborated tutorial or links about memcached. I have checked Danga … but haven’t found it to be useful for a beginner level database operation.
Thanks in advance.
November 20th, 2009 at 5:30 am
what about using database triggers to keep cache up to date?
March 18th, 2010 at 11:47 pm
Hi, thanx for the tutorial, i’m looking for a method to save temporary data (cache), i think memcache is the solution 🙂
April 27th, 2010 at 5:02 am
How does memorycache behaves when we cluster the servers by means of Apache web server
July 8th, 2010 at 10:59 am
Why do your examples use “new Memcache” and not “new Memcached” ?
$mcsrv = new Memcached ;
$mcsrv->connect(‘localhost’,11211) or die (‘could not connect’) ;
This code fails because the Memcached (with the ‘d’ on the end) doesn’t have a ->connect() method.
July 9th, 2010 at 1:12 am
When this article was written, there was only one memcached client extension for PHP, called “memcache”, hence creating a “new Memcache” and using the connect() method. More recently, a 2nd extension called “memcached” has been released, which seems to be the one your are trying to use.
February 9th, 2011 at 10:53 am
Hey really nice article, you are really showing simplest way of using memcache.
February 27th, 2011 at 1:44 am
I consider this page as my first memcache tutorial,
thanks.