Fast Rebuilds

As I mentioned in the previous post, comment posting has been a bit slow lately. I timed it at 35 sec, which is much better than some other weblogs (where I leave for a few minutes after posting a comment) but I think is not acceptable. The problem lies with the fact that all index pages as well as the relevant archives are rebuilt when a comment is posted. As the weblog becomes bigger, the problem becomes more acute. However, there is no reason for a lot of these pages to be rebuilt on a new comment since they don’t have any information related to comments in them.

Using the MTTimer plugin, I timed the rebuild times of all of my templates.

The most time consuming one is the “Most Visited Entries Index.” This template uses the MostVisited plugin. I described how I am now rebuilding this template only once a day through a cron job in my previous post.

The other template that could be optimized was the “Most Commented Entries Index.” This creates a list of the top 10 entries based on the number of comments. I was using the technique described by Scripty Goddess. When I changed it to a query statement using the MTSQL plugin, the rebuild time dropped from 3.82 sec to 0.11 sec.

Among the other index templates, the RSS, Atom and Master Archive ones do not need to be rebuilt after a comment. Unfortunately, Movable Type does not provide a way to specify that. There is a mt rebuild type mod which allows you to specify if an index template should be rebuild when there is a new comment, trakback or entry. Once I apply that mod, I would shave a further 4 sec off of comment posting time.

Looking at the “Main Index” template in detail, we see that the following parts take most of the rebuild time.

More than half the time is taken by the CommentLeaders plugin which lists the most prolific commenters.

Looking at the archives, the most time consuming are the category archives. The average rebuild time for a category archive is 2.83 sec while the longest is 6.54 sec (for “Islam and Other Religions”). Since some posts are filed into multiple categories, all those category archives have to be rebuilt when a comment is posted. This is a big potential problem in MT since the category archives will get really big over time.

The monthly archives are more manageable. The average rebuild time there is 2.43 sec and a maximum one of 4.39 sec. However, they are not the potential time bomb that category archives are.

Another delay in comment processing is caused by MT-Blacklist but that is an absolute necessity as I get spam comments almost daily.

UPDATE: mt rebuild type mod has been insstalled. If you have any problems commenting, pinging or otherwise, please let me know. My email is on the sidebar.