One of the only things that caused me to struggle when working with WordPress is the search feature. As a result, lots of people have taken the time to try to improve the functionality overall, but the issue I’ve got is a bit more rooted. Don’t get me wrong, I’m not trying to knock WordPress at all, but I’ve realized that in the past few months I’ve done nothing but sing it praises when in fact there’s always two sides to a story.
The fundamental issue
The underlying issue I’ve got with the WordPress search is it being database powered. In the time of of Widgets, dynamic content, and comments, a database powered search system is going to limit your results.
Plugin authors have done a lot to improve the situation by releasing improvements such as Search Everything and Search Unleashed. Both plugins offer a really great improvement to the stock WordPress search functionality by including important things like Pages, comments, and more. They give you a bit more fine-grained control over what shows up on a results page.
Unfortunately, though, the plugins don’t cover every base. Even though you can include categories and tags in the search algorithm, there will always be a bit more on the table in the form of dynamic content and widgets. Wouldn’t it be great if you could implement an on-page search system in WordPress?
Enter Sphider
Some time ago when working within a legacy content management system, I had to deal with a much more dire situation when it come to searching. The best I could do was link to Google and force the site: flag to limit the search to the client website. Awful.
I know how to program, but writing a search engine from the ground up isn’t very high on my “Want to do” list. Luckily, the open source world again saved the day. I stumbled upon Sphider one day and I must admit, I was super impressed from the get-go.
Sphider is defined as:
Sphider is a lightweight web spider and search engine written in PHP, using MySQL as its back end database. It is a great tool for adding search functionality to your web site or building your custom search engine. Sphider is small, easy to set up and modify, and is used in thousands of websites across the world.
Perfect. After a two-minute installation I was up and running. I added the client site to the index, spidered the site, and was generating really awesome results in about 10 minutes overall.
Sphider and WordPress
It wasn’t until a few weeks ago when I found Sphider for WordPress – eureka! The plugin integrates Sphider to the best of its ability into the WordPress admin all the while making life a bit easier by piggybacking on WordPress’ database (removing the need for the quick setup). That really helps when moving a site from your development or staging server to production.
I can’t leave out a quick bit I discovered while poking around the Sphider forums and Sphider for WordPress page; the founding author is none other than Scott Kingsley Clark who is also responsible for Pods, my favorite WordPress plugin of all time.
Sure, a couple drawbacks
As with everything, there are drawbacks to implementing a system like Sphider. The benefit to WordPress’ search functionality being database-powered is its ability to be “live” — as content is updated, so are search results. With Sphider it’s a bit of a different story.
To update Sphider’s index, you’ll need to set up a cron job to periodically re-index your website. I typically set up a job to run every few hours on highly dynamic websites, or let it go to once a day on less busy projects. At worst, a search query may be outdated by a few hours which really isn’t a deal breaker when you consider how much more targeted the results are.
A great solution, a great service
While the WordPress default search isn’t the sharpest tool in the shed, it can be greatly improved with a plugin or two. If you’re looking to really raise the bar, I’ve found Sphider to be a completely capable solution, even though there are a couple minor drawbacks.
It’s a great service to offer your client if their website is content-heavy as well. Improving the available search functionality will help them by helping their website visitors to find the content in question that much faster.
What are your thoughts on the default WordPress search functionality? Should Automattic work to include something like Sphider in WordPress core? Should it be left up to plugins? Is it worth looking into when people are more likely originating from a search engine anyway? What do you think?
Comments
WP default search is truly dire: if you type in a phrase it’ll simply look for every word in that phrase. I’ve used Search Unleashed in the past and found it to be really good: it highlights search word and phrases as well as improving the basic search (it works with Lucene as well as offering an enhanced WP search).
Sphider looks very interesting, so thanks for pointing us to it. If you don’t want to run a Cron job I guess you could simply re-index whenever you add a new post on a personal site.
Search isn’t particulalrly sexy but really important, especially on very dense, content–heavy sites. I’d say it’s probably the #1 priority for WP, but because of the democratic approach it takes to adding new features perhaps it doesn’t get many votes.
[…] Improving WordPress’ Search with Sphider mondaybynoon.com/2009/12/14/improving-wordpress-search-with-sphider/ […]
[…] Improving WordPress’ Search with Sphider […]
Yes, WordPress is wonderful and terrible, both at the same time, and the built in search is worse than useless.
Even more so if you use WPMU as the apparent search seems to search no more than the admin blog … nothing more.
Apparently investors dropped more than $25m on WordPress so I hope they use some of it to improve their software.
It looks like Sphider can offer sitewide searches, I’ll have a play and try it on Buddypress … if anyone has any experience, please let me know.
I’ve never really had a problem with the WordPress search other than it doesn’t highlight the phrases searched for – although this can be achieved with little work. You can also of course alter the query string to only search a category, page title etc.
However, you’re right that it is worth exploring more plugins that extend search – once you find a solution/combination that works for you, use it on all future sites!
Another alternative is of course Google Search which also allows you to monetize your search results through Adsense.
Sphider looks really cool, I may be exploring it as an option on an upcoming project.
Have you ever tried the Relevanssi plugin? It’s pretty a pretty solid search option as well. The one thing that really draw’s me to Sphider is the ability to search through PDF content.
Thanks for the write-up it’s very helpful to know you support it.
I actually have played around with Relavanssi a bit, but it’s still based on records in the database as opposed to on-page content itself if I’m not mistaken. I work a lot with Pods so database-based search plugins don’t return large sections of more appropriate results when working with Pods.
WordPress search functionility sux and I didnt got the result I wanted from the plugins Search Unleasaed and Search Everthing.
Sphider Sounds good but The wordpress plugin hasnt been updated from long time, it say Compatible up to: WordPress 2.7.1