cc licensed flickr photo shared by jbelluch

I’m hopeful I have an optimum (or optimummer) search solution for my blog- not that I care if you can find anything here, but its important that I do. Back in March I vented my frustration about the limits of WordPress search, yet despite the experiments with plugins, none of them really worked for me, and one of them just spun wheels trying to index my site, and just ended up bloating the database.

But in about 45 minutes of tinkering this afternoon, I have an approach that gives me exactly what I wanted, and does not need any plugins. This will work only for a self hosted WordPress blog, and you will have to do a little bit of manual page creation, touch the code, and edit your templates.

Is anyone still there?

The first thing is to set up your own Google Custom Search (CSE) for your blog or site. I already outlined the key steps in my previous post.

One thing is to go to the Search and Feel section of CSE and set the hosting option for iFrame:

Next, go to Get Code area of CSE. You will be creating a URL for a place where the search results will appear. This will NOT work if you point it to a WordPress Page type. Why? Well, I found out by trying. The thing is, if you create a WordPress page with a URL like http://cogdogblog.com/found that is actually not the real URL; WordPress redirects it there; the actual url us something like http://cogdogblog.com/index.php?p=543 Even if you use that in Google’s CSE it wont work, because CSE appends to the URL another “?” and search parameters, which won’t work right because there are two “?” characters in there (that is the signal to a web page that there is following a “?” a string of variable names and values.

To get CSE to work right, you have to point it to a static URL. Something that sites outside of WordPress (but I’ll show you how to integrate it so it looks like it is part of the blog.

So I am going to make my own PHP page for my search results, mine is named “foundbones.php”. But before you do that, decided the name for your page (like “found.php”) and provide a URL that has that hanging off your blog URL, http://www.mydomain.com/blog/found.php — enter into the Get Code form, choose a placement for the ads (I put mine in the right), and then copy both chunks of code- one is the code for your search form, the other is the code that generates the results. I just put both in a text file as a scratch pad.

The first thing you will do is to set up the search form on your blog. Now this might get tricky depending on how your template places the search form- it might be in the header.php template, it might be its own searchform.php template– on my site, I had just used the search widget.

First thing I did was to pull the standard search widget out of my sidebar. Then I created a new text widget, gave it atile, and placed the search code form I copied from CSE in the widget (you may need to tinker with the form element search text field to make it fit nicely:

At this point we just replaced the standard wordpress search with our new form. To get it to work, we now have to build the results page. You will need a text editor to do this, create a new file and call it whatever name you decided the search URL would be, e.g. “found.php”.

No here is the magic. There is a way for a web page you make yourself to invoke the wordpress template tools to add the header, footer, sidebars, everything to make it look like part of your blog.

The first lines take care of hooking into WordPress, and then we issue the standard call to insert whatever is in the header template.

The goal here is to build the basic shell of your template; which is typically, in WordPress-ese, the header, content area, sidebars, and footer. The exact structure will need to match your template, so its best to look at something like a template in your theme folder like single.php which is usually what is used to theme a single post. You will need to match the div structure for the content area- for my blog, the shell looked like:

With this, I can put whatever I want in the content area, and it will look like the rest of my blog.

A few mods I did from here was to remove the get_sidebar() line as I wanted to maximize the width of the output, and did not need the sidebars. I also poked around the stylesheet style.css file to find a content id that specified full page width content (this may be done in another way with different templates), so I changed the content div to:

So now all I needed to do was insert the second set of code I copied from CSE in the content area so it formatted the results of the search (I can added a line of text before this to explain, so my revised template looks like:

And thats really the minimum. I went back to my home page, and ran a search, and wham! It goes through google, but displays the results on my custom page.

Hmm, not quite optimal- it displayed the results, but did not include the form field again that might allow me to refine the results.

That’s easy, I can just add the code for the search form into my custom page. That does work, to place the search form, but it did not put the words I searched on into the box, which is what I want to be able to refine the search. But that was easy- as the URl that ends up here is contrived of my own URL http;//cogdogblog.com/foundbones.php and then a “?” and a string of google search parameters; I can just use PHP to extract the search terms from the query. This bit now inserts the search form with the search terms sitting in the box (this is just the code for the search form; not the additional code for the “q” form field:

Now that is almost the whole package, but I had another thought- what if someone really wants to use the built in WordPress search– it’s still there, and available. This is again done by knowing something of the structure of WordPress search URLs- you can push a link to a search by simple using something like — http://cogdogblog.com/index.php?s=dog+treats == you just string together the search terms onto the s=….. part.

I just added a little bit of text in my output, so it includes this text:

This leverages some pho knowledge- I can get the value of a variable “q” that is coming into the page (that is always in a Google URL what the search terms are); in PHP, these are accessible via the $_GET array, and since this will be a string in PHP with the blanks in between, we add the function urlencode to convert the blanks to “+” that the link needs, this is just what we need to output right after the “s=” part of the link.

Okay, so I have given you the option to go from the Google search results to the WordPress ones, what about a round trip? Easy too. You just need to add a bit of code to the template that formats your WordPress search results; search.php

You are looking to put something right after the header for that page, which might look something like:

This is helpful because it shows the way WordPress echos the list of search terms; we just add our same CSE search form right below the results, and use that function (the_search_query()) to populate the CSE search terms field:

And now I have both search result pages going back and forth to each other.

Why am I bothering with this? With 7 years of blog posts, I find the WordPress search primitive to find things in my pile. There is not way to search on phrases, or use the Google modifiers to force one word to be in (or not be in the results). If I am searching for three key phrases, I will get results that have only one of them in there, and worse, the results are sorted by reverse chronology, not relevance, making it hard to find older content.

Now I can use all the tricks I use on Google searches, PLUS, it can search among the other content on my site, which is in other directories and not part of the WordPress realm (I have a few directories not indexable by Google using a robots.txt file).

And now, I think I have the best of both sniffers on my blog, both Google and WordPress search tools

cc licensed flickr photo shared by jbelluch

The post "Found. The WordPress Search Solution. Hewn by Hand." was originally rescued from the bottom of a stangant pond at CogDogBlog (http://cogdogblog.com/2010/04/found/) on April 24, 2010.

9 Comments

  • Joss Winn josswinn.org

    Nice hack. Did you try the ‘search api’ plugin which was an Automattic sponsored GSOC project? It is bundled with an optional Google search plugin or alternatively a MySQL plugin. I think it is intended to be core code at some point. http://wordpress.org/extend/plugins/search

    • Alan Levine aka CogDog cogdogblog.com

      Yeah, I tried it after you suggested it. Using the Google search worked, but there was some issue I had with formatting the output; and the database search flat out did not work at all for me. With my current solution, I have both as options, which was not possible with any of the plugins I tried.

      The search in WordPress has not changed since the first day I used it; that’s just not right.

  • D'Arcy Norman darcynorman.net

    that’s much easier than the one-click install for the excellent Relevanssi search plugin. quite a few hurdles to get Google to own searching on your blog… ;-)

    • Alan Levine aka CogDog cogdogblog.com

      I never claimed my approach was easier than a plugin (i started with a warning) and hardly see a few edits as “hurdles”. In fact, I think people should know how to tinker their templates and sites, otherwise the limits of what they can do are defined by off the shelf plugins.

      Based on your recommendation, I gave Relevanssi a try, but could never get it to complete the indexing– I gave it about 7 tries where it would bop out to a 404 during the indexing phase, and I would go back and ask it to start from where it left off.

      I gave up, and deleted the giant table it had created. I’m glad it works for you, and am not claiming there is a flaw in the plugin; but I could not get it to work.

      • D'Arcy Norman darcynorman.net

        that was pre-caffeinated snark. sorry. first reaction was “wow, that’s a long howto go get decent search working. built-in search is clearly broken.”

        but, getting a full google search working, integrated in your theme? that’s pretty sweet. they’re indexing the cogdogblog anyway, might as well make them earn their keep…

  • TI Fácil blog tifacil.blog.com

    Good morning, thanks for the interesting article? Is your blog a free theme or custom? I am intrigued by your blog. Is it possible to include this article on one of my sites?, i will of course linkback to this blog. regards

  • hello there from istanbul Turkey this article is very good thanks for sharing i was looking for Barış YardımoÄŸlu from bing and found myself here, i will come again. By the way Baris Yardimoglu is one of your followers too :)

  • looking good… imo Thelonious rocks… by the way WordPress 3.0 is out Joomla 1.6 is in Beta3 Drupal 7 — Alpha5 (expected Beta soon) exciting times for php open source CMSs yuppiii

Leave a Comment

All fields are required. Your email address will not be published.