Website internal search - are you making the most of the results?


Last year we changed Central Bedfordshire Council's website search engine to Zoom. It's not expensive, it doesn't have a flash look to it (though it can do!), but it does one thing...the job. That is, if you know how to make your internal search engine work for you.

I was recommended a book specialising in internal search - Search Analytics for your Site: Conversations with your Customers. This book walks you through the well of knowledge you have at your finger tips from the searches your customers are performing.

It explains how to analyse the results and make the most from them. You can then use this knowledge to amend your page structure / content in order that the most relevant pages are coming up in the search. Do you know what the most popular search is on YOUR website? Do you know how many customers are using your website search engine?

Well, on ours it is Council tax and 1 in 3 use search. That is a LOT of customers. So, we make sure that the correct page comes up first. OK, we are a council, so Council tax is pretty obvious. But, because we are a Council we offer over 500 services, so making sure the main customers searches come up tops is vital. 

My work started to make sure the top 20% of the most popular searches produced the best fit page in the top 3 results. This is ongoing work, which I do every week. But, it reaps the rewards as 88% are in position 1, with 99% in position 1-4 - yeah, shame on me for not being 100%! But I am working on it.

The next phase of work was to get rid of the noise - basically we have 3000 PDFs on the website and only 800 pages. Customers want to see web pages - my stats show 16 times more page views for web pages than PDFs. So last week we changed the search engine to default to web pages and show the top 10 results only. 

Now, I am expanding the search reach to ensure the top 25% of the most popular searches return the relevant result - may not sound like a lot. But, consider this. The top 25% of all searches is just 230 search terms. This is out of 58,000 unique searches performed on the website this year! Imagine trying to do it all. It would be impossible. 

With this knowledge, you need to make a manageable and measurable task out of the wealth of information. So, focus on the low hanging fruit - the top 20-25%. 

My next phase of work with be to do some cluster analysis - e.g. take all the associated search terms for key customer contact and ensure these are producing top 1-3 pages in results. I started this last week for "Benefits". So this involved some filtering of my data to see all the key search terms - housing benefit, council tax benefit, benefits, housing benefit form etc. It took a short time to filter and about an hour to amend a few key pages with titles, metadata and keywords to make sure it worked. Not a lot of time for such a large service area.

I'll be continuing to do this, measure results, tweak the site - as for one thing, it is a never ending job. But, it is worth doing, as you can prove you are improving the customer experience - its all in the numbers and no-one can argue with it!

I have also been lucky enough to get a few top tips along the way from the author of the book, Louis Rosenfeld via email and also through his Twitter account

I'd be interested to hear other's experiences of internal search - how you are ensuring top quality results and any top tips you might have.



Security level: Public

More Blog Entries

Long live email marketing

I intended writing this blog a while back but didn't get around to it until I read this...

Google site links - taking control of your listings

Hi We get a lot of traffic to our "Contact us" pages on our website. There are two reasons...


Former Member 9 Years Ago
Hi Alan I really like yur post. As a search vendor I encourage customers to do exactly what you do. Removing the PDF's alltogether seems drastic to me - have you had any reactions form users or staff? Maybe you will find our blog useful in your work: / Barbara
Alan Ferguson 9 Years Ago
Hi Barbara I am already a Site Improve customer. Great product - couldn't operate our website without it! Simple answer to your question is 0 complaints. We have applied a filter which means that the first search results will have PDFs filtered out. Customers can choose to see all results if they want to. We have 4 times more PDFs on the website than HTML web pages. But customers look at 16 times more HTML pages than PDFS. To me that screams out, get rid of the PDFs. Consider a PDF relating to a consultation which has 100 pages. Does a customer want to read that? Possibly, but more than likely they want to see our consultations web page and comment on it. I was fighting a losing battle getting the Consultations HTML page above long winded PDF document which was mentioning "consultation" a great deal more. Our internal search engine is not that sophisticated. Maybe as our developers understand it more, we can tweak to give an overall better performance. But at the moment no complaints on filtering out PDFs so I am getting the results I want. Cheers Alan