New Keyword Research Process & Free and Paid-for Keyword Research Tools
Update: Google Keyword Tool has changed slightly...for the better
Google always tries to be helpful and suggests a long list of words that you might want to consider.
But SEOs often only want keyword search volumes and competition data for just the keywords they enter. Now you can do this by ensuring you search for an 'exact match' and ensuring you check "Only show ideas containing my search terms" after you've typed in some keywords (one per line).
Then when you get the results sort them by 'relevance' and your keywords will be listed at the top of the screen and when you download to .csv they'll be at the very top of a long list of other keywords that Google suggests.
Hint: Make sure you login to your Google Adwords account to see the full list.
Then click Search and you'll receive a list of search volumes for your chosen keywords, which you can download in Excel format.
Notice on the left hand side of the screen, you can choose between match types of Broad, [Exact] or "Phrase". It's interesting to compare the search volumes for broad and exact terms. In the case of the screenshot above, we can see that the broad keyword luxury shoes has 8,100 global monthly searches; whereas an exact search for luxury shoes only receives 2,400 global monthly searches.
End of update
To mark the launch of our new keyword research process and tools within Analytics SEO, I thought it would be a nice idea to review a number of third party keyword research tools on the market to start a debate on which keyword research tools are the best and why.
First, please indulge me a moment and allow me to tell you a little bit about our new keyword research process and the tools we have built within Analytics SEO.
The Analytics SEO keyword research process is meant to complement your use of external sources of keyword ideas and suggestions.
We have recently re-thought this process and the way our software tackles keyword research and selection. We think the new simplified approach makes our SEO software much more effective at suggesting the right campaign keywords.
The Analytics SEO Continuous Keyword Research Process
Firstly, please note the use of the word continuous! In a world of real-time search, you need to be reviewing what’s happening with your site’s position in the SERPs and you need to be in a position to react quickly to changes or opportunities as they arise.
Analytics SEO aims to help you with this process by automatically notifying you about new keywords that are bringing organic or pay-per-click visits or sales conversions. By analysing these new keywords you can stay up-to-speed and make minor modifications to your ongoing SEO campaign keywords to ensure you are always trying to optimise for the keywords that are going to give you the best return.
For this reason, you can now add, amend or delete keywords as your search engine optimisation campaign progresses; and you can see our latest recommendations at any time which reflect changes to your site in the search engine results pages.
Keyword Research Sources
Our starting point for keyword research is to identify as many good sources of keywords as possible. Keyword sources we use in Analytics SEO:
Your website – our spider crawls your site and extracts keywords.
Your competitors’ websites - our spider crawls their sites and extracts commonly occurring keywords.
Google Analytics keywords – once you have configured the Google Analytics data feed into Analytics SEO, our software automatically picks up the keywords that are bringing you organic search traffic.
Manual keywords – You can also add any keywords manually by simply copying and pasting them, or typing them into a form.
External Tools – You can also copy and paste data directly from a spreadsheet downloaded from a third party keyword tool; and you can add keyword competitiveness and estimated organic search volumes to help in the subsequent analysis.
We split this into 2 parts:
Keyword Analysis – Current Keywords
Keyword Analysis – New Keywords (sometimes referred to as ‘Dream keywords’)
We take all the different keywords from the different sources and we put them all into one table. OK, it’s quite a big table, but it really does help to have all this data in one place; and we have created some automated pre-defined filters to help you sort through this list of keyword data.
One thing you must do yourself is to import some estimated search volumes and figures for the strength of competition into the tool. Analytics SEO then uses this data to makes its recommendations.
Once you have added Keyword competitiveness and estimated search volumes, Analytics SEO will do a series of calculations and try and identify, and group keywords together to help you select the right ones for your campaign. It does this by creating a number of pre-defined filters to help you.
Our pre-defined Filters are explained below:
Long tail keywords – multiple word phrases that have relatively low search volumes.
High demand keywords – very competitive ‘head keywords’ that have large search volumes.
New opportunities – keywords where your site is not ranking but has potential.
Room for improvement – keywords where your site ranks, but could do a lot better.
Our recommendations – keywords we would probably choose if we were running your SEO campaign.
Traffic potential – keywords that have the largest potential incremental gain in natural search engine visits.
ROI potential – keywords that will convert better based on your Google Analytics conversion data.
PPC potential – keywords that should convert better based on your PPC campaign conversion data from Google Analytics.
Campaign keywords – keywords that you have selected to watch, track and optimise for in this SEO campaign.
Show all - this lists all keywords in the table.
Landing Pages - this lists all the pages in your site that are attracting natural search visits. You can select a page to see which keywords brought organic search visits to that page. (Please note, this tells you the total organic visits each keyword generated to the whole site not to that individual landing page).
Now you can see some example data, let’s look at each type of keyword analysis in turn.
Keyword Analysis – Current Keywords
Initially focusing on the keywords that your site is already ranking naturally for, or keywords that are working in terms of visits and conversions via a PPC campaign, has two main benefits.
i) You are already ranking for these organic keywords so the search engines understand that your site is relevant – you can narrow the focus of your SEO campaign to try and optimise your site or specific pages within your site to make it even more relevant and trusted for a chosen sub-set of these existing keywords that have the greatest potential. It’s simply a case of whether you can rank better you’re your existing keywords.
ii) Estimating ROI – If you track organic and PPC conversions in Google Analytics then some simple analysis will show you which keywords have the greatest potential sales uplift for you and hence offer the greatest opportunity to get a great ROI from your SEO campaign. In the case of PPC keywords, you know that consumers are finding your site relevant in the advertising listings; so it’s just a case of working out what it will take to rank organically for the subset of your PPC campaign keywords that work best.
If you just focused initially on this subset for your first phase of website optimisation, then you would stand a good chance of making real headway in the short-term. (Do remember, you can always run a longer-term SEO campaign for ‘head –tail’ keywords that are more competitive simultaneously in Analytics SEO, simply by creating a new campaign from the “Settings” tab).
Here are some example outputs from this process:
Keyword Potential Table
This table shows the additional incremental visits or conversions you could make if you improved the rank in Google for this keyword.
Competitive League Table – Generic Keywords
This answers one simple question; “How many more potential visitors are there for the organic keywords that I already rank for today?”
By looking at your Google Analytics visit data by keyword, and checking your competitors’ ranks, Analytics SEO can estimate and produce a competitive league table benchmarking your position against your competitors. The traffic index estimates how many more (or less) visitors another site has than your site based on the basket of campaign keywords selected. Analytics SEO also tracks this league table for you, so you can use it as a measure of success for your SEO campaign.
Keyword Analysis – New Keywords (sometimes referred to as ‘Dream keywords’)
If you are looking to run a campaign to try and identify the keywords with the biggest potential for your business. Then the pre-defined filters in Analytics SEO can help.
Perhaps you are looking for long-tail keywords that might convert better? Perhaps you want to use converting PPC campaign keywords in your organic campaigns? Or perhaps you are planning a more aggressive and resource intensive campaign to target some of the highest volume and most competitive keywords out there?
Whatever you are trying to do, Analytics SEO has a filter to help.
Further Keyword Research Tools
If you want to take your keyword research even further then you can also use some complimentary free or paid keyword tools. Analytics SEO supports any of them, so long as you import the data by copying and pasting the data from a spreadsheet that a tool gives you in the correct column order you should be fine.
The real question is which one do you use and how good is the data (especially the estimated search volumes). Let’s examine them in a little more detail, starting with Google.
Google Keyword Tools
Google has 3 different keyword tools, I’m not sure why since they only have one search engine; but anyway let’s take a very brief look at each in turn.
Google’s Search-based Keyword Tool (Cost £ Free)
This suggests new keyword ideas based on actual Google search queries matched to specific pages of your website. You can enter your website and/or a few starting keywords to get your results. If you are logged into your Google AdWords account then you get more keyword suggestions; and it pays to do this as I’ve seen as few as 11 keywords suggested for a site that did not have a Google AdWords account.
The tool should give you data for 100 keywords based on that site. However, if you login to your AdWords account then you can see more keywords.
What I do like about this tool above Google’s other tools, is that the estimated search volumes seem a little more realistic; and this clearly is important when considering which ones to target for an SEO campaign.
If you do sign-in with your AdWords login information to see a full list of keyword ideas customised to your website, these typically exclude those already in your account! This should not be too much of a problem if you having configured a link between Google Analytics and Analytics SEO as our software will automatically pull in a list of converting PPC keywords anyway.
Google Keyword Tool (Cost £ Free)
This is designed to build a master list of new keywords for Google’s Adwords customers’ Ad Groups. It also outputs detailed keyword performance statistics like advertiser competition and search volume.
To get estimated search volume data simply enter your keyword(s) into the "Word or Phrase" box. You can also type your website address into the box provided.
Ensure the Locations: and Languages are set to your desired target market and click "Search".
Then click the "Download" drop down box above the table of data that appears and click "Download all".
Open in Excel and delete the columns that you do not need. You only need 3 columns:
You should be left with 3 columns; the first column containing your keywords, the second containing the competition, and the third column containing the search volumes. E.g. Keyword, Competition, Local Monthly Searches.
It is useful for coming up with a long list of keywords.
This used to have a nice “synonym” filter which allowed you to only get search volumes and competition data for the keywords you entered. Unfortunately, now it seems you get lots of related keywords. You can of course delete the inevitable irrelevant keywords before or after you import the data into Analytics SEO.
Warning, some of the search volumes estimates seem unrealistically high. See highlighted examples further down the page.
Google Traffic Estimator (Sandbox) (Cost £ Free)
This is a new tool that is obviously being tested right now. But, there’s seems to be a paucity of data coming out of it right now. I guess it’s something we’ll keep an eye on over the next few months.
General Observations on Google’s Keyword Tools
Google only has one search engine, so why does it need to provide 3 keyword research tools? Is it trying to be deliberately obtuse just to hide the true volumes; or is it just experimenting to try and give different groups of users what they need?
Either way, a little consistency would be really helpful for everyone. Here are some examples where the data does not stack up:
- Estimated search volumes vary widely (see large anomalies in cells with amber background colour).
- The list of keywords suggested for a domain varies considerably between the tools. In one instance, I got two completely different lists of keywords.
- ‘Competition’ is measured on a scale of 0 to 1.0 on the Keyword Tool, and from 0 to 10 on the Search Based Keyword Tool!
- If you look at Estimated Clicks/Day (only available from Traffic Estimator tool) then you can see that the number of clicks is often higher than the estimated search volume! (See cells with red background).
|Keyword||Competition||Local Monthly Searches||Est # Clicks/Day Low||Est # Clicks/Day High|
|advertise your website||1600||1|
|advertise your website||1||0||1|
Key:Keywords in blue come from Search Based Keyword Tool Keywords in black from Keyword Tool Keywords in purple from Traffic Estimator - with bids set to £70.00 to get maximum volumes.
Estimated search volumes are clearly a very important data point. And in my opinion, estimates from the Search Based Keyword Tool look a lot more realistic than from the Keyword Tool.
The help withink AdWords explains the differences between the data as; "The updated Keyword Tool combines search data across Google.co.uk and all affiliated search properties. Data from the Search-based Keyword Tool is based solely on searches on Google.co.uk. Numeric data between the tools may also differ due to rounding."
But it doesn't really explain why all the tools cannot use the same data sources - surely this would be a good idea?
My last gripe is; where is the filter for excluding synonyms and new keyword suggestions? It was so helpful when all you wanted were search volumes for a defined list of keywords! (Please Google bring it back!).
If you don’t then we’ll all have to look at the other Keyword Research tools available on the market:
MS Excel 2007 Ad Intelligence Tool (Cost £ Free)
This Excel 2007 plug-in from Microsoft gives you access to keyword data from its search engine Bing and MSN.
This keyword research and optimisation tool operates by adding a toolbar into Microsoft Office Excel 2007. It provides keyword expansion, research, pricing and KPI data.
You can quickly and easily build out lists of suggested keywords and develop keyword strategies taking into consideration factors such as relevance, volume, cost history, demographic and geographic.
The only issue is that it doesn’t have the volume of search queries that Google has; although its value will grow once the full Yahoo/Bing search merger is integrated.
Wordtracker (From $59 or £28/month)
Enter a keyword phrase you want to target and Wordtracker will suggest hundreds of related phrases. You can then run them through the program and Wordtracker will compile a score for each phrase, based on the number of users searching for it and the number of websites targeting it. The higher the score, the better the keyword phrase!
Keyword Spy (From $89.95/month)
This is an interesting resource. Simply type in a keyword phrase to find websites targeting that phrase. It is useful for giving you ideas around which keyword phrases your competitors are targeting.
SEM Rush (From $19.99/month) – US, RU, DE, FR only
SEM Rush analyse 40 million keywords for 20 million domains. Their software tool allows you to find a list of Google AdWords keywords for any site with estimated searches and AdWords traffic. It’s useful for trying to unearth lower cost related keywords for optimising both your organic and your AdWords campaigns.
This is an advanced tool for keyword research junkies and also has an API you can utilise if you have the skills. It features tools such as a site analyser, misspelling and typo search as well as giving you a Keyword Effectiveness Index (KEI) rating for each keyword to try and indicate the level of competition in relation to the volume of searches.
Keyword Discovery (From $69.95/month)
Keyword Discovery is an advanced keyword research tool which claims to hold data on just 36 Billion search queries! In addition to the standard keyword research tools you would expect to see, it also has some nice tools to find Seasonal Search Trends and Related Keywords.
Other FREE methods of finding alternative keywords for your organic search campaign
Thesaurus.com (Cost £ Free)
Lastly this is always a good tool to use as it can come up with some suggestions that you might otherwise not think of.
When trying to expand your keyword list, consider singular, plurals, tenses, synonyms and most importantly buying terms such as “review, compare, buy, find, purchase”.
Customers who are searching using these search qualifiers are much more likely to be a little bit further down their research route to buying that product or service; and generally have a higher propensity to purchase.
You can also search Google for synonyms using the “~” command. Simply search for your keyword with the “~” in-front of it, e.g. ~loan. Anything that Google believes is a synonym will be highlighted in bold; and it often puts a list of terms at the bottom of the page too.
If you’ve got time on your hands, you can also do a query on www.ask.com and look at the related searches it suggests.
Finally, many people have difficulty spelling and make mistakes when entering keywords into the search engines. You can profit from this by targeting commonly misspelled keywords. These keywords are often far less competitive than the correctly spelled keyword.
Do you know of any other Keyword Research tools I’ve missed?
Have you had a chance to try Analytics SEO new keyword research process out yet?
Please add to the debate and let me know what you are using, and what you would like to see from a Keyword Research tool?
By: Laurie OToole
Login to comment
Login to comment
Start with a free account
Start with a FREE account today and find out how to automate your SEO campaigns with our easy to use software.