In Search [Episode 34]: The Best Ways to Analyze Your SERP Competitors
Don't forget, you can keep up with the In Search SEO Podcast by subscribing on iTunes or by following the podcast on SoundCloud!
July 9, 2019 |
The In Search SEO Podcast
The In Search SEO Podcast Community Question of the Week!
When it comes to your search competitors how cut-throat should you be? Is it OK to work together with the competition? If so, when?
Summary of Episode 34: The In Search SEO Podcast
This week we welcome Investing.com’s Igal Stolpner
who joins us to shed light on search competition analysis:
- How to know which competitors you should or should not focus on
- How to qualitatively gauge a competitor’s threat level
- How to be both innovative and effective when creating an SEO strategy based on your competition analysis
Plus, we look at how the length of Featured Snippets has changed and what that might mean for your site’s traffic!
A Change in the Length of Featured Snippets and What It Means for Your Site
Recently, Mordy put out a study on the length of Featured Snippets
now relative to the past. The study is a follow-up to an article Mordy wrote for Search Engine Land a few months back all about why he thought Featured Snippets would be getting slimmer
Why would Google create slimmer Featured Snippets? Well, for one because Google is getting better at understanding what it’s looking at and secondly because it has some new indexation processes that allow it to index short little snippets of content. We’re, of course, talking about Fraggles
Now, shortly after that article was posted Mordy noticed that content was making its way into the Featured Snippets really quickly. For example, he would see a Search Engine Land article and then see that content in a Featured Snippet just hours later.
At the same time, Mordy also noticed that the content in those snippets was really short. But why would Google make them shorter? Well, it could be that a) people like shorter content (people are busy and have less time to read) or b) it lets a Featured Snippet act like a Direct Answer.
To explain, a Featured Snippet is content that shows on the top of the SERP that includes a URL to where the content came from. Direct Answers are short answers to known facts (i.e., Danny Devito’s birthday). Here, there is just the answer and no URL as Google is supplying the answer themselves.
Having Featured Snippets look more like a Direct Answer makes Google come off more authoritative, which is inherently helpful to a search engine!
The issue for Google is that while it can answer some queries with Direct Answers there are many queries where Google has to rely on websites to provide the content. So it wants to give users these short answers and keep them in Google’s ecosystem but it doesn’t have the content. So what does Google do? It creates a Featured Snippet that looks functions more like a direct answer. It’s using your content to create a "Direct Answer” for all intents and purposes. And this is why, according to Mordy, Google wants shorter and slimmer Featured Snippets.
After noticing that Featured Snippets were appearing shorter Mordy decided to run a test by taking 150 old Featured Snippets and comparing them to the current snippets shown on the SERP. By old we mean Featured Snippets that appeared between 2016 and the end of Q3 of 2018. In this study, Mordy looked at two types of Featured Snippets: List Featured Snippets (numerical or bulleted) and Paragraph Featured Snippets. He didn’t look at Tables because they are a pain when counting characters. He also didn’t compare Featured Snippets if in the past they were one format and today they switched to a new format because you can’t compare their character count when they are inherently different.
For example, Mordy first looked at a 2017 version of a Featured Snippet and then its 2019 version to see if they are the same format. If they were, Mordy simply added up the number of characters in both versions and determined which Featured Snippet was longer/shorter and by how much.
Time for the results!
Mordy found that Featured Snippets are longer and shorter than they used to be (we’ll explain). Here are the averages that Mordy’s study found: List Snippets were 10% longer while Paragraph Snippets were shorter by about 14 characters.
What does all of this mean? Mordy’s theory is that Google is looking to turn the Featured Snippet into something that is a bit less clickable. Not that we’re saying Google has bad intentions, but rather they’re making it shorter so it functions more like a Direct Answer because it’s better for the user (from Google’s perspective).
In the case of paragraph Featured Snippets, less is more. Clearer, more succinct, more focused, more targeted answers will satisfy the user as a Direct Answer would. In other words, having fewer characters in the paragraph Featured Snippet shows us Google is getting better at showing only what is highly relevant in the Featured Snippet making them not as click-worthy (to what extent we don’t know).
Now imagine if List Featured Snippets were shorter? Imagine you searched how to make fudge and instead of getting eight steps you only got five of them. Would you be more or less likely to click the URL in the Featured Snippet? It makes sense to say that if you could see the entire recipe within the Featured Snippet then you would obviously not click on the URL.
So in the case of lists, more content means longer Snippets which makes for fewer clicks! In other words, it’s the same paradigm, the same construct, the same strategy. It’s simply that in one case Google needs less content in the Featured Snippet to pull it off and in the other it needs more content!
Getting Search Marketing Competition Analysis Right: A Conversation with Igal Stolpner
[This is a general summary of the interview and not a word for word transcript. You can listen to the podcast for the full interview.]
We are set for another In Search SEO podcast interview session. Today, for your listening pleasure, we have an SMX speaker, a search marketing author, who just so happens to be the VP of Growth for investing.com,
welcome Igal Stolpner!
So I found a little secret out about you. You have a musical background, don’t you?
I do. My father is a cellist, my mother is a pianist, and I grew up playing the violin. I went to the Jerusalem Music Academy. Music was part of my life until I was 20-something when I had a funny move from music to the web because of studying web development when I was a child. Long story short, the first thing I did online was a website for musicians which at the time, from 2003-2005, was one of the largest websites for musicians in Israel and that’s what transitioned me to where I am today.
That’s amazing. By the way, you mentioned this to me, and I have to mention it to my audience, you were the 1st employee at investing.com?
Yes, that’s true, but to be honest there was one developer who started a couple of days before me but he left a few months later.
Oh, that doesn’t count. Wow. So how long have you been there for?
About 11 and a half years.
That’s pretty good!
Let’s get into deep insights in competitor analysis.
So you have a really fascinating concept that I feel gets glossed over in a lot of ways and the concept is that you view multiple competitors from multiple vantage points. You have your actual competitors that offer the same product/service to the same consumer base. You have the other sites you’re trying to rank above on the SERP and as an offshoot of that, you have your content competition. And you can really just run with this, you have your paid search competitors, for example, it’s really almost endless.
This is all wonderful, but what if you’re a local business or a small business in general... how can you possibly keep up with all of your various forms of competition?
You know a lot of answers in our industry start with, "It depends,” and I think that’s true here as well. My answer to a small business is going to be the same for a medium sized company as it’s not like they’re going to hire five analysts to focus only on their competitors. It’s all a question of focus, like the 80/20 rule. You have to focus on the competitors that are responsible for most of the threats. From there it’s just building a process that fits your business.
From speaking on quantity I want to jump to quality.
Let me ask, we spend a lot of time talking about identifying competitors from a quantitative perspective. So and so sells more than I do or such and such site ranks higher than I do. And we have all sorts of tools to help us identify our competitors from a quantitative perspective. However, I feel this is a big problem. It’s very easy to rely on a tool to point out your competition from a quantitative perspective, but how do you pull out your competitors from a qualitative perspective?
That is, how do you qualify the "threat level” of a competitor at scale? How do you ascertain the level of potential problems a competitor may cause you at scale? For example, a tool can tell you who is ranking above you, but it can’t really tell you if the traffic that site earns would have really gone to you had you ranked higher as it’s entirely possible that your site would not meet user needs despite ranking higher on the SERP.
Or, just to offer another example, a site may rank well ahead of you, but have a horrible UX that causes users to bounce. Why dedicate so much to outdoing a site that isn’t actually stealing any of your business?!
How do you look at competitor analysis from a qualitative perspective?
First, we have to look at how do SEO tools
identify competitors. These tools don’t know everything, they just see what’s on the SERP. So if these websites are repeatedly seen next to mine then they are more likely to be my competitors. Very often this strategy works or at least it’s a great starting point, but you have to do more research.
Very often I’ll see SEOs only look at the tools without doing more research like registering to the competitor’s site, reading their content, buying from the site, or even playing around to see their true features and true value.
And you’re right in what you said. You may rank higher but is that your main business goal? In the end, it’s all about the business goals. This is where it starts. You have to connect the SEO competitors from a SERP perspective to what makes sense for your business.
By the way, you mentioned UX, and I agree it’s something that’s being overlooked in search. And I’m not talking about whether a button is orange or green, bigger or smaller. It’s about the whole experience. As SEOs, we do understand this as it is part of who we are, but we need to look a little deeper. Maybe we should sit with the product team and understand the features better. Ask yourself, "Is that truly an opportunity or is this even a threat?”
And think of how many times you see a website you think is your competitor, but your company’s board of directors don’t even care about this website. You need to connect with others in your business to hold these things together.
Yeah, I was speaking with Stoney DeGeyter about the nature of language as part of the UX
and that gets totally overlooked. You might have a competitor with great website design, but there’s so little that goes into analyzing the language they use, the language you use, the language that targets users. There’s so much that goes into analyzing competitors from a qualitative point of view.
When I think of analyzing a competitor from a qualitative point of view, brand recognition comes to mind. How do you deal with a competitor, who may not rank as well as you do, who may not have as much content covering as many topics as you do, but has greater brand recognition?
I think about this a lot, that is, how important the branding point of view is. It’s not really a ranking factor, but the strength of brand awareness is massive.
Let me ask you, do all SEOs have access to a search volume tool
I would assume, yes.
I would assume too. If you do that can you track competitor’s branded searches?
I would also assume, yes.
So if you could do that you can do two things. You can a) look at how many searches there are for your competitors and b) you can also look at ranked searches that are associated with specific topics.
Let me give you an example, people search for best buy laptops
and they do that because they know that Best Buy has a great variety of laptops they can buy online. Now, this obviously helps Google understand the connection between Best Buy and laptops, but it also helps us marketers understand that among other things Best Buy is known for their laptops.
Branded searches are a very useful metric to look at. It’s also fascinating to find your branded searches share over your whole list of keywords and to compare it over time (this year versus the year before). This is something that can help a lot of SEOs understand the brand power of competitors a bit better.
I know you have a concept called the lean visibility test and I think it relates to all of this. Can you share a little about that?
The concept is simple and it relates to what I related to earlier about sitting with one of your product folks to understand your competitors better. What product and UX experts do before launching a new website or app is go through a process called the usability test. So what is it? First, they prepare for the test and conduct it with a few people, not just one, and then they analyze the results.
What we could do is do the same thing for competitors. Now you can’t do it for every single query, every single page, and every single competitor, but the lean visibility test would give you a big chunk of the answer that you’re looking for. It has three steps that are very simple. The first is to start with a search query. Put yourself in the user’s shoes and ask questions like, "How long does it take me to find the price filter? Does this require registering? Was this article really helpful or not?" Then you try and answer for yourself which competitor out of all of the results that you have on the SERP provides the very best experience.
If you do that for your top five competitors then you would understand things you wouldn’t have understood by just looking at the stats. You understand what’s going on behind the scenes. Why this user prefers this competitor over what you have. And while this is a product related question I feel like today more than ever it can actually help us understand things better.
I want to go back and discuss quantitative competition analysis for a second because there’s a lot of misconception there as well. For example, Google’s John Mueller has said that page speed, while there are "objective” measurements, is all relative to the other sites you’re competing with. How do you balance out looking at absolute versus relative metrics and so forth when dealing with competition analysis?
You touched upon a subject that I really like and it’s a subject that I actually realized not from other SEOs, but rather from our developers. So for a very long time, people would come to the developers saying it’s not fast enough and they will do actions like changing the code, improve the servers, or move the servers from one country to another and we would go up from 15 seconds to 12. Great.
I then realized that none of this is really important unless you’re looking at your competitors. Meaning, a website like investing.com which provides real-time data for lots of different stocks will never have the same site speed as a blog because they don’t have the same content to begin with.
Basically, I think that absolutely everything in your industry is in comparison with who you compete with and that means site speed or bounce rate. What’s a good bounce rate to have? What’s a good number of backlinks? There is no magic answer. It’s all in comparison to who you compete with.
You also need to remember that there is more than one competitor list because you could start with a specific query and then realize the larger your website is the more queries you have and different queries bring different competitors so your list of competitors only gets bigger and bigger. And you have your business competitors to look into too.
In the end, choose a list of sites you want to compete with and see if you are faster than they are. Site speed is one example, but I feel like it really clears things because there are no true benchmarks. It’s all in comparison to your competitors. And it’s not that you’re getting a better view from an SEO perspective, but it also explains what’s going on behind the scenes from the user perspective.
What I want to harp on is this problem when analyzing the competition is that there is a monkey see monkey do mentality. Meaning, we’ll take a look at our SEO tools and see what our site is doing or not doing relative to the competition and then try to replicate whatever the competitive landscape is doing that we’re not. And it’s just easier that way as opposed to doing the creative option that no one is doing.
I want to ask you, what would you choose? Would you focus on making it that you’re up to par with the competition and doing what they’re doing or would you focus on trying to find a new competitive avenue that no one has targeted at all yet?
I feel like good SEOs or good marketers will do both as you can’t do just one. First of all, going for a trend is perfectly fine. There are trends that already began and while you may be late you have to join. The thing I would do though is to try doing it a bit differently and a bit better. To ignore it entirely just because you want to be innovative is staying behind and very awful. On the other hand, if you’re only looking at others and not being creative then you’re not creating anything new and not taking a chance.
The biggest problem though is that very often when being creative you have no data to back it up with. You need to prove to your VP Product or CEO why to go ahead with your idea. And when you do have data to back it up that’s a great starting point that builds the case for you.
So I would go with Option A if I was like most websites, but to ignore Option B would not make much sense and the same goes vice versa.
I agree and that’s what’s hard about it, to act on theory. We live in our metrics or quantitative data world and we ignore that innovation. I’m not saying you need to land on Mars but to do something small, something experimental, as it’s always good to experiment.
I’ll tell you this. When speaking about creative content this is something we do very often where you come up with an idea and you add an analysis view to it like what keywords to use or what image to have inside the general topic. But everything else is new because, obviously, the last thing we want is to copy others.
Ironically, I feel in the SEO industry this is one of those areas where we fall back on. I feel like there isn’t one website where I can go for really innovative SEO articles and content news on a daily basis. It’s the same thing over and over again and once in awhile you’ll find an interesting piece.
I feel that one should write about things that already exist. If it’s a very popular evergreen topic you want to rank for it makes sense to write about it. Or if you’re a news publication and everyone’s writing about something that just happened you would also want to be there to write about it so you can rank in Google News and Top Stories. Even better would be to bring another angle or a piece of evidence or an image for statistics, something that would rank you better.
I find that innovative articles often don’t succeed because of SEO, but rather because people share them and they become viral. People speak about them because of their innovative angle or their very interesting points of analysis. These articles aren’t usually written for SEO or the high search volume on Google.
Let’s continue talking about theory because I want to talk to you about how you conceptually look at competitors. How you look at them holistically.
Let me ask you an oddball question. I have this ‘weird’ vantage point. I spend most of my time analyzing Google, SEO, etc. So I’m a bit of an idealist, I don’t have that killer instinct. I’m happy to collaborate with my competition as I have done so many times. That said, I do think, for various reasons, what while in the short term my ‘teamwork’ approach does me no good it pays off in big ways in the long term.
Am I out of my mind? In the age of the digital world where there are so many interactions and so many layers of complexity, there’s still this sort of old school outlook of it’s me versus them in this cutthroat environment. They're the competition. Got to destroy them. Does that still work?
That is, how do you relate to competition on the ideational level?
I think that even for the same online business there could be very different approaches with different competitors. For example, you could have a list of 10 competitors and you’ll choose to be super competitive with these three because of the better potential and with the others, we’ll be a lot more holistic and work with them. Meaning, to have some kind of collaboration that can eventually help both sides.
Again, I think this goes back to business goals. In the end, you have the CEO who wants to achieve X. It could be traffic, leads, a better experience, improving a funnel, or whatever it is. It’s always back what matters to the business. So take some competitors, see what works with some of them and see what works with the other list.
I don’t think you’re crazy, Mordy. It’s just I think that both ways could definitely work. The secret with a constant competitive intelligence is to understand what’s the best approach for every single site. On the other hand, what sites should you simply ignore because they’re irrelevant or because you can’t compete with them? And if you can’t compete with them then what are doing? Ignoring is one option but you need to list it, you need to officially say what you’re deciding with such a competitor.
Off the cuff let me ask. How you interact with your competitors or how you interact with the industry, in general, is perceived by the consumer. So how does that factor in? It might be that interacting or collaborating with competitor A is not in my best interest dollars and cents wise... in the immediate quantitative sense it’s not going to work out well for me. How do you factor in how your interaction with your competition is perceived by the consumer?
So you need to take all things considered. I can give you an example where business guys would come to me and say, "Okay, we want to work with this website because the content department wants this type of content covered. We're not going to go into this right now. It's a new trend. We're going take this content to someone else. They want canonical tags or they want backlinks. Is that okay with you, Igal?” And I say, "I need to think about that.” What could this mean for the business? You can't just give an answer without understanding the whole thing. And very often it's a no because that’s our practice for our SEO. And you have to take both things into consideration.
Business development, in general in companies, has a big part in it because they want their own things while the sales department also wants their own agenda to be covered and addressed. And we have our own, which is very often different. The approach is to find the balance that works in the best possible way for the company. And yes, sometimes, to be very honest, it's not you as the marketer who makes the decision. Sometimes it will be the management.
How do you holistically measure your competition's prowess? For example, you can have a competitor who is great on social media, but not so much on the SERP. You can have a competitor whose branding is spectacular but has a lot of content gaps. How do you create an overall hierarchy of competitors?
It will never be fully holistic, but it can be pretty close. I think that a very good starting point is pages. When you're looking at top pages you very quickly understand what they rank for and what they get traffic to. I think it is important to differentiate between organic traffic and the rest of the traffic because from a user point of view it's less of a thing, it's more of something that we SEOs are looking at. When you understand the whole list of pages and where this traffic is going you can go towards keywords. And with keywords, for us SEOs it's definitely a potential of things we could rank for, but it can also tell you a lot about things that are new for them or even trends. In fact, looking at a competitor’s keywords over time can help you uncover the trends they’re covering.
Now it only works if you list this and you are looking at that for months. You can't just look at it once and understand everything. If you do it just once you will understand what their current focus is or what currently works for them. And then I would add one more thing to it because when we're speaking about pages we're very often not seeing the whole thing and one thing that can really help is subfolders. Not all SEO tools provide a subfolder view, but a few of them do. And if you do that, you are basically looking at the categories of the site and you understand what they actually dominate.
From there, I'll take you to another thing entirely, which we at investing.com do every single month. We list the traffic changes. Just the one number in sessions for our top competitors. That allows us to basically see what's going on. One or two months ago, we saw one of our top competitors decreasing by over 20% and the trend is very similar to most websites. 6, 5, 4, 3% up or down.
Now, there was this one site that went down by 20%. We didn’t know yet what happened, but we knew there is something to look into. So we let someone dive deeper until they figured it out. And it's something that helps you really track everything that is going on. I feel this is a big part of that, but definitely, top pages have a very big part because they often both help you understand what they are truly known for, what they rank for, and also what changed from one month to another.
Yeah. I love trends. I'm a big trends person. I think that we’re also not focused enough on trends even for rank tracking. I did a piece recently about the idea of tracking your rank at one point versus another point
versus another point is ridiculous. Who cares where you're ranking? You have no idea where the trend really is. You have no idea where your volatility really is. You have no idea where your stability is. It really applies across the board. It applies to everything. Trends are the way to go.
Before we move on, I have to ask you, what are some other tactics we have not hit on yet for analyzing, identifying, and outmaneuvering the competition?
I'll tell you what. If I have to choose one, I would probably go into crawling through competitors. I feel like in the SEO industry we speak so much about crawling our own websites to figure out the problems like 404s and 301s, but we barely do that for competitors.
Now, if you crawl competitors, there's so much that you can find. To give a few examples, one obvious thing would be to see the whole list of their titles, descriptions, and URL structures because you will never manually find your competitor's titles and see what's the common ground for all of them. Usually, and especially for larger websites, sites use templates for their titles and descriptions and when you crawl the website you can just get a whole list in one place.
Then I really like to think about the question, "Why do they noIndex/nofollow something?” because most crawlers today can give you a whole list of all the URLs that site noindexes/no follows. So if you see they noindex a page that you do index you have to do a little brainstorming and ask why.
Crawl depth is a big thing as well. It helps to understand how far their pages are from the home page or any relevant subfolder. The domains they are linking to is also good to know and can help you can understand things easily. You will never be able to find all the domains they're linking to, but a crawler can easily give you a whole list of all the domains they’re linking to. Both the number and specifically the sites.
Crawling competitors is not something I do on a daily basis, but it's something we do from time to time, especially in periods when we go through changes, and I feel like they can really be helpful for SEO.
If I can just plug in Rank Ranger as we do offer a tool called the SEO Monitor
what title changes a competitor has made over the course of the last 30 days and how each time it affected rank. Looking into what the competitors are doing with their titles, their meta descriptions, or even their URL structure is a tremendous insight into how they’re thinking.
This is very interesting because if you are able to find a correlation between this and ranking changes, it can be very big for you. And you know how often people say how in SEO the titles are not what they used to be from a rankings point of view. Of course, they’re not, but they still matter. And larger websites that have hundreds of thousands of URLs, a little change in their template can make a major change in rankings. So optimizing these is part of the day to day for enterprise SEO, for sure.
Yeah. For example. I don't remember the keyword was or the type of query, but I saw when you added the year into the title, all of the sudden, you're rankings just shot up.
Something we've noticed a few months ago was one of our competitors made all of their titles shorter by about 20% and all of a sudden they improved not all rankings, but some specific rankings. It’s definitely something you want your SEO team to be on top of.
Right and that’s a good point. It's not across the board. You can’t add the word 2019 onto whatever. It will depend on the industry you're in, what site you are, what the keywords are, what the niche is, and all that sort of stuff.
Optimize It or Disavow It!
If you could either identify your competitors by looking at who ranks above you or by which competitor does better on social media, which would you pick?
I would go with rankings without a doubt. I have very mixed feelings on social media these days. On the one hand, I love social media, but for my brand and for a lot of large brands the things that are happening around social media aren’t great anymore. Long story short, I feel, without a doubt, there’s so much more potential from improving your ranking from 4 to 3, then from seeing post X work a little bit better on Facebook than your competitor’s post.
I couldn’t agree more. Thank you, Igal, this was great I really do appreciate it.
Thank you, Mordy. I really had fun. Have a good one.
Google is Standardizing Robots.txt:
First, Google proclaimed that it is working to make the Robots Exclusion Protocol an official web standard
. Then Google announced you will not be able to use robots.txt
to tell the search engine not to index a page! Google did list some alternatives such as using a 404 status code or the Search Console Remove URL tool. These changes will go into effect on September 1st.
Google Expanding App Ads on iOS:
Google says it is going to be showing more app ads in iOS
New Place Topic Tags Being Shown in the Local Panel:
Say hello to "Place Topics” in the Local Panel. Google is now looking at reviews, pulling out themes and highlights, and placing "tags” in the Local Panel
so that a user can get a quick look at what a business is about. It’s like having a description of the business in tag form.
Per Mordy, there is a worry about how good a job Google will do when pulling out the descriptions. What it may pull up may not be reflective at all of what your business does. It would be nice if Google allowed the business owners to add their own topic tags.
Google will Stop Pulling the Plug on Old Search Console Reports:
Google seems to have indicated that no other "old” Search Console reports will be redacted
Fun SEO Send Off Question
What does Google put on its hot dog?
Sapir knows that Google will never eat a hot dog. Google is not a commoner! Google would get a prestige catering service and would go for a ribeye steak.
On the other hand, Mordy says it will put all of the condiments on and he means ALL. Mustard, sauerkraut, ketchup, pickles, relish, onions, cheese, the works. One condiment for every SERP feature!
Tune in next Tuesday for a new episode of The In Search SEO Podcast