Atomic Media text

Atomic Media

Archive for the ‘seo news’ Category

« Older Entries | Newer Entries »

Google testing new featured snippet layouts

Thursday, June 16th, 2022

Google has recently been testing new formats, layouts and interfaces for its featured snippet slot in the search results. Generally, Google shows a single source for a featured snippet but since 2018 Google has occasionally shown multifaceted featured snippets for some queries.

Many featured snippets. Google is testing showing many featured snippets, more than one, more than two – but up to four different featured snippets. Here are some tests spotted by William Alvarez on Twitter and Brodie Clark on Twitter – showing this in action.

Card Style Google Featured Snippets

Block and list view Google Featured Snippets

Why we care. Generally, Google will show a single site or source for a featured snippet. But when Google begins to show two or more sources in the featured snippet position – the “position zero” location – that can change how valuable that position is for site owners and SEOs.

These interface tests are worth keeping an eye on because the featured snippet position is one that is generally sought after to achieve by most SEOs and site owners. Changing the design, showing more than one featured snippet, can all impact your click through rate on that position and ultimately your traffic and conversions driven from Google Search.

The post Google testing new featured snippet layouts appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Microsoft to implement advertiser identity verification

Thursday, June 16th, 2022

To combat digital payment fraud and online scams, Microsoft is implementing a new advertiser identity verification program.

Rising threats. The pandemic has changed the way many businesses cater to consumers and transact online. Not surprisingly, this has also caused an increase in the presence of fraud and scams. To crack down on these efforts, Microsoft is introducing its advertiser identity verification program.

Launch. A spokesperson for Microsoft told us “We’ve already been running a limited test for several months. This next phase will expand to a small number of countries and verticals, but comprehensive release (ramp up and timing) is still being determined. In the future, we’ll enable all advertisers to verify proactively, but for now, we’re curating participation.”

The program will be available in Australia, Brazil, Canada, France, Germany, India, the Netherlands, Poland, Mexico, Spain, Sweden, Switzerland, Ukraine, the United Kingdom, and the United States. 

Facebook recently implemented a two-factor authentication program to help protect user accounts. Google also has its own required verification program which users have 30 days to initiate and complete.

How it works. Microsoft has taken steps to ensure that the identity verification process is simple and automated. Advertisers will use government-issued, photograph-included personal identity documents or business-related documents issued by appropriate regulatory authorities to complete the process. 

Microsoft Advertising will request advertiser identity verification for one or more of your accounts via email. You will have 30 days from receipt to complete verification-related tasks.

What Microsoft says. In a blog post released today, Microsofts Neha Garg & Sandeep Krishnan said “This is one of our key investments to enhance digital advertising safety, which we hope to accomplish in partnership with our advertisers. We encourage all our advertisers to complete this process to fight misleading ads and deter bad actors. We’ll continue to use our machine-learning algorithms and systems alongside the domain knowledge of our human experts to further enhance this program.” 

When a searcher views an advertisers details, they will see the following information.

Identity verification help. You can read the official help documentation here.

Why we care. Microsoft advertisers should complete the verification requirements as quickly as possible to avoid interruptions in their ad delivery.

The post Microsoft to implement advertiser identity verification appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Using digital PR to earn links and rank for your target keywords

Thursday, June 16th, 2022

So what comes first, the chicken or the egg?

Links continue to be an important organic search ranking factor. However, Google frowns on “link building” and states that the best way to build links is to create great content.  

If you build it, they will come. But how will they find you if you don’t rank, to begin with?

Ideally, when your site starts gaining authority and ranking well, your pages will start showing up in the SERPs, and you’ll gain links naturally.  

However, you can’t achieve this without some initial outreach and digital PR.

Let’s review the steps to create a digital outreach campaign.

Preparation

You’ll need to do some prep work before starting outreach.  

Browser extensions

Install the following extensions to your Chrome browser:

Select outreach personas

You’ll need to determine who will be your outreach coordinator and make sure they have a strong social media presence.

From having a good bio and profile image to having active social profiles, it’s important that they are credible and trustworthy to elicit answers.

Make sure your outreach persona has:

All of these are signs of authority and will influence whether people open and answer the emails.

Select outreach targets

What pages should be promoted to receive inbound links? What keywords/anchor text should be used for the links?

In the past, it was common to build links using keywords in the anchor text. This can result in link penalties, so it’s important to vary your URL/Anchor text combinations to avoid being on the receiving end of manual action from Google.

It’s important to use a combination of brand terms, noise anchors and miscellaneous words for inbound links.

You should continuously track your pages and keywords using a tool (e.g., Semrush or Google Search Console) to identify potential targets for link building.  

You can select pages that are trending up, keywords that are within striking distance, as well as keywords that are sliding in rankings.

Build your outreach list

The first step is to create a list of potential sites to reach out to.  

Use sites from your content research

During your topic and audience research, you probably came across many sites that discussed relevant topics about your industry. In addition to adding all of these sites to your list of potential target sites, use tools like BuzzSumo to further expand that list.


Get the daily newsletter search marketers rely on.

Processing…Please wait.

See terms.


Using search operators to find sites on Google

The goal here is to uncover sites that have shared similar content in the past. To do so, you want to use search operators to build queries that will uncover the types of sites you need.

For example, for a site that offers online degrees, you could start off with broad keywords like “education,” “college,” and “online universities.” You’ll then want to qualify those terms by adding search operators, such as “education” + “infographics.”

Knowing how to search will greatly impact the effectiveness of your outreach campaign, so it’s definitely worth spending the time to gain more in-depth knowledge on how to use search operators.

Here are some ways to use search operators to turn up great outreach opportunities:

Quotation marks (”…”) for exact-match searches.  

The tilde (~…) before a word to generate similar topics and ideas.  

Inurl:guest-___ tends to yield good results, as many sites put “Guest Post/Blog/Writer/etc” in the title of the post, which ends up appearing in the URL.

The full search terms might come out looking like this:

• “guest post” “real estate”

• “write for us” real estate ~auction

• “foreclosures” inurl:guest-post

• ~home auctions “guest blog”

Now that you know how to find keywords, it’s time to start using them!

Create a spreadsheet with potential link opportunities

By using BuzzSumo and searching Google with search operators, you’ll be able to create a list of potential sites to reach out to and pitch your content. 

Once you have collected and added them all to a spreadsheet, your next step will be to verify the quality of these sites to ensure that they have strong authority metrics. 

You want to look at:

If you don’t check these metrics, you may waste valuable time and resources contacting and writing for sites that won’t help your link profile.

This is where your Chrome browser extensions can help by placing this information at your fingertips.

Find contact information

Now that you’ve added the metrics and have modified your list of target sites to make sure they are qualified, you’ll want to look for contact information.

Don’t stop by going to the “Contact Us” page. Look for other contact info such as direct email addresses, social media profiles and possibly even WHOIS info. You have to be thorough in this step.

Sending an email to the main site’s contact form may mean that your request ends up in the customer service department, or worse, their spam folders, where it doesn’t get forwarded to the right people.

Without trying to spend too much time navigating around the site, look carefully for their contributor guidelines and add the URL to your spreadsheet for tracking purposes. For example, there may not be a page for contributors, but perhaps the Twitter profile for the editor may be listed on the site, so you can use Twitter to contact the editor directly.

Writing pitch letters

People receive a lot of junk in their email, and many sites are constantly receiving requests for guest posting from humans and spambots alike. For this reason, it’s crucial that your pitch letters stand out and showcase transparency.

Here are some of the elements that help pitch letters stand out: 

Now let’s discuss the letter itself:

FROM: Use a company email address to make your email more credible and trustworthy.

SUBJECT: Short, catchy subjects will draw the “open”

BODY: Now, you need to mix all of the elements discussed above into the email:

It’s important to spend enough time crafting these outreach letters to have a chance to succeed.

Start outreach

Using email marketing tools such as Mailchimp or ActiveCampaign is not an effective way to send outreach emails. Many will go to the spam folder or will cause your email server to be banned.

Instead, use your email server or Gmail to send out your outreach emails. This is why it’s important to be thorough and meticulous in the steps above.

With a list of target sites and email templates, start sending out emails and marking the date that you sent the emails in a spreadsheet so you can keep track of what sites were contacted and when.

Once you start receiving responses to your emails, it’s easy for your inbox to become absolute chaos.  

Methodical organization is key at this stage to keep all conversations moving forward.

Label emails

When you receive a response, go ahead and label your emails. Here are examples of labels you could use:

This way, you can separate emails that require responses and move the ones that don’t need responses to a different folder.

Categorize positive responses

Create a folder and/or label for all of the positive responses. Some of the folders can be:

Now that everything is labeled and organized, you can actually take the actions required for each conversation. These will require sending topic ideas to people who accepted your request, ordering content with your writers, and sending follow-up emails.

Topic ideas

From the first email response, you need to cultivate a relationship with other website owners.  

When they respond and agree to accept an article, the first step is to spend some time on their site, becoming familiar with the type of content they share. What is the site’s theme? Who is their target audience?

You’ll need to spend some time researching so that the title you pitch will be relevant, interesting and appealing to their audience.

The ideas you pitch should also include something thematically relevant to your target landing page and keywords so that the link placement to your site is natural.

If you use BuzzSumo, you can try searching multiple relevant keywords, and BuzzSumo will sort the articles by total shares, which can help you surface the content that resonates with their audience.

Deliver content

As soon as the target site approves your topic, it’s time to get your article written. You may have a dozen open conversations taking place simultaneously; it’s important that you get your content written and sent to the site in a timely manner, or you may lose the chance of getting published.

Ideally, your articles should be at least 750 words long, include a link to your site and to other relevant authoritative sites, and contains a couple of links to other relevant pages of the target’s site.

Add optimized images and cite their source!

Following up

When you send articles to be published to other websites, you may need to send a few follow-ups to make sure the target sites received and has all the information they need. You may want to create a task or reminder for yourself to follow up a few times until the article goes live.

Sometimes, you may not hear from the site owner, but if you review the site, your article may already be published.  

If you don’t hear back from them, it could be that the article wasn’t relevant, and they preferred to drop the communication. Or it could be they went on vacation or got busy, and your emails drowned in their inbox.

Send out 2-3 follow-ups, and if you don’t hear back after that, you can mark your article as available and pitch it to a different site.

Promoting your published content

Getting content published on other sites can be an arduous process, but gaining editorial placements in targeted, authoritative is worth it!

Now, you’ll want to promote your published article.

Share the article in your social media profiles and tag the site where your article was published.  

If you have a budget for content promotion, consider boosting the post with Facebook ads to increase the readership and traffic to your article.  

You can also use each published content piece as “social proof” when pitching other sites.

Influencers and media

Influences have two things that every brand needs:

Consumers have learned to trust influencers and follow their recommendations, and guess what? So does Google.

Working with influencers can be a powerful way to gain authoritative links and citations, as well as gain social proof and increase brand awareness. These links and mentions will diversify your link profile and improve your site’s authority score. 

Let’s go through the steps toward creating and implementing an influencer marketing strategy:

Identify relevant influencers

Develop your influencer marketing strategy

Pitch influencers

Track the results of your campaign

Now that you have the inside scoop on influencer marketing, it’s time to get researching!  

Finding influencers and media contacts

  1. Research: You can use tools (e.g., BuzzSumo, Traackr, Izea) to create a database of possible influencers to contact.
  2. Create a connection: Before sending them a pitch, you should consider following them on social media and liking or commenting on their content. This way, your name will become visible in their feed before you contact them
  3. Personalize your pitch:  When you do reach out, make sure you send them a personal email, so it’s obvious you’re not sending an automated email to a million people.
  4. Agreement: Once you’ve talked to the influencer and negotiated an agreement, we highly recommend getting the details and deliverables signed to create accountability.  Some influencers may ask for a payment, others for product and others can become affiliates.  

Adding authoritative links and citations to your backlink profile can have a substantial impact on your site’s visibility.  

Digital PR

You should always be thinking about producing unique content that will be of genuine benefit to your audience and industry.

Some ways of producing this content include:

Once you produce these assets, you can promote them using outreach and work with influencers to help you extend the reach of these assets.

It’s a powerful way to earn media mentions and coveted links.

Track your progress

Track your outreach efforts by looking at your backlink profile periodically and seeing how your authority score changes over time.

What links have had a significant positive impact on your score? What is your link velocity? Are you continuously earning more links than you’re losing?

It’s important to keep a close eye so you can amplify your strengths and overcome weaknesses in your link profile.

The post Using digital PR to earn links and rank for your target keywords appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Apple’s ad business may reach $6 billion by 2025

Wednesday, June 15th, 2022

JP Morgan lead analyst Samik Chatterjee thinks Apple could bring in $6 billion in revenue from mobile advertising in 2025.

Market projections. According to Apple Insider, the mobile advertising market is expected to exceed $400 billion in 2024, which is about a 33% increase from the $288 billion it claimed in 2021.

Privacy concerns and speculations. The release of iOS14 and the introduction of App Tracking Transparency (ATT) caused a supposed $10 billion in lost revenue for Meta. Chatterjee mentions in the article that this change “drove headwinds for incumbent advertising platforms,” and we can’t help but wonder if this was Apple’s plan all along.

Chatterjee goes on to predict that the headwinds will lead to a reallocation of advertising funds – many of which will be spent on Apple Search Ads, which directly benefits their advertising business and revenue.

Earlier this month we reported on a new pricing model for Apple’s search campaigns, Cost Per Tap (CPT). The new model will replace the less popular CPM campaign standard, and reflect the typical cost-per-click model that advertisers are familiar with on other platforms such as Google and Facebook. This change could be an early indication that Apple has big plans for its ad network.

The future of Apple advertising. There is no word on whether Apple has plans to implement an audience network. However, given their large network of users, it wouldn’t be completely out of their wheelhouse. The largest slice of the Apple pie, however, still comes from Apple Search Ads, which Chatterjee says will account for about $4.1 billion in revenue in 2025.

Why we care. The implementation of iOS14 didn’t stop Apple from tracking our data, it just stopped them from sharing it. Given Chatterjee’s ad revenue predictions, we wonder if Apple really prioritizes privacy and customer experience, or if they have other plans to take over the digital ad landscape. With 825 million paid subscriptions and a user base of over 1 billion, they certainly have the leverage.

The post Apple’s ad business may reach $6 billion by 2025 appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Webinar: Protect your PPC spend against ad fraud

Wednesday, June 15th, 2022

By 2023, the global cost of digital advertising fraud will reach $100 billion. Invalid traffic and fraud are consuming your budget, leaving you with few genuine leads and poor advertising ROI.

Learn how to verify advertising engagements and proactively block invalid traffic with ad fraud expert Adam French, who will share his insights into creating better outcomes for digital advertising efforts. He will also discuss how organizations can deploy end-to-end protection and detection capabilities to maximize ROI.

Register today for “Protect Your Paid Advertising Spend Against Ad Fraud and Invalid Traffic,” presented by TrafficGuard.

The post Webinar: Protect your PPC spend against ad fraud appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Search marketers agree: automation the least favorite part of PPC

Wednesday, June 15th, 2022

Whether you love it or hate it, PPC is a part of digital marketing that just isn’t going away any time soon. I’ve been working in PPC for over 10 years and I’ve seen it all. ETA’s, RSA’s, cost-per-touch? (looking at you, Apple), broad match modifiers (RIP), and attribution to name just a few of the most recent changes.

Earlier this month we asked you “What’s your least favorite part of PPC?”

We received an overwhelming number of answers ranging in everything from Google support to agencies (no offense taken), to clients with unrealistic expectations. But one answer stuck out: Google automation.

Let’s dive in.

Automation can be your best friend or a nightmare. Learning how to navigate and find a balance between machine learning and manual management has been on a lot of marketers’ minds lately. (Did you catch Brad Geddes keynote on day 2 of SMX Advanced?)

Here’s what you said:

Google support. Many of the answers we received specifically named Google support as lacking when it comes to offering help. As hard as they may try, most times they miss the mark.

We’re starting to see a trend here.

But Google isn’t the only offender when it comes to paid advertising. Facebook and Microsoft make a cameo also.

What about clients? Agencies unite when it comes to managing client expectations.

 More least favorites. While these didn’t quite fit in any category, we thought they deserved an honorable mention.

Why we care. It’s nice to know that we’re not alone when we get frustrated with our jobs. Whether we work for agencies or in-house marketing teams, we all face similar issues and concerns. It’s important in marketing (now more than ever), to adapt to changes, but that doesn’t make them any less maddening.

The post Search marketers agree: automation the least favorite part of PPC appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




How to find the balance between creativity and automation in PPC

Wednesday, June 15th, 2022

Day 2 of SMX Advanced 2022 has kicked off and this morning’s keynote by Brad Geddes was all about leveraging automation in your ad campaigns.

Geddes is a PPC expert and co-founder of Adalysis, and the author of “Advanced Google AdWords,” the most advanced book ever written about Google’s advertising program. Geddes has worked with many of the world’s leading companies in managing and perfecting their PPC management and workflows.

Where is the balance?

In his keynote, Geddes went into great detail on the difference between humans and machines, what their strengths are, and how they can work together to create a winning ad campaign.

“Google and Microsoft aren’t taking away your control,” Geddes said. “They’re giving you more management options. You don’t have to fight the machine, but your job is to find the balance.”

So what is the difference?

Realizing what humans and machines do well and playing to those strengths is key. Humans are really good at:

Machines, on the other hand, are really good at:

Referencing recent PPC survey results, Geddes reminded us that ad managers are happy with the results when it comes to scripts and bidding. Managers have neutral feelings when it comes to RSAs, data-driven attribution, and local campaigns.

Not surprisingly, managers are generally unhappy with automation surrounding discovery campaigns, the insights tab and auto-applied recommendations.

All in all, machines are really good with numbers, but not with insights and intent. That’s where humans come in.

Is anyone surprised? I didn’t think so.

What do we want? Balance of data insights combined with a machine’s ability to crunch numbers and make predictable outcomes!

When do we want it? Now!

Using guardrails

Geddes described guardrails as boundaries advertisers put around automation.

Google doesn’t always get it right and adding guardrails helps advertisers “leverage automation within our own framework of tolerance and profitability.” Geddes goes into depth on how to use these to fine-tune your campaigns.

A few of the most common guardrails are:

Let’s talk strategy

One of the most important takeaways in Geddes’ keynote is how to address key parts of your funnel with proper messaging and measurement.

How is the message changed based on previous user interaction? Is there cross-channel integration to consider?

But how do you create a plan around keywords, audiences and landing page experience?

The right campaign builds for almost any account size & type

Consider your campaign type, budget and strategic goals. Then think about your strategy.

Geddes explained the exact campaign builds he uses for his own accounts which include:

Complications with strategy or campaign type (such as smart shopping or video) require additional considerations. However, as Geddes explained, this campaign build can work for everyone and uses several types of automation at the same time. 

When should we override the automation

Geddes said that in reality, especially with RSAs, we have more control than we think. We aren’t forced to use machine learning. Account success is based upon KPIs, not on how much automation you’re using.

Don’t undervalue humans

Sometimes you want full automation, but Geddes laid out some considerations before diving in. 

But wait, there’s more

There’s so much more to Geddes’ excellent keynote. Check it out for yourself – it’s not too late to register for SMX Advanced. Simply register for free here to watch the full keynote on-demand.

And there’s still plenty of time to experience the rest of this year’s 100% free and virtual edition of SMX Advanced. Register today to watch all the other great SEO and PPC sessions on the agenda today – and ask your questions of the speakers in our live Overtime Q&A. 

The post How to find the balance between creativity and automation in PPC appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




New shopping links for Google AdSense

Tuesday, June 14th, 2022

There’s a new Labs feature in town for Google AdSense called shopping links. 

What are shopping links. Shopping links are placed in your page’s main content by hyperlinking existing words. Whenever Google detects a shoppable product mention, users will be shown shopping ads for that product.

Where will shopping links show. Google will only place shopping links on pages where there are detectable product mentions. Since this is a new lab feature, Google will limit the amount of traffic enabled to view shopping links to 10%. 

Opt-out. You can opt-out of showing shopping links by going to your AdSense account > Optimization > Labs and turning off shopping links. 

Find out more. You can read the full help doc here

Why we care. Advertisers using AdSense will now have another way of monetizing their website. Since only 10% of content will be linked to shopping ads, the new feature shouldn’t be overly intrusive, but advertisers should still keep an eye out for irrelevant links and unrelated results. 

The post New shopping links for Google AdSense appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




5 tips to develop authoritative content for Google and searchers

Tuesday, June 14th, 2022

Day 1 of SMX Advanced 2022 opened today with a keynote by Jaimie Clark, VP of SEO at Centerfield. It was packed full of insights around creating authoritative content that Google ranks and searchers need. 

The Wirecutter migration: SEO + content + rebrand

Before Clark joined Centerfield, she worked at the New York Times. In 2016, New York Times Co. bought The Wirecutter, a product recommendation site.

She led the migration of thewirecutter.com to nytimes.com/wirecutter in 2020. This migration resulted in Wirecutter doubling its search traffic within a year.

But two years of SEO work preceded the migration. And it started with a new mission: to help people effortlessly choose and buy the stuff they need to live a better life.

How they did this: 

Here are five tips Clark shared during her SMX Advanced keynote on how to develop authoritative content for Google and searchers.

1. Focus on depth, not breadth

Clark said Wirecutter had a “mile-wide, inch-deep approach” to product reviews.

But their content strategy shifted in 2018 to go a mile deep. They started creating:

2. Build topical authority

One of Wirecutter’s goals was to own search results for the term “vacuums.” They had about five reviews on different types of vacuums (handheld, cordless, robot, steam, stick). 

Clark said they looked at search data to find additional use cases to identify and fill content gaps. They also researched comparisons and supporting content people were searching for. 

“And we always updated the topics page – the ‘best vacuum’ page, to link to and address each of those use cases as anything underneath was updated,” Clark said.

3. Have a purpose

There should be a clear, user-centric reason why a page or website was created.

That purpose can be singular or multifaceted.

By Google’s definition, each page should have a clear beneficial or helpful purpose for users, Clark said. 

Questions to ask yourself:

“We don’t want to help one random person make one decision. Instead, we wanted people to really Wirecutter their homes,” Clark said. “Helping them discover what they needed day to day, bigger picture, longer-term, tied to the largest events in their lives.”

4. Present content in the most useful way

You should make clear what’s more important by the way you present your content. You should then lead people deeper for people to dive into specific use cases as needed.

There are two main categories of content: 

5. Demonstrate your E-A-T 

Here’s how Wirecutter approached E-A-T:

Expertise

Structured data was added to author pages to reinforce that Wirecutter’s content was written by experts or enthusiasts who demonstrated expertise in a specific space. 

Author pages started with a paragraph up top, with details about the writer:

The pages also included social media links (e.g, LinkedIn), an email contact and a feed of their latest articles.

Authoritativeness

These are the “tried-and-true” on-page elements for how Wirecutter generally structured its pages:

Wirecutter also preserved the original publishing date timestamps, using schema. Why?

They believed this way to demonstrate longevity and make sure Google understood that even though the page moved to a new location, it is the same authoritative piece of content that was first written by the same person years earlier.

Trustworthiness

This is a measure of two things:

Register now for SMX Advanced

Missed Clark’s keynote live? No worries! Simply register for free here to watch Clark’s full keynote on-demand.

And there’s still plenty of time to experience the rest of this year’s 100% free and virtual edition of SMX Advanced. You can watch all the other great SEO and PPC sessions on the agenda today and tomorrow – and ask your questions of the speakers in our live Overtime Q&A. 

The post 5 tips to develop authoritative content for Google and searchers appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




Why 100% indexing isn’t possible, and why that’s OK

Tuesday, June 14th, 2022

When it comes to topics like crawl budget, the historic rhetoric has always been that it’s a problem reserved for large websites (classified by Google as 1-million-plus webpages) and medium-sized websites with high content change frequency.

In recent months, however, crawling and indexing have become more common topics on the SEO forums and in questions posed to Googlers on Twitter.

From my own anecdotal experience, websites of varying size and change frequency have since November seen greater fluctuations and report changes in Google Search Console (both crawl stats and coverage reports) than they have historically.

A number of the major coverage changes I’ve witnessed have also correlated with unconfirmed Google updates and high volatility from the SERP sensors/watchers. Given none of the websites have too much in common in terms of stack, niche or even technical issues – is this an indication that 100% indexed (for most websites) isn’t now possible, and that’s OK?

This makes sense.

Google, in their own docs, outlines that the web is expanding at a pace far outstretching its own capability and means to crawl (and index) every URL.


Get the daily newsletter search marketers rely on.

Processing…Please wait.

See terms.


In the same documentation, Google outlines a number of factors that impact their crawl capacity, as well as crawl demand, including:

From conversations with Google’s John Mueller on Twitter, the popularity of your URL isn’t necessarily impacted by the popularity of your brand and/or domain.

Having had first-hand experience of a major publisher not having content indexed based on its uniqueness to similar content already published online – as if it is falling below both the quality threshold and doesn’t have a high enough SERP inclusion value.

This is why, when working with all websites of a certain size or type (e.g., e-commerce), I lay down from day one that 100% indexed is not always a success metric.

Indexing tiers and shards

Google has been quite open in explaining how their indexing works.

They use tiered indexing (some content on better servers for faster access) and that they have a serving index stored across a number of data centers that essentially stores the data served in a SERP.

Oversimplifying this further:

The contents of the webpage (the HTML document) document are then tokenized and stored across shards, and the shards themselves are indexed (like a glossary) so that they can be queried quicker and easier for specific keywords (when a user searches).

A lot of the time, indexing issues are blamed on technical SEO, and if you have a noindex or issues and inconsistencies preventing Google from indexing content, then it is technical, but more often than not – it’s a value proposition issue.

Beneficial purpose and SERP inclusion value

When I talk about value proposition, I’m referring to two concepts from Google’s quality rater guidelines (QRGs), these being:

And combined, these create something I reference as the SERP inclusion value. 

This is commonly the reason why webpages fall into the “Discovered – currently not indexed” category within Google Search Console’s coverage report.

In the QRGs, Google makes this statement:

Remember that if a page lacks a beneficial purpose, it should always be rated Lowest Page Quality ­regardless of the page’s Needs Met rating or how well­-designed the page may be.

What does this mean? That a page can target the right keywords and tick the right boxes. But if it’s generally repetitive to other content and lacks additional value, then Google may choose not to index it.

This is where we come across Google’s quality threshold, a concept for whether a page meets the necessary “quality” to be indexed. 

A key part of how this quality threshold works is that it’s almost real-time and fluid.

Google’s Gary Illyes confirmed this on Twitter, where a URL may become indexed when first found and then dropped when new (better) URLs are found or even given a temporary “freshness” boost from manual submission in GSC.

Working out whether you have an issue

The first thing to identify is if you’re seeing the number of pages in Google Search Console’s coverage report being moved from included to excluded.

This graph on its own and out of context is enough to cause concern amongst most marketing stakeholders.

But how many of these pages do you care about? How many of these pages drive value?

You’ll be able to identify this through your collective data. You’ll see if traffic and revenue/leads are decreasing in your analytics platform, and you’ll notice in third-party tools if you’re losing overall market visibility and rank.

Once you’ve identified if you are seeing valuable pages dropping out of Google’s index, the next steps are to understand the why and Search Console breaks down excluded into further categories. The main ones you need to be aware of and understand are:

Crawled – currently not indexed

This is something I’ve encountered more with e-commerce and real estate than any other vertical.

In 2021 the number of new business applications registrations in the U.S. broke previous records, and with more businesses competing for users, there is a lot of new content being published – but likely not a lot of new and unique information or perspectives.

Discovered – currently not indexed

When debugging indexing issues, I find this a lot on e-commerce websites or websites that have deployed a considerable programmatic approach to content creation and published a large number of pages at once.

The main reasons pages fall into this category can be down to crawl budget, in that you’ve just published a large amount of content and new URLs and grown the number of crawlable and indexable pages on the site exponentially, and the crawl budget that Google has determined for your site isn’t geared to this many pages.

There’s not a lot you can do to influence this. However, you can help Google through XML sitemaps, HTML sitemaps and good internal linking to pass page rank from important (indexed) pages to these new pages.

The second reason why content may fall into this category is down to quality – and this is common in programmatic content or e-commerce sites with a large number of products and PDPs that are similar or variable products.

Google can identify patterns in URLs, and if it visits a percentage of these pages and finds no value, it can (and sometimes will) make an assumption that the HTML documents with similar URLs will be of equal (low) quality, and it will choose not to crawl them.

A lot of these pages will have been created intentionally with a customer acquisition objective, such as programmatic location pages or comparison pages targeting niche users, but these queries are searched in low frequency, will likely not get many eyes, and the content may not be unique enough versus the other programmatic pages, so Google will not index the low-value proposition content when other alternatives are available.

If this is the case, you will need to assess and determine whether the objectives can be achieved within the project resource and parameters without the excessive pages that are clogging up crawl and not being seen as valuable.

Duplicate content

Duplicate content is one of the more straightforward and is common in e-commerce, publishing and programmatic.

If the main content of the page, which holds the value proposition, is duplicated across other websites or internal pages, then Google won’t invest the resource in indexing the content.

This also ties into the value proposition and the concept of beneficial purpose. I’ve encountered numerous examples where large, authoritative websites have had content not indexed because it is the same as other content available – not offering unique perspectives or unique value propositions.

Taking action

For most large websites and decent-sized medium websites, achieving 100% indexing is only going to get harder as Google has to process all existing and new content on the web.

If you find valuable content being deemed below the quality threshold, what actions should you take?

The post Why 100% indexing isn’t possible, and why that’s OK appeared first on Search Engine Land.

Courtesy of Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing




« Older Entries | Newer Entries »