How to prevent spam links from being indexed on the web

HongTuyen27 · ‎12-18-2022

Hello everyone

Hello everyone, I want to ask if google is indexing the search spam pages as shown below. Even though I blocked on robots.txt, and the <head> tag on the page already had a noindex tag, the page was still indexed. Any experts can help me with this, what should I do next?

SocialAutoPost · ‎12-19-2022

If you have added a noindex tag to the page and blocked the page in your robots.txt file but the page is still being indexed by Google, there could be a few reasons for this.

One possibility is that the page is being accessed and indexed by Google through a different method, such as through a link from another website. In this case, you may want to try reaching out to the owner of the website linking to the spam page and ask them to remove the link.

Another possibility is that the noindex tag or robots.txt directive is not being properly implemented or is being overridden by other directives. You can check the source code of the page to make sure that the noindex tag is present and properly formatted, and you can also check the robots.txt file to ensure that it is blocking the spam page as intended.

If you have checked these things and the spam page is still being indexed, you may want to consider using the Google Search Console to submit a request to remove the page from the search results. To do this, follow these steps:

Go to the Google Search Console and sign in with your Google account.
Select your website from the list of verified properties.
In the left-hand menu, click on "Coverage" under the "Index" heading.
Scroll down to the "Valid with warnings" section and click on the URL of the spam page.
Click on the "Request removal" button.

This will initiate a request to remove the spam page from the search results. It may take some time for the request to be processed and for the page to be removed from the search results.

I hope this helps! Let me know if you have any questions.

Was the reply helpful? Click Like to let me know!
Was your question answered? Mark it as an Accepted Solution
Auto post your Shopify products to Instagram, Facebook, Pinterest, LinkedIn and Twitter.
App: Social Schedular

vader_art · ‎10-01-2024

The reply from SocialAutoPost is somewhat misleading. Disallowing something in the robots.txt file doesn't prevent Google from indexing the URL. It just tells them they can't access the code/content on that page. But the URL can and often does get indexed. But because they now can't access the code, they will never see your noindex robots meta tag. So, actually, you don't want that disallow rule in the robots.txt file. Allow the bots in so they can find the noindex. You can also submit the removal request. The downside is the crawl budget wastage. The upside is the lack of indexed spam links. The upside outweighs the downside, in my opinion. Unfortunately, I'm yet to figure out how to get rid of this issue altogether.

How to prevent spam links from being indexed on the web

How to prevent spam links from being indexed on the web

Community Blog Articles