How can I edit robots.txt

[House_Of_Hoye]

I need to edit robots.txt file because, the "https://www.jeremy-hoye.co.uk/collections/unique-engagement-rings"  page not getting link juice value.  How to modify robots file? 

[Jason]

You can't edit the robots.txt file.
I'm curious though - what would you change to add extra value?

[George_Jobs]

Well it seems Shopify deliberatly undermining clients' effort to have trheir sites proplry indexed @ Goolge, and stubornly refuse to provide a solution.

Why would Shopify ban indexing of i.e. /collections & therefore products , /blogs ? or images ? 

This is returned from Google crawler (15 pages indexed, 198 exluded from indexing) :

Warning: 15 Indexed, though blocked by robots.txt

Crawling anomalies
Warning: Google chose different canonical than user

# we use Shopify as our ecommerce platform

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /XXXXXXXXX/checkouts
Disallow: /carts
Disallow: /account
Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*
Disallow: /*design_theme_id*
Disallow: /*preview_theme_id*
Disallow: /*preview_script_id*
Disallow: /discount/*
Disallow: /apple-app-site-association
Sitemap: https://XXXXXXX.com/sitemap.xml

# Google adsbot ignores robots.txt unless specifically named!
User-agent: adsbot-google
Disallow: /checkout
Disallow: /carts
Disallow: /orders
Disallow: /XXXXXXXXX/checkouts
Disallow: /discount/*
Disallow: /*design_theme_id*
Disallow: /*preview_theme_id*
Disallow: /*preview_script_id*

User-agent: Nutch
Disallow: /

User-agent: MJ12bot
Crawl-Delay: 10

User-agent: Pinterest
Crawl-delay: 1

[Luuk_Harleman]

2018 and still not possible to edit robot and/or sitemaps? Unbelievable 

[TrafficSolder]

But there will be option to edit robots.txt for your store ? or i would like to recommend ask with support.

If still there is not option to edit robots.txt or sitemap than it's not helpful seo guys.

[Furqan]

still Not possible in 2019
https://www.lcd4laptop.co.uk

[Dresscheap]

Can not edit Robots.txt? then how can I submit my sitemap?

https://myswardrobe.com/

[Don]

Hi there @Dresscheap!

Don here from Shopify. 🙂

You can learn how to find and submit your sitemap here on our help site. 

It would not be necessary to edit your robots.txt file in order to do so.

All the best!

Don

[jacktyler3]

Hi @Don,

I have Google crawling some old pages that are no longer supposed to exist, and still indexing them. How can I tell google not to crawl them if I can't edit the robots.txt file? They are already not submitted in the sitemap...

[Don]

Hi there @jacktyler3!

Don from Shopify once again. 🙂

If you'd like to add pages to your robots.txt file so that they are not crawled you can do so by following the steps outlined here on our help site.

Note that it is not possible to remove pages from this file, only to add them.

All the best!

Regards,

Don

[jacktyler3]

Hi @Don , I'm still having a hard time getting this issue fixed. Would you be able to help? The page is https://www.owlvenice.com/collections/all/bone-broth and it is showing as a site extension for our site in search results. If you search "owl venice" you will see it. It just doesn't look good - it shows as "bone-broth" and I can't remove the dash because it's not a page with a title that I can change.

[Don]

Hi again @jacktyler3!

Don from Shopify once again. 🙂

It will take a few days from when you make changes like this to your site before you can expect them to be reflected online in terms of search engine results.

You can check if Google has indexed your site to see its current state or you can manually request a re-crawl from Google to prompt this process sooner. 

As long as you have followed the coding steps given previously in that guide then search engines will pick up on this change in time and should not index the pages you have indicated in your code.

All the best!

Regards,

Don

[eliasjaxson0001]

I am curious to know the exact method.

[Don]

Hi there @eliasjaxson0001!

Don from Shopify once again. 🙂

If you are referring to the process for adding more pages to your robots.txt file so that they are not indexed by search engines you can learn how to do so by reading this guide I linked earlier.

Bear in mind, as above, it will take at least a few days for search engines to re-crawl your site to fully reflect any changes like this you have made.

All the best!

Regards,

Don

[beeeeeee]

is it possible to edit robots.txt now??? I find this quite limiting that shopify has blocked off google from crawling blogs. No wonder my pages were not indexed!!! even doing a force crawl doesnt guarantee indexing. 

[emaster]

We have just migrated our website https://www.fashionplaza.com.au/ from Magento to Shopify.  There are  100' s of pages in the new website getting indexed like fashionplaza.com.au/products/gold-blue-mens-tungsten-ring?variant=28888256938077  it is very easy to add one line of code in robots to stop indexing all pages with the pattern " ?variant= " I wonder how it can be done without editing robots files?

[Don]

Hi there @emaster!

Don from Shopify once again. 🙂

As in my previous replies, it is possible to hide more pages from search engines as outlined in the guide I have been sharing here.

While it is not possible to edit the robots.txt file to exclude pages, it is possible to take steps for further pages to prevent them from being indexed.

Check out the guide that has been posted here and see how you can put this in place for your needs.

Regards,

Don

[UltimateSkins]

So do I understand it correctly that I can't delete those exclusions? I have submitted the sitemap of my website (https://ultimate-skins.com/). Still most products/collections are getting excluded. 

[Don]

Hi there @UltimateSkins!

Don from Shopify once again. 🙂

That is correct; as outlined above it is not possible to edit the robots.txt file, only to de-index further pages not already excluded from indexing by that file.

Our guide to how this works can be found here.

All the best!

Regards,

Don

[mrunalD]

Hi,

My website https://markandscribe.com/ is not getting indexed by google, google search not showing meta description in google search. Check the screenshot. can you please suggest some solution? Google search console says it's blocked by robot.txt