How can I edit robots.txt

Solved
House_Of_Hoye
New Member
1 0 1

I need to edit robots.txt file because, the "https://www.jeremy-hoye.co.uk/collections/unique-engagement-rings"  page not getting link juice value.  How to modify robots file? 

 

Accepted Solution (1)

Accepted Solutions
Trevor
Community Moderator
3115 404 635

This is an accepted solution.

Hello!

As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

Due to the age of the topic, I will be locking this thread and marking it as resolved to help direct anyone that lands on this page. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.


Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

View solution in original post

Replies 92 (92)
Jason
Shopify Expert
10440 169 2081

You can't edit the robots.txt file.
I'm curious though - what would you change to add extra value?

I jump on these forums to help and share some insights. Not looking to be hired, and not looking for work.
Don't hand out staff invites or give admin password to forum members unless absolutely needed. In most cases the help you need can be handled without that.
★ http://freakdesign.com.au ★
George_Jobs
Excursionist
26 0 14

Well it seems Shopify deliberatly undermining clients' effort to have trheir sites proplry indexed @ Goolge, and stubornly refuse to provide a solution.

Why would Shopify ban indexing of i.e. /collections & therefore products , /blogs ? or images ? 

This is returned from Google crawler (15 pages indexed, 198 exluded from indexing) :

Warning: 15 Indexed, though blocked by robots.txt

Crawling anomalies
Warning: Google chose different canonical than user

 

# we use Shopify as our ecommerce platform

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /XXXXXXXXX/checkouts
Disallow: /carts
Disallow: /account
Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*
Disallow: /*design_theme_id*
Disallow: /*preview_theme_id*
Disallow: /*preview_script_id*
Disallow: /discount/*
Disallow: /apple-app-site-association
Sitemap: https://XXXXXXX.com/sitemap.xml

# Google adsbot ignores robots.txt unless specifically named!
User-agent: adsbot-google
Disallow: /checkout
Disallow: /carts
Disallow: /orders
Disallow: /XXXXXXXXX/checkouts
Disallow: /discount/*
Disallow: /*design_theme_id*
Disallow: /*preview_theme_id*
Disallow: /*preview_script_id*

User-agent: Nutch
Disallow: /

User-agent: MJ12bot
Crawl-Delay: 10

User-agent: Pinterest
Crawl-delay: 1
Luuk_Harleman
New Member
3 0 8

2018 and still not possible to edit robot and/or sitemaps? Unbelievable 

Crisp Golf
TrafficSolder
New Member
1 0 0

But there will be option to edit robots.txt for your store ? or i would like to recommend ask with support.

If still there is not option to edit robots.txt or sitemap than it's not helpful seo guys.

Furqan
Tourist
3 0 2

still Not possible in 2019
https://www.lcd4laptop.co.uk

Dresscheap
New Member
1 0 1

Can not edit Robots.txt? then how can I submit my sitemap?

https://myswardrobe.com/

Don
Shopify Staff
1598 107 216

Hi there @Dresscheap!

 

Don here from Shopify. :)

 

You can learn how to find and submit your sitemap here on our help site

 

It would not be necessary to edit your robots.txt file in order to do so.

 

 

All the best!

Don

Don | Social Care @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

jacktyler3
New Member
6 0 0

Hi @Don,

 

I have Google crawling some old pages that are no longer supposed to exist, and still indexing them. How can I tell google not to crawl them if I can't edit the robots.txt file? They are already not submitted in the sitemap...

Don
Shopify Staff
1598 107 216

Hi there @jacktyler3!

 

Don from Shopify once again. :)

 

If you'd like to add pages to your robots.txt file so that they are not crawled you can do so by following the steps outlined here on our help site.

 

Note that it is not possible to remove pages from this file, only to add them.

 

All the best!

 

 

Regards,

Don

Don | Social Care @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog