All things Shopify and commerce
Hello,
I am using Ahrefs SEO for my Shopify store. Recently, the Ahrefs team informed me about an update to their crawler bot, requiring their IP address to be added to the whitelist.
Additionally, the robots.txt file on the website is returning a 403 Forbidden error.
I need assistance with adding the Ahrefs IP address to the whitelist. Can this be done via an API? I'm open to other suggestions as well.
Hello meenak149,
It seems there might be a bit of confusion in your question. There’s no need to whitelist an IP address for Ahrefs specifically in Shopify. If you're asking about allowing Ahrefs bots to crawl your website, you can achieve this by modifying the rules for bots in your theme.
To allow Ahrefs bots explicitly, you can include the following in your theme.liquid file under the <head> section:
<pre>
User-agent: AhrefsSiteAudit
Allow: /
User-agent: AhrefsBot
Allow: /
</pre>
and following is the robots.txt editing answer https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt
Do open this link and follow the instructions.
Thank you,
Hope this helps!
Learn how to expand your operations internationally with Shopify Academy’s learning path...
By Shopify Feb 4, 2025Hey Community, happy February! Looking back to January, we kicked off the year with 8....
By JasonH Feb 3, 2025Expand into selling wholesale with Shopify Academy’s learning path, B2B on Shopify: Lau...
By Shopify Jan 28, 2025