Semrush Crawl

New Member
1 0 0

Hi I'm using semrush to crawl our websites - We have circa 5000 urls to crawl -

But It is very long - nearly 3 days !

Do you have the same experience ? Do you have a solution ? another crawler to suggest ?

Thanks for the help


Shopify Partner
233 21 64

Hey no one answered you but thought I'd just jump in.

Two alts are ScreamingFrog or Sitebulb are pretty good options, although they're not free, they're super good value for what they can give you. ScreamingFrog has extremely granular and technical exploratory capabilities for digging into the nitty-gritty, meanwhile Sitebulb has some great issue detection features.

As far as cloud-crawlers go the all in one apps like SEMrush, Ahrefs, Moz etc all tend to have pretty basic crawl features. Much better quality options for software if you go for a dedicated crawler like ContentKing, DeepCrawl, Botify or OnCrawl if you want something more ongoing rather than ad-hoc.

Full time Shopify SEO guy, based in NZ. Sometimes freelance outside the 9-5.