Where do spiders check?

New Member
3 0 1

Hello, i'm still a bit green on how google works.

Could someone please tell me whether search spiders just check your homepage for updates and will overlook other pages of your website if the homepage has not been updated?

0 Likes
Excursionist
220 0 13

Hi Natasha,

I suggest you start learning SEO, it's crucial for running a webshop. Maybe you could dive into Google Webmaster tools. You can add your sitemap (all liks and subpages from your webshop, Shopify automatically generates it) and ask Google to crawl your webpage. Just put ' /sitemap.xml' behind your URL. In my case, it would be http://transparantehoesjes.nl/sitemap.xml

Learn about H-elements, comments, meta tags (index, follow, nofollow en noindex = important), etc etc.

Good luck!

Need a smartphone case? https://www.transparantehoesjes.nl
0 Likes
Shopify Expert
246 0 38

Search engine spiders/bots crawl to whataver pages they can find. From your home page to your product pages. they follow the links on your pages.

They repeately crawl all pages to check to see if things are new. That is true for all pages. But they tend to come back to the most important pages more often.

Technical SEO Consultant - Specialising in setting up Structured Data for Rich Snippets, Google Analytics for advanced reports and the Google Search Console for efficient crawling.
0 Likes
Highlighted
Shopify Staff
Shopify Staff
23 0 0

Hey Natasha,

Tomas from the Shopify Guru team here!

Google's search when crawling sites will check all the pages it has found, not solely your homepage. If you want to learn more about Google Search, you should check these links, Google own support doc on How Google Search Works and our own guide on Google integration.

If you want to follow Thomas' advice and submit your sitemap to Google, you can find our guide on how to do so here.

I hope this helps!

Tomas W

0 Likes
Excursionist
224 0 14

Hello Natasha, 

In addition to what everyone else has said, it will be useful to have SEO experts make things clearer and easier for you. 

Go ahead to install SEO Doctor https://apps.shopify.com/kudobuzz-seo. The app is makes SEO very easy to understand while showing you which parts of your various pages you need to improve. 

Wish you the best. 

SEO Doctor App (https://apps.shopify.com/kudobuzz-seo)
0 Likes
Shopify Partner
3 0 0

Agreed. Google as most web crawlers follows links. First it looks at your robots.txt file if there is one in order to know if it is ok to crawl your site and/or which areas are off limits to it. Then it will look for a sitemap.xml or sitemap.txt file and follow the links in that or if not present the index.html file or whatever is the index page (such as index.htm or index.php). It will follow links found on a page it is crawling as well. Too, each file or page on your site has a server time stamp which shows the last time it was updated and mos tlikely it uses this as well to determine if a page has been updated since it last crawled it.

Shopify Partner - Search Engine Marketing Specialist - In Business since 1998
0 Likes
New Member
3 0 0

Hi Thomas, I might have this issue going on with my site: candydistrict.com. How can I check it? Thanks, Tanya

0 Likes
Shopify Expert
246 0 38

Tanya, you may get more responses if you start your own question.

Your site is indexed as can be confirmed by a site search in Google:

site:candydistrict.com

If you think you have issues, then the first thing to do is verify your website with the Google Search Console and see if any issues are being reported.

Technical SEO Consultant - Specialising in setting up Structured Data for Rich Snippets, Google Analytics for advanced reports and the Google Search Console for efficient crawling.
0 Likes
Shopify Partner
3 0 0

Tanya,


Which Thomas are you posting your question too, there are two of us in this thread. :)

Also, exactly what issue do you mean? This post doesn't really reference a specific issue per se far as I can see so perhaps a little more clarification as to the issue you think you have?

If you are referring to which pages google has indexed of your site you can do a search in google and see which pages it has crawled and indexed like this: site:candydistrict.com

 

also when you do this search in google next to each green link is a small down arrow. Clicking it shows the word "cached" clicking that word shows the page in googles cache which reveals more information such as when google last crawled that page

Shopify Partner - Search Engine Marketing Specialist - In Business since 1998
0 Likes
Tourist
4 0 2

This is very interesting, could anyone tell me how many, or what proportion of a sites pages you would expect google to index please?    I have just checked Webmaster tools for my site and it shows 349 pages submitted but just 48 indexed, is this normal or am I doing something wrong?    

If I do the search suggested above.... site:josery.com   .... that produces 395 results, so maybe all is well...

Thanks,

Tim Jones

0 Likes