FROM CACHE - en_header

Load more than 1000 products on a collections page

ParekhAbhishekN
New Member
4 0 0

Hello,

I am trying to create a custom view for collection which does not use any layout and it should load all the products in the collection. The requirement is to iterate through all products and get some specific information. I am updating the pagination limit to

collection.all_products_count

whose numeric value is greater than 1000. But when I iterate over all the products using 

{% for product in collection.products %}

or using index based referencing such as 

{% for index in (1..collection.all_products_count) %}

, It only allows me access to first 1000 products in the collection. I want to know what is a workaround for it and How can I get more than 1000 products when collection contains more than 1000 items.

 

Thank you ,

Abhishek Parekh

Replies 8 (8)
KarlOffenberger
Shopify Partner
1873 184 869

Hi, there's a hard limit on 1000. No way around that in Liquid.

Begs the question, why would you want to display that many items anyway? Are you sure there isn't a more effective and efficient way to do it?

Just my 2c

Liked this post? You might also like our fantastic upsell apps Candy Rack, Last Upsell post-purchase and Candy Cart or offer free gifts with Gift Box. All made with ❤️  and care by Digismoothie
ParekhAbhishekN
New Member
4 0 0

Thank you Mr Offenberger for your reply. 

As I pointed out in question description, I need product specific information for all the products contained by a collection; which is different for each product and needs to be displayed on the custom collection view. This is an extreme scenario and current limit is preventing the code to function as expected. 

Thanks you,

KarlOffenberger
Shopify Partner
1873 184 869

I am pretty sure you will not be able to circumvent that limit within Liquid. 1000 is the roof and Liquid doesn't do chimneys.

However, you could use JavaScript and Storefront API client side.

Liked this post? You might also like our fantastic upsell apps Candy Rack, Last Upsell post-purchase and Candy Cart or offer free gifts with Gift Box. All made with ❤️  and care by Digismoothie
Nick
Community Moderator
Community Moderator
4524 434 990

Hi there, 

Nick here from Shopify. 

I looked into this for you from Shopify's end and can confirm the information in Karl's reply. Something to note is that any collection or even a store with that many products will slow it down. Is there any way you could break the products into multiple collections by any chance? I feel this could be the best solution. Karl also mentioned using Javascript and Storefront API which is something I would not be overly comfortable with myself. But I wanted to message and confirm Karls original reply. 

Hope this helps. 

All the best, Nick

Nick | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

KarlOffenberger
Shopify Partner
1873 184 869

Totally agree with Nick - regardless which way you'd implement this, it is a performance killer. With Storefront API on the client side you will most definitely run in to major throttling issues. You could create a performant variation by developing an app that fetches these collection items and caches them - whether you choose redis, elastic search or even file IO depends only on how you wish to query these items later on so you don't have to hit Shopify APIs each time. Then you can use app proxies to and redirection to serve up the massive list of collection items from your cache or pre-rendered file.

However, that cache needs to be updated as collection items and item details change which is why a read through / write through cache wouldn't be very suitable. A coallesced write behind cache would be better or simply a schedule sync job at say 3AM daily. But that means you'd be running stale data and need to account for that. See... it gets complicated real quick and finding the right kind of compromise is a tightrope walk at best.

So again, it really begs the question why do you need this? I understand you need more than 1000 collection items limit. That's just a technical requirement of your current use case. What is the use case? Are you sure this is the only or best solution?

For instance, if you need this because you think Google will be happier, then I can assure you it wont be. Good sitemaps and structured data in your product pages will make Google a lot happier.

If you need this for data sync then there are sensible approaches as well.

Ultimately it's not about what you're trying to accomplish, but why?

Liked this post? You might also like our fantastic upsell apps Candy Rack, Last Upsell post-purchase and Candy Cart or offer free gifts with Gift Box. All made with ❤️  and care by Digismoothie
ParekhAbhishekN
New Member
4 0 0

Hello Mr Offenberger,

The custom view I am creating needs to have information about all the products in a collection and user needs to have ability to perform action based on the different attributes products have. e.g. Lets say loading all the tags associated with all the products inside collection with count of each tag; which I see it impossible to get unless we load all the products of collection at a time. 

Please, let me know if there is any collection attribute or StoreFront API that can be used to query and I might be missing it.

Thank you, 

Abhishek Parekh 

PS : I also tried using map filter to get product attribute as a string and using it to manipulate but it also seems to return values for only first 1000 products. 

e.g. 

{%- assign variable_name = collection.products | map: 'attribute' | join: 'delimiter that will be used to differentiate' %}
KarlOffenberger
Shopify Partner
1873 184 869

So basically, if I understand correctly, you would like aggregates such as count over products grouped by collection and N other product properties so you can display these totals as users filter through products?

DSLR Lenses     (48)

Type
--------------------
Fixed / Prime   (21)
Macro            (5)
Tele / Zoom     (10)
Wide Angle      (12)

Tags
--------------------
Kit             (12)
Consumer        (15)
Pro             (21)

Manufacturer
--------------------
Canon           (14)
Leica            (4)
Nikon           (11)
Samyang          (5)
Sigma            (9)
Tamron           (4)
Zeiss            (8)

I am not going to say it is impossible, but I am going to say it is such an extremely inefficient feature to implement with what we have at hands as devs in Shopify that, considering its marginal value to users, I'd absolutely talk you or whoever else out of it.

If you still want it - don't waste your time trying to find a work around or hidden feature with Liquid. It's not possible to implement above in a Liquid template.

If you still want it, but don't want to create an app - it is possible using Storefront API (at least in theory) but even the most "optimized" solution (aka hack) I can think of would still be very inefficient. Not to mention the implementation cost because it would involve service workers, caching GraphQL client, progressive enhancement.

If you still want it, but need it to be efficient - it is possible if you essentially duplicate the collection and product schema in your own database to query specifically for aggregates. Or something like Firebase if that floats your boat. But like I said, now you have a data store to maintain, an app as well, the infrastructure etc etc. All that just to display counts of filters? Meeeh!

As a dev I know how some features with little user value are super cool to do, but ultimately they're a waste of time and money that goes by unnoticed. Sorry to be so frank, just being honest about it.

Best wishes!

Liked this post? You might also like our fantastic upsell apps Candy Rack, Last Upsell post-purchase and Candy Cart or offer free gifts with Gift Box. All made with ❤️  and care by Digismoothie
ParekhAbhishekN
New Member
4 0 0

Thank you Mr Offenberger again for your detailed reply.

I ended up using inefficient way of StoreApi and Ajax calls to implement the functionality. I ended up segmenting collection in to subcollection that contain less than 1000 products and then loading the required information using AJAX calls. It would be good if there was a way to load specific information about items inside collection without loading the actual products themselves using somekind of querying language. 

Thank you for everyone's replies and help.