I think the order is wrong here: the problem is more important than the tool. It may sound like a trivial distinction, but I really believe this is important. When you put the tool first, you just look for places where to use it, without thinking about what will be accomplished.
So my main suggestion is to first define the problem you wish to solve, and then think about what tools are appropriate.
I would like to give some examples of problems I tried solving using web scraping:
- when developing an app, we scraped the whole app marketplace to see what are the prices of apps in that category. We wanted to see what are the usual prices for apps.
- when we started writing blogs, we didn't know what our competitors wrote about, so we scraped their blog pages to see the titles of all blog posts they wrote...
- SEO: we wanted to see what domains rank for a given set of keywords (and what is their ranking)
Hi @Octoparse, this is Richard - CRO expert at PageFly.
I'd say that I agree with @adrianmisak reply. However, from my experience, the most common practices of e-commerce scraping tool are to:
It's a good way. Usually, the selected data is available in CSV format. As you can see, web scraping can be a lot beneficial in extracting product data from e-commerce websites, no matter how big the data is.
Explore the 30-30-30 rule, a dynamic social media strategy for new businesses. Learn how t...By Trevor Sep 20, 2023
Discover how to leverage the often overlooked footer of your ecommerce site to gain custom...By Skye Sep 15, 2023