Over 13 years, Zyte has built a whole ecosystem around web scraping: an open-source scraping framework, headless browser, data collection services, and more.Smart Proxy Manager makes one – albeit a very important – piece of its data collection puzzle.
The tool was born as an in-house solution to a nagging problem.As web scrapers, the guys at Zyte dealt with proxies every day.However, no matter which provider they chose, they would soon run into issues with blocks and reliability.Eventually, they stopped focusing on what IPs they had, and instead put all their efforts toward how they worked. What emerged was the ‘the world’s smartest proxy network’.
In this review, we’ll try to find out if Zyte can compete with the dominant residential proxy networks.I’ll take a closer look at Smart Proxy Manager’s features, performance, user experience, and customer support.If you’re time-constrained, the provider’s key features with pros & cons are just below.
Payment options: Credit card, PayPal
Trial: 14 days
Datacenter proxies are Zyte’s main product – or, at least, the basis for it.What you’re really buying is the proxy management features mounted on top, and the provider will constantly remind you to ‘focus on the data, not proxies’.
So, we can’t know for sure how many IPs there are.Nor is there much information about the location coverage: the guys at Zyte told us there are 12 countries with more than 1 thousand IPs, and it can get other locations on demand.That’s likely because Zyte rents proxies from other providers, so it can accommodate to your needs.
In short, what we see is pretty unspectacular.The interesting part is that which we don’t see – Zyte’s proxy management logic.Here are some of its underlying features:
Request throttling – Smart Proxy Manager looks at the website you’re scraping, estimates its load, ban history, and then limits the request rate just enough so you wouldn’t get banned. Automatic retries – Smart Proxy Manager keeps a database of different ban types, captchas, and response codes.Once it detects a block, the system automatically switches the IP and retries the request.Cookies and sessions – the proxies can be configured to store cookies, so that you wouldn’t need to do so in your crawler.This feature isn’t available with the cheaper plans, which simply discard the cookies.
Zyte doesn’t treat residential proxies as a standalone product.Rather, it’s a paid add-on for clients that need to deal with the most unruly websites.While we had access to a custom Enterprise plan, it didn’t include the residential proxy functionality.I would assume it follows the same proxy management logic as the datacenter IPs.
Positioning itself as a data provider that also happens to sell proxies, Zyte has a corresponding pricing model.It charges not by IPs (the datacenter standard) or traffic (99% of residential proxy providers) but rather by successful requests.That’s right: you only pay for the data you get.
There are a few gotchas here and there: for example, a redirect counts as a successful request. But these are all details; in general, the pricing model is transparent and simple to understand. I would only question the feature lock.I’m not sure if Zyte has decided that smaller customers don’t need things like more concurrent connections and IP whitelisting, or if it’s simply an attempt to upsell.
Here are Zyte’s plans:
The cheapest Starter plan starts from $29 per month. This will give you 50k requests, with the ability to top up for an extra fee.The following plan, Basic, gives 200k requests for $99.The most popular Advanced plan costs $349 for 2.5M requests.So, it’s nearly three times more efficient. The Enterprise plan starts from $999 and is tailored according to your needs.
We ran Smart Proxy Manager through multiple performance tests to see if its proxies were stable, fast, and could access some of the major websites.
Proxy Quality Test
As expected, the IPs came from a data center.They all used the IPv4 protocol, which is still considered to be superior to IPv6.We made a lot of requests but could identify only 51 proxies. We don’t know if it’s because these were the only IPs we received or if Zyte’s proxy management logic decided there was no need to assign more for a website that wouldn’t block them.
Throughout days of testing, SmartProxy Manager proved to be very stable, and 99.91% requests we made were successful.This is a great result.
Success Rate Test
The proxies did especially well with e-commerce websites: Amazon, eBay, and Craigslist all had a 100% success rate.Search engines proved harder to crack, especially Google (86.72%).The hardest challenge for Smart Proxy Manager was social media platforms, Instagram in particular.This is understandable, considering how abused Instagram is by datacenter IPs.
Due to Zyte’s proxy management measures, we experienced few outright blocks.Whenever the proxies failed, it was mostly because of timeouts (over 30s).Increasing the timeout threshold would have likely made the success rate even better at the expense of speed.