METHODOLOGY

How we pick

Each guide names picks for specific jobs and backs every one with facts you can check: prices, terms, documented features and dated third-party tests.

How picks are chosen

What counts as evidence

Sources and their commercial interests

No source is perfect, and several have commercial interests. We cite them as third-party tests, not gospel.

01 / EVIDENCE BEHIND THE PICKS

Proxies for web scraping

Audience: Developers running their own scrapers (Scrapy, Playwright, Crawlee) who pay for residential bandwidth by the GB. Products compared: PacketStream, Evomi Core, Rayobyte Premium, Webshare, Decodo, SOAX, Oxylabs residential, IPRoyal, Bright Data residential. Read the guide →

Proxies for web scraping: picks by use case and the evidence for each (checked October 2026)
Use case Pick Evidence Main caveats
Simplest flat bill that never expires PacketStream
  • One price for everything: $1.00/GB for rotating and sticky traffic, with no tiers and no feature multipliers.
  • Pay-as-you-go with no subscription, and purchased balance never expires.
  • HTTPS, SOCKS5 and HTTP on one gateway and country targeting with a _country-XX suffix; the same balance pays for its Google Search, Fetch and MCP APIs.
—
Best-tested network on pay-as-you-go Decodo Runner-up: Oxylabs
  • 99.92% success, 0.53 s average and 2,853,543 unique IPs in Proxyway’s April 2026 test.
  • Pay-as-you-go available; no mandatory KYC for standard use; 24/7 support.
  • SOCKS5, 24-hour sticky sessions, city, ZIP and ASN targeting.
Four times the entry price of Evomi Core or PacketStream. PAYG traffic expires after 12 months; plan traffic does not roll over; wallet balances are non-refundable. Part of Source.ai (with Oxylabs, Webshare and ScrapingBee) since October 2026.
Long sticky sessions IPRoyal
  • Sticky sessions up to 7 days, the longest we found (most rivals cap at 24 hours or less).
  • Pay-as-you-go from 1 GB and traffic never expires.
  • City and ISP targeting, SOCKS5 (TCP only), 24/7 support.
Expensive at small volumes since long-running coupons ended. Success fell to 96.11% (P95 4.13 s) in Proxyway’s April 2026 test. KYC needed to unlock more ports, PayPal and some domains.
Finest geo-targeting (ZIP, ASN, carrier) Bright Data residential Runner-up: Oxylabs
  • City, ZIP, ASN and carrier targeting; claims 400M+ monthly IPs in 195 countries.
  • Pay-as-you-go with no minimum, plus unblocker, browser and pre-built scraper products on one account.
  • Proxyway’s residential test (March 2024) showed 98.96% success and 1.12 s average response.
The strictest KYC we cover: verified companies only, corporate email, possible ID check or video call. The highest list price per GB here.

02 / EVIDENCE BEHIND THE PICKS

Search & fetch APIs for AI agents

Audience: Developers giving an LLM agent (Claude Code, Cursor, Codex and similar) live Google results and page contents through MCP, on a budget. Products compared: PacketStream Search + Fetch, Bright Data SERP API + MCP, ScrapingBee, Firecrawl, ZenRows, Crawlbase. Read the guide →

Search & fetch APIs for AI agents: picks by use case and the evidence for each (checked October 2026)
Use case Pick Evidence Main caveats
Google search, fetch and MCP on one prepaid balance PacketStream Search + Fetch + MCP
  • $0.0025 per successful search or fetch, and failed calls cost nothing.
  • Google web, news, shopping, places, autocomplete, Scholar and AI Mode results as JSON or Markdown; Fetch returns a page’s main content as Markdown with its title, description and final URL.
  • A hosted MCP server (packetstream_search, packetstream_fetch) documented for Claude Code, Codex CLI, Cursor, VS Code and Claude Desktop.
  • Draws on the same never-expiring prepaid balance as its $1/GB proxies, with no subscription.
—
Pay-as-you-go Google search, no subscription Bright Data SERP API + MCP
  • The lowest pay-as-you-go Google search price we found: $1.50 per 1,000 requests, and you pay only for success.
  • 5,000 free records a month with no card; Scale plan $499/mo (380K requests, then $1.30 per 1,000).
  • An MCP server with search, scrape-as-Markdown and browser tools, hosted or self-hosted, with 5,000 free requests a month.
MCP pricing beyond the 5,000 free monthly requests is not stated in its docs overview. Bright Data’s account verification (KYC) is the strictest we cover.
Most complete search + fetch toolkit ScrapingBee API + remote MCP
  • Google search (eight result types) and a sub-second Fast Search, plus fetch with JavaScript rendering, premium proxies, screenshots and extraction: 18 MCP tools.
  • On a subscription, 25,000 searches fit the $49 Freelance plan ($1.96 per 1,000), alongside JS-rendering fetches.
  • 84.47% on 15 protected sites in Proxyway’s 2025 benchmark (tested Oct 2025).
Subscription only; unused credits expire, so light use still costs $19 a month. The API key travels in the MCP URL; slowest product in Proxyway’s 2025 test (25.46 s). Part of Source.ai (Oxylabs’ group).
Free monthly allowance for page fetching and browser control ZenRows API + MCP Runner-up: Firecrawl
  • A recurring free tier of 5,000 credits a month, no card.
  • Hosted MCP with OAuth 2.1 or a Bearer key, plus 30+ browser-control tools (navigate, click, fill forms, run JavaScript).
  • Failed requests are never charged.
No web search tool: pair it with a search API. Proxyway 2025: 70.39% at 2 req/s, but 31.76% at 10 req/s because of plan concurrency limits.
LLM-ready Markdown/JSON and site crawling Firecrawl API + MCP
  • Scrape, crawl, map and search endpoints built around Markdown and JSON output for LLMs.
  • Hosted MCP with API key, OAuth or keyless access; the core is AGPL-3.0 and self-hostable.
Last of ten in Proxyway’s 2025 protected-site benchmark (33.69%). 403/404 pages still cost a credit; no credit rollover below the Scale plan.
Pay-as-you-go fetching with JavaScript rendering Crawlbase API + MCP
  • No subscription needed and only successful requests are charged.
  • Renders JavaScript and returns HTML, Markdown or screenshots.
No search tool and no hosted MCP server (local npx only). Proxyway 2025: 56.72%, with 0% on five of 15 protected sites; difficulty multipliers up to ×20.

03 / EVIDENCE BEHIND THE PICKS

Web scraping APIs

Audience: Teams who would rather pay per page than run their own scraping stack, from public pages wanted as clean Markdown to sites behind Cloudflare, Akamai, DataDome or Kasada. Products compared: PacketStream Fetch API, Zyte API, Bright Data Web Unlocker, Oxylabs Web API, ScrapingBee, ZenRows, ScraperAPI, Crawlbase, Firecrawl. Read the guide →

Web scraping APIs: picks by use case and the evidence for each (checked October 2026)
Use case Pick Evidence Main caveats
Public pages as clean Markdown, every error free PacketStream Fetch API
  • Billed only when content comes back: every error is free, including target-site errors, timeouts and rate limits.
  • Returns the page’s main content as Markdown, plus the final URL, HTTP status, title and description; follows up to 5 redirects.
  • Long pages come back in sections: 50,000 characters by default (up to 500,000), with start_index to page through the rest; a country parameter picks the exit country.
  • No subscription: the same never-expiring prepaid balance pays for its proxies and Search API.
—
Strongest current independent evidence Zyte API
  • First of ten in Proxyway’s Web Scraping API Report 2025: 93.14% on 15 protected sites (tested October 2025).
  • 98.38% and fastest (6.61 s) in the 2024 edition; best fingerprint result among the scraping APIs in ScrapeOps’ January 2026 test (80.48).
  • Pay-as-you-go, only successful responses billed; Scrapy integration and Scrapy Cloud hosting.
Each site is priced by tier: browser renders cost $1.01 per 1,000 on the easiest tier and $16.08 on the hardest. Trustpilot 4.2 from 28 reviews, 39% 1-star, mostly about billing.
Flat, predictable price with rendering included Bright Data Web Unlocker
  • One flat $1.50 per 1,000 whatever the target needs; pay only for success; 5,000 free requests a month.
  • 97.90% success in Proxyway’s 2024 report, where it never failed a run.
Did not take part in Proxyway’s 2025 benchmark, so its best evidence is from 2024. ScrapeOps’ January 2026 fingerprint test gave it 41.43, against Zyte’s 80.48. Bright Data’s KYC is the strictest we cover.
Lowest JavaScript-rendering rate on a subscription Oxylabs Web API
  • $1.22 per 1,000 rendered pages on larger plans, with premium routing included.
  • First in Proxyway’s 2024 report (98.50%) and third in 2025 (85.82%).
  • /scrape and /search share one credit pool; Google results from $0.50 per 1,000.
Rebuilt on 14 October 2026, so every benchmark predates the current API. Subscription only, no pay-as-you-go; 0% on G2 and 38.71% on Allegro in 2025. Part of Source.ai since October 2026.
Every feature on every plan, with excellent docs ScrapingBee
  • JS rendering, premium and stealth proxies, geotargeting, screenshots and AI extraction on every plan from $19/mo.
  • 84.47% in Proxyway’s 2025 benchmark (fourth of ten); Proxyway calls its docs and UX excellent.
Slowest in the 2025 test (25.46 s) and 5% on G2. Premium + JS costs $4.90 per 1,000 on Freelance; credits expire; no weekend support.

04 / EVIDENCE BEHIND THE PICKS

No-code web scrapers

Audience: Analysts, marketers and founders who need data from websites without writing or maintaining code. Products compared: Apify, Octoparse, ParseHub. Read the guide →

No-code web scrapers: picks by use case and the evidence for each (checked October 2026)
Use case Pick Evidence Main caveats
Cheaper residential bandwidth for Apify Actors PacketStream proxies in Apify
  • $1.00/GB flat against $8/GB ($7 on higher plans) for Apify’s own residential proxy, with no subscription and a balance that never expires.
  • Apify accepts your own proxies: in an Actor’s Input tab, open “Proxy and browser configuration” and enter your proxy URLs. It works in Actors whose input offers proxy configuration.
  • PacketStream’s HTTP endpoint takes the username and auth key in the proxy URL (http://USER:AUTH_KEY@proxy.packetstream.io:31112); add _country-XX to the key to choose a country.
—
Ready-made scrapers for popular sites Apify
  • 19,000+ ready-made scrapers ("Actors") you configure with a form, plus schedules, webhooks and an API.
  • Residential and Google SERP proxies available on the same account; built on the open-source Crawlee library.
Usage-based bills are hard to predict: Proxyway saw one run cost more than $60 for a tenth of the results. In Proxyway’s 2025 tests, dedicated Actors failed on around half of the protected targets.
Point-and-click builder with cloud runs Octoparse Runner-up: ParseHub
  • A no-code desktop app with a visual builder; tasks run locally or in Octoparse’s cloud.
  • Residential proxies ($3/GB) and CAPTCHA solving ($1–1.50 per 1,000) available as add-ons; 5-day money-back.
Monthly, quarterly and annual billing are offered (annual saves 16%), and the page doesn’t make clear which period the $69 assumes. Trustpilot 4.2 from 97 reviews, but 31% are 1-star (surprise charges, cancellation).

05 / EVIDENCE BEHIND THE PICKS

Open-source scraping frameworks

Audience: Developers who want full control and are willing to run servers and buy their own proxies. Products compared: Scrapy, Playwright, Puppeteer, Crawlee. Read the guide →

Open-source scraping frameworks: picks by use case and the evidence for each (checked October 2026)
Use case Pick Evidence Main caveats
Residential proxies with setup guides for Scrapy, Playwright and Puppeteer PacketStream
  • Publishes integration guides for Scrapy, Playwright and Puppeteer, plus official code for Python, Node, Go and curl.
  • HTTPS, SOCKS5 and HTTP endpoints on one gateway, with sticky sessions at no extra cost.
  • $1.00/GB flat for rotating and sticky traffic, and purchased balance never expires.
—
Browser crawling with proxies and sessions built in Crawlee
  • Wraps Playwright, Puppeteer and plain HTTP crawling with a session pool and proxy rotation (ProxyConfiguration).
  • blockRequests() drops CSS, images and fonts to cut bandwidth; available for JavaScript/TypeScript and Python.
blockRequests() is Chromium-only. Maintained by Apify; smaller community than Scrapy.
High-volume HTML crawling in Python Scrapy
  • A purpose-built crawler: fast, asynchronous and extensible; v2.19.0 released 10 Sep 2026.
  • No browser means the lowest bandwidth per page, which is what you pay proxies for.
No JavaScript without scrapy-playwright or Zyte API. SOCKS proxies only through the HTTPX download handler.
Scripting a real browser in any major engine Playwright Runner-up: Puppeteer
  • Chromium, Firefox and WebKit from several languages; 1.64.1 released 14 Oct 2026.
  • Proxy per browser or per context (HTTP(S) or SOCKS5) and route.abort() to block heavy assets.
Not a crawler: queueing, retries and storage are yours to build.

Update policy

Prices are re-checked on the vendor's own page before every content update, and the "prices verified" date at the top of each page changes only when that happens. Picks are reviewed at each update, and every test result is shown with its date.