Wayback Machine Scraper

List every Internet Archive (Wayback Machine) snapshot of a URL or an entire domain as clean, structured JSON. Give it a URL or hostname and this Actor returns each capture's timestamp, direct snapshot URL, HTTP status code, MIME type, content digest and byte length — filterable by date and status, with optional de-duplication. It reads the Archive's public CDX index, so there's no account and no API key. Built for SEO audits, OSINT and historical research where you need the full capture history, fast.

Features

• 🗂️ Full capture history — every archived snapshot of a URL, or a whole domain, from the Wayback CDX index.
• 🎯 Flexible match types — exact, prefix, host or domain (all subdomains) to scope exactly what you list.
• 📅 Date filtering — bound results with fromDate / toDate (YYYY or YYYYMMDD) to focus on a period.
• ✅ Status filter — set onlyStatus200 to exclude redirects and errors and keep only successful captures.
• 🧹 De-duplication modes — collapse by unique URL, one-per-day, or unique content (digest), plus automatic removal of exact duplicate captures.
• 🔗 Ready-to-open links — each record includes a snapshotUrl that opens the archived page directly.
• 📈 Scales to domains — cap output with maxSnapshotsPerUrl (up to 100,000 per input).

What you get

One record per snapshot, for example:

{
  "inputUrl": "apify.com",
  "originalUrl": "https://apify.com/",
  "timestamp": "20230115094512",
  "isoDate": "2023-01-15T09:45:12Z",
  "statusCode": "200",
  "mimeType": "text/html",
  "digest": "GKQ3XY7ABC5D6EF7GH8IJ9KLMN0OPQRS",
  "length": 48213,
  "snapshotUrl": "https://web.archive.org/web/20230115094512/https://apify.com/"
}

Output schema

Input

{
  "urls": ["apify.com", "https://example.com/pricing"],
  "matchType": "exact",
  "fromDate": "2020",
  "toDate": "2023",
  "onlyStatus200": false,
  "collapse": "none",
  "maxSnapshotsPerUrl": 1000
}

Use cases

• Audit a site's historical page changes and URL structure for SEO.
• Recover lost content by locating the exact archived snapshot to restore.
• OSINT and due diligence: trace how a page or brand presented itself over time.
• Detect when pages appeared, changed (by content digest) or went offline.

Pricing

Pay-per-event: you're billed a small amount per run and per item scraped — you only pay for what you get. See the Pricing tab for current rates.

FAQ

Do I need an account or API key? No. The Actor reads the public CDX index of the Internet Archive Wayback Machine, so there's no account, login, or API key required.

How many snapshots can I list per run? Up to 100,000 per input URL or domain, controlled by maxSnapshotsPerUrl. Use host or domain match types to enumerate captures across an entire site.

Is scraping the Wayback Machine legal? The Actor collects only publicly available capture metadata from the Internet Archive's CDX API. It is an independent tool, not affiliated with the Internet Archive — please respect their terms and use it responsibly.

What's the output format? A JSON dataset with one record per snapshot (timestamp, original URL, status code, MIME type, content digest, byte length, and a ready-to-open snapshotUrl). Export as JSON, CSV, or Excel.

Can I filter or limit results? Yes. Use matchType, fromDate/toDate, onlyStatus200, the collapse de-duplication modes (url/daily/content), and maxSnapshotsPerUrl.

Can I connect the Wayback Machine Scraper to other tools? The Wayback Machine Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. It works with Make, Zapier, Slack, Airbyte, GitHub, Google Drive and many more, plus the Apify API, JavaScript/Python clients and MCP. Or use webhooks to trigger an action whenever a run finishes.

Pair archival history with live web-diagnostics tools:

• Bulk URL Status Checker — check which of those historical URLs are still live today.
• Domain Inspector — WHOIS/RDAP, DNS, and SSL details for any domain.
• Website Content Crawler — crawl a live site into clean text for analysis or RAG.
• Bulk Image Downloader — fetch and store images at scale from a URL list.

Notes

This Actor reads publicly available capture metadata from the Internet Archive's Wayback Machine CDX API. It is an independent tool and is not affiliated with, endorsed by, or sponsored by the Internet Archive. It targets public data only — please respect the Internet Archive's terms and use it responsibly.

You pay only for what the run returns. Set a maximum cost per run and it stops cleanly at that limit.