Lobsters Scraper

hipersoft/lobsters-scraper

Scrape stories from Lobsters (lobste.rs): title, link, score, comments, submitter, tags and description. Pull the Hottest or Newest feed, or filter by tag. One clean row per story as JSON, CSV or Excel.

Run this scraper →See the API call

Total users

Monthly users

Total runs

Per story scraped

$0.0015

Last rebuilt

Oct 26, 2026

Overview

Scrape stories from Lobsters (lobste.rs), the computing-focused link-aggregation community, as clean, structured data. Pull the Hottest or Newest feed, or filter by tag — and get one row per story with title, link, score, comment count, submitter, tags and description. Export to JSON, CSV or Excel, or pull straight from the API.

Perfect for tech news monitoring, trend and topic tracking, content curation, developer research, and building link datasets for dashboards or LLMs.

What you get

One flat, ready-to-use record per story:

Field	Description
`shortId`	Lobsters story ID
`title`	Story title
`url`	The linked article
`lobstersUrl` / `commentsUrl`	Lobsters page and comments
`score`	Story score / upvotes
`commentCount`	Number of comments
`submitter`	Who submitted it
`tags`	Story tags
`description`	Submitter's description (plain text)
`createdAt`	When it was posted (ISO 8601)
`source`	Feed or tag it came from
`collectedAt`	When the row was scraped

How to use

{
  "feed": "hottest",
  "tags": ["rust", "security"],
  "maxItems": 200
}

Feed — Hottest or Newest (used when no tags are set).
Tags — filter by one or more Lobsters tags (e.g. programming, ai, security).
Max stories — cap the total (0 = no limit; the scraper pages through the feed).

Example output

{
  "shortId": "abc123",
  "title": "Thoughts on the Future of Browsers",
  "url": "https://example.com/browsers",
  "lobstersUrl": "https://lobste.rs/s/abc123",
  "score": 46,
  "commentCount": 20,
  "submitter": "orib",
  "tags": ["browsers"],
  "createdAt": "2026-09-20T14:00:00.000-05:00",
  "collectedAt": "2026-09-20T16:40:00.000Z"
}

Common use cases

Tech news monitoring — track what's trending in computing and security.
Topic & tag tracking — follow specific tags over time.
Content curation — build clean digests of the best stories.
Developer research — mine links, discussions and scores.
Datasets — build structured link datasets for analysis or model training.

Frequently asked questions

Do I need any login, account or key? No. Just pick a feed or tags and run.

Can I filter by topic? Yes — add Lobsters tags like rust, programming or security.

How many stories can I get? Set Max stories to whatever you need (0 = no limit) — the scraper pages through the feed.

What format is the data? A flat table you can export as JSON, CSV or Excel, or fetch from the Apify API.

Related scrapers

Building a tech-content toolkit? Pair this with our Hacker News scraper and DEV.to scraper — all with the same clean, one-row-per-item output.

Input

Field	Type	Description
`feed` Feed	string · hottest, newest	Which feed to scrape when no tags are given. Default: `"hottest"`
`tags` Tags	array	Filter by Lobsters tags — e.g. 'rust', 'programming', 'security', 'ai'. Each tag is scraped and results combined. Leave empty to use the feed above.
`maxItems` Max stories	integer	Maximum number of stories to collect (0 = no limit). Default: `100`

API

curl -X POST "https://api.apify.com/v2/acts/hipersoft~lobsters-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("hipersoft/lobsters-scraper").call(run_input={})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('hipersoft/lobsters-scraper').call({});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Pricing

Event	Price	Paid plans from
Story scraped Charged for each Lobsters story returned.	$0.0015	$0.0008

You pay only for what the run returns. Set a maximum cost per run and it stops cleanly at that limit.

Related scrapers

Stack Overflow & Stack Exchange Scraper

Scrape questions from Stack Overflow and any Stack Exchange site by tag or full-text search: title, tags, score, views, answer count, accepted answer, author, dates, link and full question body. Fast clean HTTP via the Stack Exchange API; optional key for higher quota.

117 runs · 1 users · $0.004 / question scraped

Steam Reviews Scraper

Scrape user reviews and ratings for any game on Steam. Get the review text, recommended/not-recommended, helpful & funny votes, author playtime, purchase type, language, early-access flag and dates. One clean row per review as JSON, CSV or Excel.

45 runs · 2 users · $0.0008 / review scraped

Rotten Tomatoes Scraper

Scrape Rotten Tomatoes movie and TV data: Tomatometer critics score, audience score, sentiment, title, year, genres, content rating, director, cast, description and poster. Just pass titles or URLs. One row per title as JSON, CSV or Excel.

38 runs · 2 users · $0.002 / title scraped

AniList Scraper

Scrape anime and manga data from AniList: title, score, popularity, format, status, episodes/chapters, genres, studios, season, start/end dates, description and cover image. Search by title. One row per title as JSON, CSV or Excel.

38 runs · 2 users · $0.001 / title scraped

Readability Scorer

Score how easy any text is to read. Get Flesch Reading Ease, Flesch-Kincaid Grade, Gunning Fog, SMOG, Coleman-Liau and ARI, plus word, sentence and syllable counts and a plain-English difficulty label. Score one text or many at once. Output JSON, CSV or Excel.

37 runs · 2 users · $0.001 / text scored

Chess.com Player Scraper

Scrape Chess.com player data in bulk: name, title, country, followers, join date, and ratings for rapid, blitz, bullet and daily plus win/loss/draw records, best ratings, tactics and puzzle rush. Just pass usernames. One row per player as JSON, CSV or Excel.

37 runs · 2 users · $0.001 / player scraped

Need a source that isn't here?

Send the site and the fields. We build it and you get it first.

Request a build