just-scrape

Web search, scraping, extraction, crawling, and monitoring via ScrapeGraph AI CLI.

scrapegraphai ↓ 245k

Claude Code

View source ↗

What it does

API key: Set SGAIAPIKEY, use a .env file, use ~/.scrapegraphai/config.json, or complete the interactive prompt.
Credits: Remaining ScrapeGraph AI credits. Each operation consumes credits.
Use scrape for raw page formats: markdown, html, screenshot, branding, links, images, summary.
Use scrape -f json -p "<prompt>" or extract -p "<prompt>" for AI-structured output.
Use extract when the task is only structured data. Use scrape when mixed formats are needed in one call.
search -p can extract structured data from search results. Do not re-scrape those URLs unless results are incomplete.
crawl already fetches per-page formats. Do not re-scrape every crawled URL unless a second pass is required.

Requirements & configuration

API key: Set SGAIAPIKEY, use a .env file, use ~/.scrapegraphai/config.json, or complete the interactive prompt.
3. Extract - Need structured JSON from a known URL with an AI prompt and optional schema.
4. Crawl - Need bulk content from an entire site section.
5. Monitor - Need scheduled page-change tracking with optional webhook notifications.
| Need | Command | When |

Configuration: SGAI_API_KEYSESSION_COOKIEAPI_TOKENWEBHOOK_URLSGAI_API_URLSGAI_TIMEOUTSGAI_DEBUGJUST_SCRAPE_API_URL

Derived from the skill's own SKILL.md documentation · extracted 2026-07-23

Web search, scraping, extraction, crawling, and monitoring via ScrapeGraph AI CLI.

Source

Repository: https://github.com/scrapegraphai/just-scrape

just-scrape FAQ

What does the just-scrape skill do?

Search, scrape, crawl, extract structured data, and monitor web pages via the ScrapeGraph AI CLI. Use when the user asks to search the web, scrape a webpage, grab content from a URL, extract JSON from a site, crawl documentation or site sections, monitor a page for changes, inspect request history, check ScrapeGraph credits, or validate API setup. API key: Set SGAIAPIKEY, use a .env file, use ~/.scrapegraphai/config.json, or complete the interactive prompt. Credits: Remaining ScrapeGraph AI credits. Each operation consumes credits.

What does just-scrape require?

API key: Set SGAIAPIKEY, use a .env file, use ~/.scrapegraphai/config.json, or complete the interactive prompt. 3. Extract - Need structured JSON from a known URL with an AI prompt and optional schema. 4. Crawl - Need bulk content from an entire site section. 5. Monitor - Need scheduled page-change tracking with optional webhook notifications. | Need | Command | When | Configuration keys: SGAI_API_KEY, SESSION_COOKIE, API_TOKEN, WEBHOOK_URL, SGAI_API_URL, SGAI_TIMEOUT, SGAI_DEBUG, JUST_SCRAPE_API_URL.

How do I install just-scrape?

Run: npx -y skills add https://github.com/scrapegraphai/just-scrape --skill just-scrape --agent claude-code — the source lives at github.com/scrapegraphai/just-scrape.

Maintain just-scrape?

Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.

[![just-scrape on getagentictools](https://getagentictools.com/badge/skills/scrapegraphai-just-scrape-just-scrape.svg)](https://getagentictools.com/skills/scrapegraphai-just-scrape-just-scrape?ref=badge)

npx agentictools info skills/scrapegraphai-just-scrape-just-scrape

The second line is the CLI lookup for this page — handy in READMEs and docs.