scrapy-consistency

AI agent skill for Scrapy web crawling. Spider vs CrawlSpider patterns, XPath/CSS selectors, item pipelines and loaders, handling…

Guidogl Apache-2.0 updated 1mo ago
Claude Code
View source ↗

AI agent skill for Scrapy web crawling. Spider vs CrawlSpider patterns, XPath/CSS selectors, item pipelines and loaders, handling pagination and login flows, AUTOTHROTTLE and politeness settings, async/Twisted gotchas, middleware for proxies and user agents, and exporting clean structured data.

Contains

1 plugin.

Install

/plugin marketplace add Guidogl/scrapy-consistency

Then browse and install with /plugin install.

Source

Repository: https://github.com/Guidogl/scrapy-consistency

Maintain scrapy-consistency?

Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.

[scrapy-consistency on getagentictools](https://getagentictools.com/plugins/guidogl-scrapy-consistency?ref=badge)