spark-optimization
spark optimization
Claude Code
What it does
- Optimizing slow Spark jobs
- Tuning memory and executor configuration
- Implementing efficient partitioning strategies
- Debugging Spark performance issues
- Scaling Spark pipelines for large datasets
- Reducing shuffle and data skew
- Enable AQE - Adaptive query execution handles many issues
Derived from the skill's own SKILL.md documentation · extracted 2026-07-23
spark-optimization FAQ
What does the spark-optimization skill do?
Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines. Optimizing slow Spark jobs Tuning memory and executor configuration
How do I install spark-optimization?
Run: npx -y skills add https://github.com/wshobson/agents --skill spark-optimization --agent claude-code — the source lives at github.com/wshobson/agents.
Maintain spark-optimization?
Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.
[](https://getagentictools.com/skills/wshobson-agents-spark-optimization?ref=badge) npx agentictools info skills/wshobson-agents-spark-optimization The second line is the CLI lookup for this page — handy in READMEs and docs.
