scrapegraph-sdk

scrapegraph-sdk

πŸ•·οΈ Official Scrapegraph API SDK: Effortlessly extract content from any website. AI-powered. πŸ€– Hassle-free web scraping made simple.

Stars: 57

Visit
 screenshot

Official SDKs for the ScrapeGraph AI API - Intelligent web scraping and search powered by AI. Extract structured data from any webpage or perform AI-powered web searches with natural language prompts. The SDK offers features such as SmartScraper for data extraction, SearchScraper for AI-powered web search, Markdownify for converting webpages to markdown, SmartCrawler for intelligent crawling, AgenticScraper for automated browser actions, and more. It provides seamless integration with popular frameworks and tools, supports Python and JavaScript SDKs, LLM frameworks, low-code platforms, and offers core features like AI-powered extraction, structured output, multiple data formats, high performance, and enterprise-grade security.

README:

🌐 ScrapeGraph AI SDKs

License Python SDK JavaScript SDK Documentation

Official SDKs for the ScrapeGraph AI API - Intelligent web scraping and search powered by AI. Extract structured data from any webpage or perform AI-powered web searches with natural language prompts.

Get your API key!

Features

  • πŸ€– SmartScraper: Extract structured data from webpages using natural language prompts
  • πŸ” SearchScraper: AI-powered web search with structured results and reference URLs
  • πŸ“ Markdownify: Convert any webpage into clean, formatted markdown
  • πŸ•·οΈ SmartCrawler: Intelligently crawl and extract data from multiple pages
  • πŸ€– AgenticScraper: Perform automated browser actions with AI-powered session management
  • πŸ“„ Scrape: Convert webpages to HTML with JavaScript rendering and custom headers
  • ⏰ Scheduled Jobs: Create and manage automated scraping workflows with cron scheduling
  • πŸ’³ Credits Management: Monitor API usage and credit balance
  • πŸ’¬ Feedback System: Provide ratings and feedback to improve service quality

πŸš€ Quick Links

ScrapeGraphAI offers seamless integration with popular frameworks and tools to enhance your scraping capabilities. Whether you're building with Python or Node.js, using LLM frameworks, or working with no-code platforms, we've got you covered with our comprehensive integration options..

You can find more informations at the following link

Integrations:

πŸ“¦ Installation

Python

pip install scrapegraph-py

JavaScript

npm install scrapegraph-js

🎯 Core Features

  • πŸ€– AI-Powered Extraction & Search: Use natural language to extract data or search the web
  • πŸ“Š Structured Output: Get clean, structured data with optional schema validation
  • πŸ”„ Multiple Formats: Extract data as JSON, Markdown, or custom schemas
  • ⚑ High Performance: Concurrent processing and automatic retries
  • πŸ”’ Enterprise Ready: Production-grade security and rate limiting

πŸ› οΈ Available Endpoints

πŸ€– SmartScraper

Using AI to extract structured data from any webpage or HTML content with natural language prompts.

πŸ” SearchScraper

Perform AI-powered web searches with structured results and reference URLs.

πŸ“ Markdownify

Convert any webpage into clean, formatted markdown.

πŸ•·οΈ SmartCrawler

Intelligently crawl and extract data from multiple pages with configurable depth and batch processing.

πŸ€– AgenticScraper

Perform automated browser actions on webpages using AI-powered agentic scraping with session management.

πŸ“„ Scrape

Convert webpages into HTML format with optional JavaScript rendering and custom headers.

⏰ Scheduled Jobs

Create, manage, and monitor scheduled scraping jobs with cron expressions and execution history.

πŸ’³ Credits

Check your API credit balance and usage.

πŸ’¬ Feedback

Send feedback and ratings for scraping requests to help improve the service.

🌟 Key Benefits

  • πŸ“ Natural Language Queries: No complex selectors or XPath needed
  • 🎯 Precise Extraction: AI understands context and structure
  • πŸ”„ Adaptive Processing: Works with both web content and direct HTML
  • πŸ“Š Schema Validation: Ensure data consistency with Pydantic/TypeScript
  • ⚑ Async Support: Handle multiple requests efficiently
  • πŸ” Source Attribution: Get reference URLs for search results

πŸ’‘ Use Cases

  • 🏒 Business Intelligence: Extract company information and contacts
  • πŸ“Š Market Research: Gather product data and pricing
  • πŸ“° Content Aggregation: Convert articles to structured formats
  • πŸ” Data Mining: Extract specific information from multiple sources
  • πŸ“± App Integration: Feed clean data into your applications
  • 🌐 Web Research: Perform AI-powered searches with structured results

πŸ“– Documentation

For detailed documentation and examples, visit:

πŸ’¬ Support & Feedback

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.


Made with ❀️ by ScrapeGraph AI

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for scrapegraph-sdk

Similar Open Source Tools

For similar tasks

For similar jobs