Best Python automation repositories

The API to search, scrape, and interact with the web at scale. 🔥

Matched: scraping

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Matched: scraping

Scrapy, a fast high-level web crawling & scraping framework for Python.

Matched: scraping

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

Matched: scraping

AIHawk aims to easy job hunt process by automating the job application process. Utilizing artificial intelligence, it enables users to apply for multiple jobs in a tailored way.

Matched: scraping

Python scraper based on AI

Matched: scraping

Elegant Scraper and Crawler Framework for Golang

Matched: scraping

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Matched: scraping

An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.

Matched: scraping

The headless browser for AI agents and web scraping

Matched: scraping