Comic Crawl

Scrape comics from scanlation sites.

STATUS: TESTED (NEED TO TEST WITH OTHER SOURCES)

Prerequisites:

• Makefile
• Go Version >= 1.24
• Docker (optional, for Cloudflare bypass)

Working Sources:

 Asurascans
 Utoon
 Madarascans
 Drakescans
 Webtoon
 Flamecomics
 Rizzfables

Not Working:

 Flaresolverr implementation

Individual Components:

Run from step 1 to last (if wanted to run the whole project)

1. Fetch series from scanlator sites:

make series

2. Filter series using MangaUpdates data:

make filter

3. Fetch All Series Chapter & Images URL (From Scanlator)

make scrape

4. Download All series in proper format (need to have Series Chapter & Image URL)

make download

5. Generate Metadata for download series

make metadata

Development

Install tools:

make install-tools

Code quality checks:

make fmt      # Format code
make lint     # Run linters
make test     # Run tests
make all      # Format, lint, and test

Pre-commit hook:

make pre-commit

Optional: Cloudflare Bypass

For sites protected by Cloudflare:

docker-compose up