slubby

中文文档

About Slubby

slubby is a go-based, componentized, highly extensible, fast, crawler development component library that can be used with slub scaffolding tools to achieve fast and fast release of go language crawlers.

Features

• Componentization, where you are free to replace supporting components to get different representations.

• Good scalability, support multiple process intermediate parts, efficient expansion of crawler function.

• Quickly develop crawlers with slub scaffolding.

• More common crawler component support.

Require

• go 1.15

• go mod

Installation method (slub)

# This method will install slub crawler scaffolding, and we will use slub to create slubby projects.

go install github.com/wheat-os/slub@latest

Give it a try