web-scan

Intro

A program that generates a combination of letters from the alphabet and composes an address for accessing the site.After that, it reads the data from the title and description of this site and saves it to the structure. At the end, the program displays all the titles and descriptions of the available sites.

Usage

golang/cmd/go

go get github.com/digital-technology-agency/web-scan

Data store

• sqlite - SQLite
• jsoneachrow - Json Each row

Generator type

• simple - Simple generator type

Init configuration file

$ ./wscan init

Configuration file config.json

{
  "process_count": 1,
  "alphabet": "abcdefgefghijklmnop",
  "url_len": 5,
  "concurrency_count": 5,
  "data_store_type": "sqlite",
  "generator_type": "simple",
  "protocol_types": [
    "http",
    "https"
  ]
}

Run

$ ./wscan -configuration_file config.json