Crawl Website | Scout | Documentation

BFS crawl from start_url, bounded by max_pages + max_depth.

AuthorizationBearer

Your API key, sent as a Bearer token.

This endpoint expects an object.

start_urlstringRequired

max_pagesintegerOptional1-200Defaults to 20

max_depthintegerOptional0-5Defaults to 2

same_host_onlybooleanOptionalDefaults to true

include_patternslist of strings or nullOptional

exclude_patternslist of strings or nullOptional

Successful Response

422

Unprocessable Entity Error

$	curl -X POST https://core.usescout.sh/v1/crawl/website \
>	-H "Authorization: Bearer <token>" \
>	-H "Content-Type: application/json" \
>	-d '{
>	"start_url": "https://www.example.com"
>	}'