Crawls | Screpy

Crawls

Start website crawls and retrieve crawl status, progress, and completed audit summaries with the Screpy REST API.

Treat crawl status as part of the workflow

Start a crawl only after you have selected the right project and checked the current account allowance. Then poll the crawl status until it is complete before relying on its summary or using that crawl in a comparison.

A crawl that is still running is not a final technical baseline. Keep the crawl UID returned by the API so later crawl-data requests read the matching result set.

List a project's crawls

GET /projects/{project_uid}/crawls

Authorization

bearerApiKey

AuthorizationBearer

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

Path Parameters

project_uid*string

Accessible project UID. Discover it with GET /projects; do not guess it.

Match ^[a-z0-9]{10}$

Query Parameters

limit?integer

Maximum records to return. Defaults to 50.

Range 1 <= value <= 100

Default 50

cursor?string

Opaque cursor returned by the prior response. Do not construct or alter it.

Length length <= 512

Response Body

200 application/json

401 application/json

403 application/json

404 application/json

422 application/json

429 application/json

{
  "data": [
    {
      "uid": "string",
      "status": "string",
      "target_url": "http://example.com",
      "max_pages": 0,
      "depth": 1,
      "health_score": 0,
      "seo_score": 0,
      "pages_total": 0,
      "critical_issues": 0,
      "warnings": 0,
      "notices": 0,
      "queued_at": "2019-08-24T14:15:22Z",
      "started_at": "2019-08-24T14:15:22Z",
      "finished_at": "2019-08-24T14:15:22Z",
      "created_at": "2019-08-24T14:15:22Z"
    }
  ],
  "links": {
    "first": "http://example.com",
    "last": "http://example.com",
    "prev": "http://example.com",
    "next": "http://example.com"
  },
  "meta": {
    "path": "http://example.com",
    "per_page": 0,
    "next_cursor": "string",
    "prev_cursor": "string"
  }
}

Start a crawl

POST /projects/{project_uid}/crawls

Starts a crawl with the project's current settings. It may consume the account's crawl allowance.

Authorization

bearerApiKey

AuthorizationBearer

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

Path Parameters

project_uid*string

Accessible project UID. Discover it with GET /projects; do not guess it.

Match ^[a-z0-9]{10}$

Response Body

201 application/json

401 application/json

403 application/json

404 application/json

409 application/json

429 application/json

503 application/json

{
  "data": {
    "uid": "string",
    "status": "string",
    "target_url": "http://example.com",
    "max_pages": 0,
    "depth": 1,
    "health_score": 0,
    "seo_score": 0,
    "pages_total": 0,
    "critical_issues": 0,
    "warnings": 0,
    "notices": 0,
    "queued_at": "2019-08-24T14:15:22Z",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "created_at": "2019-08-24T14:15:22Z"
  }
}

Get crawl status and progress

GET /projects/{project_uid}/crawls/{crawl_uid}

Authorization

bearerApiKey

AuthorizationBearer

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

Path Parameters

project_uid*string

Accessible project UID. Discover it with GET /projects; do not guess it.

Match ^[a-z0-9]{10}$

crawl_uid*string

Crawl UID belonging to the selected project. Discover it with GET /projects/{project_uid}/crawls.

Match ^[a-z0-9]{16}$

Response Body

200 application/json

401 application/json

403 application/json

404 application/json

429 application/json

{
  "data": {
    "crawl_uid": "string",
    "status": "string",
    "state": "string",
    "status_label": "string",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "last_synced_at": "2019-08-24T14:15:22Z",
    "progress": {
      "pages_crawled": 0,
      "pages_total": 0,
      "max_pages": 0,
      "current_depth": 0
    }
  }
}

Get a completed crawl summary

GET /projects/{project_uid}/crawls/{crawl_uid}/summary

Authorization

bearerApiKey

AuthorizationBearer

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

Path Parameters

project_uid*string

Accessible project UID. Discover it with GET /projects; do not guess it.

Match ^[a-z0-9]{10}$

crawl_uid*string

Crawl UID belonging to the selected project. Discover it with GET /projects/{project_uid}/crawls.

Match ^[a-z0-9]{16}$

Response Body

200 application/json

401 application/json

403 application/json

404 application/json

409 application/json

429 application/json

{
  "data": {
    "crawl_uid": "string",
    "status": "string",
    "state": "string",
    "status_label": "string",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "last_synced_at": "2019-08-24T14:15:22Z",
    "progress": {
      "pages_crawled": 0,
      "pages_total": 0,
      "max_pages": 0,
      "current_depth": 0
    },
    "health": {
      "score": 0,
      "confidence": 0,
      "coverage": 0
    },
    "issues": {
      "critical": 0,
      "warnings": 0,
      "notices": 0,
      "total": 0
    },
    "totals": {
      "pages": 0,
      "indexable_pages": 0,
      "noindex_pages": 0,
      "links": 0,
      "images": 0,
      "unique_urls": 0
    }
  }
}