# Crawls

Start website crawls and retrieve crawl status, progress, and completed audit summaries with the Screpy REST API.

## [Treat crawl status as part of the workflow](/content/docs/api/crawls#treat-crawl-status-as-part-of-the-workflow/index.html)

Start a crawl only after you have selected the right [project](/content/docs/api/projects/index.html) and checked the current account allowance. Then poll the crawl status until it is complete before relying on its summary or using that crawl in a comparison.

A crawl that is still running is not a final technical baseline. Keep the crawl UID returned by the API so later [crawl-data](/content/docs/api/crawl-data/index.html) requests read the matching result set.

## [List a project's crawls](/content/docs/api/crawls#list-a-projects-crawls/index.html)

GET `/projects/{project_uid}/crawls`

### [Authorization](/content/docs/api/crawls#authorization/index.html)

`bearerApiKey`

AuthorizationBearer <token>

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

### [Path Parameters](/content/docs/api/crawls#parameters-path/index.html)

project_uid*string

Accessible project UID. Discover it with `GET /projects`; do not guess it.

Match `^[a-z0-9]{10}$`

### [Query Parameters](/content/docs/api/crawls#parameters-query/index.html)

limit?integer

Maximum records to return. Defaults to 50.

Range `1 <= value <= 100`

Default `50`

cursor?string

Opaque cursor returned by the prior response. Do not construct or alter it.

Length `length <= 512`

### [Response Body](/content/docs/api/crawls#response-body/index.html)

### 200  `application/json`

### 401  `application/json`

### 403  `application/json`

### 404  `application/json`

### 422  `application/json`

### 429  `application/json`

```json
{
  "data": [
    {
      "uid": "string",
      "status": "string",
      "target_url": "http://example.com",
      "max_pages": 0,
      "depth": 1,
      "health_score": 0,
      "seo_score": 0,
      "pages_total": 0,
      "critical_issues": 0,
      "warnings": 0,
      "notices": 0,
      "queued_at": "2019-08-24T14:15:22Z",
      "started_at": "2019-08-24T14:15:22Z",
      "finished_at": "2019-08-24T14:15:22Z",
      "created_at": "2019-08-24T14:15:22Z"
    }
  ],
  "links": {
    "first": "http://example.com",
    "last": "http://example.com",
    "prev": "http://example.com",
    "next": "http://example.com"
  },
  "meta": {
    "path": "http://example.com",
    "per_page": 0,
    "next_cursor": "string",
    "prev_cursor": "string"
  }
}
```

### [Start a crawl](/content/docs/api/crawls#start-a-crawl/index.html)

POST `/projects/{project_uid}/crawls`

Starts a crawl with the project's current settings. It may consume the account's crawl allowance.

### [Authorization](/content/docs/api/crawls#authorization/index.html)

`bearerApiKey`

AuthorizationBearer <token>

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

### [Path Parameters](/content/docs/api/crawls#parameters-path/index.html)

project_uid*string

Accessible project UID. Discover it with `GET /projects`; do not guess it.

Match `^[a-z0-9]{10}$`

### [Response Body](/content/docs/api/crawls#response-body/index.html)

### 201  `application/json`

### 401  `application/json`

### 403  `application/json`

### 404  `application/json`

### 409  `application/json`

### 429  `application/json`

### 503  `application/json`

```json
{
  "data": {
    "uid": "string",
    "status": "string",
    "target_url": "http://example.com",
    "max_pages": 0,
    "depth": 1,
    "health_score": 0,
    "seo_score": 0,
    "pages_total": 0,
    "critical_issues": 0,
    "warnings": 0,
    "notices": 0,
    "queued_at": "2019-08-24T14:15:22Z",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "created_at": "2019-08-24T14:15:22Z"
  }
}
```

### [Get crawl status and progress](/content/docs/api/crawls#get-crawl-status-and-progress/index.html)

GET `/projects/{project_uid}/crawls/{crawl_uid}`

### [Authorization](/content/docs/api/crawls#authorization/index.html)

`bearerApiKey`

AuthorizationBearer <token>

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

### [Path Parameters](/content/docs/api/crawls#parameters-path/index.html)

project_uid*string

Accessible project UID. Discover it with `GET /projects`; do not guess it.

Match `^[a-z0-9]{10}$`

crawl_uid*string

Crawl UID belonging to the selected project. Discover it with `GET /projects/{project_uid}/crawls`.

Match `^[a-z0-9]{16}$`

### [Response Body](/content/docs/api/crawls#response-body/index.html)

### 200  `application/json`

### 401  `application/json`

### 403  `application/json`

### 404  `application/json`

### 429  `application/json`

```json
{
  "data": {
    "crawl_uid": "string",
    "status": "string",
    "state": "string",
    "status_label": "string",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "last_synced_at": "2019-08-24T14:15:22Z",
    "progress": {
      "pages_crawled": 0,
      "pages_total": 0,
      "max_pages": 0,
      "current_depth": 0
    }
  }
}
```

### [Get a completed crawl summary](/content/docs/api/crawls#get-a-completed-crawl-summary/index.html)

GET `/projects/{project_uid}/crawls/{crawl_uid}/summary`

### [Authorization](/content/docs/api/crawls#authorization/index.html)

`bearerApiKey`

AuthorizationBearer <token>

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

### [Path Parameters](/content/docs/api/crawls#parameters-path/index.html)

project_uid*string

Accessible project UID. Discover it with `GET /projects`; do not guess it.

Match `^[a-z0-9]{10}$`

crawl_uid*string

Crawl UID belonging to the selected project. Discover it with `GET /projects/{project_uid}/crawls`.

Match `^[a-z0-9]{16}$`

### [Response Body](/content/docs/api/crawls#response-body/index.html)

### 200  `application/json`

### 401  `application/json`

### 403  `application/json`

### 404  `application/json`

### 409  `application/json`

### 429  `application/json`

```json
{
  "data": {
    "crawl_uid": "string",
    "status": "string",
    "state": "string",
    "status_label": "string",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "last_synced_at": "2019-08-24T14:15:22Z",
    "progress": {
      "pages_crawled": 0,
      "pages_total": 0,
      "max_pages": 0,
      "current_depth": 0
    },
    "health": {
      "score": 0,
      "confidence": 0,
      "coverage": 0
    },
    "issues": {
      "critical": 0,
      "warnings": 0,
      "notices": 0,
      "total": 0
    },
    "totals": {
      "pages": 0,
      "indexable_pages": 0,
      "noindex_pages": 0,
      "links": 0,
      "images": 0,
      "unique_urls": 0
    }
  }
}
```