> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apimart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Grok Imagine 2.0 Ext Layers and Region Editing

> Use segment to retrieve object layers and precise masks, then edit polygons, boxes, or detected objects with region_edit.

<Info>
  Both `segment` and `region_edit` use the existing asynchronous image endpoint. Save the returned `task_id`, then poll [Get task status](/en/api-reference/tasks/status); the create request does not return final layers or images.
</Info>

<Warning>
  Never expose an API key in a browser bundle, LocalStorage, a URL, or frontend logs. Call APIMart through your backend or BFF.
</Warning>

<RequestExample>
  ```bash cURL theme={null}
  curl --request POST \
    --url https://api.apimart.ai/v1/images/generations \
    --header 'Authorization: Bearer <token>' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header 'Idempotency-Key: 6baf0940-25d6-4ec2-9131-925250840fa7' \
    --data '{
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "source_task_id": "<COMPLETED_SINGLE_IMAGE_TASK_ID>",
      "include_mask_rle": true,
      "cached_only": false
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 theme={null}
  {
    "code": 200,
    "data": [{ "status": "submitted", "task_id": "task_..." }]
  }
  ```
</ResponseExample>

## Operation overview

| Purpose                                                                 | Key input                       | Completed result                   | Billing                    |
| ----------------------------------------------------------------------- | ------------------------------- | ---------------------------------- | -------------------------- |
| `segment`: Detect objects and retrieve layers, boxes, and precise masks | `source_task_id`                | `image_id`, `image_url`, `objects` | Free                       |
| `region_edit`: Edit a polygon, rectangle, or detected object            | `image_id`, `prompt`, selection | New URL and `image_id`             | Charged per completed task |

```text theme={null}
task_id → segment(source_task_id) → image_id + mask_rle
        → selection_regions → region_edit → new task_id + image_id
```

<Note>
  `source_task_id` and `image_id` are not interchangeable. `segment` takes the source task ID; `region_edit` takes the image asset ID. To segment an edited image, use the completed `region_edit` task ID as the next `source_task_id`.
</Note>

## Request headers

Use `Authorization: Bearer <APIMART_API_KEY>`, `Content-Type: application/json`, and `Accept: application/json`.

`Idempotency-Key` is optional and strongly recommended for paid `region_edit` requests. It accepts 1–191 visible ASCII characters; UUID is recommended. Use a new key for each new logical operation. A network retry of the same request must reuse the original key and identical body. If a paid edit has an indeterminate result, do not retry automatically with a new key.

## Asynchronous task flow

A successful create request returns HTTP `200` and `data[0].task_id`. Poll `GET /v1/tasks/{task_id}?language=en` every 2 seconds, back off to at most 5 seconds, and set a 10-minute overall timeout. Stop old polling when the source image changes.

<Warning>
  A task query can return HTTP `200` while `data.status` is `failed`. Always determine success from `data.status` and display `data.error` when present.
</Warning>

## `segment`

### Request parameters

| Field              | Type    | Required | Default | Description                                                   |
| ------------------ | ------- | :------: | ------- | ------------------------------------------------------------- |
| `model`            | string  |     ✅    | —       | Fixed to `grok-imagine-2.0-ext`                               |
| `operation`        | string  |     ✅    | —       | Fixed to `segment`                                            |
| `source_task_id`   | string  |     ✅    | —       | A completed, single-image Grok task owned by the current user |
| `include_mask_rle` | boolean |     —    | `true`  | Return COCO compressed RLE; keep `true` for precise editing   |
| `cache_only`       | boolean |     —    | `false` | Check segmentation cache only; do not call upstream on a miss |
| `cached_only`      | boolean |     —    | `false` | Upstream cache hint; not a local cache guarantee              |
| `refresh`          | boolean |     —    | `false` | Bypass cache; do not use in the normal editor flow            |

`segment` does not need `prompt`. Do not send `image_id`, `image_index`, `billing_model_name`, `n`, `size`, or `response_format`. `cache_only=true` and `refresh=true` are mutually exclusive.

### Request examples

<Tabs>
  <Tab title="Fetch layers">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "source_task_id": "<COMPLETED_SINGLE_IMAGE_TASK_ID>",
      "include_mask_rle": true,
      "cached_only": false
    }
    ```
  </Tab>

  <Tab title="Cache probe">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "segment",
      "source_task_id": "<COMPLETED_SINGLE_IMAGE_TASK_ID>",
      "include_mask_rle": true,
      "cache_only": true
    }
    ```
  </Tab>
</Tabs>

A cache miss is still a successful task. Use `cache_status` (`hit` or `miss`) or `from_cache`; do not infer a hit from `cached`.

### Completed response

For `segment`, `data.result` is the segmentation result directly; it is not wrapped in `images`.

```json theme={null}
{
  "code": 200,
  "data": {
    "id": "task_...",
    "status": "completed",
    "progress": 100,
    "cost": 0,
    "credits_cost": 0,
    "result": {
      "source_task_id": "task_...",
      "image_id": "6b78a7c3-5f9e-4a3d-a928-9e9b42113a3f",
      "image_url": "https://.../source.jpg",
      "from_cache": true,
      "cache_status": "hit",
      "objects": [{
        "index": 0,
        "name": "red sports car",
        "box_xyxy": [38.1, 689.8, 945.8, 1065.4],
        "score": 0.9765625,
        "mask_size": [1792, 1008],
        "mask_url": "",
        "mask_rle": { "size": [1792, 1008], "counts": "..." }
      }]
    }
  }
}
```

| Field                 | Description                                                    |
| --------------------- | -------------------------------------------------------------- |
| `result.image_id`     | Asset ID used by `region_edit`                                 |
| `result.image_url`    | HTTP(S) URL aligned with `image_id`                            |
| `objects[].index`     | Original server index; preserve it when using `object_indices` |
| `objects[].box_xyxy`  | Mask pixel box `[x1,y1,x2,y2]`                                 |
| `objects[].score`     | Detection confidence; may be `null`                            |
| `objects[].mask_size` | Always `[height,width]`; never hard-code dimensions            |
| `objects[].mask_rle`  | COCO compressed RLE for precise outlines                       |
| `objects[].mask_url`  | Optional mask image URL; may be empty                          |

An object without a valid `mask_rle` or `mask_url` can only use approximate box editing.

## Decode `mask_rle`

`mask_rle.counts` is a COCO compressed-count string, not Base64 or zlib. It expands in column-major order; the first run is background, followed by alternating foreground and background runs.

The following TypeScript converts it into a browser-friendly row-major binary mask:

```ts theme={null}
export interface CocoRLE {
  size: [height: number, width: number];
  counts: string;
}

export interface BinaryMask {
  width: number;
  height: number;
  data: Uint8Array; // data[y * width + x]
}

function decodeCompressedCounts(counts: string): number[] {
  const runs: number[] = [];
  let cursor = 0;
  while (cursor < counts.length) {
    let value = 0;
    let shift = 0;
    let more = true;
    while (more) {
      if (cursor >= counts.length) throw new Error("Truncated COCO RLE counts");
      const current = counts.charCodeAt(cursor++) - 48;
      value |= (current & 0x1f) << shift;
      more = (current & 0x20) !== 0;
      shift += 5;
      if (!more && (current & 0x10) !== 0) value |= -1 << shift;
    }
    if (runs.length > 2) value += runs[runs.length - 2] ?? 0;
    if (value < 0) throw new Error(\`Invalid COCO RLE run: \${value}\`);
    runs.push(value);
  }
  return runs;
}

export function decodeCocoRLE(rle: CocoRLE): BinaryMask {
  const [height, width] = rle.size;
  if (!Number.isInteger(height) || !Number.isInteger(width) || height <= 0 || width <= 0) {
    throw new Error(\`Invalid mask size: \${JSON.stringify(rle.size)}\`);
  }
  const pixelCount = width * height;
  if (!Number.isSafeInteger(pixelCount) || pixelCount > 32_000_000) {
    throw new Error(\`Mask exceeds the frontend safety limit: \${pixelCount}\`);
  }
  if (!rle.counts) throw new Error("Missing COCO RLE counts");

  const data = new Uint8Array(pixelCount);
  const runs = decodeCompressedCounts(rle.counts);
  let position = 0;
  let foreground = false;
  for (const run of runs) {
    if (position + run > data.length) throw new Error("COCO RLE exceeds mask_size");
    if (foreground) {
      for (let offset = 0; offset < run; offset++) {
        const index = position + offset;
        const y = index % height;
        const x = (index - y) / height;
        data[y * width + x] = 1;
      }
    }
    position += run;
    foreground = !foreground;
  }
  if (position !== data.length) throw new Error("COCO RLE does not cover the mask");
  return { width, height, data };
}
```

Decode large masks in a Web Worker. Never send complete `mask_rle.counts` values to logs, analytics, URLs, or error reporting.

### Convert masks to precise selections

```ts theme={null}
interface SelectionBoundary { points: number[] }
interface SelectionRegion { outer: SelectionBoundary; holes?: SelectionBoundary[] }
```

Trace connected components and holes, simplify the contours, and normalize every point to `0–1`. Each ring needs at least 3 distinct points, must have non-zero area, and must not self-intersect. Keep at most 16 largest regions per layer and 400 points per ring.

<Warning>
  `mask_size` is `[height,width]` and uses source-mask coordinates, not CSS display coordinates. With `object-fit: contain`, subtract letterbox offsets, scale against the actual drawn area, and clamp the final values to `0–1`.
</Warning>

Reading source-image or `mask_url` pixels requires CORS. Set `crossOrigin = "anonymous"` before `src`, or fetch a Blob. Decoding `mask_rle` directly avoids this dependency.

## Edit a region: `region_edit`

### Request parameters

| Field               | Type         | Required | Description                                                                             |
| ------------------- | ------------ | :------: | --------------------------------------------------------------------------------------- |
| `model`             | string       |     ✅    | Fixed to `grok-imagine-2.0-ext`                                                         |
| `operation`         | string       |     ✅    | `region_edit`                                                                           |
| `image_id`          | string       |     ✅    | Source asset ID; first use `segment` result `image_id`, then use the latest edit result |
| `prompt`            | string       |     ✅    | Non-empty instruction describing the desired change                                     |
| `selection_regions` | array        |    \*    | Normalized `0–1` polygons with `outer` and optional `holes`; recommended                |
| `boxes`             | number\[]\[] |    \*    | Rectangles `[x1,y1,x2,y2]`; pixel boxes require `mask_size`                             |
| `object_indices`    | integer\[]   |    \*    | Original `objects[].index` values; approximate box editing only                         |
| `mask_size`         | integer\[]   |    \*    | Required for pixel boxes; `[height,width]` with positive integers                       |

At least one of `selection_regions`, `boxes`, or `object_indices` must be non-empty. The API accepts combinations, but the frontend should use one method per request.

<Warning>
  Do not send `billing_model_name`, `size`, `aspect_ratio`, `source_aspect_ratio`, `source_size`, or `image_urls`. Omit `n` or set it to `1`; omit `claim_asset` or set it to `false`; omit `response_format` or set it to `url`. Base64 output and `stream=true` are unsupported.
</Warning>

### Selection methods

| Method              | Selection source         | Precision              | Recommended use                   |
| ------------------- | ------------------------ | ---------------------- | --------------------------------- |
| `selection_regions` | Frontend polygons        | Exact, including holes | Production layer or brush editing |
| `boxes`             | Frontend rectangles      | Box approximation      | Box tool or MVP                   |
| `object_indices`    | Original segment indices | Box approximation      | Quick integration testing         |

<Tabs>
  <Tab title="Precise polygon">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the selected car to bright red and preserve the rest",
      "selection_regions": [{
        "outer": { "points": [0.12, 0.20, 0.48, 0.20, 0.48, 0.61, 0.12, 0.61] },
        "holes": [{ "points": [0.30, 0.35, 0.38, 0.35, 0.38, 0.45, 0.30, 0.45] }]
      }]
    }
    ```

    `points` can be flat or nested pairs. Every value must be finite and within `0–1`; each ring needs at least 3 coordinate pairs.
  </Tab>

  <Tab title="Normalized box">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the car inside the box to bright red",
      "boxes": [[0.04, 0.385, 0.938, 0.594]]
    }
    ```
  </Tab>

  <Tab title="Pixel box">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the car inside the box to bright red",
      "boxes": [[40, 689.6, 945.9, 1064.4]],
      "mask_size": [1792, 1008]
    }
    ```
  </Tab>

  <Tab title="Object index">
    ```json theme={null}
    {
      "model": "grok-imagine-2.0-ext",
      "operation": "region_edit",
      "image_id": "<SOURCE_IMAGE_ID>",
      "prompt": "Change the selected car to bright red",
      "object_indices": [0]
    }
    ```

    Indices must come from the segment response for the same `image_id`. Do not replace them with indices from a filtered, sorted, or grouped frontend array.
  </Tab>
</Tabs>

### Completed response

```json theme={null}
{
  "code": 200,
  "data": {
    "id": "task_...",
    "status": "completed",
    "progress": 100,
    "cost": 0.016,
    "credits_cost": 0.16,
    "result": {
      "images": [{
        "url": ["https://.../result.jpg"],
        "image_ids": ["<NEW_IMAGE_ID>"],
        "items": [{
          "url": "https://.../result.jpg",
          "image_id": "<NEW_IMAGE_ID>",
          "source_image_id": "<SOURCE_IMAGE_ID>",
          "role": "region_edit"
        }],
        "expires_at": 1787040000
      }]
    }
  }
}
```

Prefer `result.images[0].items[0]`. For legacy responses, pair `url[0]` with `image_ids[0]` only when the arrays have equal lengths. Continue only after obtaining both an HTTP(S) URL and a new `image_id`.

Use `expires_at` as the source of truth for URL expiry; do not hard-code a number of hours. Download or persist assets needed for long-term display.

## Continuous editing

After an edit completes, update the displayed URL, the current asset ID, and the source task ID together, then clear old layers and polling state.

* Segment again: use this `region_edit` task ID as `source_task_id`
* Edit again: use the newly returned `image_id`
* Never pass `image_id` to `segment`, and never keep editing the previous image ID.

## Error handling

| HTTP / status                   | Common cause                                                                     | Handling                                                                      |
| ------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| 400 invalid source or operation | Wrong operation, unusable source task, or `image_id/image_index` sent to segment | Validate the operation and use the current user's completed single-image task |
| 400 invalid selection           | Empty prompt, missing selection, invalid polygon, box, or object index           | Validate the prompt and selection before submission                           |
| 400 unsupported option          | Invalid `claim_asset`, `n`, output format, size, or stream mode                  | Remove unsupported fields and use URL output                                  |
| 401 / 403                       | Invalid key or missing model permission                                          | Check the server-side key and account access                                  |
| 402                             | Insufficient balance for a paid edit                                             | Ask the user to top up before retrying                                        |
| 409                             | Idempotency request is in progress, changed, or indeterminate                    | Follow the response; do not switch keys automatically                         |
| 429 / 5xx                       | Rate limit or temporary service failure                                          | Honor `Retry-After` and retry with bounded backoff                            |
| failed / task\_failed           | Asynchronous execution failed                                                    | Stop polling and show `data.error.message`                                    |

## Billing

* `segment` is free and completes with `cost=0` and `credits_cost=0`, but still requires authentication and a valid source task.
* `region_edit` is paid. Use the completed task's `cost` and `credits_cost`; do not hard-code prices in the frontend.
* Never send the internal field `billing_model_name`.

## Frontend checklist

* Keep the API key only in the backend or BFF.
* Send only `source_task_id` to `segment`; do not send `image_id` or `image_index`.
* Use segment's `image_id` for `region_edit` and provide at least one selection method.
* Use `selection_regions` for precise production editing; `object_indices` is only a box approximation.
* Always parse `mask_size` as `[height,width]` and account for display scaling and letterboxing.
* Reuse the original idempotency key for the same network retry and validate both the output URL and new `image_id`.