Page Observe

POST /sessions/{session_id}/page/observe

python

Python

from notte_sdk import NotteClient, actions

notte = NotteClient()
with notte.Session() as session:
    # Observe the state of a webpage

    _ = session.execute(
        actions.Goto(url="https://google.com/travel/flights")
    )
    obs = session.observe()
    print("Page Title:", obs.metadata.title)
    print(
        "Available Actions:\n",
        obs.space.description,
    )
    # # Example Output:
    # # Flight Search
    # * I1: Enter departure location
    # ...

import { NotteClient } from "notte-sdk";

const notte = new NotteClient();
await notte.Session().use(async (session) => {
  // Observe the state of a webpage
  await session.execute({
    type: "goto",
    url: "https://google.com/travel/flights",
  });
  const obs = await session.observe();
  console.log("Page Title:", obs.metadata.title);
  console.log(
    "Available Actions:\n",
    obs.space.description
  );
  // Example Output:
  // Flight Search
  // * I1: Enter departure location
  // ...
});

curl -X POST "https://api.notte.cc/sessions/$session_id/page/observe" \
-H "Authorization: Bearer $NOTTE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"min_nb_actions": null,
"max_nb_actions": 100,
"instructions": null,
"perception_type": null
}'

200

{
  "started_at": "2023-11-07T05:31:56Z",
  "ended_at": "2023-11-07T05:31:56Z",
  "metadata": {
    "title": "<string>",
    "url": "<string>",
    "viewport": {
      "scroll_x": 123,
      "scroll_y": 123,
      "viewport_width": 123,
      "viewport_height": 123,
      "total_width": 123,
      "total_height": 123
    },
    "tabs": [
      {
        "tab_id": 123,
        "title": "<string>",
        "url": "<string>"
      }
    ],
    "timestamp": "2023-11-07T05:31:56Z"
  },
  "screenshot": {
    "raw": "<string>",
    "bboxes": [
      {
        "x": 123,
        "y": 123,
        "width": 123,
        "height": 123,
        "scroll_x": 123,
        "scroll_y": 123,
        "viewport_width": 123,
        "viewport_height": 123,
        "iframe_offset_x": 0,
        "iframe_offset_y": 0,
        "notte_id": "<string>"
      }
    ],
    "last_action_id": "<string>"
  },
  "space": {
    "description": "<string>",
    "interaction_actions": [
      {
        "type": "click",
        "category": "Interaction Actions",
        "description": "Click on an element of the current page",
        "id": "",
        "selector": "<string>",
        "press_enter": true,
        "text_label": "<string>",
        "timeout": 5000
      }
    ],
    "actions": [
      {
        "value": {},
        "type": "form_fill",
        "category": "Special Browser Actions",
        "description": "Fill a form with multiple values. Important: If you detect a form requesting personal information, try to use this action at first, and otherwise use the regular fill action. CRITICAL: If this action fails once, use the regular form fill instead."
      }
    ],
    "browser_actions": [
      {
        "value": {},
        "type": "form_fill",
        "category": "Special Browser Actions",
        "description": "Fill a form with multiple values. Important: If you detect a form requesting personal information, try to use this action at first, and otherwise use the regular fill action. CRITICAL: If this action fails once, use the regular form fill instead."
      }
    ],
    "markdown": "<string>"
  }
}

{
  "detail": [
    {
      "loc": [
        "<string>"
      ],
      "msg": "<string>",
      "type": "<string>",
      "input": "<unknown>",
      "ctx": {}
    }
  ]
}

Authorizations

Authorization · string · header · required

The access token received from the authorization server in the OAuth 2.0 flow.

Headers

x-notte-request-origin · string | null

x-notte-sdk-version · string | null

Path Parameters

session_id · string · required

Body

application/json

min_nb_actions · integer | null

The minimum number of actions to list before stopping. If not provided, the listing will continue until the maximum number of actions is reached.

max_nb_actions · integer · default: 100

The maximum number of actions to list after which the listing will stop. Used when min_nb_actions is not provided.

instructions · string | null

Additional instructions to use for the observation.

perception_type · enum<string> | null

Whether to run with fast or deep perception

Available options: fast, deep

Response

200

application/json

Successful Response

started_at · string<date-time> · required

ended_at · string<date-time> · required

metadata · SnapshotMetadata · object · required

Metadata of the current page, i.e url, page title, snapshot timestamp.

Show child attributes

screenshot · Screenshot · object · required

Base64 encoded screenshot of the current page

Show child attributes

space · ActionSpace · object · required

Available actions in the current state

Show child attributes