From ced10b92b4b7866004bc3965f5500131d0d7e294 Mon Sep 17 00:00:00 2001 From: kathayl Date: Mon, 8 Dec 2025 17:48:25 -0800 Subject: [PATCH 1/2] docs(browser-rendering): add waitUntil guidance for JS-heavy pages - Add new partial explaining gotoOptions.waitUntil for JavaScript-heavy pages - Add partial to all REST API endpoint docs - Clean up timeouts reference page with dedicated waitUntil options table - Fix default waitUntil value to domcontentloaded per API reference --- .../browser-rendering/reference/timeouts.mdx | 29 ++++++++++++++----- .../rest-api/content-endpoint.mdx | 5 ++++ .../rest-api/json-endpoint.mdx | 2 ++ .../rest-api/links-endpoint.mdx | 2 ++ .../rest-api/markdown-endpoint.mdx | 5 ++++ .../rest-api/pdf-endpoint.mdx | 5 ++++ .../rest-api/scrape-endpoint.mdx | 5 ++++ .../rest-api/screenshot-endpoint.mdx | 5 ++++ .../browser-rendering/rest-api/snapshot.mdx | 5 ++++ .../javascript-heavy-pages.mdx | 16 ++++++++++ 10 files changed, 71 insertions(+), 8 deletions(-) create mode 100644 src/content/partials/browser-rendering/javascript-heavy-pages.mdx diff --git a/src/content/docs/browser-rendering/reference/timeouts.mdx b/src/content/docs/browser-rendering/reference/timeouts.mdx index 837083892156d8..47a3abe8b256fe 100644 --- a/src/content/docs/browser-rendering/reference/timeouts.mdx +++ b/src/content/docs/browser-rendering/reference/timeouts.mdx @@ -11,14 +11,27 @@ If any of these timers exceed their limit, the request returns a timeout error. Each timer controls a specific part of the rendering lifecycle — from page load, to selector load, to action. -| Timer | Scope |Default |Max | -| -------------------------------------- | --------------- | --------------- | --------------- | -| `goToOptions.timeout` | Time to wait for the page to load before timeout. | 30 s | 60 s | -| `goToOptions.waitUntil` | Time until page load considered complete:
`load` = full page load, including resources, like images and stylesheets.
`Event.domcontentloaded` = waits until the DOM content has been fully loaded, fires before the page `load` event.
`Event.networkidle0` = there are no active network connections for at least 500 ms.
`Event.networkidle2` = there are no more than two active network connections for at least 500 ms. | — | — | -| `waitForSelector` | Time to wait for a specific element (any CSS selector) to appear on the page. | null | 60 s | -| `waitForTimeout` | Additional amount of time to wait after the page has loaded to proceed with actions. | null | 60 s | -| `actionTimeout` | Time to wait for the action itself (for example: a screenshot, PDF, or scrape) to complete after the page has loaded. | null | 5 min | -| `PDFOptions.timeout` | Same as `actionTimeout`, but only applies to the [/pdf endpoint](/browser-rendering/rest-api/pdf-endpoint/). | 30 s | 5 min | +| Timer | Scope | Default | Max | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------ | ------- | ------ | +| `goToOptions.timeout` | Time to wait for the page to load before timeout. | 30 s | 60 s | +| `goToOptions.waitUntil` | Determines when page load is considered complete. Refer to [`waitUntil` options](#waituntil-options) for details. | `domcontentloaded` | — | +| `waitForSelector` | Time to wait for a specific element (any CSS selector) to appear on the page. | null | 60 s | +| `waitForTimeout` | Additional amount of time to wait after the page has loaded to proceed with actions. | null | 60 s | +| `actionTimeout` | Time to wait for the action itself (for example: a screenshot, PDF, or scrape) to complete after the page has loaded. | null | 5 min | +| `PDFOptions.timeout` | Same as `actionTimeout`, but only applies to the [/pdf endpoint](/browser-rendering/rest-api/pdf-endpoint/). | 30 s | 5 min | + +### `waitUntil` options + +The `goToOptions.waitUntil` parameter controls when the browser considers page navigation complete. This is important for JavaScript-heavy pages where content is rendered dynamically after the initial page load. + +| Value | Behavior | +| ------------------ | ------------------------------------------------------------------------ | +| `load` | Waits for the `load` event, including all resources like images and stylesheets | +| `domcontentloaded` | Waits until the DOM content has been fully loaded, which fires before the `load` event (default) | +| `networkidle0` | Waits until there are no network connections for at least 500 ms | +| `networkidle2` | Waits until there are no more than two network connections for at least 500 ms | + +For pages that rely on JavaScript to render content, use `networkidle0` or `networkidle2` to ensure the page is fully rendered before extraction. ## Notes and recommendations You can set multiple timers — as long as one is complete, the request will fire. diff --git a/src/content/docs/browser-rendering/rest-api/content-endpoint.mdx b/src/content/docs/browser-rendering/rest-api/content-endpoint.mdx index 2936eae2b20cc8..33ffe41f474e10 100644 --- a/src/content/docs/browser-rendering/rest-api/content-endpoint.mdx +++ b/src/content/docs/browser-rendering/rest-api/content-endpoint.mdx @@ -83,6 +83,11 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts//browser- Many more options exist, like setting HTTP headers using `setExtraHTTPHeaders`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/content/methods/create/) for all available parameters. + + + diff --git a/src/content/docs/browser-rendering/rest-api/links-endpoint.mdx b/src/content/docs/browser-rendering/rest-api/links-endpoint.mdx index 5843e286b0f406..95cd305de9bcc2 100644 --- a/src/content/docs/browser-rendering/rest-api/links-endpoint.mdx +++ b/src/content/docs/browser-rendering/rest-api/links-endpoint.mdx @@ -255,6 +255,8 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts//browser- }' ``` + + diff --git a/src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx b/src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx index 3eca3538a17071..ae7a04719883a0 100644 --- a/src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx +++ b/src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx @@ -116,6 +116,11 @@ curl -X 'POST' 'https://api.cloudflare.com/client/v4/accounts//browse } ``` + + /browser- --output "dynamic-header-footer.pdf" ``` + + + /browser- Many more options exist, like setting HTTP credentials using `authenticate`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/screenshot/methods/create/) for all available parameters. + + /browser- product="browser-rendering" /> + + Date: Tue, 9 Dec 2025 09:17:28 -0800 Subject: [PATCH 2/2] docs(browser-rendering): clarify networkidle as simplest solution, waitForSelector for advanced users --- .../partials/browser-rendering/javascript-heavy-pages.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/partials/browser-rendering/javascript-heavy-pages.mdx b/src/content/partials/browser-rendering/javascript-heavy-pages.mdx index d409a968692d89..dfd4af526bb3fd 100644 --- a/src/content/partials/browser-rendering/javascript-heavy-pages.mdx +++ b/src/content/partials/browser-rendering/javascript-heavy-pages.mdx @@ -2,7 +2,7 @@ For JavaScript-heavy pages or Single Page Applications (SPAs), the default page load behavior may return empty or incomplete results. This happens because the browser considers the page loaded before JavaScript has finished rendering the content. -To ensure the browser waits for dynamic content to render, use the `gotoOptions.waitUntil` parameter set to `networkidle0` or `networkidle2`: +The simplest solution is to use the `gotoOptions.waitUntil` parameter set to `networkidle0` or `networkidle2`: ```json { @@ -13,4 +13,4 @@ To ensure the browser waits for dynamic content to render, use the `gotoOptions. } ``` -For more details on `waitUntil` options and other timeout settings, refer to [REST API timeouts](/browser-rendering/reference/timeouts/). +For faster responses, advanced users can use `waitForSelector` to wait for a specific element instead of waiting for all network activity to stop. This requires knowing which CSS selector indicates the content you need has loaded. For more details, refer to [REST API timeouts](/browser-rendering/reference/timeouts/).