Extract Pages From BrowserStack Sessions

Extract the URLs navigated in BrowserStack Automate sessions from network logs.

Each step is a standalone script and an importable function. Use them one at a time, or run the full pipeline with extractPages.js.

projects → builds → sessions → network logs → page URLs

Setup

1. Clone the repo and cd into it.

2. Set credentials:

   export BROWSERSTACK_USERNAME=<your-username>
   export BROWSERSTACK_ACCESS_KEY=<your-access-key>

3. Requires Node.js 18+ (uses built-in fetch). No npm install needed.

Standalone scripts

Script	Purpose	Usage
getProjects.js	List Automate projects	node getProjects.js
getBuilds.js	List builds for a project	node getBuilds.js <projectId>
getSessions.js	List sessions for a build	node getSessions.js <buildId>
getNetworkLogs.js	Extract page URLs from a session	node getNetworkLogs.js <sessionId>

Examples:

npm run projects
npm run builds -- 1915071
npm run sessions -- b2f90cb5682bc65547a01b75c2db77494167b6f9
npm run network-logs -- a20c680df1918de4b0227403aa62452987f06407

IDs can also be passed via env vars: BROWSERSTACK_PROJECT_ID, BROWSERSTACK_BUILD_ID, BROWSERSTACK_SESSION_ID.

Orchestrated pipeline

Run the full chain (or start mid-pipeline by supplying an ID):

# All projects → builds → sessions → page URLs
npm run extract

# One project
npm run extract -- --projectId 1915071

# One build
npm run extract -- --buildId b2f90cb5682bc65547a01b75c2db77494167b6f9

# One session
npm run extract -- --sessionId a20c680df1918de4b0227403aa62452987f06407

# Cap builds per project
npm run extract -- --projectId 1915071 --buildLimit 10

# Custom output path (default: output/pages.json)
npm run extract -- --projectId 1915071 --out results/my-pages.json

Results are written to output/pages.json by default (override with --out). Each entry is { projectId, buildId, sessionId, urls }.

Import as a library

import { getProjects } from './getProjects.js';
import { getBuilds } from './getBuilds.js';
import { getSessions } from './getSessions.js';
import { getPageUrls } from './getNetworkLogs.js';
import { extractPages } from './extractPages.js';

const projects = await getProjects();
const builds = await getBuilds(projects[0].id);
const sessions = await getSessions(builds[0].hashed_id);
const urls = await getPageUrls(sessions[0].hashed_id);

// Or the whole pipeline:
const results = await extractPages({ projectId: 1915071, buildLimit: 5 });