SDK/TypeScript

TypeScript SDK

Full-featured TypeScript/JavaScript SDK for controlling Android phones via ScreenMCP.

npm install @screenmcp/sdk

Installation

npm

npm install @screenmcp/sdk

yarn

yarn add @screenmcp/sdk

pnpm

pnpm add @screenmcp/sdk

Quick Start

example.ts

import { ScreenMCPClient } from "@screenmcp/sdk";

const phone = new ScreenMCPClient({
  apiKey: "pk_your_api_key_here",
  deviceId: "your-device-id", // optional
});

await phone.connect();

// Take a screenshot
const { image } = await phone.screenshot();
// image is a base64-encoded WebP string

// Tap on the screen
await phone.click(540, 1200);

// Type text
await phone.type("Hello world");

// Get the UI tree
const { tree } = await phone.uiTree();
console.log(tree);

await phone.disconnect();

Constructor Options

TypeScript

new ScreenMCPClient(options: {
  apiKey: string;          // Required. Your API key (pk_... format).
  apiUrl?: string;         // API server URL. Defaults to https://screenmcp.com
  deviceId?: string;       // Target device ID. Optional; auto-selects if omitted.
  commandTimeout?: number; // Per-command timeout in ms. Defaults to 30000.
  autoReconnect?: boolean; // Auto-reconnect on disconnect. Defaults to true.
})

Option	Type	Default	Description
`apiKey`	string	--	Required. Your API key starting with pk_.
`apiUrl`	string	https://screenmcp.com	ScreenMCP API server URL.
`deviceId`	string	undefined	Target device ID. If omitted, the first available device is used.
`commandTimeout`	number	30000	Timeout per command in milliseconds.
`autoReconnect`	boolean	true	Automatically reconnect on WebSocket disconnect.

Connection

await phone.connect();    // Discover worker and open WebSocket
await phone.disconnect(); // Close connection gracefully

connect() calls the ScreenMCP API to discover the optimal worker, then opens a WebSocket and authenticates.

disconnect() closes the WebSocket connection and stops any auto-reconnection attempts.

Properties

phone.connected       // boolean - is WebSocket connected
phone.phoneConnected  // boolean - is the phone online
phone.workerUrl       // string | null - current worker URL

API Reference

All 15 command methods are async and return a Promise. Commands are sent to the phone over the WebSocket connection and resolved when the phone returns a response.

screenshot()

Take a screenshot of the phone screen. Returns a base64-encoded WebP image string.

screenshot(): Promise<{ image: string }>

Returns: { image: string } -- base64-encoded WebP image data.

click(x, y)

Tap on the screen at the given coordinates.

click(x: number, y: number): Promise<void>

Parameter	Type	Description
`x`	number	X coordinate
`y`	number	Y coordinate

Returns: void

longClick(x, y)

Long-press at coordinates for approximately 1000ms.

longClick(x: number, y: number): Promise<void>

Parameter	Type	Description
`x`	number	X coordinate
`y`	number	Y coordinate

Returns: void

drag(startX, startY, endX, endY)

Perform a drag gesture from one point to another.

drag(startX: number, startY: number, endX: number, endY: number): Promise<void>

Parameter	Type	Description
`startX`	number	Start X coordinate
`startY`	number	Start Y coordinate
`endX`	number	End X coordinate
`endY`	number	End Y coordinate

Returns: void

scroll(direction, amount?)

Scroll the screen in a given direction.

scroll(direction: "up" | "down" | "left" | "right", amount?: number): Promise<void>

Parameter	Type	Description
`direction`	"up" \| "down" \| "left" \| "right"	Scroll direction
`amount`	number (optional)	Scroll distance in pixels

Returns: void

type(text)

Type text into the currently focused input field.

type(text: string): Promise<void>

Parameter	Type	Description
`text`	string	The text to type

Returns: void

getText()

Read the text content from the currently focused element.

getText(): Promise<{ text: string }>

Returns: { text: string }

selectAll()

Select all text in the currently focused field.

selectAll(): Promise<void>

Returns: void

copy()

Copy the currently selected text to the clipboard.

copy(): Promise<void>

Returns: void

paste()

Paste clipboard contents into the focused field.

paste(): Promise<void>

Returns: void

back()

Press the Android back button.

back(): Promise<void>

Returns: void

home()

Press the Android home button.

home(): Promise<void>

Returns: void

recents()

Open the recent apps / app switcher view.

recents(): Promise<void>

Returns: void

uiTree()

Get the accessibility tree of the current screen. Returns an array of UI node objects with bounds, text, and clickable state.

uiTree(): Promise<{ tree: any[] }>

Returns: { tree: any[] } -- array of accessibility node objects.

camera(facing?)

Take a photo using the phone camera. Defaults to the rear camera.

camera(facing?: "front" | "rear"): Promise<{ image: string }>

Parameter	Type	Description
`facing`	"front" \| "rear" (optional)	Camera to use. Defaults to "rear".

Returns: { image: string } -- base64-encoded WebP image data.

Custom Commands

Use sendCommand() to send any command, including future commands that may not yet have a dedicated method.

TypeScript

// Send a custom or future command
const result = await phone.sendCommand("screenshot", { quality: 50 });
console.log(result);

Event Handling

The client extends EventEmitter and emits the following events:

Event	Callback	Description
`connected`	() => void	WebSocket connection established
`disconnected`	() => void	WebSocket connection closed
`error`	(err: Error) => void	Connection or protocol error
`phone_status`	(online: boolean) => void	Phone came online or went offline
`reconnecting`	() => void	Attempting automatic reconnection
`reconnected`	(workerUrl: string) => void	Successfully reconnected

Example: Event listeners

const phone = new ScreenMCPClient({ apiKey: "pk_..." });

phone.on("connected", () => {
  console.log("Connected to worker");
});

phone.on("phone_status", (online) => {
  console.log(`Phone is ${online ? "online" : "offline"}`);
});

phone.on("error", (err) => {
  console.error("Connection error:", err.message);
});

phone.on("reconnecting", () => {
  console.log("Reconnecting...");
});

phone.on("reconnected", (workerUrl) => {
  console.log("Reconnected to:", workerUrl);
});

await phone.connect();

Error Handling

Commands throw standard JavaScript errors on failure. Use try/catch to handle them. Common error scenarios include authentication failure, command timeout, and phone disconnection.

TypeScript

import { ScreenMCPClient } from "@screenmcp/sdk";

const phone = new ScreenMCPClient({ apiKey: "pk_..." });

try {
  await phone.connect();

  const { image } = await phone.screenshot();
  console.log("Screenshot captured:", image.length, "bytes");

} catch (err) {
  if (err instanceof Error) {
    console.error("Failed:", err.message);
  }
} finally {
  await phone.disconnect();
}

Authentication errors

Thrown during connect() if the API key is invalid or revoked.

Command timeout

Thrown when a command does not receive a response within commandTimeout ms (default 30s). This can happen if the phone is locked or the screen capture permission was not granted.

Connection lost

If the WebSocket disconnects mid-command, the pending promise is rejected. With autoReconnect: true, the SDK will automatically attempt to reconnect in the background.

Advanced Usage

Save a Screenshot to Disk

save-screenshot.ts

import { ScreenMCPClient } from "@screenmcp/sdk";
import { writeFileSync } from "fs";

const phone = new ScreenMCPClient({ apiKey: "pk_..." });
await phone.connect();

const { image } = await phone.screenshot();
writeFileSync("screenshot.webp", Buffer.from(image, "base64"));

await phone.disconnect();

Reconnection Behavior

With autoReconnect: true (the default), the SDK automatically reconnects when the WebSocket connection drops. You can listen for reconnection events and disable auto-reconnect if needed.

reconnection.ts

const phone = new ScreenMCPClient({
  apiKey: "pk_...",
  autoReconnect: true, // default
});

phone.on("disconnected", () => {
  console.log("Connection lost, will auto-reconnect...");
});

phone.on("reconnected", (workerUrl) => {
  console.log("Back online via:", workerUrl);
});

await phone.connect();

// To disable auto-reconnect, pass autoReconnect: false
// and handle reconnection manually:
const manual = new ScreenMCPClient({
  apiKey: "pk_...",
  autoReconnect: false,
});

manual.on("disconnected", async () => {
  // Custom reconnection logic
  await new Promise((r) => setTimeout(r, 5000));
  await manual.connect();
});

Monitor Phone Connection

monitor.ts

const phone = new ScreenMCPClient({ apiKey: "pk_..." });

phone.on("phone_status", (online) => {
  console.log(`Phone is ${online ? "online" : "offline"}`);
  if (online) {
    // Phone just came online, safe to send commands
  }
});

phone.on("error", (err) => {
  console.error("Error:", err.message);
});

await phone.connect();

← SDK Overview Python SDK →