How XY Works (at a glance)

1. Capture real work
   Use the Browser Agent to observe how work is actually done in web apps—recording clicks, fields, and navigation with DOM + computer vision. Export the recording into structured steps.

2. Add Integrations & Files
   Plug into EHRs, billing systems, engagement tools, collaboration apps, storage, and productivity suites. Attach reference docs (policies, templates) that Agents can consult.

3. Assemble a Workflow
   Combine recorded steps with backend/API actions and decision logic. Workflows are config-driven, chaining Agents, tools, and model calls for end-to-end outcomes.

4. Run with HitL
   Execute deterministically where possible; invoke LLM reasoning only for ambiguous states; route exceptions to human-in-the-loop queues and resume after review.

5. Orchestrate at scale
   XY’s architecture separates the Next.js web app from Python gRPC services and a message-driven orchestrator (RabbitMQ), with PostgreSQL + cloud object storage for state and files.

6. Measure & improve
   Track runs, review exceptions, and iterate recordings/selectors. Promote co-pilot flows to supervised → autonomous modes as confidence grows.