Skip to main content

Recording Workflows

Learn how to create, save, and execute automated workflows using the XY Chrome Extension. This guide covers everything from basic recording to advanced features like variables and iteration.

Watch the recording portion of this video (1:50-5:21) for a visual walkthrough of recording your first workflow. For installation instructions, see the Installation Guide.

Overview​

The XY Chrome Extension enables you to record your interactions with web pages and transform them into reusable automated workflows. Every click, navigation, and action you perform is captured and can be replayed automatically, saving time and ensuring consistency in repetitive tasks.

Getting Started: Your First Workflow​

Prerequisites​

Before you begin recording workflows, ensure you have:

  • The XY Chrome Extension installed in your browser
  • Valid login credentials for the XY platform
  • Access to the web application you want to automate

Step 1: Open and Login​

  1. Click the XY icon in your Chrome Extensions toolbar to open the plugin
  2. Login using your credentials to unlock the plugin's features
  3. Return to the original tab where you want to record your workflow

Step 2: Start Recording​

  1. Click the "Start Recording" button in the extension panel

  2. Configure multimodal recording options (first time or when needed):

    • Video/Screenshots: Capture visual recordings of your actions (recommended: ON)
    • Audio: Record voice narration to add context (recommended: ON, unless in noisy environment)
    • Browser Code: Capture DOM-level interactions and component data (recommended: ON)
    • You can toggle each option on or off based on your needs

    Start Recording Options Configure multimodal recording options before starting

  3. First-time permissions (if prompted):

    • Microphone access: If audio recording is enabled, Chrome will ask for microphone permission
    • Screen recording: Chrome will ask which screen/tab/window to record
    • Select your preferred microphone and screen source
    • Choose "Allow" for the current session or "Always allow" for future recordings

  4. Select recording source:

    • Choose Tab, Window, or Screen to record
    • Select the specific tab or window you want to record
    • Click Share to begin recording

  5. Recording begins: The extension will now capture all your actions with the selected data streams

  6. Real-time feedback: You'll see steps appearing in the extension panel as you interact with the page

Step 3: Perform Your Actions​

Navigate through your application and perform the tasks you want to automate:

  • Navigate to sections: For example, go to the Patients section
  • Open specific records: Click on individual items like patient records
  • Interact with the page: Every action is automatically captured as a step

Note: Each action you take appears as a step on the right-hand side of the extension panel, giving you real-time feedback of what's being recorded.

Step 4: Add Manual Steps (Optional)​

While recording, you can manually add special steps:

  • Capture Page Content: Click to insert a step that captures the current page content
  • This captured content can be reused later in the workflow or in other processes

Step 5: Save Your Workflow​

  1. Click "Stop" to end the recording
  2. Recording complete message will appear confirming the recording was captured
  3. Give your workflow a descriptive name that will help you identify it later
  4. Click "Save" to store your workflow

Completed Recording After stopping, name and save your recording

Step 6: Execute Your Workflow​

  1. Find your saved workflow in the list of recordings
  2. Click on it to open the workflow details
  3. Click "Execute" to run the workflow
  4. Watch as the workflow automatically follows the exact steps you recorded

Congratulations! You've successfully built and executed your first workflow.

Multimodal Recording Options​

XY Browser Agent supports multimodal data streams, allowing you to capture comprehensive recordings with multiple types of data:

Video/Screenshots Capture​

  • Records visual screenshots of your actions as you interact with web pages
  • Provides visual context for each step in your workflow
  • Essential for understanding what happened during recording
  • Recommendation: Keep enabled for best results

Audio Recording​

  • Captures voice narration while you perform actions
  • Allows you to add verbal context and explanations
  • Useful for documenting complex workflows or providing instructions
  • Recommendation: Keep enabled unless you're in a noisy environment

Browser Code Capture​

  • Records DOM-level interactions and component data
  • Captures the underlying structure of web pages
  • Essential for reliable workflow execution and error handling
  • Recommendation: Keep enabled for technical accuracy

Configuring Recording Options​

When you start a recording, you can toggle each data stream on or off:

  • All streams enabled (recommended): Provides complete context with video, audio, and code-level data
  • Video + Code only: Good for quiet environments where audio isn't needed
  • Code only: Minimal recording with just the essential browser interactions
Best Practice

For most workflows, keep all three options enabled. This provides the richest data for both understanding and executing your workflows. Only disable audio if you're in a noisy environment or don't need verbal context.

First-Time Setup​

The first time you record with audio or video enabled, Chrome will request permissions:

  1. Microphone permission: If audio is enabled, Chrome will ask which microphone to use

    • Select your preferred microphone
    • Choose "Allow" for this session or "Always allow" for future recordings

  2. Screen recording permission: Chrome will ask which screen/tab/window to record

    • Select Tab to record a specific browser tab
    • Select Window to record a specific application window
    • Select Screen to record your entire screen
    • Choose the specific tab/window/screen you want to capture
    • Click Share to begin recording

These permissions are remembered for future recordings, so you won't need to grant them every time.

Practical Example: Appointment Booking Workflow​

See how the XY Chrome Extension streamlines complex tasks like appointment booking with intelligent automation and error handling.

Watch a complete appointment booking workflow demonstration showcasing recording, replay, and AI-powered error correction.

What This Example Demonstrates​

This practical example walks through creating an appointment booking workflow that includes:

Recording Phase​

  • Starting the workflow recording to capture all interactions
  • Searching for a person in the system
  • Creating a new appointment with the booking interface
  • Filling out appointment details:
    • Reason for visit
    • Appointment date
    • Preferred time
  • Saving the appointment to the system
  • Stopping and saving the workflow for future use

Key Features Highlighted​

Real-Time Step Logging

  • Every action is instantly logged in the side panel
  • Complete visibility into what's being recorded
  • Easy review of captured steps before saving

Instant Replay Capability

  • Saved workflows are immediately available for replay
  • Execute the same appointment booking process with one click
  • Consistent, error-free repetition of complex tasks

AI-Powered Error Detection and Correction

  • The AI automatically detects when elements have changed
  • Intelligently fixes errors without manual intervention
  • Ensures workflows remain functional even as applications update

Why This Matters​

This appointment booking example showcases how the XY Chrome Extension handles real-world healthcare workflows. Instead of manually clicking through multiple screens and forms for each appointment, you can:

  1. Record the process once
  2. Replay it whenever needed
  3. Trust the AI to handle minor changes and errors automatically

This approach saves time, reduces errors, and ensures consistency across all appointment bookings.

Advanced Features: Variables and Iteration​

Once you're comfortable with basic recording, you can leverage more powerful features to create dynamic workflows that adapt to different data sets.

Learn how to save variables and iterate over arrays of elements in your workflows.

Working with Variables​

Variables allow you to capture and store data from web pages for later use in your workflow.

Adding AI Steps for Variable Capture​

  1. Start recording your workflow as usual
  2. Navigate to a page with multiple items (e.g., a patient list)
  3. Click "Add Step" and choose "Add AI Step"
  4. Write an AI prompt that describes what to capture

Example AI Prompt:

Read the page and store the chart ID, clickable element, 
and index of each patient in the memory named 'patients'

The AI will process the page and extract the requested information into variables. Processing time depends on the number of items on the page.

Creating Loops with ForEach​

ForEach loops allow your workflow to repeat actions for multiple items automatically.

Setting Up a ForEach Loop​

  1. After capturing your variables, click "Add Step" and select "ForEach"
  2. Enter the variable name you want to iterate over (e.g., "patients")
  3. Continue recording the actions you want to repeat for each item
  4. Click "Exit ForEach" when you've finished recording the loop actions
  5. Stop and save your workflow with a descriptive name

Verifying Your Advanced Workflow​

After creating a workflow with variables and loops:

  1. Open Browser Workflows in the XY Workflow Agent

  2. Find your saved workflow and click the "Detail" button

  3. Review the workflow structure:

    • Check that all steps are present
    • Verify variables are correctly named
    • Ensure ForEach substeps are properly configured
    • Confirm action indices are set correctly

  4. Save any changes you make during verification

Testing Advanced Workflows​

  1. Return to your target application
  2. Load your workflow in the extension
  3. Click "Execute" to run it
  4. Watch as the workflow:
    • Captures variables from the page
    • Iterates through each item
    • Performs the recorded actions for each element

Key Concepts Recap​

Basic Workflow Recording​

  • Open XY Extension and login
  • Click "Record Workflow"
  • Perform your actions
  • Add manual steps as needed
  • Stop and save with a descriptive name
  • Execute to replay the workflow

Variables and Iteration Features​

  • AI Steps: Capture and store page data as variables
  • ForEach Loops: Iterate over arrays of elements
  • Variable Management: Store and reuse data throughout your workflow
  • Workflow Verification: Review and adjust steps before execution

Best Practices​

  1. Start simple: Master basic recording before moving to advanced features
  2. Use descriptive names: Make workflows easy to identify and understand
  3. Test thoroughly: Always verify your workflow works as expected
  4. Check variables: Ensure variable names and data are correct before execution
  5. Review steps: Use the Detail view to understand and adjust your workflow

Next Steps​

Now that you understand the fundamentals of workflow recording and advanced features like variables and iteration, you're ready to:

  • Create more complex automation sequences
  • Build workflows that handle dynamic data
  • Automate repetitive tasks across your applications
  • Explore additional XY platform capabilities

This is only the beginning of what's possible with the XY Chrome Extension. As more features become available, you'll be able to create even more powerful and sophisticated automations.

Accessing and Sharing Recordings​

Once you've saved a recording, you can access it in multiple ways and share it with others.

Viewing Recordings in the XY Web App​

All your recordings are automatically synced to your XY web app account:

  1. Navigate to Browser Agents in the XY web app
  2. Go to Recorded Browser Workflows section
  3. Find your recording in the list

Recorded Browser Workflows View all your recorded workflows in the XY web app

Playing Back Recordings​

  1. Click on a recording to open it
  2. View the recording details:
    • See all recorded steps
    • View screenshots/video from the recording
    • Review the browser code interactions
    • Access advanced settings

Recording View in Web App Play back recordings and review all steps in the web app

  1. Click "Play" to watch the recording playback
  2. Review steps: Each step shows what action was performed and when

Sharing Recordings​

You can share recordings with team members, XY support, or other stakeholders:

  1. Open the recording in the XY web app
  2. Click the "Copy Link" button (or similar sharing option)
  3. Share the link via:
    • Slack: Paste the link in a Slack message or channel
    • Email: Include the link in an email message
    • Other platforms: Use the link anywhere you need to share the recording
Sharing Use Cases
  • Team collaboration: Share recordings with colleagues to demonstrate workflows
  • Support requests: Share recordings with XY support to troubleshoot issues
  • Documentation: Create a collection of recordings representing different workflow types for XY to analyze
  • Training: Use recordings as training materials for new team members

Managing Task Execution​

The extension also provides tools for managing task execution:

Manage Task Execution Manage and monitor task execution through the extension interface

From the extension menu, you can:

  • View active task executions
  • Monitor workflow progress
  • Access execution history
  • Manage workflow queues