How to Fix Ranorex Test Failures on Virtual Machine (VM) and Remote Desktop Sessions During Nightly Runs

Automated UI testing is one of the biggest productivity multipliers in test-automation. But when you’re running UI-driven tests on a virtual machine (VM) or via remote desktop sessions and seeing intermittent failures, you end up chasing flaky results instead of shipping reliable releases.

At Enhops, we rely heavily on Ranorex for clients with complex UIs and diverse environments. Recently we encountered a puzzling pattern: nightly UI test runs triggered by Task Scheduler on a VM kept failing — black screens, tests not starting, and no consistent root-cause.

In this article we’ll walk through why those failures happened, how we fixed them, and what you should check to make your Ranorex-based automation stable on VMs and remote desktops.

Common symptoms & environment

Nightly runs of Ranorex UI tests, triggered by Microsoft Task Scheduler on a VM, consistently resulted in failures. The reports were filled with black screens, and test cases didn’t execute as expected.

Investigation and root-cause steps

Investigation 1: Simulate user activity via PowerShell

Hypothesis: The VM might be locking itself, causing test failures. To counter this, we created a PowerShell script to simulate constant NUMLOCK key toggling, mimicking user activity to keep the VM active.

Outcome:

• The script failed because the VM relies on the local machine’s keyboard and mouse inputs via RDP.
• Minimizing or closing the RDP session caused the interactive session to be lost.

Key Insight: The PowerShell script required an open and maximized RDP session to function, making it unsuitable for unattended execution.

Investigation 2: Run Ranorex Agent without RDP session

Hypothesis: The issue might not stem from VM sleep or lock settings but from how the tests were triggered.

Approach: We replicated the nightly execution scenario by triggering Ranorex tests using the Ranorex Agent without establishing an RDP connection.

Outcome:

• Test results matched the nightly execution failures: black screens and unsuccessful test cases.
• Sleep settings were confirmed as “Never,” ruling out inactivity as the cause.

Key Insight: The VM requires an active interactive session to execute Ranorex UI tests successfully.

Investigation 3: Use tscon to transfer the session to the VM console

Hypothesis: An interactive session is essential for Ranorex test execution. We explored transferring the RDP session to the VM’s physical console to maintain this session.

Approach:

1. Established an RDP connection to the VM.
2. Used the tscon command to transfer the local interactive session to the VM console.
3. Disconnected the RDP session using tscon command and triggered tests via the Ranorex Agent.

Outcome:

• The VM maintained an active interactive session indefinitely.
• Tests executed successfully without requiring an open RDP session.

Key Insight: The tscon command enabled the VM to run tests autonomously, replicating the nightly execution environment.

The winning formula – solution summary

Root Cause:

• Ranorex UI tests fail on a VM when there’s no active interactive session.
• Minimizing or closing an RDP connection disrupts the session, leading to failures.

Solution:

• Use the tscon command to transfer the RDP session to the VM’s physical console.
• This ensures the VM retains an interactive session indefinitely, independent of an RDP connection.

Benefits:

1. Nightly test executions triggered by MS Task Scheduler run successfully without manual intervention.
2. The VM remains test-ready, ensuring uninterrupted automation processes.

Conclusion

Many teams face flaky or inconsistent test results when running automation on VMs.

Virtual machines often have slower performance, screen resolution mismatches, or missing dependencies. These factors can throw off automation tools like Ranorex, especially for UI-driven tests. Ensure VMs are properly configured, use consistent display settings, and optimize your test scripts with smart waits and retries. Running tests in headless mode or on dedicated test VMs can also reduce failures significantly.

Test Automation with Ranorex

At Enhops, we’ve seen firsthand how Ranorex accelerates test automation with its robust capabilities and seamless integrations. It’s our preferred tool for client projects that demand flexibility, precision, and scalability. Whether you’re looking to implement Ranorex or troubleshoot existing setups, our team has the experience to guide you through. Let us help you unlock the full potential of your automation strategy.