How to Fix Ranorex Test Failures on Virtual Machine (VM) and Remote Desktop Sessions During Nightly Runs

Test failures are a common hurdle in automation testing. They require testers to investigate the root cause and re-execute tests to ensure reliability. Each tool and framework have its own approach to identifying and resolving failed tests.

At Enhops, we rely heavily on Ranorex—a flexible, highly customizable tool that’s well-suited for diverse client needs. But even the most reliable tools face challenges.

We recently encountered an issue with Ranorex UI tests failing intermittently during unattended runs on a virtual machine (VM) and Remote Desktop Machines. These failures were inconsistent and difficult to replicate, making troubleshooting especially tricky.

This blog talks in detail about the problem and solution we implemented to make our test runs more reliable.

The Problem at Hand

Nightly runs of Ranorex UI tests, triggered by Microsoft Task Scheduler on a VM, consistently resulted in failures. The reports were filled with black screens, and test cases didn’t execute as expected.

Investigation 1: PowerShell Script to Prevent VM Lock

Hypothesis: The VM might be locking itself, causing test failures. To counter this, we created a PowerShell script to simulate constant NUMLOCK key toggling, mimicking user activity to keep the VM active.

Outcome:

• The script failed because the VM relies on the local machine’s keyboard and mouse inputs via RDP.
• Minimizing or closing the RDP session caused the interactive session to be lost.

Key Insight: The PowerShell script required an open and maximized RDP session to function, making it unsuitable for unattended execution.

Investigation 2: Replicating the Nightly Execution Environment

Hypothesis: The issue might not stem from VM sleep or lock settings but from how the tests were triggered.

Approach: We replicated the nightly execution scenario by triggering Ranorex tests using the Ranorex Agent without establishing an RDP connection.

Outcome:

• Test results matched the nightly execution failures: black screens and unsuccessful test cases.
• Sleep settings were confirmed as “Never,” ruling out inactivity as the cause.

Key Insight: The VM requires an active interactive session to execute Ranorex UI tests successfully.

Investigation 3: Leveraging the tscon Command

Hypothesis: An interactive session is essential for Ranorex test execution. We explored transferring the RDP session to the VM’s physical console to maintain this session.

Approach:

1. Established an RDP connection to the VM.
2. Used the tscon command to transfer the local interactive session to the VM console.
3. Disconnected the RDP session using tscon command and triggered tests via the Ranorex Agent.

Outcome:

• The VM maintained an active interactive session indefinitely.
• Tests executed successfully without requiring an open RDP session.

Key Insight: The tscon command enabled the VM to run tests autonomously, replicating the nightly execution environment.

The Winning Formula

Root Cause:

• Ranorex UI tests fail on a VM when there’s no active interactive session.
• Minimizing or closing an RDP connection disrupts the session, leading to failures.

Solution:

• Use the tscon command to transfer the RDP session to the VM’s physical console.
• This ensures the VM retains an interactive session indefinitely, independent of an RDP connection.

Benefits:

1. Nightly test executions triggered by MS Task Scheduler run successfully without manual intervention.
2. The VM remains test-ready, ensuring uninterrupted automation processes.

Conclusion

Many teams face flaky or inconsistent test results when running automation on VMs.

Virtual machines often have slower performance, screen resolution mismatches, or missing dependencies. These factors can throw off automation tools like Ranorex, especially for UI-driven tests. Ensure VMs are properly configured, use consistent display settings, and optimize your test scripts with smart waits and retries. Running tests in headless mode or on dedicated test VMs can also reduce failures significantly.

Test Automation with Ranorex

At Enhops, we’ve seen firsthand how Ranorex accelerates test automation with its robust capabilities and seamless integrations. It’s our preferred tool for client projects that demand flexibility, precision, and scalability. Whether you’re looking to implement Ranorex or troubleshoot existing setups, our team has the experience to guide you through. Let us help you unlock the full potential of your automation strategy.