Skip to content

Conversation

@asolntsev
Copy link
Contributor

@asolntsev asolntsev commented Dec 6, 2025

User description

🔗 Related Issues

e.g. failed build: https://github.com/SeleniumHQ/selenium/actions/runs/19987626355/job/57324241360?pr=13739

💥 What does this PR do?

Fixes few flaky Ruby tests

🔧 Implementation Notes

This PR adds few "waits" to Ruby tests.

🔄 Types of changes

  • Bug fix (backwards compatible)

PR Type

Bug fix, Tests


Description

  • Replace flaky sleep calls with explicit waits in Ruby tests

  • Add wait_for_element() helper to ensure elements are rendered before interaction

  • Add wait_for_title() helper for page navigation assertions

  • Fix DevTools log event handling to extract args properly


Diagram Walkthrough

flowchart LR
  A["Flaky Tests<br/>with sleep calls"] -->|Replace sleep| B["Explicit Waits<br/>wait_for_element"]
  A -->|Replace sleep| C["Explicit Waits<br/>wait_for_title"]
  B --> D["Stable Tests<br/>element_spec.rb"]
  C --> D
  E["DevTools Log Event"] -->|Extract args| F["Fixed Log Handling<br/>devtools_spec.rb"]
Loading

File Walkthrough

Relevant files
Bug fix
element_spec.rb
Replace direct element finds with wait helpers                     

rb/spec/integration/selenium/webdriver/element_spec.rb

  • Replace 30+ direct driver.find_element() calls with wait_for_element()
    helper to ensure elements are rendered
  • Replace sleep 0.5 calls with wait_for_title() helper for page
    navigation assertions
  • Improve test reliability by waiting for elements to be available
    before interaction
+55/-52 
devtools_spec.rb
Fix DevTools log test with waits and arg extraction           

rb/spec/integration/selenium/webdriver/devtools_spec.rb

  • Remove 5 sleep 0.5 calls from console log test
  • Change log event handler to extract log.args[0] instead of storing
    full log object
  • Replace hardcoded assertions with wait.until checks for log values
  • Improve test stability by waiting for expected log entries
+5/-12   
Enhancement
helpers.rb
Add wait_for_title helper method                                                 

rb/spec/integration/selenium/webdriver/spec_support/helpers.rb

  • Add new wait_for_title() helper method with 5-second timeout
  • Helper waits until driver title matches expected value
  • Provides reusable pattern for page navigation assertions
+5/-0     

@asolntsev asolntsev requested a review from p0deje December 6, 2025 20:49
@selenium-ci selenium-ci added the C-rb Ruby Bindings label Dec 6, 2025
@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Dec 6, 2025

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
No audit logs: The added test code performs actions and assertions without introducing any audit logging
for critical actions, though as test code this may be acceptable and outside the scope of
production audit trails.

Referred Code
it 'notifies about log messages' do
  logs = []
  driver.on_log_event(:console) { |log| logs.push(log.args[0]) }
  driver.navigate.to url_for('javascriptPage.html')

  driver.execute_script("console.log('I like cheese');")
  driver.execute_script('console.log(true);')
  driver.execute_script('console.log(null);')
  driver.execute_script('console.log(undefined);')
  driver.execute_script('console.log(document);')

  wait.until { logs.include?(true) }
  wait.until { logs.include?(nil) }
  wait.until { logs.include?({ 'type' => 'undefined' }) }
  wait.until { logs.include?('I like cheese')}
end

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Minimal wait handling: The new wait_for_title helper lacks explicit error context on timeout, relying on generic
wait exceptions which may hinder debugging though this may be standard in test utilities.

Referred Code
def wait_for_title(title:)
  wait = Wait.new(timeout: 5)
  wait.until { driver.title == title }
end

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Dec 6, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Consolidate multiple waits into one

Consolidate multiple wait.until calls into a single block. This can be achieved
by defining an array of expected logs and waiting until all of them are present
in the logs array.

rb/spec/integration/selenium/webdriver/devtools_spec.rb [112-115]

-wait.until { logs.include?(true) }
-wait.until { logs.include?(nil) }
-wait.until { logs.include?({ 'type' => 'undefined' }) }
-wait.until { logs.include?('I like cheese')}
+expected_logs = [true, nil, { 'type' => 'undefined' }, 'I like cheese']
+wait.until { (expected_logs - logs).empty? }
  • Apply / Chat
Suggestion importance[1-10]: 6

__

Why: The suggestion correctly proposes consolidating multiple wait.until calls into a single, more efficient and readable check, which is a good practice for improving test code quality.

Low
Learned
best practice
Replace hardcoded wait timeout

Use a shared, configurable timeout (and a shorter poll interval) instead of a
hardcoded 5 seconds to keep waits consistent and adjustable across tests.

rb/spec/integration/selenium/webdriver/spec_support/helpers.rb [105-108]

-def wait_for_title(title:)
-  wait = Wait.new(timeout: 5)
-  wait.until { driver.title == title }
+DEFAULT_WAIT_TIMEOUT = 10 # seconds
+
+def wait_for_title(title:, timeout: DEFAULT_WAIT_TIMEOUT, interval: 0.1)
+  Wait.new(timeout: timeout, interval: interval).until { driver.title == title }
 end
  • Apply / Chat
Suggestion importance[1-10]: 6

__

Why:
Relevant best practice - Guard time-based waits by computing and clamping timeouts and centralizing timeout values to avoid brittle tests.

Low
  • Update

@asolntsev asolntsev force-pushed the fix/flaky-ruby-tests-2 branch from d07716a to 275cf06 Compare December 6, 2025 21:29
Wait until the page gets loaded and the element ges rendered.

Example of failure:

```ruby
Selenium::WebDriver::Element raises if different element receives click
     Failure/Error: expect { driver.find_element(id: 'contents').click }.to raise_error(Error::ElementClickInterceptedError)

       expected Selenium::WebDriver::Error::ElementClickInterceptedError, got #<Selenium::WebDriver::Error::NoSuchElementError:"no such element: Unable to locate element: {\"metho...it: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#nosuchelementexception"> with backtrace:
         # ./rb/lib/selenium/webdriver/remote/response.rb:63:in `add_cause'
         # ./rb/lib/selenium/webdriver/remote/response.rb:41:in `error'
         # ./rb/lib/selenium/webdriver/remote/response.rb:52:in `assert_ok'
         # ./rb/lib/selenium/webdriver/remote/response.rb:34:in `initialize'
         # ./rb/lib/selenium/webdriver/remote/http/common.rb:103:in `new'
         # ./rb/lib/selenium/webdriver/remote/http/common.rb:103:in `create_response'
         # ./rb/lib/selenium/webdriver/remote/http/default.rb:103:in `request'
         # ./rb/lib/selenium/webdriver/remote/http/common.rb:68:in `call'
         # ./rb/lib/selenium/webdriver/remote/bridge.rb:625:in `execute'
         # ./rb/lib/selenium/webdriver/remote/bridge.rb:493:in `find_element_by'
         # ./rb/lib/selenium/webdriver/common/search_context.rb:71:in `find_element'
         # ./rb/spec/integration/selenium/webdriver/element_spec.rb:34:in `block (3 levels) in <module:WebDriver>'
         # ./rb/spec/integration/selenium/webdriver/element_spec.rb:34:in `block (2 levels) in <module:WebDriver>'
     # ./rb/spec/integration/selenium/webdriver/element_spec.rb:34:in `block (2 levels) in <module:WebDriver>'
```

See https://github.com/SeleniumHQ/selenium/actions/runs/19987626355/job/57324241360?pr=13739 for example.
instead of sleeping for 0.5s, just wait for the needed title.
instead of sleeping for 2.5s, just wait for the needed logs.
@asolntsev asolntsev force-pushed the fix/flaky-ruby-tests-2 branch from 275cf06 to 92fbd13 Compare December 6, 2025 22:42
@asolntsev asolntsev marked this pull request as draft December 6, 2025 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants