[🐛 Bug]: Videos are not always recorded due to video container not being aware of some sessions.

[SergeyMokhov]

What happened?

I am running selenium grid with a video container on CI (Linux). The test suite runs over a 100 tests sequentially, one by one.
Every test opens and closes a new browser window and sets a unique name for the video. After the run I expected to see one video for every test ran, when in fact amount of videos is random, with many missing. (20 to 86, rarely 100+)

I was able to reproduce the same behaviour locally on MacOs, by starting the containers and manually running the same test multiple times in a row, checking the amount of videos after each execution. All possible logging was enabled, but still insufficient for me to get to the bottom of the problem. My best efforts resulted in conclusion that video container is not always aware a new session has been created to start recording it.

Full debug logs are attached. log.log
There test was ran 3 times, ID of the session that was not recorded is 7ee3e3d1-3f69-4022-ab5c-5f5537dd1433. IP of the video container in the logs is 172.18.0.4

To reproduce the issue, run reproduce.sh from this project I created specifically for this
You will need to install gradle to run the script.
The script will start the containers and run selenium test in a loop, checking amount of videos generated every iteration.

Command used to start Selenium Grid with Docker (or Kubernetes)

docker compose -f "docker-compose.yaml" --project-name test up --wait;

Relevant log output

selenium-hub     | 21:08:39.172 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 48ef380c-161d-47f4-8697-8171e0618491 
selenium-hub     | 21:08:43.428 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 7ba51047-8ecd-4f2a-87ce-1835319ee384 
selenium-hub     | 21:08:46.760 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 7ee3e3d1-3f69-4022-ab5c-5f5537dd1433

Operating System

MacOs, Linux.

Docker Selenium version (image tag)

selenium/hub:4.25.0-20241024

Selenium Grid chart version (chart version)

No response

[github-actions]

@SergeyMokhov, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[VietND96]

Thanks for your details. Let me try to reproduce and find out the fix if possible

[VietND96]

I saw the Node config as below

firefox-1  | 
firefox-1  | [node]
firefox-1  | session-timeout = "300"
firefox-1  | override-max-sessions = true
firefox-1  | detect-drivers = false
firefox-1  | drain-after-session-count = 0
firefox-1  | max-sessions = 2
firefox-1  | 
firefox-1  | [[node.driver-configuration]]
firefox-1  | display-name = "firefox"
firefox-1  | stereotype = '{"browserName": "firefox", "browserVersion": "133.0", "platformName": "Linux", "moz:firefoxOptions": {"binary": "/usr/bin/firefox"}, "se:containerName": ""}'
firefox-1  | max-sessions = 2
firefox-1  |     

It means, at a time, Node could take up to 2 sessions of concurrency, which is opening in the same display.
However, the video recording still supports something basic like One container is needed per each container where a browser is running. This means if you are running 2 Nodes/Standalone containers (2 sessions in parallel), you will need 2 video containers, the mapping is 1-1. - that noted in README https://github.com/SeleniumHQ/docker-selenium?tab=readme-ov-file#video-recording.
I haven't checked your shell script yet. But you can try to configure the script that will spawn a pair node-video containers for each single test.

[SergeyMokhov]

Thank you for the suggestion. In my current setup all the test are sequential, and there is only one test running at any given time. Some do briefly open a second browser window and the video captures it too in the same file, which is desired outcome in this case. The issue is reproducible even with tests that utilize a single tab though.

[VietND96]

I updated something and get a successful run - SergeyMokhov/selenium-missing-videos-bug#1

[SergeyMokhov]

I checked out your branch and got the same result as you did, successful 100 iterations.
I did some further experimentation, and it appears to be that setting SE_NODE_MAX_SESSIONS to anything other than 1 messes up the video recordings, even if the second session is never created.

For my use case it would be benefitial to be able to open a second session even on the same display, because it simplifies the tests. I'll have to rewrite those tests that open second session to logout/login as a different user instead. In some cases I will probably get away with creating another tab instead. Or add another pair of browser-recorder containers and deal with 2 sets of videos (and increased resource consumption).

I understand that this might not qualify for a bug anymore, but please consider this ticket as an avenue for future improvements. Documenting that max sessions will affect recordings in that manner might be worth doing, mapping is 1-1 was not convincing enough for me not to do it.

Please, let me know if you decide to look into this issue further.

Either way, I appreciate your support, thank you for the assistance in this investigation.

[VietND96]

I suspect something like when SE_NODE_MAX_SESSIONS=2, it would go to a situation where the test just finished, and the next test is coming immediately (since the script is without delay). Both sessions could be present at the same time with an array list in the node endpoint response.
In the video container, the script extracts the index [0] in the array, so it causes the missing.
Based on your tests, I will improve the video recorder further.

[VietND96]

All you mentioned is still valid since a request to optimize for the mapping 1-1 convention was also raised in another discussion: #2452.