Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[🚀 Feature]: Optimize video recorder containers (Phase 1) #2458

Open
VietND96 opened this issue Nov 5, 2024 Discussed in #2452 · 0 comments
Open

[🚀 Feature]: Optimize video recorder containers (Phase 1) #2458

VietND96 opened this issue Nov 5, 2024 Discussed in #2452 · 0 comments
Milestone
4.27.0

Comments

@VietND96
Copy link
Member

VietND96 commented Nov 5, 2024 •
edited
Loading

Phase 1

Instead of reusing FFmpeg image is built in other project. Now, an image RecorderBase is built, including FFmpeg, codec x264, and other dpes for x11 recording (keep it minimal)
NodeBase is built on top of RecorderBase (which includes recording tools) in each Node and Standalone images

Discussed in #2452

Originally posted by saidoyk November 1, 2024
I have a Selenium Grid setup with 20 Chrome containers and 20 video recording containers monitoring each one. This setup consumes 80GB of memory. Do I need 20 video containers? Can a single video container monitor multiple Chrome pods? Also, my docker-compose file is too long may I simply it easily or do I need a scalable automation script?

version: '3'
services:
  jaeger:
    image: jaegertracing/all-in-one:1.60
    ports:
      - "16686:16686"
      - "4317:4317"
  chrome_01:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6900:5900"
  chrome_02:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6901:5900"
  chrome_03:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6902:5900"
  chrome_04:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6903:5900"
  chrome_05:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6904:5900"
  chrome_06:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6905:5900"
  chrome_07:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6906:5900"
  chrome_08:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6907:5900"
  chrome_09:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6908:5900"
  chrome_10:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6909:5900"
  chrome_11:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6910:5900"
  chrome_12:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6912:5900"
  chrome_13:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6913:5900"
  chrome_14:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6914:5900"
  chrome_15:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6915:5900"
  chrome_16:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6916:5900"
  chrome_17:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6917:5900"
  chrome_18:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6918:5900"
  chrome_19:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6919:5900"
  chrome_20:
    
      
      
    image: selenium/node-chrome:latest
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317
    ports:
      - "6920:5900"


  chrome_01_video:
    image: selenium/video:latest
     
    depends_on:
      - chrome_01
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_01
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto

  chrome_02_video:
    image: selenium/video:latest
     
    depends_on:
      - chrome_02
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_02
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
       
  chrome_03_video:
    image: selenium/video:latest
    depends_on:
      - chrome_03
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_03
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_04_video:
    image: selenium/video:latest
    depends_on:
      - chrome_04
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_04
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_05_video:
    image: selenium/video:latest
    depends_on:
      - chrome_05
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_05
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_06_video:
    image: selenium/video:latest
    depends_on:
      - chrome_06
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_06
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_07_video:
    image: selenium/video:latest
    depends_on:
      - chrome_07
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_07
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_08_video:
    image: selenium/video:latest
    depends_on:
      - chrome_08
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_08
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_09_video:
    image: selenium/video:latest
    depends_on:
      - chrome_09
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_09
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_10_video:
    image: selenium/video:latest
    depends_on:
      - chrome_10
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_10
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_11_video:
    image: selenium/video:latest
    depends_on:
      - chrome_11
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_11
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_12_video:
    image: selenium/video:latest
    depends_on:
      - chrome_12
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_12
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_13_video:
    image: selenium/video:latest
    depends_on:
      - chrome_13
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_13
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_14_video:
    image: selenium/video:latest
    depends_on:
      - chrome_14
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_14
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_15_video:
    image: selenium/video:latest
    depends_on:
      - chrome_15
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_15
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_16_video:
    image: selenium/video:latest
    depends_on:
      - chrome_16
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_16
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_17_video:
    image: selenium/video:latest
    depends_on:
      - chrome_17
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_17
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_18_video:
    image: selenium/video:latest
    depends_on:
      - chrome_18
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_18
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_19_video:
    image: selenium/video:latest
    depends_on:
      - chrome_19
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_19
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto
  chrome_20_video:
    image: selenium/video:latest
    depends_on:
      - chrome_20
    
      
      
      - "/app/Selenium-grid/videos:/videos"
    environment:
      - DISPLAY_CONTAINER_NAME=chrome_20
      - SE_NODE_GRID_URL=http://selenium-hub:4444
      - SE_VIDEO_FILE_NAME=auto

  selenium-hub:
    image: selenium/hub:latest
    
      
      
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    depends_on:
      - jaeger
    environment:
      - SE_ENABLE_TRACING=true
      - SE_OTEL_TRACES_EXPORTER=otlp
      - SE_OTEL_EXPORTER_ENDPOINT=http://jaeger:4317


```</div>
@VietND96 VietND96 added this to the 4.27.0 milestone Nov 5, 2024
@VietND96 VietND96 mentioned this issue Nov 15, 2024
Open
@VietND96 VietND96 changed the title [🚀 Feature]: Optimize video recorder containers [🚀 Feature]: Optimize video recorder containers (Phase 1) Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.27.0
Development

No branches or pull requests
1 participant
@VietND96