Support XDG Desktop Portal / Pipewire screencasting

[Ferdi265]

This is the "standard" screencasting method for wayland. It's less convenient than the two sway-specific protocols in that you cannot specify the monitor or region directly, but it is the only method that will work on all compositors.

Tasks related to less direct selection of capture targets:

• allow specifying no monitor if xdg-portal backend is chosen
• research how region selection could work with xdg-portal
• allow selecting a window to capture if supported by the portal implementation

Implementation Tasks:

• Add DBus library (sd-bus / basu)
• Add Screencast portal client code
• add libpipewire
• add PipeWire client code for creating a stream
• add dmabuf/shm code for taking stream frames and passing them to EGL
  • code for dmabuf is there, but for some reason doesn't work (black screen, no GL errors)

[Ferdi265]

Current progress tracked in branch feature-xdg-portal

[Ferdi265]

Current status: the initial codebase is there, but rendering the imported DMA-BUF still seems to fail in some way (the resulting texture is black).

Currently unimplemented are stream format negotiation and shm buffers. Of course this also needs to be cleaned up a whole lot and made more robust.

[Ferdi265]

I'm currently at a dead end; none of the debug info provided by MESA / GL is of any help to me right now (most debug flags don't output anything, since there aren't any GL errors, and the driver specific debug flags vomit out endless parameter dumps that aren't very useful), and the import of the DMA-BUF from pipewire succeeds, but drawing results in a black screen.

I am not sure where the problem is, but so far I have only tested this on an Intel system. If you would like to test this, use these commands:

git checkout feature-xdg-portal
cmake . -B build -DWITH_XDG_PORTAL_BACKEND=ON
cd build
make
./wl-mirror -v -b xdg-portal eDP-1

If this results in anything other than a black screen on your system, please let me know.

[Ferdi265]

I recently found some ugly use after free bugs in my pipewire feature branch, but after fixing those the black screen is still there, so that was not the cause it seems.

[Kaboom22]

Just compiled this branch (feature-xdg-portal) and tried this on my all AMD system

OS: Pop!_OS 22.04 LTS x86_64
Kernel: 6.6.6-76060606-generic
DE: GNOME 44
WM: Mutter
CPU: AMD Ryzen 9 7950X3D (32) @ 5.759GHz
GPU: AMD Radeon 5700 XT
Wayland

the only unlisted requirement i needed to compile was "libdrm-dev". When running ./wl-mirror -v -b xdg-portal HDMI-A-1 (or anything else for that matter) i get:

debug: main::main(): initializing stream
debug: main::main(): initializing wayland
debug: wayland::on_registry_add(): wl_compositor (version = 5, id = 1)
debug: wayland::on_registry_add(): wl_drm (version = 2, id = 2)
debug: wayland::on_registry_add(): wl_shm (version = 1, id = 3)
debug: wayland::on_registry_add(): wl_output (version = 2, id = 4)
error: wayland::on_registry_add(): wl_output received before xdg_output_manager
debug: main::cleanup(): deallocating resources
debug: wayland::cleanup(): destroying wayland objects
Segmentation fault (core dumped)

If i put the block with comment "// create xdg_output object" before the block with "// check for xdg_output_manager" i get:

debug: main::main(): initializing stream
debug: main::main(): initializing wayland
debug: wayland::on_registry_add(): wl_compositor (version = 5, id = 1)
debug: wayland::on_registry_add(): wl_drm (version = 2, id = 2)
debug: wayland::on_registry_add(): wl_shm (version = 1, id = 3)
debug: wayland::on_registry_add(): wl_output (version = 2, id = 4)
Segmentation fault (core dumped)

But I'm not sure if that's any better. Happy to test anything you can do to get Gnome working 👍

[Ferdi265]

Hi! Thanks for testing that branch!

the only unlisted requirement i needed to compile was "libdrm-dev"

Yes, the XDG portal branch AFAIK depends on the headers for libdrm for the drm_fourcc.h header file for format codes. I need to either replace that or document the dependency.

error: wayland::on_registry_add(): wl_output received before xdg_output_manager

This is mainly because the registry event handling code was written very early on when I understood little about Wayland's object system. I need to rewrite that code significantly in order to make it more robust. GNOME sends globals in a different order than sway and wl-mirror currently does not handle that gracefully (on the TODO list).

This branch sadly doesn't work yet, even on sway, because of a really weird black screen issue with EGL / pipewire / DMA-BUFs that I haven't been able to figure out the cause of.

Also, even if this branch works, you probably won't be able to move/resize the window on GNOME, since wl-mirror does not have client-side window decorations (yet). This is fixed in the feature-libdecor branch though, which will be merged soon.

TL;DR: There is still lots to do and a lot of stuff to fix, but progress is being made, and I hope I can get this to work in the near-ish future, but I can't guarantee anything since I'm pretty stuck with a basically undebuggable bug (the black screen).

[Kaboom22]

A lack of move/resize or decor isn't a problem for me, my use case is copying a fake 4800X1200 monitor onto three real 1600X1200 monitors to create a Eyefinity setup that is robust without having to do a variety of tweaks and hacks for each game or doing weird X11 things that won't be future proof. Wl-mirror seems perfect for that as i can fit the window to the three monitors and copy the fake, and run the game in exclusive full screen to capture the input.

But anyway i won't hijack your thread, let me know if you need to test a debug build or something.

[Ferdi265]

My best guess for what is going wrong with the black screen in the feature-xdg-portal branch is the hacked-up format "negotiation" (hardcoded formats copied from OBS).

I need to rewrite that whole code to actually negotiate formats since that is likely what's causing the issues.

Documentation: https://docs.pipewire.org/page_dma_buf.html

[Ferdi265]

I'm not sure what happened, but as of today, with sway-git 1.10.r7384.05e895c-1, wlroots-git 0.19.0.r7159.775817e27-1, and pipewire 1.2.2-1, the xdg-portal branch has magically started working!

I need to go back and clean it up, and also verify what the first versions are this works with, as well as test on more different GPU setups (testing this on my AMD integrated graphics right now).

[Ferdi265]

Tested this together with Libdecor now, also works on KDE and Gnome (see #17)