Regarding OBS, Plugins, and Flatpak

From a spur of the moment idea with the Mrs., I am prepping for a family game night with Jackbox 7. I bought a prevous version of it on the Nintendo Switch as it was the most portable way to play it with friends and family.

Jackbox has suggestions on how to play remotely most likely due to COVID-19. They have setup instructions for Google Hangouts and Zoom. Curiously, Jitsi, an open source alternative is not represented.

I’ve decided to try out different online conferencing software to determine which one works the best, while giving open source solutions preference in my testing.

My first test was on Jitsi. I discovered a few interesting quirks while testing it out with a friend. Jitsi does share the game screen, but it will disable my camera as soon as I start a screen sharing. There isn’t an option to share the audio of the application within Jitsi, but the microphone picked up my PC’s audio. Also, both the screen sharing and application sharing only showed a few frames every second on my friend’s screen. Not great. Maybe that’s why Jackbox didn’t list them?

I then tried Zoom. Zoom at least displays the video and the webcam at the same time. There’s a checkbox on application sharing for sharing application audio, and to optimize for full-screen videos. This ended up being much closer to the desired experience for my friend than Jitsi, but the overall framerate of the game was slow (maybe around 5 to 10 frames per second) while my webcam looked fine.

What else to do?

I assumed for bandwidth reasons that these video conferencing applications don’t prioritize the video for application sharing as high as the web camera, so if I could send both the camera and screen feeds together, perhaps I can increase the framerate.

Using OBS a bit in the past, I figured this was the best place to start.

OBS Studio

I’ve used OBS in Windows for screen recording, so I was pretty familiar with the interface. I won’t go into any major details on setting up the view in this post, as I think the configuration is the more interesting part anyways.

Using the Fedora Software manager, OBS Studio is available in the RPM Free repository and Flathub. I started with the RPM Free version, but the application had major issues capturing any screen display. I assume that it has something to do with a combination of Wayland and some downline dependency.

The flatpak version at least worked out of the box. I was able to start screen sharing and inserting my camera on the feed.

I also played around with streaming on Twitch, but I thought that would be too confusing to have people join in a video conference and a Twitch stream at the same time. I found the video quality was excellent, but there was approximately a 9 second delay with the stream even with low-latency settings enabled in Twitch.

Ok, all the inputs are good! The last part is to redirect this video out to a camera, and then I could use any conferencing application I want!

V4l2loopback

Some quick searching yielded a plugin on the OBS website to do exactly what I needed, but on Windows… The Linux plugin is called v4l2sink. It required V4l2loopback.

V4l2loopback lets you create “dummy cameras.” You then can feed whatever to the camera souces for downline applications. Cameras can be found in the file system under /dev/camera* when v4l2loopback is set up properly, you should see an additional camera listed.

Fedora doesn’t have a package for it in DNF, so I built it straight from source: https://github.com/umlaeute/v4l2loopback

Building was pretty easy, the only additional step outside of the instructions that I had to do was install the kernel headers:

sudo dnf install kernel-headers
sudo dnf install kernel-devel

Doing another check under /dev/camera* showed the new camera. Success!

obs-v4l2sink Plugin

Now I have a dummy camera, but it doesn’t output anything. The final piece is to have OBS feed it video with the v4l2sink plugin. This ended up being the most difficult step in the whole process.

The instructions for v4l2sink on their GitHub page are pretty straightforward, but were for building on an Ubuntu/Debian platform; Fedora didn’t have a package called libops-dev.

Since I couldn’t find the package, I thought if I downloaded all the dependencies to build OBS that may install the required dependencies for the plugin. Depencies for OBS listed are conveniently located on their GitHub page.

The libspeexdsp-devel package wasn’t found and the link to their source code is dead, so I decided to skip it. Unfortunately, this did not fix my plugin depencies. Oh well, time to try something else!

After a bit of guesswork browsing the available packages on DNF, I did find the equivalent to libobs-dev and was able to install it with this:

sudo dnf install obs-studio-devel
sudo dnf install obs-studio-libs

Getting It All to Work Together

With dependencies installed and plugin built, I eagerly launched OBS again to set up v4l2sink. It didn’t show up. I figured that it may have something to do with Flatpak, since I know it runs in a sandbox environment for security purposes.

There aren’t many articles about Flatpak OBS plugins; I first tried the suggestions from this article, but didn’t work. Ugh! The only clue I received was from the OBS console mentioning that it cannot find the files for it. I also tried dumping the plugin within my .config, .app, and /usr folders to no luck.

The ultimate fix was located within a still open¹ bug report for OBS. Essentially, you need to build a flatpak package for the plugin to use for the flatpack version of OBS. I didn’t quite understand what’s happening behind the scenes, but I followed all the steps and voilà! The plugin is finally visible in OBS!

The dummy camera was available within Zoom, but not within Jitsi or Signal. One final tweak was needed on the v4l2loopback that was mentioned on their Github page:

sudo modprobe v4l2loopback exclusive_caps=1

Outcome

After all that work I would have liked to say that it was a complete success. I did manage to screenshare my dummy camera feed within Jitsi and even Signal video calls, but when I started to alt-tab into other applications, my desktop feed turned black.

I also forgot to consider that the audio feed from conferencing was not combined with the microphone on my webcam. I’m sure that there’s probably a way to mix in the audio as well, but this seems like way too much effort at this point.

I did see that there is a screen sharing capability with “Steam Play with Friends.” I will test that out to see if the performance is any better. If not, I will probably stick to Zoom for this time and figure out an alternative later.

Regardless, it was a good learning experience to understand how Linux controls webcams, and I even managed to learn a few new tricks within OBS, so it definitely wasn’t a total loss!

Day 10 of #100DaysToOffload