Comprehensive Guide to Enabling AV1 Encoding, Plex Deployment & GPU Management with Tdarr on Unraid 7.0
This comprehensive guide provides a step-by-step approach to optimizing your media library through AV1 encoding while efficiently managing GPU resources between Plex and Tdarr on Unraid. By following this guide, you'll learn how to drastically reduce video file sizes, saving valuable storage space, and automate the allocation of GPU resources to ensure smooth and uninterrupted playback for Plex users.
In addition to encoding and resource management, this guide covers essential information on setting up and configuring necessary plugins, importing and applying AV1 encoding flows, troubleshooting common issues, and implementing backup and recovery strategies. Whether you're a seasoned Unraid user or new to media server management, this guide equips you with the knowledge and tools needed to maximize your server's performance and efficiency.
This requires Unraid 7.0 (which is in beta at this time). I have an AMD 7900 - 3 Intel ARC Cards - 64GB DDR5 RAM - Two 4TB NVME Drives - 350+ TB of drives running many Docker containers and have zero problems. I run no VMs nor any passthrough (which always complicates things). Remember, it's always at your own risk.
**Apple TV Users: ** Force Apple TV to Play AV1 Natively! No Transcoding from the GPU @ https://github.com/plexguide/AV1-AppleTV
- Data Savings with AV1 Encoding
- AV1 Drawbacks
- Upgrading to Unraid 7.0 and Installing Required Plugins
- Deploying Plex with Intel ARC GPU Support
- AV1 Tdarr Flow
- Optimizing AV1 Encoding Settings
- Tdarr Node Killer Script
- Experimental: Running the Script on Other Operating Systems
- Backup and Recovery Tips
- Summary
Running this setup with three ARC GPUs has shown significant data savings over two weeks. With AV1 encoding, a savings of 37TB was achieved, covering only 10-15% of the library.
Explanation: AV1 encoding can drastically reduce storage needs. For example, a 300TB library could be reduced to 75-100TB, making it an efficient solution for large media libraries.
For more information on the potential drawbacks of using AV1 encoding, including device compatibility issues and increased resource usage during transcoding, please visit the AV1 Drawbacks page.
Before setting up the AV1 Tdarr Flow or the Tdarr Node Killer Script (totally optional and not needed), ensure that you are running Unraid 7.0 and have the necessary plugins installed to monitor and manage your Intel ARC GPU.
The first plugin to install is the Intel GPU TOP by ich777. This plugin is essential for monitoring your Intel ARC GPU’s performance.
- Plugin Developer: ich777
- Installation: Available through the Unraid Community Applications
- GitHub Repository: Intel GPU TOP by ich777
Next, install the GPU Statistics plugin by b3rs3rk. This plugin provides detailed statistics on GPU usage, helping you verify that your Intel ARC GPU is working correctly.
- Plugin Developer: b3rs3rk
- Installation: Available through the Unraid Community Applications
With these plugins installed, you can monitor your Intel ARC GPU during transcoding and other tasks. Below are examples of what you can expect to see when your Intel ARC GPU is in use:
In Unraid, you need to add the Intel ARC GPU to your Plex Docker template as a device. Ensure that this is done correctly by adding the device at the end of your template configuration:
After adding the GPU to the Docker template, you need to configure Plex to ensure it uses the Intel ARC GPU for transcoding.
- Turn Off HDR Tone Mapping: This setting should be disabled because, currently, HDR transcoding will not work with the Intel ARC GPU. If HDR Tone Mapping is enabled, Plex will default to using your CPU for transcoding, which could cause confusion if you see high CPU usage.
If everything is configured correctly, Plex should use the Intel ARC GPU for transcoding. You can verify that your GPU is being used by checking the Plex dashboard during transcoding:
Insight: Properly configuring Plex to use your Intel ARC GPU can significantly improve transcoding performance while reducing CPU load. This is especially beneficial when handling multiple streams or when you want to optimize power consumption and system efficiency.
The AV1 encoding flow is a process that converts video data into the AV1 format, known for its high efficiency and excellent compression. The flow involves:
- Source: Your original video file.
- Input Processing: Preparing the video by adjusting resolution, color space, and more.
- Encoding: The AV1 encoder compresses the video data, making the file smaller without losing too much quality.
- Output: The final, compressed AV1 video file is ready for streaming or storage.
Note: This flow should work on any operating system that has an Intel ARC GPU card. You just need to ensure that the ARC GPU is exposed in your Tdarr Docker container.
To use the AV1 flow, you need to import it into Tdarr. The JSON file for the AV1 flow can be found here. Use the lastest version, right now is v2 for better quality.
-
Adding a New Flow: In Tdarr, click on "Flows" at the top of the interface, then click the "Flow+" button to add a new flow.
-
Scroll to the Bottom: Scroll to the very bottom of the page to find the import option. It can be easy to miss, so make sure you scroll all the way down.
-
Import the Flow: Copy the exact JSON content from the AV1 flow JSON file and paste it into the import field.
-
Enabling the Flow for Libraries: After importing the flow, you need to enable it for each library. Go to the "Library" tab in Tdarr, click "Transcode Options" for each library, and change the transcode option from "Classic Plugin" to "AV1" or whatever you named the flow.
Note: This step is crucial as the flow will not work until it is applied to the libraries.
To get the best results with AV1 encoding, consider the following tips:
-
Balancing Quality and Compression: Adjust the CRF (Constant Rate Factor) and bit rate settings to find the right balance between video quality and file size. A lower CRF value will increase quality but also file size, while a higher CRF value will reduce quality but save more space.
-
Hardware Acceleration: Ensure that your Intel ARC GPU is being utilized for hardware-accelerated encoding. This can significantly speed up the process and reduce the load on your CPU.
-
Testing Settings: Run a few test encodes with different settings to determine what works best for your library. Different types of content may require different settings to achieve optimal results.
The Tdarr Node Killer Script is designed to manage your GPU resources between Plex and Tdarr efficiently. It ensures that when Plex starts transcoding, the Tdarr node using the same GPU is automatically stopped, preventing any conflicts.
You can find the script here.
- Monitoring Plex: The script continuously monitors Plex for any transcoding activity. When Plex starts transcoding, the script kills the Tdarr node using the same GPU.
- Five-Minute Check Cycle: After stopping the Tdarr node, the script checks every 5 seconds for 5 minutes to see if Plex has finished transcoding.
- If no transcoding is detected during this time, the Tdarr node is restarted.
- If transcoding is detected at any point, the timer resets, and the script continues monitoring.
This setup allows you to put the GPU back to work when Plex is idle, while ensuring Plex users always get priority when transcoding is required.
-
Tdarr Node Running, No Plex Transcoding
Explanation: The Tdarr node is running, and Plex is not currently transcoding any videos. This means Tdarr is using the GPU resources for video processing tasks.
-
Script Monitoring for Plex Transcoding
Explanation: The script continuously checks if Plex is transcoding. At this point, no transcoding is detected, so Tdarr continues using the GPU.
-
Plex User Starts Transcoding
Explanation: A Plex user starts watching a video, causing Plex to begin transcoding. This might happen if the video is in a format like AV1, H.264, or H.265, which requires transcoding for older devices or specific user settings.
-
Script Detects Plex Transcoding, Stops Tdarr Node
Explanation: The script detects that Plex is transcoding and stops the Tdarr node. This action frees up the Intel ARC GPU so that Plex can use it exclusively for transcoding.
-
Tdarr Node Is Stopped
Explanation: Inside Tdarr, you can see that the node has been stopped by the script. This ensures that Plex has full access to the GPU for efficient transcoding.
-
Tdarr Node Dead
Explanation: The Tdarr node is completely stopped, ensuring that Plex has exclusive access to the GPU.
The script doesn't immediately restart the Tdarr node after Plex stops transcoding. Instead, it checks every 5 seconds for 5 minutes to ensure that Plex isn't going to start transcoding again. This prevents the Tdarr node from constantly stopping and starting, which could be inefficient.
-
Countdown Before Restarting Tdarr Node
Explanation: The script is counting down, checking every 5 seconds to see if Plex starts transcoding again. If Plex does start, the timer resets, ensuring that the Tdarr node stays off as long as Plex needs the GPU.
-
Tdarr Node Restarted After 5 Minutes
Explanation: After 5 minutes with no Plex transcoding detected, the script restarts the Tdarr node. The process then continues to check if Plex starts transcoding, so the node can be stopped again if needed.
-
Tdarr Node Coming Back Online
Explanation: Inside Tdarr, you can see that the node is coming back online after being restarted by the script.
-
Tdarr Node Fully Online
Explanation: The Tdarr node is now fully operational and visible on the dashboard. The script will continue to monitor Plex and manage the node as needed.
If you encounter issues during setup or operation, here are some common problems and solutions:
-
Plex Not Using GPU for Transcoding:
- Verify GPU Configuration: Ensure the Intel ARC GPU is correctly added to the Plex Docker template and that the correct device path is used.
- Check Logs: Review Plex logs for any errors related to GPU transcoding. This can help identify if Plex is defaulting to CPU transcoding due to a misconfiguration.
- Driver Issues: Ensure that the necessary drivers for Intel ARC are installed and up to date. If using Unraid, make sure the Intel GPU TOP plugin is correctly installed and running.
-
Tdarr Node Not Restarting After Plex Stops Transcoding:
- Script Debugging: Run the script manually and monitor the output to ensure it’s detecting the end of Plex transcoding correctly.
- Check for Conflicting Processes: Ensure that no other processes are interfering with the Tdarr node or preventing it from restarting.
-
High CPU Usage During Transcoding:
- HDR Tone Mapping: Verify that HDR tone mapping is disabled in Plex settings if you're using the Intel ARC GPU. This is a common cause of high CPU usage as Plex defaults to CPU transcoding for HDR content.
- Check GPU Utilization: Use the GPU Statistics plugin to monitor GPU utilization and ensure the GPU is being used efficiently.
While this script is designed to work seamlessly on Unraid, it can technically work on any operating system that supports Docker and systemd services. The steps provided below can be adapted for use on systems like Ubuntu, CentOS, or any other Linux distribution that uses systemd.
-
Save the Script: Save your Tdarr Node Killer Script as
tdarr_node_killer.shin/usr/local/bin/.sudo nano /usr/local/bin/tdarr_node_killer.sh
-
Set the Proper Permissions:
Ensure that the script has the correct permissions to execute. Use the following commands:
sudo chmod +x /usr/local/bin/tdarr_node_killer.sh sudo chown root:root /usr/local/bin/tdarr_node_killer.sh
-
Create a Service File: Create a service file for the script:
sudo nano /etc/systemd/system/tdarr_node_killer.service
-
Add the Following Content:
[Unit] Description=Tdarr Node Killer Script After=network.target [Service] Type=simple ExecStart=/bin/bash /usr/local/bin/tdarr_node_killer.sh Restart=on-failure [Install] WantedBy=multi-user.target
-
Reload Systemd:
sudo systemctl daemon-reload
-
Start and Enable the Service:
sudo systemctl start tdarr_node_killer.service sudo systemctl enable tdarr_node_killer.service
By following these steps, the script will run automatically on startup and ensure that it stays active in the background, managing your GPU resources efficiently.
Before making significant changes to your Unraid setup, Plex configurations, or Docker containers, it's essential to create backups. Here are some tips:
-
Backup Plex Configurations:
- Use the built-in Plex backup tools to save your library metadata, watch history, and other settings.
- Regularly back up the Plex configuration folder (
/config) to ensure you can restore your setup if needed.
-
Backup Docker Containers:
- Create backups of your Docker container templates in Unraid. This makes it easy to redeploy containers with the same settings if something goes wrong.
- Consider using Unraid’s built-in backup features or plugins like CA Backup/Restore Appdata to automate this process.
-
Backup Unraid Configuration:
- Regularly back up your Unraid flash drive. This contains your Unraid license, configuration, and other critical settings.
- Use the Unraid GUI to download a backup of your flash drive, and store it in a safe location.
-
Recovery Testing:
- Periodically test your backups by restoring them to ensure they work as expected. This is crucial for verifying that your backups are reliable.
By following these backup and recovery tips, you can ensure that your setup remains stable and that you can recover quickly from any issues.
The Tdarr Node Killer Script is designed to intelligently manage GPU resources between Plex and Tdarr. By monitoring Plex transcoding activity and controlling the Tdarr node, the script ensures that your Intel ARC GPU is used efficiently. This guide provides a visual walkthrough of the process, making it easy for beginners to understand how the script works and how to configure it for their own use.