[feature request] Implement AV1 support

[cromefire]

First and foremost: Yes it is still a draft and you may of course reject this idea, but due to it already having player support (Kodi on FireTV Max), seeming relatively straight forward (is seems to be based on the already existing ITU-T T.35 Metadata/SMPTE ST 2094-40) and seeming relatively stable for a few months, I feel like this would be a great addition as it allows conversion of existing h265 content to AV1 (or back if needed, although currently it doesn't seem like there's a lot of content).

Specification (draft): https://aomediacodec.github.io/av1-hdr10plus/
Sample file I have verified working (HDR10+ metadata arriving at the TV) with Kodi (Nexus) on my FireTV Max: https://github.com/AOMediaCodec/av1-hdr10plus/blob/ecdafef3eaf104135a6b28aabd960552e8209c7d/HDR10_SIZZLE_PLUS_UP-FINAL_02_AV1.mp4
AV1 bitstream parser library: https://github.com/yohhoy/av1parser

The bitstream parser above seems to already support ITU-T T.35 metadata and your code already supports the parsing of the metadata from bytes from (hopefully) the same format. So in theory it should be as easy as "only" creating some conditions to support more than one codec and some verification that it works. One additional problem might arise from the seeming lack of a .av1 format or something similar, all the AV1 people seem to be using .ivf files (that also seem to be really low overhead over a raw bitstream), but luckily there seems to be a rust implementation from rav1e. I have considered trying to integrate it myself, but as I lack knowledge of the format of the metadata the chances of success would be pretty low.

[quietvoid]

See yohhoy/av1parser#14

I don't have any plan to support AV1 until the spec is finalized.
Ideally we also need an encoder with support as reference.

[quietvoid]

xiph/rav1e#3000 should be working fine to generate test streams.
I'm not sure when I'll get around to this.

[cromefire]

Happy to test stuff, with Kodi, the FireTV and Panasonic OLED, I can actually test whether HDR10+ metadata is arriving at the TV (and it already works with AV1 & HDR10+).

[quietvoid]

Good to know. I can also test on a Samsung TV.

[sven-pke]

Any news about this?

[quietvoid]

Nope.

[thePublicer]

there is support to add hdr10+ to av1 see https://github.com/HandBrake/HandBrake/blob/master/NEWS.markdown

[thePublicer]

also any plans to extract hdr10plus from vp9?

[JaitinPrakash]

the official spec has reached a state where i believe you can now implement it.
https://aomediacodec.github.io/av1-hdr10plus/

[quietvoid]

The complexity isn't in the HDR10+ spec but rather parsing AV1 bitstreams.
There aren't stateful parsers that are usable so I'd probably have to implement one from scratch.

[sven-pke]

https://gstreamer.freedesktop.org/documentation/gst-plugins-bad-codecparsers/gstav1parser.html?gi-language=c I think Gstreamer is also available in Rust. Could that be an option?

[quietvoid]

GStreamer implies an additionnal dependency at runtime and I'd rather avoid that since it's more complicated for end users.
I also can't seem to find Rust bindings to the codec parsers plugin, anyways.

[JaitinPrakash]

handbrake seems to have a semi-functional system in place. my phone properly detects and decodes av1 hdr10+ videos, although mediainfo only reports hdr10.

[cromefire]

my phone properly detects and decodes av1 hdr10+ videos

Do you have any good way to test that? In the past I've found you can't really tell well on the phone, except maybe with your eyes, although that's hard-ish to see.

[JaitinPrakash]

I took an hdr10+ video on my phone, encoded it with regular svt-av1 and with handbrake, and difference in color saturation, contrast, and overall vibrancy was very clear.

[cromefire]

Do you have test files?

[JaitinPrakash]

not at the moment, and it's midnight right now, so no good bright footage. I'll get some stuff to you when the sun is up. in the mean time, HDR10+ technologies plus up video is on YouTube.

[cromefire]

Okay cool.

YouTube BTW doesn't really have HDR10+ support (according to my docs and my tests), you can only upload HDR10+ content there and I think it will try and use that metadata to create a good approximation of the HDR10+ version. That's why I always test on my TV, it'll display whenever it receives HDR10+.

[JaitinPrakash]

here's a basic sample file from ff pictures GmbH, an hdr mastering company. have confirmed the hdr file is hdr10+ with mediainfo, video player, and hdr10plus_tool.

https://ff.wetransfer.com/downloads/0f596c6233476b3a74a7005f2912235320210201160114/257cf4

[cromefire]

Yeah that one I know, but it's so incredibly hard to see the differences I'm always unsure if there are any or if I'm just imagining it.
(Seems to work though I think, but it's also HEVC it seems)

[JaitinPrakash]

Handbrake SVT-AV1 HDR10+: https://files.catbox.moe/qsc8wl.mkv

FFMPEG SVT-AV1 HDR10: https://files.catbox.moe/5e3qih.webm

Samsung Camera HDR10+ HEVC Source: https://files.catbox.moe/udbnuy.mp4

[JaitinPrakash]

They're not great, but with overcast skies i can't get sharp sunlight/shadow lines.