I've been successfully using the Intel Media SDK GStreamer plugins on Windows for a while now (specifically this branch), but I've encountered an error and I'm wondering if someone can help me out.
What I'm trying to do is generate BGRA output from either the h264 or h265 encoders, but some negotiation with downstream elements in the pipeline is failing. NV12 works just fine, and so I know I've got things hooked up right. The failure occurs when wanting the encoder to generate any other colorspace but NV12.
A bit more detail
I've got an RTSP stream that I'm reading, and I'm wanting to draw it onto some Windows Presentation Foundation control, which only allows BGRA/RGBA/RGB/etc. I'm using the plugin in code, but I can replicate the problem using just gst-launch command:
gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=BGRA' ! autovideosink
This will fail. The decodebin does use the Intel Media plugins when building the pipeline. If you enable debugging through GST_DEBUG=4 or higher, you will get the following error/warning:
WARN GST_PADS gstpad.c:4230:gst_pad_peer_query:<mfxdec_h264-0:src> could not send sticky events
The net effect is the pipeline never fully transitions to the play state. If I watch the GPU and CPU, I can see it's chewing, but downstream elements don't get any data.
Flipping BGRA to NV12, the stream will work: the pipeline transitions to PLAY and downstream elements get data and I can view my stream:
gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink
When I inspect the MFX GStreamer elements, they all advertise they can handle BGRA. For example, here are the pad outputs generated by gst-inspect when examining the H265 encoder:
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h265
alignment: au
profile: { (string)main, (string)main-10 }
stream-format: byte-streamSRC template: 'src'
Availability: Always
Capabilities:
video/x-raw(memory:MFXSurface)
format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2, (string)ENCODED }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1
I'm running on Windows 10. My CPU is a i7-6700K @ 4GHZ.
Any insight or thoughts would be appreciated. I'm digging through the Media SDK's GStreamer plugin code right now to find the source of the issue, but I'm hoping someone can offer a pointer or two. Thanks.
Edit:
I have found that the problem at least manifests itself at this point in the code. The call to gst_video_decoder_finish_frame() returns GST_FLOW_NOT_NEGOTIATED if BGRA is specified, GST_FLOW_OK if NV12 is specified.