afaerber has quit [Remote host closed the connection]
afaerber has joined #linux-amlogic
AUser has quit [Ping timeout: 240 seconds]
AUser has joined #linux-amlogic
<ndufresne>
Ely, I've looked up the colorimetry issue, and the answer is that gst currently fails if the drivers returns 0 as colorspace
<ndufresne>
but it also raised some larger issues in gst/v4l2 colorimetry
<ndufresne>
Our mains issue in gst is that v4l2 does not let us enumerate the colorimetry in an efficient way
<ndufresne>
I'll probably just disable colorimetry enumeration and colorimetry matching from now on, I'll leave the code that sets the colorimetry on the driver when available, and simply warn if it miss-match
<Ely>
ndufresne: Thanks for the investigation!
<Ely>
I do the exact same thing as venus, does that mean they'll have the same problem too ?
<Ely>
Basically colorspace is initialized at 0, then it's set by s_fmt, then we return the same one
<Ely>
I'm guessing you're not setting it all the time or something ?
<ndufresne>
no, venus do copy the colorimetry from S_FMT(OUTPUT) to G_FMT(CAPTURE), if you do, it does not work
<Ely>
I already do this in my driver Oo. Wait let me check just in case.
<ndufresne>
but at 4K there is a special case for colorimetry in gst, because we pick a transfer function for BT709 that is extended for 12 bit precision
<ndufresne>
Ely, when I call G_FMT after passing the sps/pps/keyframe, the colorimetry is 0
<ndufresne>
on venue it would be whatever I have passed to S_FMT(OUTPUT)
<ndufresne>
which is as wrong as anything else really
<ndufresne>
colorimetry is a bit unusable in V4L2 api atm
<Ely>
Yeah I already copy colorspace information from OUTPUT to CAPTURE
<Ely>
same code as venus
<ndufresne>
hmm, maybe I'm off on that, the warning of Unknown enum v4l2_colorspace 0 is after Acquired caps: video/x-raw, ...
<ndufresne>
Ely, btw, each time you see "Unknown enum v4l2_colorspace 0", it means you have a driver bug btw
<ndufresne>
this is a totally invalid value, there is a value for "default" that isn't zero
<Ely>
yup maybe I should initialize it to non zero
<Ely>
like 601 or something
<ndufresne>
no, there is a _DEFAULT, when you don't know, this is the right one to set
<ndufresne>
but if the OUTPUT queue have a format, G_FMT/TRY_FMT should always set it to the OUTPUT colorspace
<Ely>
yes that is already done
<ndufresne>
because we trust the containers over the bitstream (for gst, we also parse this from the bitstream)
<ndufresne>
Ely, not on TRY_FMT apparently
<Ely>
Indeed, only on g_fmt
<Ely>
just like venus :D
<ndufresne>
then it probably fails on venus too
<ndufresne>
(the DB820c is so unstable, that I didn't have a chance to do a lot of 4K testing)
<Ely>
V4L2_COLORSPACE_DEFAULT = 0,
<ndufresne>
really, maybe I'm confusing APIs ...
<ndufresne>
I whish I wrote the colorimetry in gst/v4l2, this is confusing me so much ...
<Ely>
hehe
<ndufresne>
sometimes, you receive patches that looks good, and works most of the time, and finally endup with hard to fix issues
<ndufresne>
and the value of the enum is not printed in the doc ....
focus_ has quit [Quit: Leaving]
<Ely>
Huh indeed
<ndufresne>
ok, not enough time today unfortunatly, I'll continue this later, I'll fix couple of traces so we get both gst/v4l2 values, everything we TRY/S/G_FMT
<ndufresne>
but overall, we get "bt709" as input, which I believe get read back has something else because the transfer function in gst is different at 4K ...
<ndufresne>
gst/v4l2 is a not a one one match, mostly because of v4l2 legacy "colorspace"
sputnik__ has quit [Remote host closed the connection]
trem has quit [Read error: Connection reset by peer]
<ndufresne>
Ely, ok, I finally found out where the asymetry came from, it was BT709, if you set that on the v4l2, and read it back, using gst helpers, you get two different results ;-P
<ndufresne>
was a bit myfault, I received V4L2_XFER_FUNC_709, I would always bump it to GST_VIDEO_TRANSFER_BT2020_12 instead of GST_VIDEO_TRANSFER_BT709 in 4K, because they are the same, just that the second is more precise. But to stay symmetric, I need to bump it only when V4L2_COLORSPACE_BT2020
<ndufresne>
now, don't ask me why iPhones uses V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_BT2020 for 4K content, it generally the wrong thing to do ...
<ndufresne>
(unless if a bug on gst parser ...)
<Ely>
ndufresne: cool that you found the root cause!!
<Ely>
I thought 2020 was mostly set with 10-bit hdr while 709 remained for regular 8-bit non hdr
<ndufresne>
ok, that comes from the ISOMP4 demuxer, and there is no support for the BT2020_12 variant added in gst 1.16 ...
<ndufresne>
Ely, yes, and clearly the guy who added these in gst thought it was harmless, but he created a mess ...
<ndufresne>
but it remains that I need to disable all colorimetry probing in gst, cause the enumeration of the colorimetry leads to incomplete list, I'm worried this will keep breaking from time to time ...
<ndufresne>
Ely, but your right, as this is 8bit content, using bt709 is right
<ndufresne>
anyway, got a fix now, will post in 1.14 and master, I can provide you a patch for 1.12 i you need
<ndufresne>
Ely, btw, the number of reference frames is set to 25, is that hardcoded ?
<Ely>
ndufresne: Yes please a 1.12 patch would be nice if not too much trouble
<Ely>
Mmh how do you mean ? in the file ?
<ndufresne>
(it's a one liner ;-P)
<ndufresne>
Ely, well, I notice I have 25 reference (queued) frames in the encoder with that iphone video
<ndufresne>
that's insanely high, H264 should never require more then 16
<Ely>
I'm not sure I understand
<Ely>
Where do you get that number from ?
<Ely>
dmesg ? gst ?
<Ely>
Do you have 25 buffers queued in ?
* Ely
needs more information to compute
<ndufresne>
hmm, it's hard coded to 8, ok the decoder ignores some frames apparently
<ndufresne>
Ely, so gst request 3 capture buffer, the driver returns 8
<ndufresne>
then I queue 25 OUTPUT buffer before the first CAPTURE buffer comes out
<ndufresne>
in gst, it means the reference frames queue is at level 25
<Ely>
Are you doing the output/capture in an async fashion ?
<ndufresne>
oh I see, that's because of you ring buffer ...
<Ely>
in the driver both are decorrelated
<Ely>
you could queue 80 OUTPUT buffers if you want
<ndufresne>
well, probably not in 4K
<Ely>
:D
<ndufresne>
I think it's bad for drivers to do that
<ndufresne>
both chromium and gst will be sluggish, as the internal ref frame queues are designed with small numbers in mimnd
<ndufresne>
you'll also get huge CPU spike at start and over parsing effect
<ndufresne>
Ely, don't you have the DPB size ?
<Ely>
Well my driver just queues and DONEs the buffers as long as there is free space in the VIFIFO
<Ely>
There's nothing wrong with queuing up a lot of data
<Ely>
but I did enhance the way I do that in my latest commits
<ndufresne>
sure, I'm just saying it's super hard for userspace (which in byte-stream are not parsing the stream) to deal with the efficiently
<Ely>
instead of relying on a 24 count semaphore (which is probably why you see 25), I now rely on vififo available space
<ndufresne>
Ely, well, yes, accepting to queue a lot will have serious effect on seeking performance
<Ely>
mh I guess you're right, although this will have to be tested, the decoder is really fast so hopefully it should be negligible
<ndufresne>
this is of course never an issue in trully zero-copy drivers, so MFC and Venus don't have this side effect
<Ely>
otherwise I'll implement back a hard-fixed max buffers
<ndufresne>
the speed of the decoder does not matter, because the threads the dequeues is slowed down by the display
<ndufresne>
so you have an upstream thread the queue at full speed, and downstream thread the is slowed down by synchronization and rendering
<Ely>
but as soon as you seek, then you discard the "left" buffers that have a past timestamp right ?
<Ely>
like, the decoder will return you the 25 buffers it had queued, you discard them, and wait for proper pts
<ndufresne>
yes, but then, each time to start playing back after a seek, you have a huge and large CPU spike
<ndufresne>
instead of parsing a maximum of 16 frames in burst, you endup parsing as you said 80 (maybe the entire file)
<Ely>
because of parsing/copying the bitstream in OUTPUT buffers ?
<Ely>
I see
<ndufresne>
it's parsing and depayloading (which copied most of the time)
<Ely>
The biggest problem I've had so far is that I cannot block in a QBUF. If I do so, both ffmpeg / gst will freeze because somehow they also stop queuing back CAPTURE buffers (which are needed for proper decoder operation)
<ndufresne>
it's also a lot of useless disk IO when playing back from HD
<ndufresne>
Ely, no QBUF is only called after select/poll says it's ok to do so
<ndufresne>
so you don't really block QBUF, you will instead delay the current buffer DONE state until you have a CAPTURE buffer DONE state
<ndufresne>
the result is that when you DPB is full, then input/output runs in lock step, without wasting any resources
<ndufresne>
but for that you need a key information, which is the number of encoded buffers needed to produce first image
<ndufresne>
probably not urgent for now, but something to keep in mind
<Ely>
Not sure about that number, probably n + 1 where n is the number of packets in a PB block
<Ely>
anyway, I'll set a max input queue to 16 since it's the max number of consecutive bframes
<ndufresne>
that's true for H264, should be as you said N+1, so probably 17
<ndufresne>
in MFC, the firmware returns N + 1 already ;-P
<Ely>
MFC is for which chip ?
<ndufresne>
Samsung
<ndufresne>
it's the first M2M codec driver to have been upstreamed in linux
<ndufresne>
(v4l2 m2m)
<ndufresne>
the framework you use didn't exist back then, that's why they are not using it
<ndufresne>
Ely, looks like the image quality is good from this 4K stream
<ndufresne>
though, I got less then 1 fps ;-P
<ndufresne>
to get going, I believe that enabling the display overlay, with scaling and 4K would be very exciting ;-P
<ndufresne>
(or making an m2m driver around ge2d, that could do to ...)
<Ely>
I think we're getting close to a zero copy video pipeline
<Ely>
ge2d M2M would indeed be nice but I don't think it's planned for anytime soon
<Ely>
also we still don't know if we can keep nv12, or if we'll have to add the tiled pixfmt.. It'll all depend on what the display IP can process
Ntemis has joined #linux-amlogic
sputnik_ has joined #linux-amlogic
Ntemis has quit [Remote host closed the connection]
focus_ has quit [Quit: Leaving]
focus_ has joined #linux-amlogic
sputnik__ has joined #linux-amlogic
trem_ has quit [Quit: Leaving]
sputnik_ has quit [Ping timeout: 264 seconds]
sputnik_ has joined #linux-amlogic
sputnik__ has quit [Ping timeout: 260 seconds]
sputnik__ has joined #linux-amlogic
sputnik_ has quit [Ping timeout: 240 seconds]
sputnik_ has joined #linux-amlogic
sputnik__ has quit [Ping timeout: 256 seconds]
sputnik_ has quit [Remote host closed the connection]
sputnik_ has joined #linux-amlogic
sputnik__ has joined #linux-amlogic
sputnik_ has quit [Ping timeout: 260 seconds]
sputnik_ has joined #linux-amlogic
sputnik__ has quit [Ping timeout: 268 seconds]
sputnik__ has joined #linux-amlogic
sputnik_ has quit [Ping timeout: 260 seconds]
sputnik__ has quit [Remote host closed the connection]
sputnik__ has joined #linux-amlogic
mag has quit [Quit: Bye]
sputnik__ has quit [Read error: Connection reset by peer]
mag has joined #linux-amlogic
sputnik_ has joined #linux-amlogic
tingoose has quit [Ping timeout: 260 seconds]
tingoose_ has quit [Ping timeout: 276 seconds]
<ndufresne>
Ely, true, but I think we should keep it, and simply let userspace choose, a bit like CODA driver is doing
<ndufresne>
we could have I420, NV21, NV12 and tiled variants for all of these (even though, we will probably limit the amount of new formats, as it's a lot of work to document and all)
<ndufresne>
I wonder what VPP_POSTBLEND_ENABLE is ...