AN75779 get nv12 stream

[zjjnsst]

We are trying to add NV12 format for our USB camera. 

We use 32bits gpif and transfer 1080p image.

But we can't get correct stream.

Can we transfer data from FPGA to FX3 using GPIF II interface like this?

[zjjnsst]

I modified the usb descriptor.

#ifdef YUY2

        0x59,0x55,0x59,0x32,            /* GUID used to identify streaming-encoding format: YUY2  */

        0x00,0x00,0x10,0x00,

        0x80,0x00,0x00,0xAA,

        0x00,0x38,0x9B,0x71,

        0x10,                           /* Number of bits per pixel used to specify color in the decoded video frame.

                                           0 if not applicable: 16 bit per pixel */

#endif

#ifdef NV12

        0x4E,0x56,0x31,0x32,            /* GUID used to identify streaming-encoding format: NV12  */

        0x00,0x00,0x10,0x00,

        0x80,0x00,0x00,0xAA,

        0x00,0x38,0x9B,0x71,

        0x0c,                           /* Number of bits per pixel used to specify color in the decoded video frame.

                                           0 if not applicable: 12 bit per pixel */

#endif

[JayakrishnaT_76]

Hello,

Based on my understanding, the FPGA provides a 32 bit data to FX3 and FX3 samples it. Zero padding is not done on the incoming data. Please correct me if my understanding is wrong.

I was not able to understand 1040Y and 540UV in the snapshot that you shared initially. Does this mean that the initial lines will contain the Y component of the frame and the later lines will contain the UV components? Please elaborate on this.

Also, when you say that you were not able to get the correct stream, do you mean to say that you are able to get some video stream but it is not recovered correctly by the host application? Or did you mean to say that the host application shows black screen?

Please elaborate the problem that you are facing. Also, please let me know the OS used for testing.

[zjjnsst]

Thanks for your reply.

1. Your understanding is correct.

2. I mean the first 1080 lines contain the Y component of the frame and the next 540 lines contain the UV components.

3. I'm able to get some video stream but it is not recovered correctly by the host application.

4. The OS we used is win10.

[JayakrishnaT_76]

Hello,

I am confused with the following statement in your previous response:

"I mean the first 1080 lines contain the Y component of the frame and the next 540 lines contain the UV components"

Based on your initial description, the number of lines for your video frame is 1080. So, I believe that few out of this 1080 will transfer the Y components and the remaining lines will transfer UV components.. Please correct me if my understanding is wrong.

Also, please let me know the host application used for video streaming. Is it a custom host application or is it a standard UVC application? 

[zjjnsst]

The number of lines for my video frame is 1620(1080Y+540UV).

The host application I used is potplayer.

[JayakrishnaT_76]

Hello,

Can you please share the following:

1. Resolution of video frame. I understand that V-Active (in Lines) is 1620. What exactly is the H-Active.

2. Is the FPGA processing the video data and then feeding it to the FX3 as 32 bits? If not, then please let me know how this is done.

3. As you are transferring the Y data together followed by the UV data, FX3 will store the data into the DMA buffers in the order in which the data is received. This means that the data will go out of FX3 as 1080Y lines followed by 540UV lines. So, ideally the host application should not be able to recover the original video stream. Can you please try streaming the data using Windows camera application? We should be seeing the same issue with this application too. 
You need to process the data received at the host side to recover the original video frame. 

[zjjnsst]

1. The H-Active is 1920.

2. Yes, the FPGA recive rgb bayer data from the sensor and then convert it to NV12 data and then feeding it to the FX3 as 32 bits. We have successfully transferred YUV2 data like this.

3. We have used potplayer to recover the nv12 stream of other cameras before. I think the problem is with the stream of the camera.

[JayakrishnaT_76 · Accepted solution]

Hello,

As you might be knowing, FX3 is just a bridge between the FPGA and the host. It accepts data from the FPGA as it is, stores it in the internal DMA buffers by adding UVC header, and transfers it to the host. So, it is unlikely that FX3 is causing the issue here.

Can you please provide details on how the FPGA convert the incoming video frame into 32 bits?

Please try the following to debug this issue further:

1. Configure the FPGA to send a test pattern to FX3

2. Capture one video frame at the host side using Potplayer

3. Check the difference between the frame received at the host and the frame sent by the FPGA.

[zjjnsst]

Yes I know, this problem may be caused by fpga.

As the snapshot shows, it takes 480 periods for the fpga to transmit a line of data(1920 pixels).Instead of converting 12-bit data to 32-bit, we just use the 32-bit interface to transmit 12-bit data.

Thanks for your suggestions, we will try to debug this issue.

[JayakrishnaT_76]

Hello,

Than you for the update. Please let us know if you have any more questions on this issue.