How to create a faked line in start of every video frame in CX3 firmware

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Anonymous
Not applicable

Assuming I have a 1920x1080 camera sensor connected to CX3, could it be possible to have a 1920x1081 UVC video streaming interface reported to PC by creating a faked line in start of every video frame in CX3 firmware?

Apart from modifying VS Frame Descriptor, what else should I do in CX3 firmware to make it work? Many thanks!

const uint8_t CyCx3USBSSConfigDscr[] =

{

          :

    /* Class specific Uncompressed VS Frame Descriptor 1 - 1080p@30fps */

    0x1E,                               /* Descriptor size */

    CX3_CS_INTRFC_DESCR,                /* Descriptor type*/

    0x05,                               /* Subtype: Uncompressed frame interface*/

    0x01,                               /* Frame Descriptor Index: 1 */

    0x00,                               /* No Still image capture supported */

    0x80, 0x07,                         /* Width in pixel:  1920 */

    0x38, 0x39, 0x04,                         /* Height in pixel: 1080 1081 */

    0x00, 0x80, 0x53, 0x90, 0x61, 0x3B,             /* Min bit rate (bits/s): 1080 1081 x 1920 x 2 x 30 x 8 = 995328000 996249600*/

    0x00, 0x80, 0x53, 0x90, 0x61, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

    0x00, 0x48, 0x57, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1080 1081 x 2 */

    0x15, 0x16, 0x05, 0x00,             /* Default frame interval (in 100ns units): (1/30)x10^7 */

    0x01,                               /* Frame interval type : No of discrete intervals */

    0x15, 0x16, 0x05, 0x00,             /* Frame interval 3: Same as Default frame interval */

          :

}

0 Likes
1 Solution

Gary,

Yes, you have to adjust the DMA Buffer size that meets your requirement - dmaBuffer.size > dmaBuffer.count + sizeof (fake_line). This is the only working case for you.

In other case ( dmaBuffer.size < dmaBuffer.count + sizeof (fake_line)), buffer over flow occurs. Therefore, avoid this case.

And in dmaBuffer.size = dmaBuffer.count + sizeof (fake_line) case, the host does not know when the frame ends for particular resolutions.

Becasue host expects a partial packet (less than 1024 bytes) to know that the transfer has ended. If the total frame including the fake line is exactly multiple of 1024, the host does not know the end of transfer. Hence, avoid this case too.

View solution in original post

0 Likes
8 Replies
Anonymous
Not applicable

Hi

Yes it is possible.

Please do similar modifications in the glProbeCtrl.

In the DMA Callback you can add the line, just like how headers are added in the firmware. Or you can send the header as a separate buffer to the USB, the fake line in a next separate buffer to the USB and the followed by the actual video data.

Regards,

- Madhu

0 Likes
Anonymous
Not applicable

I've made some changes in the following 3 places as below.

  1. VS Frame Descriptor
  2. glProbeCtrl
  3. DMA call back

However it doesn't work. I see lots of DMA reset events caused by the function call, CyU3PDmaMultiChannelCommitBuffer(), to commit the actual video data to USB.

Any hints on that? Thanks!

const uint8_t CyCx3USBSSConfigDscr[] =

{

          :

    /* Class specific Uncompressed VS Frame Descriptor 1 - 1081p@30fps */

    #if 0

    0x38, 0x04,                                 /* Height in pixel: 1080 */

    0x00, 0x80, 0x53, 0x3B,             /* Min bit rate (bits/s): 1080 x 1920 x 2 x 30 x 8 = 995328000 */

    0x00, 0x80, 0x53, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

    0x00, 0x48, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1080 x 2 */

    #else

    0x39, 0x04,                                 /* Height in pixel: 1081 */

   0x00, 0x90, 0x61, 0x3B,             /* Min bit rate (bits/s): 1081 x 1920 x 2 x 30 x 8 = 996249600 */

    0x00, 0x90, 0x61, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

    0x00, 0x57, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1081 x 2 */

    #endif

          :

};

/* UVC Probe Control Setting - 1080p@30FPS */

uint8_t const gl1080pProbeCtrl[CX3_APP_MAX_PROBE_SETTING] = {

          :

    //0x00, 0x48, 0x3F, 0x00,             /* Max video frame size in bytes = 1920 x 1080 x 2 */

    0x00, 0x57, 0x3F, 0x00,             /* Max video frame size in bytes = 1920 x 1081 x 2 */

          :

};

#define LINE_LENGTH (1920*2)

uint8_t gfakeLine[LINE_LENGTH], gbackupLine[LINE_LENGTH];

/* DMA callback function to handle the produce and consume events. */

void

CyCx3AppDmaCallback (

        CyU3PDmaMultiChannel *chHandle,

        CyU3PDmaCbType_t      type,

        CyU3PDmaCBInput_t    *input)

{

    CyU3PDmaBuffer_t dmaBuffer;

    CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

    if (type == CY_U3P_DMA_CB_PROD_EVENT)

    {

        /* This is a produce event notification to the CPU. This notification is

         * received upon reception of every buffer. The buffer will not be sent

         * out unless it is explicitly committed. The call shall fail if there

         * is a bus reset / usb disconnect or if there is any application error. */

        status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

        while (status == CY_U3P_SUCCESS)

        {

            /* Add Headers*/

            if (dmaBuffer.count < CX3_APP_DATA_BUF_SIZE)

            {

                CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_EOF);

                gIsSOF = CyTrue;

            }

            else

            {

                /* Start of Frame */

                if (gIsSOF)

                {

                    CyU3PMemSet(gfakeLine, 0x0, LINE_LENGTH);

                   CyU3PMemCopy(gbackupLine, dmaBuffer.buffer, LINE_LENGTH);   /* Backup the 1st line */

                    CyU3PMemCopy(dmaBuffer.buffer, gfakeLine, LINE_LENGTH);       /* Copy the faked line to DMA buffer */

                    CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);                  

                   

                   /* Commit the faked line to USB */

                    status = CyU3PDmaMultiChannelCommitBuffer (chHandle, LINE_LENGTH + 12, 0);

                    if (status != CY_U3P_SUCCESS)

                    {

                        CyU3PDebugPrint (4, "Commit faked line failed, status 0x%x \n\r", status);

                    }

                    /* Restore the original first line to DMA buffer */    

                    CyU3PMemCopy(dmaBuffer.buffer, gbackupLine, LINE_LENGTH);

                    gIsSOF = CyFalse;

                }            

                CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);

            }

            /* Commit Buffer to USB */           

            status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);

            if (status != CY_U3P_SUCCESS)

            {

                CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT, CYU3P_EVENT_OR);

                break;

            }

            else

            {

                glDmaDone++;

            }

            status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

        }

    }

           :

           :

}

0 Likes

Gary,

Can you please check whether the first commit buffer is failed? i.e. fake line commitbuffer. I guess it should commit successfully. Please confirm.

Then the second commitbuffer (actuall video data buffer) is failing.

Because, you can not do commitbuffer on the same buffer twice.

0 Likes
Anonymous
Not applicable

You're right. The first commit is OK, however the second commit is failing.

So the question becomes how could I send something extra (i.e. the faked line) to the consumer?

Which APIs should I call? Thanks!

0 Likes

Hi,

Here in this firmware you are adding fake line at the starting of the frame. You can add a line at the end of the frame as we have partial buffer there. Is it fine for your application?

Thanks & regards

Abhinav

0 Likes
Anonymous
Not applicable

Adding the fake line at the end of the frame is also fine for my application.

In theory, this approach works for Scenario 1, but it might not work for Scenario 2.

  • Scenario 1: If dmaBuffer.size >= dmaBuffer.count + sizeof (fake_line)
  • Scenario 2: If dmaBuffer.size < dmaBuffer.count + sizeof (fake_line)

I tried that with CX3 RDK and got the following results.

  • Resolution 640x481: Streaming OK, dmaBuffer.size = 24560, dmaBuffer.count = 400, sizeof(fake_line) = 640*2
  • Resolution 1920x1081: Streaming NG, dmaBuffer.size = 24560, dmaBuffer.count = 21120, sizeof(fake_line) = 1920*2

So it looks this solution depends on the resolution. May I have your suggestions for Scenario 2?

Perhaps one possible solution for Scenario 2 is to adjust DMA buffer size?

/* DMA callback function to handle the produce and consume events. */

void

CyCx3AppDmaCallback (

        CyU3PDmaMultiChannel *chHandle,

        CyU3PDmaCbType_t      type,

        CyU3PDmaCBInput_t    *input)

{

    CyU3PDmaBuffer_t dmaBuffer;

    CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

    if (type == CY_U3P_DMA_CB_PROD_EVENT)

    {

        /* This is a produce event notification to the CPU. This notification is

        * received upon reception of every buffer. The buffer will not be sent

        * out unless it is explicitly committed. The call shall fail if there

        * is a bus reset / usb disconnect or if there is any application error. */

        status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

        while (status == CY_U3P_SUCCESS)

        {

            /* Add Headers*/

            if (dmaBuffer.count < CX3_APP_DATA_BUF_SIZE)

            {

                CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_EOF);              

                CyU3PMemSet(dmaBuffer.buffer, 0xff, FAKE_LINE_LENGTH);

                dmaBuffer.count += FAKE_LINE_LENGTH;

           }

            else

            {

                CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);

            }

            /* Commit Buffer to USB */          

            status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);

            if (status != CY_U3P_SUCCESS)

            {

                CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT, CYU3P_EVENT_OR);

                break;

            }

            else

            {

                glDmaDone++;

            }

            status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

        }

    }

                     :

}

0 Likes

Gary,

Yes, you have to adjust the DMA Buffer size that meets your requirement - dmaBuffer.size > dmaBuffer.count + sizeof (fake_line). This is the only working case for you.

In other case ( dmaBuffer.size < dmaBuffer.count + sizeof (fake_line)), buffer over flow occurs. Therefore, avoid this case.

And in dmaBuffer.size = dmaBuffer.count + sizeof (fake_line) case, the host does not know when the frame ends for particular resolutions.

Becasue host expects a partial packet (less than 1024 bytes) to know that the transfer has ended. If the total frame including the fake line is exactly multiple of 1024, the host does not know the end of transfer. Hence, avoid this case too.

0 Likes
Anonymous
Not applicable

Thanks for all the comments. It's quite helpful!

0 Likes