Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

USB superspeed peripherals

SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

I am facing error handling problem with FX3-based camera which leads to significant frame loss after an error occurs.

Camera configuration: 640 x 512, 16-bit, 50fps, ~262MBit/s = 32.7MB / s

FX3 configuration: USB 2.0 only, AN75779 based firmware, SDK version 1.3.4 and 1.3.5 (unofficial release)

After the host detects an error -71 it initiates a restart of the video stream, which results in a pause of about 1000 milliseconds between issuing  CY_U3P_USB_SC_CLEAR_FEATURE and CY_FX_UVC_STREAM_EVENT which leads to a loss of 50 frames. Most likely, such a pause in case of an error -71 is set by the software on the host side, which, unfortunately, I cannot influence.

Some FX3 UVC device debug UART logs ( timestamp format: seconds.milliseconds )

 

 

 

9390.335 - UVC: Completed 5149 frames and 2 buffers, DMA(Watermark) = 2
9391.335 - UVC: Completed 5199 frames and 2 buffers, DMA(Watermark) = 3
9391.644 - Clear feature request detected..., DMA(WM) = 2
9391.645 - Application Stopped
9391.645 - UVC: Completed 0 frames and 0 buffers, DMA(Watermark) = 2
9392.645 - UVC: Completed 0 frames and 0 buffers, DMA(Watermark) = 0
9392.648 - Application Started
9392.648 - UVC: Completed 0 frames and 0 buffers, DMA(Watermark) = 0
9393.648 - UVC: Completed 49 frames and 29 buffers, DMA(Watermark) = 2
9394.648 - UVC: Completed 99 frames and 30 buffers, DMA(Watermark) = 2
9395.648 - UVC: Completed 149 frames and 30 buffers, DMA(Watermark) = 2
9396.648 - UVC: Completed 199 frames and 30 buffers, DMA(Watermark) = 2
9397.648 - UVC: Completed 249 frames and 30 buffers, DMA(Watermark) = 2
9398.648 - UVC: Completed 299 frames and 30 buffers, DMA(Watermark) = 3

 

 

 

The goal is to reduce this pause as much as possible. I am thinking about strategy described in KBA231382 article with monitoring difference between commited but not consumed DMA buffers to avoid this error at all, but it looks like I didn't quite get the idea.

The first thing I would like to get is a number of DMA buffers already sent by FX3 but not yet received by host as a threshold criteria to temporarely stop producing more data from GPIF state machine:

 

 

 

static uint8_t DMABufferWatermark = 0;
static uint8_t DMABufferWatermark_Max = 0

void CyFxUvcApplnDmaCallback (
        CyU3PDmaMultiChannel *chHandle,
        CyU3PDmaCbType_t      type,
        CyU3PDmaCBInput_t    *input
        )
{
    CyU3PDmaBuffer_t    dmaBuffer;
    CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

    if (type == CY_U3P_DMA_CB_PROD_EVENT)
    {

#ifdef FRAME_TIMER_ENABLE
        /* Received data from the sensor so stop the frame timer */
        CyU3PTimerStop(&UvcTimer);

        /* Restart the frame timer so that we receive the next buffer before timer overflows */
        CyU3PTimerModify(&UvcTimer, glFrameTimerPeriod, 0);
        CyU3PTimerStart(&UvcTimer);
#endif

        /* There is a possibility that CyU3PDmaMultiChannelGetBuffer will return CY_U3P_ERROR_INVALID_SEQUENCE here.
         * In such a case, do nothing. We make up for this missed produce event by making repeated commit actions
         * in subsequent produce event callbacks.
         */
        status = CyU3PDmaMultiChannelGetBuffer (chHandle, &dmaBuffer, CYU3P_NO_WAIT);
        while (status == CY_U3P_SUCCESS)
        {
            /* Add Headers*/
            if (dmaBuffer.count == CY_FX_UVC_BUF_FULL_SIZE)
            {
                /* A full buffer indicates there is more data to go in this video frame. */
                CyFxUVCAddHeader (dmaBuffer.buffer - CY_FX_UVC_MAX_HEADER, CY_FX_UVC_HEADER_FRAME);
            }
            else
            {
                /* A partially filled buffer indicates the end of the ongoing video frame. */
                CyFxUVCAddHeader (dmaBuffer.buffer - CY_FX_UVC_MAX_HEADER, CY_FX_UVC_HEADER_EOF);

#ifdef DEBUG_PRINT_FRAME_COUNT
                glFrameCount++;
                glDmaDone = 0;
#endif
            }

            /* Commit Buffer to USB*/
            status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + CY_FX_UVC_MAX_HEADER), 0);
            if (status == CY_U3P_SUCCESS)
            {
            	DMABufferWatermark++;
#ifdef DEBUG_PRINT_FRAME_COUNT
                glDmaDone++;
#endif
            }
            else
            {
                if(glDmaResetFlag == CY_FX_UVC_DMA_RESET_EVENT_NOT_ACTIVE)
                {
                    glDmaResetFlag = CY_FX_UVC_DMA_RESET_COMMIT_BUFFER_FAILURE;
                    CyU3PEventSet(&glFxUVCEvent, CY_FX_UVC_DMA_RESET_EVENT, CYU3P_EVENT_OR);
                }
                break;
            }

            /* Check if any more buffers are ready to go, and commit them here. */
            status = CyU3PDmaMultiChannelGetBuffer (chHandle, &dmaBuffer, CYU3P_NO_WAIT);
        }
    }
    else if (type == CY_U3P_DMA_CB_CONS_EVENT)
    {
    	if (DMABufferWatermark > 0)
    		DMABufferWatermark--;
        streamingStarted = CyTrue;
        glCommitBufferFailureCount = 0;        /* Reset the counter after data is consumed by USB */
    }
	DMABufferWatermark_Max = max(DMABufferWatermark_Max, DMABufferWatermark);
}

void UVCAppThread_Entry (uint32_t input)
{
    for (;;)
    {
        apiRetStatus = CyU3PEventGet (&glFxUVCEvent, CY_FX_UVC_STREAM_ABORT_EVENT | CY_FX_UVC_STREAM_EVENT | CY_FX_UVC_DMA_RESET_EVENT | CY_FX_USB_SUSPEND_EVENT_HANDLER, CYU3P_EVENT_OR_CLEAR, &flag, LOOP_TIMEOUT);

        ...

        CyFxUVCAppDebugPrint(4, "UVC: Completed %d frames and %d buffers, DMA(Watermark) = %d", glFrameCount, (glDmaDone != 0) ? (glDmaDone - 1) : 0, DMABufferWatermark_Max);
        DMABufferWatermark_Max = 0;
    }
}

 

 

 

It seems like DMABufferWatermark_Max is around pretty normal 2..3 even in case of -71 error and never comes close to all 8 available DMA Buffers which I expect as a pre-error criterion.

How to correctly implement the recommendations given in the article KBA231382 in the software code? 

Regards,

Sergiy

0 Likes
1 Solution
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

I greatly appreciate your support,

It seems that this problem lies only on the hardware side. I ended up replacing the USB cables, adding a small ceramic capacitor (22uF) between the VUSB/GND pins on the camera side, and adding a script to reset the USB camera from Linux before running the capture application. Also i disabled USB autosuspend on Linux side.

After all these changes, the -71 error in the logs is no longer observed, at least for now

Best Regards,

Sergiy

View solution in original post

0 Likes
11 Replies
Rashi_Vatsa
Moderator
Moderator 500 solutions authored 1000 replies posted 750 replies posted
Moderator

Hello,

It seems like DMABufferWatermark_Max is around pretty normal 2..3 even in case of -71 error and never comes close to all 8 available DMA Buffers which I expect as a pre-error criterion.

>> The no. of buffers committed will depend on the DMA buffer count configured in the firmware. Please let me know the DMA buffer count and no. of  PIB sockets associated with the DMA channel.

As USB 2.0 is used, the maximum UVC throughput in bytes (frame height × frame width × pixel size ×
frames per second) should be less than 40-MBps ( mentioned in AN75779 app note)

Please let me know if you tried the below points mentioned in the KBA

"Do the following to avoid all DMA buffers from being filled up:

  1. Allocate the maximum possible memory to the DMA buffers.
  2. Configure horizontal blanking of the image sensor as high as possible (to do this vertical blanking can be reduced thereby maintaining same frame rate). While doing this make sure that vertical blanking does not go below ~350 us."
Regards,
Rashi
0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

The no. of buffers committed will depend on the DMA buffer count configured in the firmware. Please let me know the DMA buffer count and no. of  PIB sockets associated with the DMA channel.

 

<uvc.h>
/* UVC Video Streaming Endpoint Packet Size */
#define CY_FX_EP_BULK_VIDEO_PKT_SIZE    (0x400)         /* 1024 Bytes */

/* UVC Video Streaming Endpoint Packet Count */
#define CY_FX_EP_BULK_VIDEO_PKTS_COUNT  (0x10)          /* 16 packets (burst of 16) per DMA buffer. */

/* DMA buffer size used for video streaming. */
#define CY_FX_UVC_STREAM_BUF_SIZE       (CY_FX_EP_BULK_VIDEO_PKTS_COUNT * CY_FX_EP_BULK_VIDEO_PKT_SIZE)  /* 16 KB */

/* Number of DMA buffers per GPIF DMA thread. */
#define CY_FX_UVC_STREAM_BUF_COUNT      (4)

<uvc.c>
    /* Create a DMA Manual channel for sending the video data to the USB host. */
    dmaMultiConfig.size           = CY_FX_UVC_STREAM_BUF_SIZE;
    dmaMultiConfig.count          = CY_FX_UVC_STREAM_BUF_COUNT;
    dmaMultiConfig.validSckCount  = 2;
    dmaMultiConfig.prodSckId [0]  = (CyU3PDmaSocketId_t)CY_U3P_PIB_SOCKET_0;
    dmaMultiConfig.prodSckId [1]  = (CyU3PDmaSocketId_t)CY_U3P_PIB_SOCKET_1;
    dmaMultiConfig.consSckId [0]  = (CyU3PDmaSocketId_t)(CY_U3P_UIB_SOCKET_CONS_0 | CY_FX_EP_VIDEO_CONS_SOCKET);
    dmaMultiConfig.prodAvailCount = 0;
    dmaMultiConfig.prodHeader     = 12;                 /* 12 byte UVC header to be added. */
    dmaMultiConfig.prodFooter     = 4;                  /* 4 byte footer to compensate for the 12 byte header. */
    dmaMultiConfig.consHeader     = 0;
    dmaMultiConfig.dmaMode        = CY_U3P_DMA_MODE_BYTE;
    dmaMultiConfig.notification   = CY_U3P_DMA_CB_PROD_EVENT | CY_U3P_DMA_CB_CONS_EVENT;
    dmaMultiConfig.cb             = CyFxUvcApplnDmaCallback;

 

These buffers are left unchanged, as in the original example, with total DMA memory = 16Kb x 4 x 2 = 128Kb.

As USB 2.0 is used, the maximum UVC throughput in bytes (frame height × frame width × pixel size ×
frames per second) should be less than 40-MBps ( mentioned in AN75779 app note)

Yes, I've seen that mention. As for me, 32.7 MB/s definitely has a reasonable margin in relation to the ~40 MB/s maximum.

Please let me know if you tried the below points mentioned in the KBA

"Do the following to avoid all DMA buffers from being filled up:

1. Allocate the maximum possible memory to the DMA buffers.

Yes, but raising number of DMA buffers per GPIF DMA thread to 6 (total DMA memory 192Kb) changes nothing at all: error -71 still remains, and the variable DMABufferWatermark_Max still does not exceed 2-3.

2. Configure horizontal blanking of the image sensor as high as possible (to do this vertical blanking can be reduced thereby maintaining same frame rate). While doing this make sure that vertical blanking does not go below ~350 us."

Unforunatelly it is not possible to change sensor vertical / horizontal blanking  configuration with current sensor.

In any case, I'm still trying to find a way to prevent the host from issuing CY_U3P_USB_SC_CLEAR_FEATURE.

I am starting to think that root case of error -71 (0x43) appears on host side might not be an device buffer overrun but it might be something related to physical connection interference.

Does the -71 (0x43) error appearing on the host side always mean an FX3 DMA overrun, or could this error be related to a physical connection issue that the FX3 firmware can do little about?

Regards,

Sergiy

0 Likes
Rashi_Vatsa
Moderator
Moderator 500 solutions authored 1000 replies posted 750 replies posted
Moderator

Hello Sergiy,

Yes, but raising number of DMA buffers per GPIF DMA thread to 6 (total DMA memory 192Kb) changes nothing at all: error -71 still remains, and the variable DMABufferWatermark_Max still does not exceed 2-3.

>> Please confirm if you are using the default GPIF II state machine as in AN75779. If yes, please confirm if you tried  to increase the DMA buffer size to 32KB count 3. With this change, did you also changed the RX payload size in the probe control  structure and counter values in GPIF state machine?

Does the -71 (0x43) error appearing on the host side always mean an FX3 DMA overrun, or could this error be related to a physical connection issue that the FX3 firmware can do little about?

>> The two reasons through which 0x47 is received, is mentioned in the KBA pointed above

Regards,
Rashi
0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

My bad, -71 hex is 0x47, not 0x43

Please confirm if you are using the default GPIF II state machine as in AN75779.

Yes,  I only changed GPIF bus width to 16 bit with corresponding changes to DATA/ADDR_COUNT = 8183

If yes, please confirm if you tried  to increase the DMA buffer size to 32KB count 3. With this change, did you also changed the RX payload size in the probe control  structure and counter values in GPIF state machine?

Yes, i changed DMA buffer size to 32Kb with corresponding changes in glProbeCtrl20 and GPIF DATA/ADDR_COUNT modified to 16375. I am able to see video but i still can't get rid of sporadic error -71

If this error is caused by the device, and not by the host, I would like to find a way in the firmware to prevent this by  restarting  the GPIF state machine in advance with DMA buffer clearing, losing only 1-2 frames, instead of responding to the CLEAR_FEATURE issued by the host followed by restarting video capture with loss of 50 frames.

Regards,

Sergiy

0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

My bad, error -71 should has hex 0x47 of course

Please confirm if you are using the default GPIF II state machine as in AN75779. 

Yes, with 16-bit GPIF data bus width and ADDR/DATA_COUNT = 8183

If yes, please confirm if you tried  to increase the DMA buffer size to 32KB count 3. With this change, did you also changed the RX payload size in the probe control  structure and counter values in GPIF state machine?

I tested with DMA Buffer 32KB x 3 and glProbeCtrl20 changed accordingly, ADDR/DATA_COUNT = 16375. Video stream works same as before, with sporadic -71 error in host logs and CY_U3P_USB_SC_CLEAR_FEATURE request appears on device side.

How can I catch an event or a precondition for such an event in the firmware code, which is the cause of the -71 error? As I understand it, CLEAR_FEATURE is a consequence of the host's reaction to such an event.

Regards,

Sergiy

0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

My bad, error -71 hex is 0x47, not 0x43.

Please confirm if you are using the default GPIF II state machine as in AN75779. 

Yes, just changed the width of the GPIF bus to 16 bit with ADDR_COUNT /DATA_COUNT recalculations (8183 for 16-bit)

If yes, please confirm if you tried  to increase the DMA buffer size to 32KB count 3. With this change, did you also changed the RX payload size in the probe control  structure and counter values in GPIF state machine?

I changed DMA buffer size to 32KB x 3 with corresponding changes in glProbeCtrl  and changed GPIF counters ADDR/DATA_COUNT to 16375. I can see the video exactly the same as before, but the sporadically repeated -71 error remains.

Host application log:

May 17 04:58:50 sla-alip kernel: [ 3032.960305] uvcvideo: Non-zero status (-71) in video completion handler.
May 17 04:58:50 sla-alip VTNext: V4L2_BUF_FLAG_ERROR: need to restart capture
May 17 04:58:51 sla-alip VTNext: USB Capture PixelFormat YUYV Width 640 Height 512 BytesPerLine 1280

FX3 Device Debug UART:

1586.430 - UVC: Completed 1299 frames and 15 buffers, DMA(WM) = 2
1587.430 - UVC: Completed 1349 frames and 15 buffers, DMA(WM) = 2
1587.685 - Clear feature request detected..., DMA(WM) = 2
1587.685 - Application Stopped
1587.686 - UVC: Completed 0 frames and 0 buffers, DMA(WM) = 2
1588.686 - UVC: Completed 0 frames and 0 buffers, DMA(WM) = 0
1588.689 - Application Started
1588.689 - UVC: Completed 0 frames and 0 buffers, DMA(WM) = 0
1589.689 - UVC: Completed 49 frames and 14 buffers, DMA(WM) = 2

If this error is caused by the device and not the host, I would like to find in the firmware the reason for which it is called, and restart the GPIF State Machine and clear the DMA buffers in advance, losing only 1-2 frames.

That's why I added watermark variable  to log output and expected the difference between committed buffers and consumed buffers at the time of the error to be 12 (or 6, in the case of 32Kb DMA buffers). But for some reason this doesn't happen.

Instead, the host sends a CLEAR_FEATURE_REQUEST for some reason and restarts the video, resulting in a loss of 50 frames.

Simply put, the firmware still cannot detect the moment that precedes the occurrence of an error and prevent it, like it was advised in KBA231382.

Regards,

Sergiy

0 Likes
Rashi_Vatsa
Moderator
Moderator 500 solutions authored 1000 replies posted 750 replies posted
Moderator

Hello Sergiy,

Please let me know if you have any debug prints for the DMA overflow or commit buffer failure 0x47 on the firmware side.

I see -71 in host app debug logs, please let me know how is this information sent to the host. 

To debug the reason for clear feature from host, please share the USB traces using Wireshark (.pcap) and kindly confirm if CyU3DebugPrint or any other blocking API is not called in DMA callback.

Also, please register for GPIF (CyU3PPibRegisterCallback) and Endpoint event (CyU3PUsbRegisterEpEvtCallback) callback and share the UART debug prints.

 

Regards,
Rashi
0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

No, there is no UART prints on DMA callback

Unforunatelly i'm not able to provide USB sniffer files as this host device provided with pre-compiled linux kernel with no USBMON support enabled.

I'm not sure i've got your idea right, but i did following:

1. Speeded up Debug UART with uartConfig.baudRate = CY_U3P_UART_BAUDRATE_2M;

2. Registered CyU3PPibRegisterCallback and CyU3PUsbRegisterEpEvtCallback to capture all events:

 

CyU3PPibRegisterCallback (CyFxUvcAppPibCallback, 0xFFFFFFFF);
CyU3PUsbRegisterEpEvtCallback(CyFxAppIntEpCb, 0xFFFFFFFF, 0xFF, 0xFF);

static void CyFxAppIntEpCb(
		CyU3PUsbEpEvtType evType,
		CyU3PUSBSpeed_t  usbSpeed,
		uint8_t  ebNum)
{
	if ((ebNum != CY_FX_EP_BULK_VIDEO) && (evType != CYU3P_USBEP_NAK_EVT))
	    CyU3PDebugPrintTime(4, "EP:%02X,%d", ebNum, evType);
	else {
		CyU3PDebugPrintTime(4, "NAK");
	}
}
static void CyFxUvcAppPibCallback (
        CyU3PPibIntrType cbType,
        uint16_t cbArg)
{
    if ((cbType == CYU3P_PIB_INTR_ERROR) && ((cbArg == 0x1005) || (cbArg == 0x1006)))
    {
        if (!back_flow_detected)
        {
            CyU3PDebugPrintTime (4, "Backflow detected..., DWM=%d", DMAWatermark_Max);
            back_flow_detected = 1;
        }
    }
    CyU3PDebugPrintTime (4, "PIB:Typ %d,%02X", cbType, cbArg);
}

 

Debug log output:

 

123.609 - NAK
123.609 - NAK
123.609 - NAK
123.610 - NAK
123.610 - NAK
123.611 - NAK
123.612 - NAK
123.613 - NAK
123.613 - NAK
123.614 - PIB:Typ 4,1012
123.615 - Clear feature request detected..., , DWM=3
123.615 - PIB:Typ 4,1012
123.615 - PIB:Typ 4,101A
123.616 - Application Stopped
123.616 - UVC: Completed 0 frames and 0 buffers, DWM=3
124.616 - UVC: Completed 0 frames and 0 buffers, DWM=0
124.619 - Application Started
124.619 - UVC: Completed 0 frames and 0 buffers, DWM=0

 

Position of PIB event could be random relatively to Clear Feature. DWM variable is my attempt to monitor the maximum number of occupied DMA buffers according to KBA231382 / Scenario 2. CyU3PDebugPrintTime is my wrapper for CyU3PDebugPrint which adds time in Seconds.Milliseconds format.

Anyway, it seems like this issue is more hadware-oriented, since the following two actions significantly reduce the number of errors:

1. Adding 100uF capacitor between +5V and GND close to USB connector pins on device side.

2. Reseting USB device and disabling autosuspend before capture application startup :

UVC_CAMERA=/sys/bus/usb/devices/usb1
sudo sh -c "echo -1 > /sys/module/usbcore/parameters/autosuspend"
sudo sh -c "echo 0 > $UVC_CAMERA/authorized"
sleep 0.5
sudo sh -c "echo 1 > $UVC_CAMERA/authorized"

We will try to make more changes to the camera power circuit, maybe we missed something.

Thanks,

Sergiy

0 Likes
Rashi_Vatsa
Moderator
Moderator 500 solutions authored 1000 replies posted 750 replies posted
Moderator

Hello Sergiy,

From the UART logs shared, I can see that there are multiple NAK events before the clear feature. It seems there is some issue with the USB interface. 

Also, as you mentioned that doing some changes on hardware improved the issue, the issue could be due to bad USB link. 

You can refer to AN70707 and refer to the Schematic and layout checklist

Regards,
Rashi
0 Likes
SeAn_4634356
Level 2
First solution authored 5 sign-ins 5 replies posted
Level 2

Hello Rashi,

I greatly appreciate your support,

It seems that this problem lies only on the hardware side. I ended up replacing the USB cables, adding a small ceramic capacitor (22uF) between the VUSB/GND pins on the camera side, and adding a script to reset the USB camera from Linux before running the capture application. Also i disabled USB autosuspend on Linux side.

After all these changes, the -71 error in the logs is no longer observed, at least for now

Best Regards,

Sergiy

0 Likes
Rashi_Vatsa
Moderator
Moderator 500 solutions authored 1000 replies posted 750 replies posted
Moderator

Hi Sergiy,

Glad to hear that the issue is resolved!

 

Regards,
Rashi
0 Likes