Alternatively read two camera, one would stuck

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
rossihwang
Level 3
Level 3
5 questions asked 50 sign-ins 25 replies posted
  • Hardware

    • host: tinker board 2s(rk3399),kernel 5.13

    • slave: sg130gs(1280*720, YUYV, 30fps) + fpga(spartan 6) + cyusb3014, isochronous mode

  • Problem

    • Run this script on host

      #!/bin/bash

      while :
      do
            yavta -c30 /dev/video0
            sleep 2
            yavta -c30 /dev/video1
            sleep 2
      done
    • After random time(from minutes to hours), one of these cameras would be stuck

    • USB packets captured by tcpdump indicate that the host keeps polling(isochronous) the slave while the slave keeps responding with a data packet with zero-length

    • After this, if I restart the test script, the device that is stuck will be missed

  • some trials on other settings also failed

    • ov5640(1280*720, YUYV,30fps) + cyusb3014, bulk mode

    • sg130gs(1280*720, YUYV, 30fps) + fpga(spartan 6) + cyusb3014, bulk mode

  • In addition, test with a single camera is fine

Any ideas? thanks

0 Likes
1 Solution

Hello,

Thank you for confirming that the issue was not seen when CyU3PSysEnterSuspendMode was commented out from the firmware.

Please note that commenting out the CyU3PSysEnterSuspendMode will not allow the device to go to low power mode when USB host sends a suspend. The device will come out of the suspend state only when there is some activity on bus (CY_U3P_SYS_USB_BUS_ACTVTY_WAKEUP_SRC). Commenting out the CyU3PSysEnterSuspendMode will not cause any harm.

In addition to this, we do not recommend to call CyU3PSysEnterSuspendMode when JTAG debugging is done. Please refer to this KBA https://community.infineon.com/t5/Knowledge-Base-Articles/Segger-J-Link-Known-Issues-with-FX3-CX3-FX...

To check how is CyU3PSysEnterSuspendMode  affecting, please  share the UART prints for the following conditions:

- CYUSB3KIT-003 - CyU3PSysEnterSuspendMode (not commented off) - JTAG debugging not done

- CYUSB3KIT-003 - CyU3PSysEnterSuspendMode (commented off) - JTAG debugging not done

Regards,
Rashi

View solution in original post

0 Likes
26 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Please let me know if both the cameras are connected to the host on two different USB ports or via USB hub.

Can you please share the UART debug prints from both the cameras for us to check how many DMA buffer are available with FX3 when the issue is seen. We will debug the the issue with bulk transfer first.

Also, is the issue is always seen when a camera is connected to particular USB port or is it random?

Kindly let me know if reducing the video resolution on both sensors solves the problem and which host controller is used by the host?

Regards,
Rashi
0 Likes

hello Rashi

>> Please let me know if both the cameras are connected to the host on two different USB ports or via USB hub.

 

USB device tree 

root@galaxis-dev:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/1p, 12M
   |__ Port 1: Dev 2, If 0, Class=Wireless, Driver=btusb, 12M
   |__ Port 1: Dev 2, If 1, Class=Wireless, Driver=btusb, 12M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/1p, 12M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
   |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
       |__ Port 1: Dev 36, If 1, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 1: Dev 36, If 0, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 3: Dev 35, If 0, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 3: Dev 35, If 1, Class=Video, Driver=uvcvideo, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
   |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

---------------------------------------------------------------------------------------------------------------------

>> Can you please share the UART debug prints from both the cameras for us to check how many DMA buffer are available with FX3 when the issue is seen. We will debug the the issue with bulk transfer first.

 

Should I keep the default log  output or need to enable "DEBUG_PRINT_FRAME_COUNT" or other else?

---------------------------------------------------------------------------------------------------------------------

>> Also, is the issue is always seen when a camera is connected to particular USB port or is it random?

 

No, I used different hardware connection settings(but all on the same bus, as shown above)

---------------------------------------------------------------------------------------------------------------------

>> Kindly let me know if reducing the video resolution on both sensors solves the problem and which host controller is used by the host?

 

Currently, only support one resolution,  we may add support and test for this later

the USB hardware information is as below

 

# lsusb -v 2>/dev/null | grep "^Bus\|iSerial"

Bus 002 Device 035: ID 04b4:00f9 Cypress Semiconductor Corp.  
 iSerial                 0  
Bus 002 Device 036: ID 04b4:00f9 Cypress Semiconductor Corp. USB3.2 Hub
 iSerial                 0  
Bus 002 Device 002: ID 05e3:0620 Genesys Logic, Inc. USB3.2 Hub
 iSerial                 0  
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 iSerial                 1 xhci-hcd.1.auto

 

# lsusb -t

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
   |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
       |__ Port 1: Dev 36, If 1, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 1: Dev 36, If 0, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 3: Dev 35, If 0, Class=Video, Driver=uvcvideo, 5000M
       |__ Port 3: Dev 35, If 1, Class=Video, Driver=uvcvideo, 5000M

0 Likes

Hello,

Please find my comments below

Should I keep the default log  output or need to enable "DEBUG_PRINT_FRAME_COUNT" or other else?

>> Yes, please enable DEBUG_PRINT_FRAME_COUNT and FRAME_TIMER_ENABLE and share the UART debug prints

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hello

please check the attachment

In addition, I reduced LOOP_TIMEOUT to 100 for more outputs

 

0 Likes

Hello,

Please confirm if video6_ttyUSB3_log is taken simultaneously with script_log, as I can see difference in both the logs. Please capture UART debug prints and host app logs simultaneously.

 I don't see that "Application Started" is called and then "Application Stopped" is not called. If the video frame is not sent or gets stuck in between  then it should be seen in UART debug prints as well. But u the UART debug prints every time 30 frames are sent without any issues. 

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hi Rashi

  • Surely, these logs are captured simultaneously, the stuck state is after "SUSPEND".

  • And I captured some USB packets by the tcpdump, which indicates that after the host SET_INTERFACE(bAlternateSetting = 1)(packet NO. 99), it repeatedly send ISOCHRONOUS request while the device responses with empty data, please check the attachment for the full data. Note that these packets are correspondent to the test above, but I found the behavior is the same in all tests.

  • I also upload a tcpdump packet of normal state for reference. 

Thanks

0 Likes

Hi,

I am not able to open the files shared.

Can you please share the files in .txt for us to check

Regards,
Rashi
0 Likes

Hi

You can open it with wireshark https://www.wireshark.org/

0 Likes

Hi,

I tried doing that but it says that the trace is cut short in middle of packets.

Can you please provide complete traces

Regards,
Rashi
0 Likes

Hi

this warning doesn't matter, the usb.pcap0 is the successive data of usb.pcap1. I think the cause of this warning is the wireshark can't find the start of the first transaction in usb.pcap0.

0 Likes
lock attach
Attachments are accessible only for community members.

Hi, Rashi

I did another test last weekend, the only modification is changing the delay 2s to 5s. The result is a bit different as the stuck camera was not suspended, and repeatly commit an empty DMA buffer(As your initial expectation?). Please check out the attachment for the details

 

I wonder that in this case, is it the cause of the frontend of the cyusb3014, say the sensor has no data output? Is it possible to reset the sensor to recovery from this situation?

 

Thanks

0 Likes

Hello,

I understand that the devices are directly connected to the USB host and not through the USB hub. Is my understanding correct?

From the traces, I understand that the application is in stopped state. 

In the traces, I could see the following:

Host sends SET_ INTERFACE (Alternate setting 0) > this will lead to application stop by calling CyFxUvcApplnStop ()

Then host sends SET_CUR request (COMMIT Control) > This will lead to setting the sensor for particular resolution and then calling CyFxUvcApplnStart()

Again, host sends SET_ INTERFACE (Alternate setting 1), which will check if the application is started, it will call CyFxUvcApplnStop

To confirm this, please check if this code is added in your firmware

case CY_U3P_USB_EVENT_SETINTF:
/* Stop the application before restarting. */
if (glIsApplnActive)
{
CyFxUvcApplnStop ();
}

/*  if alternate setting 1 has been selected. */
if (evdata == 0x0001)
CyFxUvcApplnStart();
break;

 

So, SET_CUR request and SET_INTERFACE requests back to back puts the firmware in CyFxUvcApplnStop  where the streaming is stopped and hence the ZLP are sent on the USB

Regards,
Rashi
0 Likes

Hi

1. These two cameras are directly connected to the host.

2. I think the original code has done well, say

- Call CyFxUvcApplnStop() on SET_INTERFACE(0) asynchronously by 

CY_FX_UVC_STREAM_ABORT_EVENT

- Call CyFxUvcApplnStart() on SET_CUR request (COMMIT Control) asynchronously by CY_FX_UVC_STREAM_EVENT

- But no explicit handle for SET_INTERFACE(1), cause already call CyFxUvcApplnStart() beforehand

- Personaly, Putting CyFxUvcApplnStop, CyFxUvcApplnStart(large code block) into 

CyFxUVCApplnUSBEventCB callback function violates the initial code design. I would like to modify as below to meet the "back to back", how do think of it?
 
```
void UVCAppThread_Entry(...) {
 
...
 
for (;;)
{
apiRetStatus = CyU3PEventGet (&glFxUVCEvent, CY_FX_UVC_STREAM_ABORT_EVENT | CY_FX_UVC_STREAM_EVENT |
CY_FX_UVC_DMA_RESET_EVENT | CY_FX_USB_SUSPEND_EVENT_HANDLER, CYU3P_EVENT_OR_CLEAR, &flag, LOOP_TIMEOUT);

if (apiRetStatus == CY_U3P_SUCCESS)
{
/* Request to start video stream. */
if ((flag & CY_FX_UVC_STREAM_EVENT) != 0)
{
if (glIsApplnActive == CyTrue) {
CyFxUvcApplnStop();
}
glIsApplnActive = CyTrue;
CyFxUvcApplnStart();
}
 
...
 
}
 

Regards

0 Likes

Hello,

- But no explicit handle for SET_INTERFACE(1), cause already call CyFxUvcApplnStart() beforehand

>> Please let me know how is  this is being handled in your firmware.

I would like to modify as below to meet the "back to back", how do think of it?

>> You mean you want to use CY_FX_UVC_STREAM_EVENT to start the stream instead of directly calling CyFxUvcApplnStart. If yes, that can be done.

Please share your firmware for us to check (sensor settings not required)

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hi

- It's merely a guess of mine, the SET_CUR_REQ(video streaming control) is always followed by a SET_INTERFACE(1), thus is no need to handle the latter.

- Please check the attachment. In addition, this version of the modification is still with the same problem

Regards

0 Likes

Hello,

My assumption that SET_INTERFACE(1) is handled in the firmware by starting the streaming was wrong. Sorry for that

From the firmware, I see that the SET_INTERFACE(1) is not handled in the firmware but we see that Application Stopped is called at the end. 

We need to know from where is Application Stopped is called. From the video6.log, following is the sequence

Application Started >> SET_CUR request from host

Application Stopped >> Alternate setting (0)

Application Started >> SET_CUR request from host

Application Stopped >> Reason not known (no debug prints from where is it called)

From the firmware shared, I could see that before calling CyFxUVCApplnAbortHandler (Application Stopped) debug prints are added. There is one more instance Application Stopped is called but no debug prints are added in firmware i.e. in CY_FX_UVC_STREAM_EVENT handler when glIsApplnActive == CyTrue

Can you please confirm from which instance is the last Application Stopped called by putting debug prints

Regards,
Rashi
0 Likes

Hi

This part of the code

rossihwang_0-1635405162860.png

is nothing to do with the video6.log, cause I just add this code for the "back to back":)

However, it may call stop here more likely

rossihwang_0-1635405505568.png

 

Regards

 

0 Likes

Hello,

Please let me know if FRAME_TIMER_ENABLE is enabled or not.

If yes, then the debug prints should be seen in the logs. If not, please try to increase the timer period to time much greater than the V_Total time such that the timer doesn't times out and also enable FRAME_TIMER_ENABLE

Regards,
Rashi
0 Likes

Hi

Sorry for my late reply

- The FRAME_TIMER_ENABLE is enabled

- For the V_Total, do you mean the glFrameTimerPeriod? I am gonna increase it to 800ms, Is it ok?

Regards

0 Likes

Hello,

- For the V_Total, do you mean the glFrameTimerPeriod? I am gonna increase it to 800ms, Is it ok?

>> V_total = 1/FPS. Yes, I think 800ms might be enough

Regards,
Rashi
0 Likes

Hi

After the test, stuck still exists.

In addition, Did you do the same test on your side? Is this problem exist on your hardware as well? 

Regards

0 Likes

Hello,

No we haven't tested streaming through two cameras with the test environment that you have shared.

To check if the issue is not with the hardware, can you test the same hardware (with only one camera which gets stuck) with different host PC and let me know if that works or try connecting the same 2 camera on different host pc

Also, as suggested before, please try using a lower resolution when both cameras are connected to host.

Regards,
Rashi
0 Likes

Hi

I reduced the resolution to 640x480, still the same problem.

In addition, I wonder that what will happen if I declare 1280x720 resolution in USB description, but actually transmit 1280x1024 data on 3014?

0 Likes

Hello,

Thank you for the update with the following hardware

Current tested hardware: sc130gs -> cyusb3kit_003 instead of  the hardware in the old thread: sc130gs -> some Xilinx FPGA(custom board) -> 3014(custom board)

From the test with the same host application , the issue is reproducible with Superspeed Explorer kit (CYUSB3KIT-003)

Please let me know if the same host "host: tinker board 2s(rk3399),kernel 5.1" is used for this test.

If yes, can you please share the UART debug prints and USB traces with CYUSB3KIT-003. Please share the UART prints from both the devices (connected to the USB host).

This is because from the v4l2 traces shared by you, I could see that device reset is called by the host.

Regards,
Rashi
0 Likes

Hello,

Thank you for confirming that the issue was not seen when CyU3PSysEnterSuspendMode was commented out from the firmware.

Please note that commenting out the CyU3PSysEnterSuspendMode will not allow the device to go to low power mode when USB host sends a suspend. The device will come out of the suspend state only when there is some activity on bus (CY_U3P_SYS_USB_BUS_ACTVTY_WAKEUP_SRC). Commenting out the CyU3PSysEnterSuspendMode will not cause any harm.

In addition to this, we do not recommend to call CyU3PSysEnterSuspendMode when JTAG debugging is done. Please refer to this KBA https://community.infineon.com/t5/Knowledge-Base-Articles/Segger-J-Link-Known-Issues-with-FX3-CX3-FX...

To check how is CyU3PSysEnterSuspendMode  affecting, please  share the UART prints for the following conditions:

- CYUSB3KIT-003 - CyU3PSysEnterSuspendMode (not commented off) - JTAG debugging not done

- CYUSB3KIT-003 - CyU3PSysEnterSuspendMode (commented off) - JTAG debugging not done

Regards,
Rashi
0 Likes
rossihwang
Level 3
Level 3
5 questions asked 50 sign-ins 25 replies posted

Any updates?

0 Likes