Bulk In data streaming hangs on CYUSB3KIT-001 DevKit connected to USB host controller ASMedia ASM3142

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Sergey
Level 3
Level 3
25 sign-ins 25 replies posted 10 replies posted

Hello,

We are facing an issue, that bulk IN data streaming hangs on CYUSB3KIT-001 DevKit (and other our devices base on FX3) connected to USB host controller ASM3142.

Issue was reproduced with FX3 sample FW "USBBulkSourceSink". After pressing "Start" in application "Streamer", device sends few USB packets and application hangs(sometimes causes BSOD on Windows). Bulk OUT works fine.

Issue has not been reproduces on other USB host controllers (such ASM2142, Intel, Fresco Logic..).

We have contacted ASMedia, they sent us latest FW, but issue still can be reproduced.

Has anyone faced the same issue? And how to fix/workaround it? 

ASM3142 had been chosen due to Power Delivery and Type C connection. 

Regards,

Sergey

 

 

0 Likes
1 Solution
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

From the crash dump it was concluded that the BSOD was not caused due to Cypress driver or host application. The crash was due to a Microsoft cng.sys driver. The issue may be related to some corruption in the windows registers. Cng.sys is trying to write windows register just before BSOD.

Best Regards,
AliAsgar

View solution in original post

0 Likes
48 Replies
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Could you send us the hardware traces and the UART debug prints when the issue is seen?

Is the same issue been faced with other FX3 devices as well.

Best Regards,
AliAsgar

 

0 Likes

Dear AliAsgar,

Thank yo for reply.

Please find attached files (ASM3142_UART_out.jpg - UART debug prints) and  FX3_DevKit_Streamer.zip - USB communication captured by USB analyzer Beagle 5000 of TotalPhase).

And yes, issue is reproducible on all our devices base on FX3.

Regards,

Sergey

 

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Please try disabling the LPM inside the setconf event using the CyU3PUsbLPMDisable() API and commenting the LPM enable API's in the bulksrcsink firmware and let us know if the issue persists? 

What is the version of the FX3 SDK being used?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

I have added function CyU3PUsbLPMDisable to CyFxBulkSrcSinkApplnStart and removed CyU3PUsbLPMEnable from TimerCb, but unfortunately nothing changed. See attached UART prints. 

We are using SDK v.1.3.4

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Could you let me know if the issue is seen with USB 2.0 as well?

Could you send us a hardware traces after disabling LPM?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

USB2.0 works fine.

Please find attached  hardware traces after disabling LPM.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

1. Could you please share the debug prints when using the bulksrcsink firmware. Make sure there are USB debug logs in it.

2. Please confirm if the cables used are the standard USB certified cables, with length less than 3m.

3. If possible, do share the traces taken with LeCroy analyser.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

1. Attached.

2. Yes, we use certified 1m(US2-AMCB-1M) cables from NewNex.

3. Unfortunately we don't have LeCroy analyzer.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Could you enable the ITP's and all other packets when taking the trace, we want to check what happens before host is initiating a U0 to SS.inactive transition.
Maybe the device is not responding by sending a Data packet eventhough host advertises its NumP as 1. We are not sure.
Hence please send a trace with LPM disable and enabling all type of packets when taking the trace.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Please find attached file. I have enabled all packets for tracing(like LPR, LUP/LDN, ITP,Traning,LFPS).

Please take a look at the end of data streaming, in both cases streaming ends with "strange" data packet of 1036 bytes sent by device, even last ACK from Host was NumP=0.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

There are no attached files to the post. Could you resend them?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Sorry for that.  Attached.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

I could not find the issue in the traces which you have sent recently. Could you try sharing the trace with ITP when the issue reproduces?

The 1036 byte data packet is not strange. Its a Data packet payload, which contains 1024 bytes of data, 8 bytes of K symbols like SDH, EPF and END, and 4 byte CRC. Please refer to Pg. 119 of the USB 3.2 specifications for more information.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

They are traces when the issue was reproduced.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

1. Can you let me know the driver version of CyUSB3.sys being used?

2. Can you confirm if the issue is seen when packet per xfer and xfers to queue are set to 1?

3. Could you share the code seen on the blue screen when the BSOD issue is seen?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

1. Driver v. 1.2.3.20

2. Works fine. Tested 1h with speed rate 20500 KBps.

3. Attached 4 last BSODs.

Regards,

Sergey

0 Likes
Sergey
Level 3
Level 3
25 sign-ins 25 replies posted 10 replies posted

Dear AliAsgar,

Would you be so kind updating me with any information?

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Sorry for the delay in response.

Could you build the streamer application with the attached streamer.h file and let us know if the issue still occurs?

Best Regards,
AliAsgar

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Any Updates?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Yes, I have tried to compile Streamer application with header file you sent and it works continuously without issues. 

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Glad to hear it worked.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

What can be next? It works, but it is too slow for our device, we need full bandwidth of 400MB/s

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey, 

Could you let us know the current data rate with the workaround?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

It is about 36000 KB/s.

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Please find the attached streamer header file.
I have declared a variable "customDelay". Please try to tune this variable such that the issue is not seen, and your desirable throughput is also attained.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

I can achieve about 250MB/s without issue, if it goes faster I got BSOD.

Regards,

Sergey

0 Likes
Sergey
Level 3
Level 3
25 sign-ins 25 replies posted 10 replies posted

Dear AliAsgar,

Would you be so kind changing status of issue to "unresolved yet"? We till facing the issue. 

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Could you let us know what is packet per xfer and xfers to queue set for the testing?
Also when the delay in the streamer.h, packets per xfer and xfers to queue are set so that 250MBps is acheived, in this case, when the packets per xfer and xfers to queue are increased, what is the result?

When there is no delay, please try the following.
1. xfers to queue as 1 and packets per xfer as 4, 32, 64.
2. packets per xfer as 1 and xfers to queue as 4, 16, 32.

Is the issue seen in both the cases or any one of the case

Do you get a higher data rate with any other devices?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

When the delay in the streamer.h:

Increasing "xfer and xfers to queue" works faster(I could achieve event 400 MB/s)   and more stable, but still not perfect, still from time to time got BSODs.

When there is no delay:

Made tests with different combinations.

Packets per Xfer x Xfers to queue - Result

1. 1x1 - works fine 18 MB/s

2. 2x2 - works fine 72 MB/s

3. 4x4 - works fine for a while, then "Failures" counters starts counting.

4. 8x8 - in a while stops with error "Xfer request rejected NTSTATUS=c000000e"

5. 16x16  - in a while stops with error "Xfer request rejected NTSTATUS=c000000e"

6. 32x32 - got BSODs after couple of packets.

7. 64x64 - Application "Streamer" crashes or BSOD.

Is it possible to reproduce issue on your side?

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

We cannot reproduce the issue at our end. From what we have seen, this issue is usually seen with a type of Asmedia controllers.

I meant, can you test with packets per xfer as 1, and varying the xfers to queue.
After this test, you can keep xfers to queue as 1, and vary packets per xfer.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Packets per Xfer = 1:

in all combinations counting "Failures" . 

Xfers to Queue = 1:

Works fine in all combinations. Max bandwidth is 370MB/s if Packets per Xfer = 32

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Thank you for the details on the issue.

I have attached a streamer.h file, where i have included a delay before the wait for xfer api in the xferloop function. Could you build the application with this streamer.h and let me know if there are no more BSODs, regardless of the packets per xfer and xfers to queue values.

Also I would like to know your end application. This will help us to cater to your needs better.
Why speed more than 250MBps is  needed.
Why is xfer to queue more than 1 needed when packets per xfer is 32 and xfer to queue as 1 is giving 370 MBps speed already?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Thank you. It woks fine and provides about 22 MB/s.

We manufacture industrial cameras with USB interface and frame rate(data rate) is very critical for it. 

In our application we commit to driver as much as possible URBs, with different lengths, due to our cameras onboard don't have buffering it is very critical to have stable data rate or camera starts loosing data.  Small amount of "Xfers in Queue" can't provide stable data rate, very sensitive to Host performance. 

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

Could you share with us the memory dump from the BSOD along with crash reports?

What is the maximum speed that can be attained using USB 2.0?

Please try to build the streamer using the below file and let us know what is printed as the lastError when the issue occurs?

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Memory dump is attached.

Max speed on USB 2.0 is about 43 MB/s

I have built streamer application, but works properly just 1x1, other combinations "Failures"

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey, 

Thank you for the memory dump.

The shared file is expected to fail. It was shared with the intention to get the "Last Error".
I have added a print to get the Last Error when the beginDataXfer fails. It will be printed before "Xfer request rejected" error line. Please let us know what is printed for the Last Error on the streamer application.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Ok, finaly I could get error message(not BSOD or "Failures" counting), but could not get  "Last Error...". Just next:

"Xfer2 request rejected. NTSTATUS = c000000e"

Regards,

Sergey

0 Likes
AliAsgar
Moderator
Moderator
Moderator
1000 replies posted 250 solutions authored 750 replies posted

Hi Sergey,

From the crash dump it was concluded that the BSOD was not caused due to Cypress driver or host application. The crash was due to a Microsoft cng.sys driver. The issue may be related to some corruption in the windows registers. Cng.sys is trying to write windows register just before BSOD.

Best Regards,
AliAsgar

0 Likes

Dear AliAsgar,

Thank you for information.

But how to explain that issue is reproducible on different PCs and not reproducible on USB host controllers (Intel, Fresco and event ASM2142(1142)?

Regards,

Sergey

0 Likes