[AN75779 FX3 + UVC] No data in DMA buffers _ 640x480 YUY2 stream @15fps

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
lock attach
Attachments are accessible only for community members.
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello, dear Infineon support community !

I am currently working on a custom UVC application based on AN75779 and CYUSB3KIT_003.

To simulate data coming from a camera sensor, an FPGA generates the signals corresponding to a 640 x 480 YUY2 video stream @15fps, 16bits/pixel, as defined in the descriptors of the original AN75779 project. (See attached Probe_FPGA_Signals.zip archive for oscilloscope probes on these signals, acquired on FX3 GPIF, and a readme.txt file with comments).

My FX3 system is recognized as a UVC device, and does appear in VLC's Capture Device interface, which is how I initiate video streams.

However, I can't seem to receive any of the input data on the host side, as shown in the following UART messages. By following this similar support thread ,  I implemented some additionnal information messages based on key flag variables, such as bufferCount which indicates the amount of data in the current DMA buffer. 

PuTTy_StreamUVC_EmptyBuffer_31-08.png

From this, I can conclude that :

  • The state machine is not blocked, as the current SM state oscillates between 2 and 5 (respectively WAIT_FOR_FRAME_START_0 and _1).
  • There is a stream of data at approximately 15 fps, which triggers both the prod and cons Events
  • The buffers of my DMA channel remain desperately empty.

Moreover, the Wireshark traces reveal that there is an ongoing stream ; but the payloads emitted by FX3, although at a rate of ~15 /s , only contain the 12-byte header , with its second byte alternating between 0x8E and 0x8F. (See  Wireshark Trace_Header 31-08.png )

Wireshark Trace_Header 31-08.png

Here are the modifications I added to the original AN75779 project :

  • Modifications to the GPIF configuration : 16-bit data bus, LD_ADR/DATA_COUNT limit modified to 8183, activated SPI and I2C, sampling on negative edge.
  • Implementation of Extension Unit controls, using SPI and I2C protocols. 
  • Additionnal flags in UART log messages

 

My questions are the following :

  • Could you please suggest any interpretation as to why no data seems to be sampled and stored into the buffers, and maybe a solution ?
  • Also, are there any expected limitations in terms of horizontal blanking ? I arbitrarily chose to wait for 160 PCLK cycles between two active LV signals, but please let me know if that's inadequate. 

 

I understand that the symptoms are nearly identical to the previously mentioned thread, where the solution was apparently a difference between the expected width/height values and the actual video signal dimensions. In the meantime, I will be trying to change UVC descriptors to close frame dimensions (640*483, for example) to see if that solves my issue, and follow once again the debug checklist from the appropriate KBA .

Please find attached the uvc.c, uvc.h, cyfxuvcdscr.c , fx3_uvc.cyfx and cyfxgpif2config.h files from my project, the uvc.c might refer to some custom functions or header files which I would rather not share on a public forum for IP reasons but can be commented without interfering with the overall program (See Project_Files.zip ). 

 

Best regards,

Enzo

0 Likes
1 Solution
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Thank you for the update that synchronizing the FPGA solved the issue and the black screen issue is due to extra data coming from the FPGA. If the FPGA sends the data equal to the frame size reported by FX3 to host and is there are no failures on Fx3 side, the video will be seen on the host app

Regards,
Rashi

View solution in original post

13 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Thank you for the details.

The firmware files shared looks fine. Please let me know how do you initiate streaming when the device is not detected by VLC. 

To debug the issue, please disable the extension unit from the USB descriptors.

Also, please confirm that the streaming from FPGA is started only when SET_CUR request or CY_FX_UVC_STREAM_EVENT is triggered from the Host application. Confirm that the FX3 is programmed before FPGA.

If the data from FPGA is 640*480, the descriptors and the probe control structure (glProbeCtrlVGA) should have the same frame resolution.

To debug the enumeration issue, please share the .pcap file (Wireshark) when the device is seen as UVC device but not detected by VLC

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello,

Thanks very much for that quick reply and those fresh ideas.

First of all, there is no enumeration problem with VLC, as my device always shows up in the list of available UVC devices.( I guess the "does appear" formulation can easily be read as a "doesn't", my bad.

As an alternative to VLC, I also developped a host application on a Linux machine which uses a library called libuvc to scan for UVC devices and send requests to manage the streaming process or extension unit controls. The results on that Linux setup are pretty much the same : device detected, stream initiated, UART messages reveal a 15fps stream of empty data.

However, I did not configure my FPGA to start streaming when the STREAM_EVENT is received by FX3 ; instead, it constantly sends data to the 16-bits data port DQ[0..15] after I press a button, and the stream is started independently on the UVC side. By the way, FPGA data flow is a simple 10-bit counter loop on DQ[0..9], with the last 6 bits on the data port connected to the ground.

This could be the source of the problem, I will try to synchronize my FPGA data sequence by toggling a GPIO right after setting the CY_FX_UVC_STREAM_EVENT and keep you updated on the results.

The best place to insert that GPIO action seems to be in the video streaming request handler (UVCHandleVideoStreamingRqts), more specifically in the SET_CUR section of the COMMIT_CTRL requests, but please let me know if I should try it differently.

Finally,  I will edit the descriptors and glProbeCtrl structures to match, specifically the max frame size which in my case should be 640*480*2 = 0x96000.

However, please note that the descriptors and glProbeCtrl values in my previous project files were exactly identical to those given in the source .zip project, as it can be downloaded today on Infineon's portal (version 13_00). If those values are not correct, it might be worth implementing and uploading this correction to the source files for future users.

I will be joining to this message the wireshark trace corresponding to the sequence from which the previous screenshots were taken (see Wireshark Trace 31-08.zip, where my FX3 device's ID is 1.26.3 and UARTmessage go to a FTDI cable at 1.13.1).

Regards,

Enzo

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

However, I did not configure my FPGA to start streaming when the STREAM_EVENT is received by FX3 ;

>> Yes, synchronizing the FPGA with FX3 might resolve the issue

his could be the source of the problem, I will try to synchronize my FPGA data sequence by toggling a GPIO right after setting the CY_FX_UVC_STREAM_EVENT and keep you updated on the results.

>> Yes, using GPIO to synchronize will be good

please note that the descriptors and glProbeCtrl values in my previous project files were exactly identical to those given in the source .zip project, as it can be downloaded today on Infineon's portal

>> I had asked to change the descriptor and glprobectrl if you are changing the resolution to 640*483 as the default an75779 supports 640*480

Please let me know if synchronizing the FPGA helps

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello, 

I did synchronize the start of my data flow by activating a GPIO at the reception of a STREAM_EVENT request, and as a result I am now receiving some data from my FX3 device (thanks !).

This GPIO action is managed in the for (;;) loop of the main app thread (UVCAppThread_Entry), just before starting the application with CyFxUvcApplnStart :

/* Request to start video stream. */
            if ((flag & CY_FX_UVC_STREAM_EVENT) != 0)
            {
                glIsApplnActive = CyTrue;
                CyU3PDebugPrint (4, "\r\n CY_FX_UVC_STREAM_EVENT : Start Application\r\n");

                //Toggle STREAM_GPIO40 as a signal for the FPGA to start streaming data
				CyU3PDebugPrint (4, "\r\n Activating STREAM_GPIO...");
				apiRetStatus = CyU3PGpioSetValue (STREAM_GPIO, CyTrue);
				if (apiRetStatus != CY_U3P_SUCCESS)	{
					CyU3PDebugPrint (4, "Could not Set LED value\r\n");
				} else {
					CyU3PDebugPrint (4, "Done. Starting video stream : \r\n\n");
				}

                CyFxUvcApplnStart();
            }

 I could obviously use some advice if you think of a more appropriate way to implement the GPIO activation.

 

However, the stream always stops after a few payloads, going back to the previous situation (empty payloads) :

UARTlog02-09 (one frame).png

Wireshark Trace 02-09 (one frame).png

 The payloads received on Wireshark have the expected size, and the data is quite coherent, except for the format : the data sent is a 10-bit counter, supposedly acquired as 16-bit values, but it looks like the result only contains the first byte of each data sample, resulting on a 8-bit counter.

I have tested the acquisition a few times now, and the number of payloads sent before the stream becomes empty is variable ; but I noticed that the UART always indicates that a backflow is detected at the beginning of the stream.

Also, the  first payload observed in Wireshark always starts at 0x46 (instead of starting at 0x00 like the actual data sent ), which might reveal that some of the data might be missed at the beginning of the acquisition process.

 

My interpretation is that there might still be some synchronization issues, probably because the input stream is starting before the application is actually ready to process any data. 

Any advice on how to solve this ? I'm thinking about adding a delay at the beginning of the FPGA sequence.

Could the backflow event be the source of the problem ? I've read in this other thread that GPIF interrupts may cause firmware stops.

What about the data format, appearing as 8-bit values instead of 16  in the payload? I thought that the 2nd byte of each measure would be stored in the buffer right after the first, but I'm not sure where my bytes measured on DQ[8..15] are in the payload.

I verified both the descriptors and the GPIF config file, and both indicate that I am configured for a 16-bit data bus.

 

Please find attached a .zip file containing debug data (UART and Wireshark traces) for two tests, one where the program worked well for several frames and one where it stopped after roughly one frame (most frequent test case). I am also joining a probe screenshot showing the time elapsed between my STREAM_GPIO activation and the first FV rising edge.

 

Best regards,

Enzo

 

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

From the logs and the details shared by you, backflow is detected when a socket is overwritten. This could happen when the data write is done beyond the available buffer size.

Please confirm if the FPGA is sending data as expected. I understand the the GPIF and FPGA both are configured for 16 bit interface and GPIF counters are configured to 8183 i.e. (16368/2 -1)

The expected Full DMA buffers  614400 /16368 = 37 and 1 partial buffer of 8784 by it l0oks like the FPGA is sending almost the half of it frame size.

It looks that the state machine shared by you in first post uses on 8 bit GPIF bus. Please configure it 16 bite if FPGA is sending out 16 bit data

Regards,
Rashi
0 Likes
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello Rashi,

Thanks again for you answer. As you mentioned, there was an issue with my data bus, which once solved allowed me to get the expected 16-bits data format (see screenshot below).

Screenshot 2022-09-05 UART_Wireshark.png

The configuration problem was that the GPIF configuration  exported its output cyfxgpif2config.h file to its own separate folder (fx3_uvc.cydsn) instead of replacing the config file at the root of the project, resulting in two distinct cyfxgpif2config.h files in my project. The file at the root of the project, containing the actual GPIF configuration for FX3, had not been updated since I downloaded the project and had a 8-bit data bus default setting. 

I can now receive a few complete frames, but the application still detects a backflow and stops after some iterations.

As per my previous post, the first payload stills misses the first 0x46 (70d) data samples, which might cause compatibility issues in the format and could mean that FX3 receives input data before being ready_ more specifically, receiving data before emptying all buffers, maybe the cause of the backfow?.

In order to solve this, I will implement a delay in my FPGA program to wait for 100 clock cycles at the reception of the STREAM_GPIO signal, before starting the input stream. 

I will keep you updated on the result as soon as possible , but please let me know if you have any other leads to solve my problem of the application stopping inexpectedly.

 

Best regards,

Enzo

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Enzo,

Please let me know if you have modified the counter values based on the GPIF bus width in the GPIF state machine.

From the logs I can frame timeout. Please try to increase timeout period to 1000ms (glFrameTimerPeriod = 1000)m and check if that helps. 

Regards,
Rashi
0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Enzo,

Please let me know if you have modified the counter values based on the GPIF bus width in the GPIF state machine.

From the logs I can frame timeout. Please try to increase timeout period to 1000ms (glFrameTimerPeriod = 1000)m and check if that helps. 

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello,

Yes, I confirm that both counters in the state machine have their limits set to 8183, as expected for a 16-bit data bus.

As you suggested, I tried increasing the timeout period (1000, 2000, even 3000ms) but that did not really solve the fact that my stream stops at some point (see attached Wireshark trace and UART log, from a test with  glFrameTimerPeriod =1000).

To this point, I am able to observe the data fromfirst few frames on Wireshark before the stream stops, and the 16-bit format is respected. In some lucky cases, I could even briefly observe a green data frame on VLC (see VLC screenshot[...].png), but that would not work in all tests.

I mentioned earlier that the first data sample was systematically 0x46 (it is actually 0x0346, now that I can see both bytes), and I implemented a delay at the beginning of the FPGA sequence to see if that would reduce the amount of data missed at the beginning of the streaming process. 

Stream start sequence with delay.PNG

This delay did not change the fact that my first data sample is still always 0x0346, so I imagine that there is something else causing this data loss, possibly in the firmware on the FX3 side.

From the Wireshark traces of my tests, I noticed that in some cases the frames would contain more data than expected, even filling a bit more than 38 buffers per frame. I will be verifying my FPGA program again to ensure that the data flow is strictly as expected.

 

The backflow_detected message is still there and I guess it might be a good step to focus on solving that.

Any suggestions on how to proceed to debug the backflow ? I've found this ticket and this KBA that mention it, and will try to find  solution from there in the meantime.

 

Thanks again for your support,

Best regards,

Enzo

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Enzo,

Thank you for the details.

Backflow will be detected when CYU3P_PIB_ERR_THR0_WR_OVERRUN/CYU3P_PIB_ERR_THR1_WR_OVERRUN is triggered. This happens when data to particular DMA buffer is written more than the DMA buffer size.

For the backflow detect error, please try to assert STREAM_GPIO after CyU3PGpifSMSwitch so that we are sure that FX3 is ready to receive the data before FPGA is started

It looks like the issue is from FPGA side. The reason for this is in the logs, I can see the current SM state is 5 and we are waiting for new frame but frame is not received before the timeout happens which causes the appstop (flushing out what we have) and appstart (start receiving the next frame) so here data loss is possible.

When Backflow detect is triggered, there is no handling done in the firmware i.e. appstop /appstart is not called. This only means that some data will be lost that is being written by FPGA on already filled DMA Buffer. Please let me know the Horizontal blanking period (i.e. LV) for the data sent by FPGA.

Regards,
Rashi
0 Likes
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello Rashi,

Thank you for all those precisions on my technical issues.

Activating STREAM_GPIO at the end of the AppStart function, right after starting the State Machine, allowed me to solve solve the missing data issue (0x0346 samples) at the beginning of the first frame, which now starts by the first expected value.

The Horizontal blanking period is set to 160 clock cycles , so about 5.93us with PCLK @ 27MHz. This value was chosen arbitrarily and can be changed. That's also the reason why in the previous Wireshark logs the acquired values would sometimes suddenly increase by 161 values, as the clock still increments my counter while LV is low.

Should the data bus be driven LOW/HIGH during blanking, with LV/FV LOW ? I thought the state of the data bus pins would not matter when LV is low, since no sampling is happening during this time.  

Regards,

Enzo

0 Likes
Nzagone
Level 3
Level 3
25 sign-ins First solution authored First like received

Hello,

Here is a quick update from my side. 

The issue is definitely from the FPGA side : examining the data received in Wireshark allowed me to find that there was an additionnal sample being received every 256 values, apparently due to some delay in my 10-bit counter when incrementing the 9th bit when the first byte reaches its limit.

With these additionnal values, there is simply more data than expected as per the format defined in the descriptor, each line having ~642 data samples instead of 640 and each frame containing about 2000 bytes too many. This is likely the source of the backflow error.

To solve this, I will modify my FPGA data input to ensure that the 640px/line format is respected ; alternatively, I am also considering a modification of the descriptors on the FX3 side to accept the 642px/line data format (for test purposes).

Concerning my previous question on the data bus during blanking, I figured it out by reading the relevant documentation again and the data bus was set LOW between data transfers since the beginning of my tests, as expected. 

 

The main issue of this whole thread, which has already been solved, was that the start of the  input video stream must be synchronized with the reception of a STREAM REQUEST on the FX3 controller. 

If I manage to solve my own data format issues on the FPGA side and no unexpected error message shows up, I think I will have the information needed to close the case.

Thanks for your patience and support,

Enzo

 

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Thank you for the update that synchronizing the FPGA solved the issue and the black screen issue is due to extra data coming from the FPGA. If the FPGA sends the data equal to the frame size reported by FX3 to host and is there are no failures on Fx3 side, the video will be seen on the host app

Regards,
Rashi