Issues sending 4 byte audio samples with 24bit resolution

[jnagy]

I am trying to send 24bit PCM to the USB host using a composite UVC and UAC device. For testing purposes, we are sending 1 data sample through GPIF at a time with the data being left-justified to MSB. This means that we have 24 out of the 32 GPIF bus bits writing data while the other 8 are tied to zero. Reading through the documentation on UAC, it seems the only modifications I need to make to the sample UAC descriptors is to modify the bSubframeSize and bBitResolution fields to 0x04 and 0x18 respectfully.

However making these modifications prevents the USB device from working. I go to record a video (and thus enable the UAC application) and the camera freezes and then returns to the non-recording state. The UAC audio interface never sends any data. Can I have some guidance on why these two modifications would stop the USB device from working? I am using UAC 1.0 which should support these two modifications. I am using the Windows Camera application for testing so I am not sure if is just that that application doesnt support 4 byte / 24bit resolution audio.

[Rashi_Vatsa]

Hello,

To understand the modifications better, please let me know if the GPIF bus width is changed to 24 bits or if it's still 32 bits but 8-bit zero padding is done. 

If zero padding is done, the DMA buffer will have padded data every clock cycle. 

You can refer to this KBA for reference  FX3 – UAC Requests Handling - Infineon Developer Community

[jnagy]

We are using a 32 bit data bus with the FPGA is sending UAC data with the 8 bits of padding. Therefore, the FPGA is sending a 32 bit word where the most significant 24 bits are PCM data and the least significant 8 bits are zeros.

What do you mean by "DMA buffer will have padded data every clock cycle"? The wording makes me think that the DMA buffer does the padding itself. I understand that if we send padded data to the DMA buffer from the FPGA, there will be padded data in the buffer. This is why I've set the UAC descriptor bit resolution to be 24. I am under the assumption that setting the subsample size to 4 and bit resolution to 24 the UVC camera will read samples 4 bytes at a time and discard the last 8 bits due to bit resolution.

[Rashi_Vatsa]

Hello,

 I understand that if we send padded data to the DMA buffer from the FPGA, there will be padded data in the buffer

>> Yes, your understanding is correct. The data sampled on GPIF interface  (with zero padding) will be stored in the DMA buffer.

Please share the descriptor file to understand the endpoint descriptor configuration and  USB Wireshark trace when the issue is seen

[jnagy]

Here is a Wireshark capture and my descriptor file. I also included a header with defines and another capture with 2 byte subsamples and 16 bit resolution. The UAC data is sample counter data, not real audio but it works on the 2 byte / 16 bit resolution data . The only difference between the 2 captures is the descriptor. The USB EP structure is that I have UVC control on EP 1, UVC streaming on EP 2 and UAC streaming on EP 3.

[Rashi_Vatsa]

Hello,

From the Wireshark traces, I do not see a SET_INTERFACE request from the Host for the Audio interface. This could mean that the USB host doesn't have enough bandwidth to initiate the audio streaming.

Also, can you please reduce the subframe size to 3 and check if the UAC streaming works.

I noticed that once there is a clear feature request for video endpoint the streaming after that is not as expected. The video frame sent is not proper. Please confirm that the clear feature request is handled properly in the firmware

[jnagy]

I tried setting the subframe size to 3 and am seeing a similar problem. I have included a Wireshark trace. Looking at the original 2 byte subframe / 16 bit resolution setting, I dont see any SET_INTERFACE for UAC either. Should there be some communication on EP0 to enable UAC mid-stream?

I am using an ISO endpoint, so wouldn't I be guaranteed bandwidth on the USB bus for Audio?

Where is this clear feature request? Is it the SET INTERFACE right after the Probe and Commit requests? I have not modified the functionality of the clear feature request handling from the original example firmware for the Slave FIFO and AN75779. And what do you mean by the video frame is not proper?

[Rashi_Vatsa]

Hello,

Please find my comments below:

I dont see any SET_INTERFACE for UAC either. Should there be some communication on EP0 to enable UAC mid-stream?

>> For the working trace,  I couldn't see the UVC GET/SET requests also which should be there before the video streaming starts. It looks like the Wireshark trace is truncated one. 

I am using an ISO endpoint, so wouldn't I be guaranteed bandwidth on the USB bus for Audio?

>> An interface with isochronous endpoints reserves bandwidth on the USB bus. As there are 2 alternate settings the USB host controller will decide which alternate setting to  choose based on the available USB bandwidth.

Also, in the firmware you shared, CyFxUACApplnStart will only be called when CY_U3P_USB_EVENT_SETINTF is received with alternate settings as CY_FX_UAC_STREAM_INTERFACE

Where is this clear feature request? Is it the SET INTERFACE right after the Probe and Commit requests? I have not modified the functionality of the clear feature request handling from the original example firmware for the Slave FIFO and AN75779. And what do you mean by the video frame is not proper?

>> Please refer to index 43091 of non working trace and from 43130 there are some partial buffers. Please confirm if the host application was not closed in between. 

Please let me know if similar problem is seen when a different Host application is used. For example, MPC-HC / VLC Media player

[jnagy]

Apologies, those captures are missing packets. I have included captures which should include everything. I was using Wireshark's filter to only capture packets for certain devices and for some reason that filtered out the SET_INTERFACE packets. These new captures contain the missing packets, including the SET_INTERFACE.

How can I debug USB bandwidth issues? I have looked for tools in the past to notify me of the bandwidth but they seem non-existent for Windows.

I am not sure what you mean by the following:

Also, in the firmware you shared, CyFxUACApplnStart will only be called when CY_U3P_USB_EVENT_SETINTF is received with alternate settings as CY_FX_UAC_STREAM_INTERFACE

Should CyFxUACApplnStart be called when that alternate setting conditional is not the case? I have not modified when this function would be called, so that conditional is from the example firmware.

The partial buffers was an issue related to an underrun error when the video camera app switches sources or was minimized/maximized during streaming. The firmware restarts the stream in response. I don't believe it is a related issue to the audio because it happens with UAC disabled.

I have tried with VLC and am experiencing the same issue.

[Rashi_Vatsa]

Hello,

Apologies for the delay in response.

The endpoint size and the data transferred in 1ms need to be configured such that the data transferred (service interval * endpoint size)  (4 *125us * 288 bytes (endpoint size)) =  Sampling fq x size of sample x num channels = 48KHz x 3 (24 bits) x 2

Please change the endpoint size to 288 bytes and bInterval to 4

Please let me know if it works with this setting

[jnagy]

It is not a problem.

I dont quite understand the formulation here. What is the full formula, where is it documented and what does it mean? What values am I using and where do they go into the descriptors? These seem to be values taken from the endpoint + super speed companion and type 1 format descriptors. I tried replacing the endpoint and super speed companion descriptor values and did not see any difference. I cannot record anything anywhere whereas when reverting back to 2byte / 16bit I had before I can get static noise out in the Camera app but not Audacity.

[Rashi_Vatsa]

Hello,

You can refer to "Standard AS Isochronous Audio Data Endpoint Descriptor" of UAC 1.0 specification which mentions that the wMaxPacketSize is determined by the audio
bandwidth constraints of the endpoint

Can you please share the USB descriptor file and WireShark traces after the changes

[jnagy]

I still don't understand how bInterval is determined.

In the formulation you gave, 4 *125 * 288 = 144,000 and 48000 * 3 * 2 = 288,000. Furthermore, the original code has bInterval of 3 and packet size of 96 bytes, giving 3 *125 * 96 = 36,000 yet it works. Looking at the "Standard AS Isochronous Audio Data Endpoint Descriptor" table (Table 4-20 of UVC 1.0 spec), it also says that bInterval must be set to 1. I tried setting it to 1 and I cant record anything. Can you help explain this because there is a lot of contradictory information here?

Is wMaxPacketSize the size per individual packet or as a group of packets? Looking at Wireshark, each ISO transfer to the host has 10 ISO packets inside it.

[jnagy]

Can you offer some explanation? Im having difficulty understanding your math.

[Rashi_Vatsa]

Hello,

Apologies for the delay in response.

As per USB spec, service interval is 2^(bInterval-1) x 125 µs so (service interval * endpoint size) will be 288 bytes/ms and 48KHz x 3 (24 bits) x 2  = 288 bytes/ms

wMaxpacketsize is amount of data the endpoint is capable if receiving/sending.

The Wireshark shows the multiple packets of wMaxPacketsize transferred

[jnagy · Accepted solution]

Ok, thanks, I found the issue. It was incorrect interval, mult and burst settings. I found this information quite hard to find and wrap my head around so for anyone who encounters this error:

1. Interval is a setting in the UAC Iso descriptor that determines the "service interval" which is how  often the endpoint will be polled in terms of USB microframes. It should be set for however often you want USB Iso transfers to be done. So for an interval of 3, the service interval will be 4 meaning that the endpoint will be polled for data every 4 USB microframes.

2. Mult is a setting in the SuperSpeed Endpoint Companion descriptor that determines how many transactions are done per poll. There can be 1-3 transactions

3. Burst is a setting in the SuperSpeed Endpoint Companion descriptor that determines how many data packets are sent per transaction. The packet is what actually carries the USB data being sent over the Iso endpoint. If burst is greater than 1, the maximum packet size must be 1024.

Once all these values are chosen, the Bytes Per Interval setting in the SuperSpeed Endpoint Descriptor must be set to max packet size * mult * burst. It represents the total maximum bandwidth that could be consumed were the maximum data to be sent in one poll of the endpoint.