Version: **
Question: Is it possible to stream video from two image sensors using one EZ-USB™ FX3?
Answer:
Yes. You can use a FX3+FPGA solution. A slave FIFO interface can be used to communicate with the FPGA
(refer to AN65974 - Designing with the EZ-USB FX3 Slave FIFO Interface for more details on slave FIFO interface). Streaming from two image sensors using one FX3 can be done by creating two DMA channels.
For example, if two DMA channels created are as follows:
DMA channel - 1 with Producer sockets as CY_U3P_PIB_SOCKET_0, CY_U3P_PIB_SOCKET_1 and Consumer socket as CY_U3P_UIB_SOCKET_CONS_1
DMA channel - 2 with Producer sockets as CY_U3P_PIB_SOCKET_2, CY_U3P_PIB_SOCKET_3 and Consumer socket as CY_U3P_UIB_SOCKET_CONS_2
The FPGA can maintain two FIFOs to store data from the two image sensors. The FPGA code should change the address lines (A0 & A1) depending upon the FIFO from which it fetches the data from and send to FX3.
The sensor1 data should be stored in FIFO 1 of the FPGA and sensor2 data should be stored in FIFO 2 of the FPGA.
During the blanking period, FPGA can switch the FIFO and change the address lines to select the appropriate DMA channel of FX3 for streaming the data; that is, if the data is coming from FIFO 1, the 2-bit address line A0:A1 should be driven with 00 or 01 alternatively i.e. in ping-pong manner to select CY_U3P_PIB_SOCKET_0 or CY_U3P_PIB_SOCKET_1 of DMA channel 1.
Similarly, if the data is coming from FIFO 2, then the FPGA should drive the 2-bit address lines A0:A1 as 10 or 11 alternatively to select CY_U3P_PIB_SOCKET_2 or CY_U3P_PIB_SOCKET_3 of DMA channel 2.
Note: Refer to sections 3 and 4 of the AN75779 - How to implement an image sensor interface using EZ-USB FX3 in a USB video class (UVC) fra... to understand ping-pong DMA buffer.
The video data received by the PIB sockets corresponding to the DMA channel as mentioned in the above example, will be transferred to USB block through DMA channel. As two video streams from two sensors will be streamed using two DMA channels, the device should enumerate as a composite device with two UVC interfaces i.e. FX3-UVC-1 and FX3-UVC-2. In the host application like AmCap, VLC media player, user will get option to choose a UVC interface to start streaming video. If streaming from both the sensors is to be started simultaneously, two instances of host application should be opened and appropriate UVC interface should be selected.
Figure 1. Composite device with two UVC interface
For the device to enumerate with two UVC interfaces, the USB descriptors should be modified to support two sets of UVC descriptors. In addition to this, the firmware should also handle UVC specific requests for both the interfaces.
A template firmware with two UVC interfaces and UVC specific request handling is attached with this KBA. The header file for GPIF state machine (cyfxgpif2config.h) used with the firmware is a dummy state machine. The user is expected to use a custom GPIF state machine header file based on the application. Two CY_U3P_DMA_TYPE_MANUAL_MANY_TO_ONE DMA channels are used in the firmware to stream two 640* 480 (VGA) YUY2 at 30 FPS video streams. As per the default AN75779 UVC firmware, the UVC header is added in the firmware. The cyu3imagesensor.c and cyu3imgagesensor.h are also created and saved in this project. The cyu3imagesensor.c file is used to configure and control the image sensor and FPGA. In this file, some structures are defined without a valid value. You need to replace these values with actual settings. Generally, used APIs such as CyFx3_ImageSensor_Sleep, CyFx3_ImageSensor_Wakeup, and CyFx3_ImageSensor_Trigger_Autofocus are also available the cyu3imagesensor.c and cyu3imagesensor.h files. Add the corresponding codes into these definitions.
Note: The firmware attached with the KBA is a template project and should be modified according to user application.