- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello guys,
We were able to successfully benefit from the application note AN75779 and use FX3 to transfer video streams from our image sensors (IMX250, IMX253, IMX420) to the PC. VLC player has been used to play the streams. Thanks for your support in that sense (thread 1, thread 2)! It is really appreciated.
We are now in the process of replacing VLC player by our custom software using UVC class drivers. In that sense, we would like to be able to dynamically change video resolution during system runtime. Here are some use-cases where this dynamic resolution change would be necessary:
1) During system start-up, we will read the type of the image sensor we have connected with FX3 and set video resolution accordingly.
2) There will be a need to change video mode or simply video resolution during runtime of the system by sending appropriate command message from PC.
In this thread you explained us how to include different resolutions into USB descriptor chain and we did it successfully. We also noticed that the last Class specific Uncompressed VS frame descriptor in the list represents the default video resolution.
Our FX3 firmware is based on AN75779 example. What are the steps/commands to take in order to change video resolution of the stream dynamically, during runtime? We can change a resolution of the image sensor successfully, we only need to inform PC about that.
As always, thank you very much for your time and efforts!
Sincerely,
Bojan.
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Bojan,
I will explain the whole procedure of changing video resolution dynamically with VLC media player
When AN75779 firmware is modified to add multiple resolution, on opening the host application (VLC) the options of the video resolutions supported by FX3 (as per modifications in the firmware) will be visible. Among the multiple resolutions one resolution will be set by default
1) To change the video resolution dynamically, one among the multiple resolutions shown in host application (VLC) need to be chosen.
2) On changing the video resolution, UVC class request will be sent by the host to FX3. Please refer to figure 6 of AN75779 and section 2.3.2.2 for the flow of the requests and response between host and device.
3) The UVC requests (sent by host) need to be handled in the firmware. Changing video resolution is for Streaming interface. Streaming interface can handle two requests CY_FX_UVC_PROBE_CTRL and CY_FX_UVC_COMMIT_CTRL requests.
- Please confirm that you are sending the probe control structures for all the video resolutions when this CY_FX_USB_UVC_GET_DEF_REQ request is sent by the host
- Under CY_FX_UVC_COMMIT_CTRL, when CY_FX_USB_UVC_SET_CUR_REQ occurs then appropriate sensor settings need to be change (according to the video resolution set) by the host
// UVC class request
if (wValue == FX3_UVC_VS_COMMIT_CONTROL)
{
switch(bRequest)
case FX3_USB_UVC_SET_CUR_REQ:
switch (CY_U3P_GET_LSB(wIndex))
{
case FX3_UVC_STREAM_INTERFACE_1:
{
CyU3PThreadSleep(900);
CyFx3UvcAppSetVideoResolution(glCommitCtrl[2], glCommitCtrl[3], frameInterval, 0);
}
break;
}
}
void
CyFx3UvcAppSetVideoResolution(
uint8_t format_index,
uint8_t resolution_index,
uint32_t frame_interval,
)
{
glFrameSizeExpected = 0;
if (CyU3PUsbGetSpeed() == CY_U3P_SUPER_SPEED)
{
if(format_index == UYVY_FORMAT_INDEX)
{
switch (resolution_index)
{
case SS_YUY2_1920x1080_INDEX:
sensor_settings_1920*1080(); //setting the sensor to stream 1920*1080
break;
case SS_YUY2_1280x720_INDEX:
sensor_settings_1280*720(); //setting the sensor to stream 1280*1720
break;
case SS_YUY2_640x640_INDEX:
sensor_settings_640x640();
break;
The fields like format_index, resolution_index etc needs to be parsed from glCommitCtrl when bRequest is CY_FX_USB_UVC_SET_CUR_REQ and wValue is CY_FX_UVC_PROBE_CTRL
Please share the modified firmware if you have more queries
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Bojan,
I will explain the whole procedure of changing video resolution dynamically with VLC media player
When AN75779 firmware is modified to add multiple resolution, on opening the host application (VLC) the options of the video resolutions supported by FX3 (as per modifications in the firmware) will be visible. Among the multiple resolutions one resolution will be set by default
1) To change the video resolution dynamically, one among the multiple resolutions shown in host application (VLC) need to be chosen.
2) On changing the video resolution, UVC class request will be sent by the host to FX3. Please refer to figure 6 of AN75779 and section 2.3.2.2 for the flow of the requests and response between host and device.
3) The UVC requests (sent by host) need to be handled in the firmware. Changing video resolution is for Streaming interface. Streaming interface can handle two requests CY_FX_UVC_PROBE_CTRL and CY_FX_UVC_COMMIT_CTRL requests.
- Please confirm that you are sending the probe control structures for all the video resolutions when this CY_FX_USB_UVC_GET_DEF_REQ request is sent by the host
- Under CY_FX_UVC_COMMIT_CTRL, when CY_FX_USB_UVC_SET_CUR_REQ occurs then appropriate sensor settings need to be change (according to the video resolution set) by the host
// UVC class request
if (wValue == FX3_UVC_VS_COMMIT_CONTROL)
{
switch(bRequest)
case FX3_USB_UVC_SET_CUR_REQ:
switch (CY_U3P_GET_LSB(wIndex))
{
case FX3_UVC_STREAM_INTERFACE_1:
{
CyU3PThreadSleep(900);
CyFx3UvcAppSetVideoResolution(glCommitCtrl[2], glCommitCtrl[3], frameInterval, 0);
}
break;
}
}
void
CyFx3UvcAppSetVideoResolution(
uint8_t format_index,
uint8_t resolution_index,
uint32_t frame_interval,
)
{
glFrameSizeExpected = 0;
if (CyU3PUsbGetSpeed() == CY_U3P_SUPER_SPEED)
{
if(format_index == UYVY_FORMAT_INDEX)
{
switch (resolution_index)
{
case SS_YUY2_1920x1080_INDEX:
sensor_settings_1920*1080(); //setting the sensor to stream 1920*1080
break;
case SS_YUY2_1280x720_INDEX:
sensor_settings_1280*720(); //setting the sensor to stream 1280*1720
break;
case SS_YUY2_640x640_INDEX:
sensor_settings_640x640();
break;
The fields like format_index, resolution_index etc needs to be parsed from glCommitCtrl when bRequest is CY_FX_USB_UVC_SET_CUR_REQ and wValue is CY_FX_UVC_PROBE_CTRL
Please share the modified firmware if you have more queries
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, RashiV_61!
Thank you very much for your reply!
The firmware we are developing is based on AN75779. We did not change the structure of the AN75779 firmware, we just added a few things on the top of it for the moment (mainly relative to I2C communication with our image sensor). We also added a few more Class specific Uncompressed VS format descriptors on appropriate place, as per your advice (link).
When we connect our system to VirtualDub player, we can see that VirtualDub recognizes all the resolutions we included within USB descriptor which proves that capability request phase (initial communication between the Host and FX3 firmware) is successfully taking place even though I am not aware where exactly in the AN75779 firmware we are responding to CY_FX_USB_UVC_GET_DEF_REQ requests. Can you point me that place, please?
Thank you for describing the process of changing resolution that is initiated by the Host. The same question again... where is the place in AN75779 firmware where we detect CY_FX_USB_UVC_SET_CUR_REQ commands from the Host? Is this somewhere within UVCAppEP0Thread_Entry() ?
Call me a dumb but I could not find where the function CyFx3UvcAppSetVideoResolution() is defined and used in AN75779 firmware. Can you point that place to me as well? (or this is something I need to add myself).
Regards,
Bojan.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Bojan,
Thank you for confirming that video resolutions are visible in host application
I am not aware where exactly in the AN75779 firmware we are responding to CY_FX_USB_UVC_GET_DEF_REQ requests. Can you point me that place, please?
>> This request is handled in AN75779 where all the resolutions supported by the device needs to be sent. As the video resolutions are seen in the host application probe control of all the video resolutions need to be sent as mentioned in the last response of this thread How to support different video resolutions - AN75779
For multiple resolution
you can parse the frame index number and send the probe control data of the appropriate resolution (as per the request from the host).
case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */
if (usbSpeed == CY_U3P_SUPER_SPEED)
{
if(frame_index == 2)
CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *glProbeCtrl_1280x720_60fps));
}... so on
where is the place in AN75779 firmware where we detect CY_FX_USB_UVC_SET_CUR_REQ commands from the Host? Is this somewhere within UVCAppEP0Thread_Entry() ?
>>CY_FX_USB_UVC_SET_CUR_REQ request is handled in UVCHandleVideoStreamingRqts. You can search for this in the firmware.
When this request comes the sensor needs to be configured for that resolution.
For multiple resolutions, CyFx3UvcAppSetVideoResolution API , is just for reference (is not in AN75779 firmware ) for you to understand what needs to be done when the request CY_FX_USB_UVC_SET_CUR_REQ occurs.
You can parse the frame index number and set the sensor for appropriate resolution.
The requested frame index number (as you are using single format so no need to parse format index) can be parsed from g1CommitCtrl global variable. For more details i.e. which array element need to parsed for frame index you can refer to UVC spec or refer the sample code in previous response
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, RashiV_61!
Thanks for your clarifications! Let me summarize my understandings so far and ask few more related questions...
When the change resolution request comes from the Host, UVCHandleVideoStreamingRqts() function will be called.
wValue = CY_FX_UVC_COMMIT_CTRL whereas
bRequest = CY_FX_USB_UVC_SET_CUR_REQ.
apiRetStatus = CyU3PUsbGetEP0Data (CY_FX_UVC_MAX_PROBE_SETTING_ALIGNED, glCommitCtrl, &readCount);
function will be called and, as a consequence, glCommitCtrl[] array will be filled with the details of the Host request.
glCommitCtrl[2] = format_index => we currently have only one format: YUY2
glCommitCtrl[3] = resolution_index
When we receive such a request from the Host, FX3 firmware should perform the following:
1) Set image sensor to start streaming desired resolution
2) Reply to the host with the following:
CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *)glProbeCtrl_XXX);
where glProbeCtrl_XXX probe we point to depends on the resolution_index. Each resolution supported should have its own glProbeCtrl_XXX settings where Video format (YUY2 in our case) as well as Video frame index (resolution_index) will be specified together with desired frame interval and max video frame size. Video frame index (resolution_index) is unique for each resolution supported.
In our case we currently have 7 different resolution specified with the following indexes:
typedef enum CyFx3SSYUY2VideoRes_t
{
YUY2_3214x2206_INDEX=1,
YUY2_4110x3006_INDEX,
YUY2_2054x1502_INDEX,
YUY2_2462x2054_INDEX,
YUY2_1934x1094_INDEX,
YUY2_1230x1026_INDEX,
YUY2_1606x1102_INDEX,
} CyFx3SSYUY2VideoRes;
Please let me know if above understanding is correct.
Thanks once again for your great support!
Cheers,
Bojan.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Bojan,
Please refer to the attachment which mentions the negotiation between host and device while changing settings dynamically
- VS_PROBE_CONTROL (SET_CUR)
This request will get the data (video resolution to be streamed) from the host using this API and store it in g1CommitCtrl
apiRetStatus = CyU3PUsbGetEP0Data (CY_FX_UVC_MAX_PROBE_SETTING_ALIGNED, glCommitCtrl, &readCount);
The g1CommitCtrl data should be parsed and stored in global variables like
Please refer to Video Probe and Commit Controls in UVC spec
glCurrentVideoFormatIndex = glCommitCtrl[2];
glCurrentVideoFrameIndex = glCommitCtrl[3];
glCurrentVideoFrameInterval = glCommitCtrl[3]; // 4bytes
- VS_PROBE_CONTROL(GET_CUR)
On this request the device will send the probe control structure of the video resolution asked by the host in the previous
This is as per the previous thread
For multiple resolution
you can parse the frame index number and send the probe control data of the appropriate resolution (as per the request from the host).
case FX3_USB_UVC_GET_CUR_REQ:
case FX3_USB_UVC_GET_MIN_REQ:
case FX3_USB_UVC_GET_MAX_REQ:
case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */
if (usbSpeed == CY_U3P_SUPER_SPEED)
{
if(glCurrentVideoFrameIndex == 2)
CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *glProbeCtrl_1280x720_60fps));
}... so on
break;
- VS_COMMIT_CONTROL (SET_CUR)
When this request is sent from host sensor needs to be configured for appropriate video resolution
case FX3_USB_UVC_SET_CUR_REQ:
switch (CY_U3P_GET_LSB(wIndex))
{
case FX3_UVC_STREAM_INTERFACE_1:
{
CyU3PThreadSleep(900);
CyFx3UvcAppSetVideoResolution(glCommitCtrl[2], glCommitCtrl[3], frameInterval, 0);
}
break;
}
}
void
CyFx3UvcAppSetVideoResolution(
uint8_t format_index,
uint8_t resolution_index,
uint32_t frame_interval,
)
{
glFrameSizeExpected = 0;
if (CyU3PUsbGetSpeed() == CY_U3P_SUPER_SPEED)
{
if(format_index == UYVY_FORMAT_INDEX)
{
switch (resolution_index)
{
case SS_YUY2_1920x1080_INDEX:
sensor_settings_1920*1080(); //setting the sensor to stream 1920*1080
break;
case SS_YUY2_1280x720_INDEX:
sensor_settings_1280*720(); //setting the sensor to stream 1280*1720
break;
case SS_YUY2_640x640_INDEX:
sensor_settings_640x640();
break;
Can we implement this advanced communication scheme within Video control interface or we need to implement some third interface between the Host and FX3 to achieve this? In Extension Unit perhaps?
>> . You can use extension unit to add vendor specific blocks to the UVC spec. This unit can have one or more inputs but only ony output. To access this extension unityou need to develop your custom host application
What I am missing here? How FX3 can tell the Host during start-up process which resolution will be played?
>> You can have more than one streaming interface. Yo can refer to UVC spec for basic details.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Bojan,
Apologies for the mistake.
I was referring to the image which i shared in the previous response. This was to make explanation more clear.
Regards,
Rashi
Rashi