RAW (mono) data streaming from FX3

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
AKS
Level 4
Level 4
100 sign-ins 10 questions asked 50 replies posted

Hi, I have interfaced the camera sensor to FX3 board. The output data is in the YUV2 format (as default in AN75779). I want to display the data in RAW format.  Does FX3 support sending RAW data through USB?
How can I do that and where I need to do changes accordingly.

Please suggest~

Best Regards,

/Ash

0 Likes
1 Solution
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

As mentioned earlier, RAW formats are not supported by UVC driver. If the GUID is set to RAW format, UVC driver might drop the data which is sent by FX3 and no video can be seen in Standard Video applications. Please refer to this KBA

As the sensor is streaming RAW16, you can stream the data as YUY2 format (16 bits/pixel). Please note this will show greenish/pinkish video stream on UVC application as the UVC driver will accept the data assuming it is YUV formatted and try to decode it.
So, it's necessary to create a custom host application to decode this RAW16 assumed to be YUV video format properly.

Regards,
Rashi

View solution in original post

21 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Please refer to section  9.4 of the app note for the instances were the changes needs to be done

https://www.infineon.com/dgdl/Infineon-AN75779_How_to_Implement_an_Image_Sensor_Interface_with_EZ-US...

As you want the output video format as YUV2, the video format in the descriptor can be kept same and accordingly the video resolution should be calculated (to populate in th descriptors)

For example:

Original video frame size: 1280*720 * 1(mono) = 921600 bytes

As video will be sent as YUV2 (16 bytes/pixel), video frame size = 640 * 720 * 2 (YUV2) = 921600 bytes

 

Regards,
Rashi
0 Likes
AKS
Level 4
Level 4
100 sign-ins 10 questions asked 50 replies posted

@Rashi_Vatsa Hi, Thanks.

I want to display mono raw data (qvga) stream, not yuv2 format. The application note defines descriptior which defines the output format as yuv2. 

How can I set output data format as raw (mono)?

Please suggest~

Best Regards,

/Ash

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

To change the output format in AN75779, please make necessary changes in GUID used to identify streaming-encoding format field and Number of bits per pixel used field of the Class specific Uncompressed VS format descriptor

Please note applications like AMCap, eCamViewer do not support RAW format streaming. Please use appropriate UVC host app to view the RAW video stream

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hi.

This defines in application note as "GUID used to identigy streaming-encoding format:YUV2" for setting output format as YUV2.

/* GUID used to identify streaming-encoding format: YUY2 */

0x59,0x55,0x59,0x32, 
0x00,0x00,0x10,0x00,
0x80,0x00,0x00,0xAA,
0x00,0x38,0x9B,0x71,

Where I can find the GUID for RAW format, any suggestions?

Also, please let me know any available host application which can stream raw data.

Regards,

/Ash

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted
0 Likes

@Rashi_Vatsa Hi,

I tried to go through it but didn't get much from it. During, I found this and tried to set it up with most of the required formats (Y16), but didn't work. https://www.fourcc.org/yuv.php

Is there any working GUID for Y16, or mono format, please suggest~

Thanks,

Regards,

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Please let me know if this works for Y8

'Y', '8', ' ', ' ',
0x00, 0x00, 0x10, 0x00,
0x80, 0x00, 0x00, 0xAA,
0x00, 0x38, 0x9B, 0x71, // End of GUID

0x08, /* Number of bits per pixel: 8 */

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hi, Thanks.

I'll give it a try and update.

well, since my data is 2byte coming on 8bit parallel port from sensor, Should I not use Y16 format? like this if it allows....

'Y', '16', ' ', ' ',
0x00, 0x00, 0x10, 0x00,
0x80, 0x00, 0x00, 0xAA,
0x00, 0x38, 0x9B, 0x71, // End of GUID

0x10, /* Number of bits per pixel: 8 */

Regards,

0 Likes

@Rashi_Vatsa Hi,

I tried with this...

'Y', '8', ' ', ' ',
0x00, 0x00, 0x10, 0x00,
0x80, 0x00, 0x00, 0xAA,
0x00, 0x38, 0x9B, 0x71, // End of GUID

0x08, /* Number of bits per pixel: 8 */

this gives me buidl error as: error: empty character constant

please suggest~

Regards,

0 Likes
lock attach
Attachments are accessible only for community members.
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

I have tried building AN75779 with the Y8 GUID and  8bits/pixel settings in the descriptor file and it builds successfully.

Please find the modified descriptor file. Please note that the only modification done in the file is GUID (Y8) and bits/ pixel

For using Y16, following are the settings

'Y', '16', ' ', ' ',
0x00, 0x00, 0x10, 0x00,
0x80, 0x00, 0x00, 0xAA,
0x00, 0x38, 0x9B, 0x71, // End of GUID

0x10, /* Number of bits per pixel: 16*/

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hi, Thanks.

May I please know which host application are you tested with to stream the raw data?

best regards,

 

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

We haven't tested streaming Y8/Y16 format as UVC applications like e CamViewer, AmCap doesn't support these formats.

You can try streaming using the Windows in- box camera

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hi,

Do you mean using the default windows10 camera app or any other? I am not sure but does it support raw streaming?

Regards.

0 Likes
AKS
Level 4
Level 4
100 sign-ins 10 questions asked 50 replies posted

@Rashi_Vatsa Hi,

I tried it with windows default camera app, but nothing came up.

Please suggest~

Regards.

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hi,

RAW formats are not supported by UVC specification.

You can stream these formats as YUY2 using UVC host applications like AmCap, MPC-HC

Although, we have seen some customers using Potplayer for streaming Y8 format on Windows but we haven't tested it at our end. You can try testing at your end

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hello.

Yes, I tried with Potplayer with Y16 format. It gives me codec error...

Ash_0-1642580257232.png

though I defined 'Y16' as format, player takes it as 'Y6', don't know why?

Regards.

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Apologies for mistake.

Can you please try this GUID and let me know if it works

'Y', '1', '6 ', ' ',
0x00, 0x00, 0x10, 0x00,
0x80, 0x00, 0x00, 0xAA,
0x00, 0x38, 0x9B, 0x71, 

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hello.

Thanks for sharing this, I tried it again and I got.

'Y','1','6',' ',
0x00,0x00,0x10,0x00,
0x80,0x00,0x00,0xAA,
0x00,0x38,0x9B,0x71,

AKS_1-1642653283996.png

Regards.

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

Please check if Y8 works fine. As mentioned earlier, we have seen some customers using Potplayer for streaming Y8 format on Windows

Regards,
Rashi
0 Likes

@Rashi_Vatsa Hello.

yes, I tried with potPlayer but failed.

AKS_1-1642996234161.png

Regards.

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

As mentioned earlier, RAW formats are not supported by UVC driver. If the GUID is set to RAW format, UVC driver might drop the data which is sent by FX3 and no video can be seen in Standard Video applications. Please refer to this KBA

As the sensor is streaming RAW16, you can stream the data as YUY2 format (16 bits/pixel). Please note this will show greenish/pinkish video stream on UVC application as the UVC driver will accept the data assuming it is YUV formatted and try to decode it.
So, it's necessary to create a custom host application to decode this RAW16 assumed to be YUV video format properly.

Regards,
Rashi