http get data in chunks

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Anonymous
Not applicable

Hi.

I am trying to download a 24KB file in 4KB chunks with http_get?

Unfortunately, the way wiced_http_get is written, even though the buffer and buffer length passed into this function are less, it just gets overwritten without any notification to the user?

On line 96, we copy the data into the buffer passed by the caller of this function. How is the user notified in the next iteration of the do..while loop?

52 wiced_result_t wiced_http_get( wiced_ip_address_t* address, const char* query, void* buffer, uint32_t buffer_length )

53 {

54     wiced_tcp_socket_t socket;

55     wiced_packet_t*    reply_packet;

56     wiced_result_t     result = WICED_ERROR;

57     wiced_result_t     rx_result;

58     char *             buffer_ptr = (char*) buffer;

59

60     if ( ( result = wiced_tcp_create_socket( &socket, WICED_STA_INTERFACE ) ) != WICED_SUCCESS )

61     {

62         return result;

63     }

64     result = wiced_tcp_connect( &socket, address, 80, 10000 );

65     if ( result != WICED_SUCCESS )

66     {

67         wiced_tcp_delete_socket( &socket );

68         return ( result );

69     }

70

71     if ( ( result = wiced_tcp_send_buffer( &socket, query, (uint16_t) strlen( query ) ) != WICED_SUCCESS ) )

72     {

73         wiced_tcp_disconnect( &socket );

74         wiced_tcp_delete_socket( &socket );

75         return ( result );

76     }

77

78     WPRINT_LIB_INFO( ("waiting for HTTP reply\n") );

79

80     do

81     {

82         rx_result = wiced_tcp_receive( &socket, &reply_packet, 5000 );

83         if ( rx_result == WICED_SUCCESS )

84         {

85             uint8_t* data;

86             uint16_t data_length;

87             uint16_t available;

88             uint32_t data_to_copy;

89

90             /* Record the fact we received a reply of some kind */

91             result = WICED_SUCCESS;

92

93             /* Copy data into provided buffer */

94             wiced_packet_get_data( reply_packet, 0, &data, &data_length, &available );

95             data_to_copy = MIN(data_length, buffer_length);

96             memcpy( buffer_ptr, data, data_to_copy );

97             buffer_ptr    += data_to_copy;

98             buffer_length -= data_to_copy;

99             wiced_packet_delete( reply_packet );

100         }

101     } while ( rx_result == WICED_SUCCESS );

102

103     wiced_tcp_disconnect( &socket );

104     wiced_tcp_delete_socket( &socket );

105

106     return ( result );

107 }

Please help?

0 Likes
1 Solution
Anonymous
Not applicable

Hi.

I was able to re-purpose the wiced_http_get to handle multiple chunks of data ... In my case, I am trying to download some audio file from the AWS S3 storage server and I exchange a JSON to confirm the url, size, and md5 checksum. After this, I repeat the do..while loop until the sum of the data_length received is equal to the size from JSON.

90   do

291     {

292        rx_result = wiced_tcp_receive( &socket, &reply_packet, 5000 );

293        if ( rx_result == WICED_SUCCESS )

294        {

295            uint8_t* data;

296            uint16_t data_length;

297            uint16_t available;

298            uint32_t data_to_copy;

299

300            /* Record the fact we received a reply of some kind */

301            result = WICED_SUCCESS;

302

303            /* Copy data into provided buffer */

304            wiced_packet_get_data( reply_packet, 0, &data, &data_length, &available );

305            printf("data_length = %u, available = %u\n", data_length, available);

306            data_to_copy = MIN(data_length, buffer_length);

307            memcpy( buffer_ptr, data, data_to_copy );

308            printf("************* received data: %s\n", data);

309            //buffer_ptr    += data_to_copy;

310            //buffer_length -= data_to_copy;

311

312            extrapolate_data_from_http_reply((uint8_t*)data, data_length);

313

314            if(chunk_index%2 == 0) {

315              buffer_ptr = (char*) app_rx_data;

316            } else {

317              buffer_ptr = (char*) http_rx_buffer;

318            }

319            ++chunk_index;

320

321            wiced_packet_delete( reply_packet );

322        }

323     } while ( rx_result == WICED_SUCCESS || num_audio_file_bytes_recvd < audio_file_size);

324

I do have a new function that takes care of extrapolating the HTTP header and body ...Also, I switch the use of buffers to accommodate a faster http vis-a-vis the saving the audio bytes on a sector-by-sector basis in the SPi-FLASH.

View solution in original post

0 Likes
1 Reply
Anonymous
Not applicable

Hi.

I was able to re-purpose the wiced_http_get to handle multiple chunks of data ... In my case, I am trying to download some audio file from the AWS S3 storage server and I exchange a JSON to confirm the url, size, and md5 checksum. After this, I repeat the do..while loop until the sum of the data_length received is equal to the size from JSON.

90   do

291     {

292        rx_result = wiced_tcp_receive( &socket, &reply_packet, 5000 );

293        if ( rx_result == WICED_SUCCESS )

294        {

295            uint8_t* data;

296            uint16_t data_length;

297            uint16_t available;

298            uint32_t data_to_copy;

299

300            /* Record the fact we received a reply of some kind */

301            result = WICED_SUCCESS;

302

303            /* Copy data into provided buffer */

304            wiced_packet_get_data( reply_packet, 0, &data, &data_length, &available );

305            printf("data_length = %u, available = %u\n", data_length, available);

306            data_to_copy = MIN(data_length, buffer_length);

307            memcpy( buffer_ptr, data, data_to_copy );

308            printf("************* received data: %s\n", data);

309            //buffer_ptr    += data_to_copy;

310            //buffer_length -= data_to_copy;

311

312            extrapolate_data_from_http_reply((uint8_t*)data, data_length);

313

314            if(chunk_index%2 == 0) {

315              buffer_ptr = (char*) app_rx_data;

316            } else {

317              buffer_ptr = (char*) http_rx_buffer;

318            }

319            ++chunk_index;

320

321            wiced_packet_delete( reply_packet );

322        }

323     } while ( rx_result == WICED_SUCCESS || num_audio_file_bytes_recvd < audio_file_size);

324

I do have a new function that takes care of extrapolating the HTTP header and body ...Also, I switch the use of buffers to accommodate a faster http vis-a-vis the saving the audio bytes on a sector-by-sector basis in the SPi-FLASH.

0 Likes