HTTP Get - Empty response body



  • Hello,

    I'm having a really weird issue where the response body of a GET http request is empty but it should have a JSON object.

    The request is done to a GoPro camera, using the Wifi API, and it's actually works, I can see my GoPro changing the state and the request response is 200, but the body is empty.

    I have been doing some testing and I found a pretty weird behaviour:

    • Any request to the GoPro API is getting empty body response using UIFlow/Microphyton (while body has content doing the request from my computer).
    • Serving a GoPro JSON object from my computer it works fine on UIFlow/Microphyton, response has content.
    • Any requests to my GoPro using Arduino IDE with WifiClient lib is getting the valid json response.

    I'm totally stuck here, not sure if M5Stack firmware or urequest lib has some kind of bug or whats it is happening. Any help is more than welcome :)

    Thanks!



  • Maybe you need set a some parameters in header of your request. like "Content-type". I use Restler app on android to testing requests.



  • Thank you for your answer, after all day testing I found the issue:

    The urequest library, which is used for the firmware, has an implementation that fails with the GoPro requests due to how it cleans the request body.

    During the clean up process of the body its split content by if not l or l == b"\r\n":,
    which probably is ok for most of the requests, but not too flexible or safe, but the GoPro use \n\n to split the Content-Type line from the actual body response.

    So in order to fix the issue, I need to modify the urequests implementation, from:
    if not l or l == b"\r\n":
    to
    if not l or l == b"\n" or l == b"\r\n":
    with this change, everything works fine.

    So my question is, it is possible to replace the urequests library included in the firmware with my fork/custom changes? I have seen how to load external libs, but I don't know the path for the urequest lib.

    Thank you!



  • I can not edit the post, sorry.

    Sample of the response bodies:

    Most common and working

    0_1592586738340_94870666-555a-4b5b-b9f4-d3075d8db6f9-image.png
    See its use all the time \r\n

    GoPro sample, and not working

    0_1592586692165_56dcbd62-936d-4bdc-a48c-f6abac19cfb3-image.png
    See it ends with \n\n b'HTTP/1.0 200 OK\r\n



  • I use urequests in raw python and i use something like this code below to parse response (i delete some unwanted characters by replacing it with empty string)

    response = urequests.get('http://...')

    if response is a json

    data = json.loads(response.text.replace('STRING_TO_REPLACE', 'NEW_STRING')

    other text

    data = response.text.replace('STRING_TO_REPLACE', 'NEW_STRING')



  • @robalstona said in HTTP Get - Empty response body:

    I use urequests in raw python and i use something like this code below to parse response (i delete some unwanted characters by replacing it with empty string)

    response = urequests.get('http://...')

    if response is a json

    data = json.loads(response.text.replace('STRING_TO_REPLACE', 'NEW_STRING')

    other text

    data = response.text.replace('STRING_TO_REPLACE', 'NEW_STRING')

    The problem is that the library is breaking the response before return it to me. When doing
    esponse = urequests.get('http://...')
    at that point the response body is already broken, that why I need to customize the library. It in deeper level the issue.