Httpx behaving differently than requests?

[EricPanDev]

the data paramater in httpx.post and httpx.client.post behave differently than requests.post

Example with discord's API:

This will not work

import httpx

header = {
    'Authorization': "Bot token",
}

files = {
    "file": ("hi.png", open("./image.png", 'rb'), "image/png")
}

payload = {
    "content": "pls work",
    "allowed_mentions": {
        "parse": []
    }
}

channel_id = "1079501847720640682"

r = httpx.post(f"https://discord.com/api/v9/channels/{channel_id}/messages", data=payload, headers=header, files=files)

print(r.status_code)
print(r.text)

However with requests, this will work fine

import requests

header = {
    'Authorization': "Bot token",
}

files = {
    "file": ("hi.png", open("./image.png", 'rb'), "image/png")
}

payload = {
    "content": "pls work",
    "allowed_mentions": {
        "parse": []
    }
}

channel_id = "1079501847720640682"

r = requests.post(f"https://discord.com/api/v9/channels/{channel_id}/messages", data=payload, headers=header, files=files)

print(r.status_code)
print(r.text)

I am unsure if this is a bug or if I need to pass these arguments differently in httpx

[EricPanDev]

Here is the console output

eric@Erics-MacBook-Air Gen5 % /usr/bin/python3 
/Users/eric/Documents/Gen5/test.py
Traceback (most recent call last):
  File "/Users/eric/Documents/Gen5/test.py", line 20, in <module>
    r = httpx.post(f"https://discord.com/api/v9/channels/{channel_id}/messages", data=payload, headers=header, files=files)
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_api.py", line 319, in post
    return request(
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_api.py", line 106, in request
    return client.request(
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_client.py", line 814, in request
    request = self.build_request(
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_client.py", line 357, in build_request
    return Request(
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_models.py", line 340, in __init__
    headers, stream = encode_request(
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_content.py", line 208, in encode_request
    return encode_multipart_data(data or {}, files, boundary)
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_content.py", line 153, in encode_multipart_data
    multipart = MultipartStream(data=data, files=files, boundary=boundary)
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_multipart.py", line 211, in __init__
    self.fields = list(self._iter_fields(data, files))
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_multipart.py", line 221, in _iter_fields
    yield DataField(name=name, value=value)
  File "/Users/eric/Library/Python/3.9/lib/python/site-packages/httpx/_multipart.py", line 50, in __init__
    raise TypeError(
TypeError: Invalid type for value. Expected primitive type, got <class 'dict'>: {'parse': []}

[mdlam92]

the APIs are slightly different, it looks like httpx typehints the data kwarg as a Mapping[str, Any] but requests is a straight dict

[EricPanDev]

I mean it works fine if the dict passed to the data paramater only has one item in httpx, but if I have 2 or more keys it raises that error

[lovelydinosaur]

Thanks for the question...

Form data (URL encoded or Multipart) can only represent plain data. Everything on the wire is just a string, so we don't want httpx to represent dict values and it's (correctly) raising a type error.

There's scope for:

• Improving the documentation.
• Improving the TypeError to point more closely at where the error is occuring.

[EricPanDev]

Wait so then what would be the equivalent in httpx? Would I pass a url encoded string to data or..?

[lovelydinosaur]

Oh that was surprising...

>>> payload = {"content": "pls work", "allowed_mentions": {"parse": []}}
>>> response = requests.post("https://example.lorg", data=payload)
>>> print(response.request.body)
content=pls+work&allowed_mentions=parse

[MtkN1]

This discussion is probably about Discord API parameters.

I'm interested in the Discord API, so I looked into it ... https://discord.com/developers/docs/reference#uploading-files

The JSON message body can optionally be provided using the payload_json parameter.

This solves the problem.

import json
...
payload_json = {"payload_json": json.dumps(payload)}
r = httpx.post(f"https://discord.com/api/v9/channels/{channel_id}/messages", data=payload_json, headers=header, files=files)

✅

---

I don't know if the form data encoding of requests is correct 🤔

[lovelydinosaur]

I've escalated this to an issue... #3127

[AneryCoft]

I think Httpx shoud be like reqests to automatically decode the byte type in the form to the str type,