Urls are not checked for control characters.

[dolfno]

There is no check on whether a url contains control characters like _CONTAINS_CONTROL_CHAR_RE = re.compile(r"[^-!#$%&'*+.^_|~0-9a-zA-Z]")` from https://github.com/urllib3/urllib3/blob/0e2c3a392d4112a66611dcdbac3e34e2f6378db0/src/urllib3/connection.py#L78

This changes the url incl query params during sending the bytes via a connection.
I expect to have a least a warning within the urls.URL or the urlspars method.

[lovelydinosaur]

You're framing a discussion in terms of the code rather than the behaviour.

Can you demonstrate an issue against the public API that you believe needs to be resolved?

[dolfno]

>>> datetime.now().replace(tzinfo=pytz.utc).isoformat()
'2023-08-03T09:16:07.559733+00:00'
>>> url = f'https://postman-echo.com/time/valid?timestamp={datetime.now().replace(tzinfo=pytz.utc).isoformat()}'
>>> httpx.get(url)
<Response [200 OK]>

What happens is that the '+' sign get replaced during the 'bytes_to_send' step without the user being aware of this.
So the actual url being:
"https://postman-echo.com/time/valid?timestamp=2023-08-03T09:16:07.559733 00:00"
(note the space instead of the +)

Off course the proper way is to encode this url first, but as a user I would like to get a warning in case I send a non-compatible url aka having the url control characters in there.

So requested behviour:

>>> datetime.now().replace(tzinfo=pytz.utc).isoformat()
'2023-08-03T09:16:07.559733+00:00'
>>> url = f'https://postman-echo.com/time/valid?timestamp={datetime.now().replace(tzinfo=pytz.utc).isoformat()}'
>>> httpx.get(url)
WARNING: Url contains non-token characters, found +
<Response [200 OK]>

[lovelydinosaur]

What happens is that the '+' sign get replaced during the 'bytes_to_send' step without the user being aware of this.

From looking into this I don't believe this is the case. Could you make sure you're using the latest versions of the package and dependencies, and demonstrate to why you believe this to be true?

Related: It would be helpful if we had better tooling around being able to debug all the way through to bytes-on-the-wire. Here's what our logging can show us...

>>> import httpx
>>> httpx.URL('https://postman-echo.com/time/valid?timestamp=2023-08-03T11:14:30.905415+00:00')
URL('https://postman-echo.com/time/valid?timestamp=2023-08-03T11:14:30.905415+00:00')
>>> u = httpx.URL('https://postman-echo.com/time/valid?timestamp=2023-08-03T11:14:30.905415+00:00')
>>> u.raw_path
b'/time/valid?timestamp=2023-08-03T11:14:30.905415+00:00'
>>> import logging
>>> logging.basicConfig(
...     format="%(levelname)s [%(asctime)s] %(name)s - %(message)s",
...     datefmt="%Y-%m-%d %H:%M:%S",
...     level=logging.DEBUG
... )
>>> 
>>> httpx.get(u)
DEBUG [2023-08-03 11:17:31] httpx - load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG [2023-08-03 11:17:31] httpx - load_verify_locations cafile='/Users/tomchristie/GitHub/encode/httpcore/venv/lib/python3.10/site-packages/certifi/cacert.pem'
DEBUG [2023-08-03 11:17:31] httpcore.connection - connect_tcp.started host='postman-echo.com' port=443 local_address=None timeout=5.0 socket_options=None
DEBUG [2023-08-03 11:17:31] httpcore.connection - connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x10a605420>
DEBUG [2023-08-03 11:17:31] httpcore.connection - start_tls.started ssl_context=<ssl.SSLContext object at 0x10a53b140> server_hostname='postman-echo.com' timeout=5.0
DEBUG [2023-08-03 11:17:31] httpcore.connection - start_tls.complete return_value=<httpcore._backends.sync.SyncStream object at 0x10a6053f0>
DEBUG [2023-08-03 11:17:31] httpcore.http11 - send_request_headers.started request=<Request [b'GET']>
DEBUG [2023-08-03 11:17:31] httpcore.http11 - send_request_headers.complete
DEBUG [2023-08-03 11:17:31] httpcore.http11 - send_request_body.started request=<Request [b'GET']>
DEBUG [2023-08-03 11:17:31] httpcore.http11 - send_request_body.complete
DEBUG [2023-08-03 11:17:31] httpcore.http11 - receive_response_headers.started request=<Request [b'GET']>
DEBUG [2023-08-03 11:17:31] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 03 Aug 2023 10:17:31 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'20'), (b'Connection', b'keep-alive'), (b'ETag', b'W/"14-W4GFGHcakus25UQnkQz7QHUcbS8"'), (b'set-cookie', b'sails.sid=s%3AbBA2FiQqM_rx-T1vMxN-4Lustc1xHmw6.1E1NlSiH9UAymBT7gbzZewutdoTdn6u%2FSggEj75Tpnw; Path=/; HttpOnly')])
INFO [2023-08-03 11:17:31] httpx - HTTP Request: GET https://postman-echo.com/time/valid?timestamp=2023-08-03T11:14:30.905415+00:00 "HTTP/1.1 200 OK"
DEBUG [2023-08-03 11:17:31] httpcore.http11 - receive_response_body.started request=<Request [b'GET']>
DEBUG [2023-08-03 11:17:31] httpcore.http11 - receive_response_body.complete
DEBUG [2023-08-03 11:17:31] httpcore.http11 - response_closed.started
DEBUG [2023-08-03 11:17:31] httpcore.http11 - response_closed.complete
DEBUG [2023-08-03 11:17:31] httpcore.connection - close.started
DEBUG [2023-08-03 11:17:31] httpcore.connection - close.complete

For httpcore we've got an example of a network recoding backend implemented here... https://www.encode.io/httpcore/network-backends/#custom-network-backends perhaps we could eg. have options on the command-line-client for record/replay?