Following a Network Request: DNS, Connections, TLS, and HTTP
When a website fails to load, find out how far the request got: did the hostname resolve, did the connection open, did TLS succeed, and did a server return an HTTP response? A 404 and a port that cannot be reached call for different investigations. A request may also pass through a proxy, so the server your browser connects to may differ from the program that handles the business operation.
Use the hypothetical URL https://api.example.com:443/health?full=1 to follow the steps. https selects secure HTTP access, api.example.com is the hostname, 443 is the port, /health is the path, and full=1 is the query. When the port is omitted, HTTPS defaults to 443 and HTTP to 80. The URL scheme supplies that convention; DNS A/AAAA records do not carry these ports. RFC 9110 §4.2
1. Resolve the hostname and interpret the cache
DNS associates domain names with typed records. An application usually asks the system's resolution interface, which can pass the query to a configured recursive resolver. When local information is insufficient, that resolver follows delegations through the domain tree to the authoritative servers responsible for the data. A cache hit can eliminate some of those queries. RFC 1034 §2.4 and §5.3
For address lookups, an A record supplies an IPv4 address; an AAAA record supplies an IPv6 address, as defined in RFC 3596 §2. A hostname can have several addresses. You still need a connection attempt to learn whether an address is reachable from your current network.
TTL, or time to live, is measured in seconds and normally governs how long a cached record can be used before its source must be consulted again. It governs cached copies rather than requiring every client to switch addresses at the same instant. RFC 1034 §3.6 A resolver with a serve-stale policy can continue returning expired records when it cannot refresh them from authoritative servers. RFC 8767 §4
Suppose a resolver caches an old address at 12:00:00 with a TTL of 300 seconds. At 12:01:00, the authoritative server changes the address and gives the new record a TTL of 60 seconds. The resolver does not refresh early or use an additional policy for serving expired data:
Lowering the TTL while changing the record cannot shorten a TTL already distributed. For a planned migration, lower it in advance and allow time for the old TTL to expire. Resolvers cache at different times. When investigating, record the hostname, record type, resolver, returned address, and remaining TTL.
The system's resolution interface can also use local configuration. Python's socket.getaddrinfo returns address information suitable for connecting, without DNS TTLs. The experiment below resolves localhost through that interface: it observes local name resolution rather than a public DNS exchange. DNS caches hold name records, whereas HTTP caches hold responses; reloading a page is not evidence that a DNS cache was cleared. HTTP caching model, RFC 9111
2. Addresses, ports, listeners, and connections
An IP address directs data toward a network endpoint; a port helps the operating system distinguish services. A TCP server binds an address and port, then listens for new connections. Binding to 127.0.0.1 accepts connections through the local IPv4 loopback path; binding to 0.0.0.0 covers all local IPv4 interfaces. The latter is a wildcard for listening: clients use an actual reachable address. Linux IPv4 socket documentation External access also depends on routing, firewalls, and port mappings.
A TCP connection is identified by a pair of endpoints, each with an address and port. Suppose a client connects from 192.0.2.20:53000 to 203.0.113.10:443. Port 53000 belongs to this client connection; 443 is the service port. Other clients can use the same destination port 443 while the operating system distinguishes their connections. TCP provides a reliable, ordered byte stream, using acknowledgments and retransmission to handle loss. An ordinary connection opens with SYN, SYN-ACK, and ACK. RFC 9293 §2.2 and §3.5
Opening a connection establishes a usable transport path. A TCP acknowledgment of bytes does not establish that the application parsed a request, checked permissions, or committed a database transaction. If the connection drops after an order submission, the client may not know whether an order was created. A client should not automatically retry a non-idempotent request unless it knows the actual semantics are safe to repeat or can establish that the original request was never applied. Idempotence means repeating a request has the same intended effect as applying it once. RFC 9110 §9.2.2
This TCP path applies to the HTTP/1.1 experiment here and to common HTTPS over TCP. HTTP/3 uses QUIC over UDP with a TLS handshake. When investigating HTTP/3, check the UDP path: reachable TCP port 443 alone cannot establish that its transport works. RFC 9114 §3
3. TLS: verify the peer and protect the channel
On a new HTTPS connection using certificates, the TLS handshake negotiates keys, and the server presents a certificate and proves possession of the corresponding private key. TLS 1.3 aims to provide authentication, confidentiality, and integrity: established channel data is readable only by the endpoints, and tampering is detectable. Data lengths can still be exposed. RFC 8446 §1 and §4.4.3
The client needs two separate checks: certificate validation, including trust-chain and validity-period checks (RFC 5280 §6.1), and a match for the host it intended to contact. For api.example.com, hostname matching uses DNS names in the certificate's subjectAltName. The client must not substitute an arbitrary name supplied by the server for its expected identity. Access by IP address requires the corresponding IP identity; a certificate for a domain on the same machine is not sufficient by itself. RFC 9525 §6
SNI, Server Name Indication, lets a client name its intended host during the TLS handshake so the server can select a certificate. It occurs at a different stage from the later HTTP Host field. RFC 6066 §3 Replacing a hostname with an IP address during a test can change SNI, certificate matching, and HTTP routing even if the connection reaches the same destination. Compare each separately.
TLS protects the channel between its two endpoints. A reverse proxy that terminates TLS can read the HTTP content; assess encryption on the proxy-to-backend connection separately. After certificate verification, the application still checks user credentials and business permissions. DNS results and established connections can be reused, so each HTTP request need not repeat name resolution, connection setup, and a full TLS handshake.
4. HTTP: methods, status codes, headers, and bodies
An HTTP request carries a method and target; a response carries a status code. Both can include headers and a body. HTTP/1.1 text lines and HTTP/2 or HTTP/3 frames express the messages differently while sharing these basic semantics. RFC 9110 §6
These definitions come from RFC 9110 §9.3. Read the method and path together: a path may accept POST while rejecting GET.
Headers refine the message's meaning. Host (often :authority in HTTP/2 and HTTP/3) identifies the target host and optional port. Content-Type gives the body's media type. Content-Length counts bytes rather than characters and helps delimit the HTTP/1.1 responses below. Authorization carries authentication credentials. Declaring a JSON media type still leaves the body to satisfy the API's requirements. RFC 9110 §7.2, §8.3, §8.6, and §11.6.2
The first status-code digit distinguishes 1xx informational responses, 2xx success, 3xx redirection, 4xx client errors, and 5xx server errors. For diagnosis, read the specific code and body. RFC 9110 §15
After receiving headers, check that the body is complete. A connection can break halfway through a response, or the JSON can fail to parse. These are later failures; a status code and a business result cannot substitute for each other.
5. Proxies and localhost: identify the caller's environment
A forward proxy sends requests on behalf of a client. A reverse proxy, or gateway, receives client requests and forwards them to a backend. RFC 9110 §3.7 Suppose a public entry point terminates HTTPS and uses HTTP to reach the backend. There are two independent connections:
If the browser receives a 502 from the entry point, it received an HTTP response on the first hop, while the backend may still have failed. Distinguish the client-to-entry connection from the entry-to-application connection, and use entry-point logs to identify the failed hop.
localhost refers to loopback in the environment making the connection, commonly IPv4 127.0.0.1 or IPv6 ::1. Name-resolution libraries should treat it specially, normally without public DNS queries. RFC 6761 §6.3 A laptop browser visiting localhost:8080 reaches a service on the laptop, not on a remote VPS.
For containers, identify the network namespace: the set of network interfaces and routes visible to a process. With a separate network namespace, 127.0.0.1 refers to the container itself; sharing the host's or another container's networking stack changes that scope. Docker networking documentation A proxy in a separate container therefore cannot use its own localhost to reach an application container automatically. It needs a reachable backend address or service name.
For configuration, continue with the VPS overview and Architecture and Safety. Cloudflare Workers explains how a managed runtime receives requests and returns responses. MCP explains tool messages and authorization when using its HTTP transport.
6. Local experiment: a working connection can return 404
Save the code below as request_trace.py and run python3 request_trace.py. It uses only Python's standard library, starts a temporary HTTP service on IPv4 loopback, makes two GET requests, and then attempts a connection to a port that is bound but not listening. Setting the server port to 0 lets the operating system choose an available port; the numbers can change between runs.
The server uses http.server, and the client uses http.client. This experiment uses plain HTTP to expose local resolution, TCP, and HTTP. Certificate checks and the TLS handshake belong to the HTTPS path described earlier.
This handler implements only GET: /health returns 200, and other paths return 404. BaseHTTPRequestHandler.path includes the query string, so urlsplit separates the path and query. This experiment ignores query parameters; /health?full=1 returns the same health-check result.
import socket
from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
from urllib.parse import urlsplit
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def do_GET(self):
if urlsplit(self.path).path == "/health":
status, body = 200, b'{"ok":true}\n'
else:
status, body = 404, b'{"error":"not_found"}\n'
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(body)
def log_message(self, format, *args):
pass
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
port = server.server_address[1]
worker = Thread(target=server.serve_forever, daemon=True)
worker.start()
conn = HTTPConnection("localhost", port, timeout=2)
try:
results = socket.getaddrinfo(
"localhost", port, socket.AF_INET, socket.SOCK_STREAM
)
print("resolve localhost (IPv4):", sorted({r[4][0] for r in results}))
print(f"listen: 127.0.0.1:{port}")
conn.connect()
print(f"TCP: {conn.sock.getsockname()} -> {conn.sock.getpeername()}")
for path in ("/health", "/missing"):
conn.request("GET", path, headers={"Host": f"localhost:{port}"})
response = conn.getresponse()
body = response.read()
print(f"GET {path} -> {response.status}; "
f"type={response.getheader('Content-Type')}; "
f"length={response.getheader('Content-Length')}; read={len(body)}")
print(body.decode("utf-8"), end="")
finally:
conn.close()
server.shutdown()
server.server_close()
worker.join()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as unused:
unused.bind(("127.0.0.1", 0))
try:
with socket.create_connection(unused.getsockname(), timeout=2):
print("unexpected connection")
except (ConnectionRefusedError, TimeoutError) as error:
print("bound port without listen:", type(error).__name__)
Output from one run on Python 3.13.15:
resolve localhost (IPv4): ['127.0.0.1']
listen: 127.0.0.1:64242
TCP: ('127.0.0.1', 64244) -> ('127.0.0.1', 64242)
GET /health -> 200; type=application/json; length=12; read=12
{"ok":true}
GET /missing -> 404; type=application/json; length=22; read=22
{"error":"not_found"}
bound port without listen: TimeoutError
64242 is the server's listening port; 64244 is the client's source port for this connection. Both requests reuse that connection and send no body. The response bodies contain 12 and 22 UTF-8 bytes respectively, including a trailing newline. length is the server's declared size; read is what the client actually received. The 404 for /missing shows a working HTTP exchange: investigate the resource path rather than treating it as a connection failure.
The final attempt timed out in this run; another system may refuse it immediately. Neither result carries an HTTP status code. Here, the experiment establishes that the port has no listener, so its cause is known. On a real network, a timeout alone cannot distinguish a listener problem from a firewall or routing problem.
7. Diagnose from the last successful layer
Record where the request starts, its URL, the actual peer, the last successful stage, and the exact error. Check each stage from the same client instead of combining laptop, container, and server observations into one imagined path.
A health check returning 200 establishes that the checked path works. If the actual task is to create an order, verify the order endpoint's authentication, input, and persisted result too. Let each step answer a concrete question before investigating later layers.