cURL beyond GET: an HTTP toolkit you already have

curl is rarely far away. When it is already on the shell, it can do much more than fetch a URL and print the body. The curl binary ships with around 257 command-line options¹. It is easy to reach for only a small handful. There is a gap there worth closing.

The first encounter with curl is almost always a one-liner that fetches a URL and prints the body. That use is fine, and it covers a real fraction of daily traffic. The trouble starts on the second invocation: a service has gone strange, a request that used to take 80ms is now taking 2.4 seconds, and the developer needs to know whether the slowness is in DNS, in the TLS handshake, in the server, or in the network in between. The reflex is to open browser devtools, or to install a heavier HTTP client, or to write a small Python script. The faster reflex is the curl already on the machine, with two flags the developer has not yet learned.

The rest of this piece walks through the handful of curl features that lift it out of "GET something and pipe it to less" territory and into a small, programmable HTTP toolkit. Timing breakdowns come first, then DNS overrides, then session cookies, then a credible smoke test, then a reference of the dozen flags worth keeping at the fingertips.

What curl is

curl is a command-line tool for moving data between a local machine and a remote server identified by a URL. The binary speaks a long list of protocols — HTTP, HTTPS, FTP, FTPS, SFTP, SCP, IMAP, SMTP, MQTT, and more². The same URL-shaped workflow reaches all of them, though individual flags still vary by protocol. Two distinct things travel under the name: the curl CLI binary, and libcurl, the shared library underneath it that ships inside browsers, package managers, telephones, and the printers in the next office over.

A first run looks unremarkable. Pointing curl at a public JSON endpoint produces the body on standard output:

# Command:
curl -s https://httpbin.org/json

# Output:
{
  "slideshow": {
    "author": "Yours Truly",
    "date": "date of publication",
    "slides": [
      {"title": "Wake up to WonderWidgets!", "type": "all"},
      {
        "items": [
          "Why <em>WonderWidgets</em> are great",
          "Who <em>buys</em> WonderWidgets"
        ],
        "title": "Overview",
        "type": "all"
      }
    ],
    "title": "Sample Slide Show"
  }
}

The body is printed; nothing else is. That is the design. curl is minimal-by-default — it does the thing the URL asks for and writes the response body to stdout, with everything else (headers, progress, timings, request internals) off unless the developer asks for it. The -s flag above hides the progress meter so the output of a piped command stays clean.

The mental model worth holding is small. curl thinks in terms of URLs, transfers, and options that control transfers. Daniel Stenberg, who has maintained curl since 1996, frames it similarly: "work on URLs, do transfers, set options to control said transfers"³. Everything else in the manual is a knob on one of those three axes.

The project began as HttpGet 0.1, written by Rafael Sagula and released on 11 November 1996⁴. Stenberg picked up maintenance and shipped 0.2 on 17 December 1996, then renamed the tool to urlget in August 1997 once it learned to fetch more than just HTTP, and renamed it again to curl — specifically, curl 4 — on 20 March 1998⁴. curl 8.3.0 shipped on 13 September 2023¹. A tool that has been continuously maintained for roughly twenty-seven years tends to have grown a few flags worth knowing.

What every byte cost: the timing breakdown

The single most useful curl invocation a developer can learn is the one that splits a slow request into the time spent on each phase. The mechanism is the -w (write-out) flag, which prints a custom template after the transfer finishes, with placeholders that curl substitutes for transfer statistics. The template can live in a file, referenced with -w "@filename".

The canonical timing template uses seven variables and looks like this:

# File: curl-format.txt
time_namelookup:    %{time_namelookup}s\n
time_connect:       %{time_connect}s\n
time_appconnect:    %{time_appconnect}s\n
time_pretransfer:   %{time_pretransfer}s\n
time_starttransfer: %{time_starttransfer}s\n
                    -----------\n
time_total:         %{time_total}s\n

# Command:
curl -w "@curl-format.txt" -o /dev/null -s "https://example.com"

# Output (illustrative, varies by network):
time_namelookup:    0.012s
time_connect:       0.034s
time_appconnect:    0.097s
time_pretransfer:   0.097s
time_starttransfer: 0.149s
                    -----------
time_total:         0.181s

The -o /dev/null discards the response body, the -s silences the progress meter, and the -w template prints exactly the fields a developer wants to see. Each variable is the cumulative number of seconds from the moment curl started until that phase finished⁵:

• time_namelookup — the time at which DNS resolution finished.
• time_connect — the time at which the TCP connection completed (the three-way handshake).
• time_appconnect — the time at which the TLS handshake finished, for HTTPS.
• time_pretransfer — the time at which curl was about to begin sending bytes.
• time_starttransfer — the time at which the first response byte arrived (this is roughly Time-To-First-Byte).
• time_total — the time at which the whole transfer finished.

Because each value is cumulative, the duration of any single phase is the difference between two consecutive variables. Cloudflare's "A Question of Timing" walks through the exact arithmetic⁶:

• DNS lookup: time_namelookup
• TCP connect: time_connect - time_namelookup
• TLS handshake: time_appconnect - time_connect
• Server processing: time_starttransfer - time_pretransfer
• Content transfer: time_total - time_starttransfer

A 2.4-second request that splits into "DNS 0.01s, TCP 0.03s, TLS 0.06s, server 2.30s" tells the developer the server is the problem. The same total split as "DNS 1.80s, TCP 0.03s, TLS 0.06s, server 0.51s" points at a misconfigured resolver. Without the breakdown, both look like the same ambiguous slow request.

curl 7.70.0 added a quality-of-life feature for pipelines: the %{json} write-out variable, which emits a single JSON object containing every available statistic. Stenberg's announcement post lays out the motivation — the old format was good for human eyes, but bad for machines, and the JSON form is meant to be piped straight into a shape-aware tool⁷. With jq on the other side of the pipe, a one-line assertion becomes possible:

# Command:
curl -sS -w '%{json}' -o /dev/null "https://example.com" | jq '{code: .http_code, total: .time_total}'

# Output:
{
  "code": 200,
  "total": 0.187
}

That single pipeline is the basic shape of every curl-driven monitoring check the rest of this article builds on.

Talking past DNS with --resolve

The second flag worth learning is --resolve, which tells curl to act as if the resolver had returned a specific IP address for a specific hostname and port. The hostname stays the same everywhere it matters — in the SNI extension of the TLS handshake, in the HTTP Host header, in certificate validation — but the connection actually reaches the IP the developer chose⁸⁵. Three concrete scenarios make the value clear.

The first is testing a new origin before DNS propagates. A team is migrating api.example.com from the old IP to a new one. The DNS record has been changed, but the world will not see it for the cache TTL. Hitting the new server right now without waiting is one flag away:

# Command (test new IP before DNS propagates):
curl --resolve api.example.com:443:198.51.100.25 https://api.example.com/health

# Output (illustrative):
{"status":"ok","version":"1.4.2"}

curl behaves as if 198.51.100.25 were the answer to the DNS query. The certificate is validated against api.example.com, the SNI tells the server which virtual host the developer wanted, and the Host header is correct — all the things that would go wrong if the developer reached for https://198.51.100.25/health and added -k to silence the certificate error.

The second is targeting one specific load-balancer backend. A service is fronted by four IPs behind a DNS round-robin. Three of them work; one returns the wrong answer for the same query. With --resolve, the developer can pin curl to each backend in turn and reproduce the bad behaviour without editing /etc/hosts and without root access:

# Command (pin to a single backend IP):
curl --resolve api.example.com:443:203.0.113.42 https://api.example.com/v1/users/42

# Output (illustrative):
{"id":42,"name":"Wrong","plan":"unknown"}

The third is bypassing a CDN to talk directly to the origin. A page is cached oddly at the edge; the developer wants to know whether the origin is producing the bad payload or whether the CDN is rewriting it. With --resolve pointing at the origin's real IP, the request skips the CDN entirely. That makes it a useful way to isolate "is it us or is it origin" questions.

A sibling flag, --connect-to, exists for the case where the developer wants to swap the connection target but keep the original hostname — including in SNI and the Host header — pointed at the destination they originally typed². The shape is --connect-to HOST:PORT:CONNECT_TO_HOST:CONNECT_TO_PORT. When the destination's TLS certificate or virtual-host routing depends on the original name being preserved everywhere, --connect-to is the right reach; for the IP-pinning case where the developer simply wants to override the resolver, --resolve is the better fit.

Sessions in three lines: cookie jars

The third feature worth a section is curl's built-in cookie handling. Two flags do the bulk of the work: -c FILE writes received cookies into the file (the "cookie jar"), and -b FILE reads cookies from the file and includes them in the request⁵. Used together, they turn a sequence of curl calls into a real session, with login, authentication, and logout all expressible in shell.

The canonical pattern is two commands. The first logs in and captures cookies; subsequent commands replay them:

# Step 1: log in and store cookies in session.txt.
curl -c session.txt \
  -d 'user=alice&pass=secret' \
  https://example.com/login

# Step 2: use the stored cookies on the next request.
curl -b session.txt \
  https://example.com/api/me

# Output (illustrative):
{"username":"alice","email":"alice@example.com","plan":"pro"}

In real use the same file is usually passed to both -b and -c on every command, so the jar reads existing cookies in and writes any new or updated cookies back out — the way a browser would handle it across a navigation:

# Read and write the same jar (browser-like behaviour):
curl -b session.txt -c session.txt \
  https://example.com/api/preferences

The jar file is plain text in Netscape cookie-file format, which is one of those specifications that has outlived its creator. A jar after the login above looks roughly like this:

# File: session.txt
# Netscape HTTP Cookie File
# https://curl.se/docs/http-cookies.html
# This file was generated by libcurl. Edit at your own risk.

#HttpOnly_example.com   FALSE   /       TRUE    1696118400      sessionid       9c7f1a2b8e4d
example.com             FALSE   /       FALSE   0               csrftoken       a14b7c

The columns are domain, whether the cookie applies to subdomains, path, secure-only flag, expiry as a Unix timestamp, name, and value. #HttpOnly_ is a curl-specific prefix marking cookies the server flagged with HttpOnly. The format is human-readable on purpose — a developer can open the jar, see what they've collected, and edit it in place if a test needs it.

A few security notes belong here. Cookie jars are plaintext on disk, which is exactly what their value implies: the contents of a jar are the keys to whatever session it represents. The standard cautions all apply. Do not commit jar files to version control. Do not pass them around over chat. Do not leave them on a shared CI runner without scrubbing them at the end of the job. And mind the redirect-and-credential combination: curl had a vulnerability, CVE-2022-27776, in which an HTTP request that included an Authorization header or a cookie could leak that credential to the same host on a different port or URL scheme⁹. The fix shipped in curl 7.83.0; the underlying lesson — be careful when combining -L (follow redirects) with sensitive credentials — is older than the bug, and worth keeping in mind whenever a one-liner mixes auth and -L.

A lightweight API smoke test

The features so far compose into something more useful than the sum of the parts. With -w for assertions, cookie jars for sessions, -d for request bodies, and --fail-with-body for shell-friendly exit codes, a hundred lines of bash can express a credible smoke test against a staging environment. The shape is straightforward: log in, verify a session call succeeds, create a resource, delete the resource, and exit non-zero on the first failure.

A complete example, kept short on purpose:

#!/usr/bin/env bash
set -euo pipefail

base="${1:?usage: smoke-test.sh BASE_URL}"
jar="$(mktemp)"
trap 'rm -f "$jar"' EXIT

step() { printf '[ %s ] %-30s ' "$1" "$2"; }
ok()   { printf 'OK\n'; }

step "1/4" "login..."
curl -fsS -c "$jar" \
  -d 'user=alice&pass=secret' \
  "$base/login" > /dev/null
ok

step "2/4" "read profile..."
who=$(curl -fsS -b "$jar" "$base/api/me" | jq -r '.username')
[ "$who" = "alice" ] || { echo "FAIL: expected alice, got $who"; exit 1; }
ok

step "3/4" "create item..."
id=$(curl --fail-with-body -sS -b "$jar" -c "$jar" \
  -H 'Content-Type: application/json' \
  -d '{"name":"smoke","value":1}' \
  "$base/api/items" | jq -r '.id')
printf '(id=%s) ' "$id"; ok

step "4/4" "delete item..."
curl --fail-with-body -sS -b "$jar" -X DELETE \
  "$base/api/items/$id" > /dev/null
printf '(id=%s) ' "$id"; ok

echo "all checks passed"

Two flags do the load-bearing work. -f (or --fail) makes curl exit non-zero on HTTP 4xx and 5xx, which composes with set -e so a failed step terminates the script⁵. --fail-with-body, added in curl 7.76.0, does the same job but still prints the response body to stdout, which means a script can preserve the server's error message in its log instead of swallowing it silently⁵. For any check that needs to know what failed, not just that something did, --fail-with-body is the better reach.

Running the script against a staging URL produces a tidy progress log:

# Command:
./smoke-test.sh https://staging.example.com

# Output:
[ 1/4 ] login...                          OK
[ 2/4 ] read profile...                   OK
[ 3/4 ] create item...                    (id=42) OK
[ 4/4 ] delete item...                    (id=42) OK
all checks passed

The pattern scales. With jq shaping the responses, a curl-driven smoke test can grow from one assertion to twenty in an afternoon — every endpoint a developer relies on, every shape it returns, every status code it should and should not produce. Run it from CI on a schedule, and a service has a working canary without a new dependency.

For this kind of script, the cookie jar belongs in a mktemp file with a trap to remove it on exit, exactly as above. A jar that lingers on disk between runs is a credential leak waiting for a misconfigured backup job to find it.

Twelve flags worth knowing

A short reference closes the practical part. These are the dozen flags worth keeping nearby; each one earns its place once curl becomes part of daily debugging.

A note on -X. The curl manpage is unusually direct: "Normally you do not need this option."⁵ Most flags imply the right method on their own — -d switches to POST, -T switches to PUT, -I switches to HEAD — and overriding the method explicitly with -X while also using one of those flags often produces requests that look subtly wrong on the wire. Stenberg's "Unnecessary use of curl -X" walks through the canonical examples of how this goes wrong, including the case where -X POST -d '' followed by a redirect causes curl to send the second request as a POST when the developer probably wanted a GET¹⁰. The reach is to let the body and method flags speak for themselves, and pull -X out only for verbs that have no shorthand — DELETE, PATCH, PROPFIND, and the rest of the long tail.

curl rewards the time spent learning it because the same habits travel well — staging, production, CI, a colleague's machine, or a freshly-provisioned VM that already has curl installed. Whatever the language and framework on disk this year, curl is often already on the shell, and the same -w pattern can diagnose slow HTTP services without another client.

The cheapest first step is to pick one diagnostic that has cost real time recently — DNS-versus-TLS-versus-server is the most common — and write the -w template for it once. Saved as ~/.curlrc or curl-format.txt, it pays back forever, on every machine the developer ever logs into, against every HTTP service they ever reach.

A binary that has been continuously maintained since 1996 is not a fashion. It is a foundation, and the flags worth knowing are the ones that turn a foundation into a tool.

---

Footnotes

1. Stenberg, Daniel. (2023). 'curl 8.3.0.' daniel.haxx.se. https://daniel.haxx.se/blog/2023/09/13/curl-8-3-0/ ↩ ↩²

2. Stenberg, Daniel. (2023). 'Everything curl.' everything.curl.dev. https://everything.curl.dev/ ↩ ↩²

3. Doerrfeld, Bill. (2023). 'Interview With curl Founder Daniel Stenberg.' Nordic APIs. https://nordicapis.com/interview-with-curl-founder-daniel-stenberg/ ↩

4. curl Project. (2023). 'History of curl.' curl.se. https://curl.se/docs/history.html ↩ ↩²

5. curl Project. (2023). 'curl(1) manual page, curl 8.3.0.' curl.se. https://curl.se/docs/manpage.html (canonical per-option sources: https://github.com/curl/curl/tree/curl-8_3_0/docs/cmdline-opts) ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

6. Cornwell, Piers. (2018). 'A Question of Timing.' Cloudflare Blog. https://blog.cloudflare.com/a-question-of-timing/ ↩

7. Stenberg, Daniel. (2020). 'curl write-out JSON.' daniel.haxx.se. https://daniel.haxx.se/blog/2020/03/17/curl-write-out-json/ ↩

8. Van Damme, Bramus. (2022). 'Force DNS resolving in cURL with the --resolve switch.' bram.us. https://www.bram.us/2022/02/10/force-dns-resolving-in-curl-with-the-resolve-switch/ ↩

9. curl Project. (2022). 'CVE-2022-27776: auth/cookie leak on redirect.' curl.se. https://curl.se/docs/CVE-2022-27776.html ↩

10. Stenberg, Daniel. (2015). 'Unnecessary use of curl -X.' daniel.haxx.se. https://daniel.haxx.se/blog/2015/09/11/unnecessary-use-of-curl-x/ ↩