jq: a Swiss Army knife for JSON on the command line

A developer is exploring an API. They curl an endpoint, and five hundred lines of JSON pour onto the terminal. The value they actually want — say, the user IDs of everyone who has been assigned an open issue — is buried three levels deep inside a nested array. Reaching for grep misses the tree structure entirely; hand-parsing with sed is a small crime against readable code; copying the response into a file and opening the browser's devtools feels absurd for what ought to be a three-line script.

The correct answer has been sitting in package managers and download pages for years. jq is a small command-line tool that treats JSON as a first-class input and output. Given a filter expression, it reads JSON on standard input and writes JSON on standard output, and the filter language is rich enough to reach, reshape, and summarise almost anything the developer is likely to throw at it.

For anyone who has written two or three ad-hoc JSON parsers in shell, jq ends the struggle. The rest of this piece walks through the handful of expressions that cover most day-to-day use, the curl + jq pattern that turns any REST API into a one-liner, and the places where jq is not the right reach.

What jq is

jq is a small command-line JSON processor. A jq program is a filter: it takes input on standard input, runs an expression over it, and produces output on standard output. The filter can be as simple as "pretty-print the input unchanged" or as elaborate as a multi-stage transformation that slices, joins, and renames its way across a nested document.

The language shape is worth a paragraph. jq filters compose with the pipe character |, exactly like shell pipes do: the output of the left-hand filter becomes the input of the right-hand filter. A jq program is read left-to-right as a sequence of stages, each one transforming what the previous stage produced. This is the single idea on which the rest of the tool is built, and it is why a developer who is comfortable with Unix pipes picks jq up quickly.

The project's own manual describes a jq program as a filter: it takes input, then produces output¹. Everything else is built on that premise. The simplest useful filter is a single dot, which means "the input, unchanged", and has the convenient side effect of pretty-printing whatever comes in:

echo '{"name":"Ada","role":"engineer","skills":["rust","spec"]}' | jq '.'

Running this command produces indented, colour-highlighted output in the terminal:

{
  "name": "Ada",
  "role": "engineer",
  "skills": ["rust", "spec"]
}

One small step up from that is a dot path. The filter .name reaches into the object and returns the value bound to the name key:

# Command:
echo '{"name":"Ada","role":"engineer"}' | jq '.name'

# Output:
"Ada"

The . on its own is the identity operator — it returns the input untouched. .name extends the identity with a lookup. Paths chain with further dots, so .user.profile.location walks three levels down a nested object in the obvious way.

A short history is in order. jq was created by Stephen Dolan². The first public releases, jq 1.0 and jq 1.1, were both tagged on 21 October 2012². jq 1.5 landed on 15 August 2015², and jq 1.6 shipped on 2 November 2018³. The project's homepage puts the pitch in a single line: "jq is like sed for JSON data"⁴ and describes it as a portable-C tool with zero runtime dependencies⁴.

Installation is uneventful. The project's homepage links to pre-built binaries and source tarballs⁴, and on most developer machines the shortest path is the platform package manager. On macOS with Homebrew, brew install jq is the canonical one-liner. MacPorts uses sudo port install jq; Debian and Ubuntu use sudo apt install jq; Fedora uses sudo dnf install jq; Arch uses sudo pacman -S jq; Alpine uses sudo apk add jq; openSUSE uses sudo zypper install jq.

A quick version check afterwards confirms the binary is on PATH:

# Command:
jq --version

# Output:
jq-1.6

On a jq 1.6 install, jq --version reports jq-1.6³.

The four moves that cover most daily use

Four building blocks do the bulk of the work. A developer who has these four under their fingers can read most jq one-liners on sight and write their own without reaching for the manual.

The first is dot navigation. Dots walk into objects; square brackets walk into arrays. The special form .[] iterates an array, emitting each element as a separate output value. Here is issues.json:

[
  {
    "number": 42,
    "title": "Crash when parsing empty input",
    "state": "open",
    "user": { "login": "ada" }
  },
  {
    "number": 47,
    "title": "Document the --slurp flag",
    "state": "closed",
    "user": { "login": "bjorn" }
  },
  {
    "number": 51,
    "title": "Typo in README",
    "state": "open",
    "user": { "login": "ada" }
  }
]

The filter .[] | .title prints every title on its own line:

# Command:
jq '.[] | .title' issues.json

# Output:
"Crash when parsing empty input"
"Document the --slurp flag"
"Typo in README"

The filter reads as "for each element of the input array, output its title field". The iteration happens on the left of the pipe; the projection happens on the right.

The second is select(condition), which keeps only the values for which the condition is truthy and drops the rest. Filtering the same issues.json down to the issues that are still open:

# Command:
jq '.[] | select(.state == "open")' issues.json

# Output:
{
  "number": 42,
  "title": "Crash when parsing empty input",
  "state": "open",
  "user": { "login": "ada" }
}
{
  "number": 51,
  "title": "Typo in README",
  "state": "open",
  "user": { "login": "ada" }
}

For each issue, jq evaluates .state == "open". If the comparison is true, the input passes through; if not, it is discarded. select is the workhorse of any search that would otherwise live inside a hand-written shell loop.

The third is object-construction shorthand. jq will happily accept a full spelling of the object literal, with every key explicitly bound, but for the common case of "keep these fields and drop the rest" there is a sugar: a bare identifier inside {} means "this key, pulled from the input". So these two filters are equivalent:

jq '.[] | {title: .title, number: .number}' issues.json
jq '.[] | {title, number}' issues.json

Both produce the same stream of narrowed objects:

{
  "title": "Crash when parsing empty input",
  "number": 42
}
{
  "title": "Document the --slurp flag",
  "number": 47
}
{
  "title": "Typo in README",
  "number": 51
}

The second form is an enormous quality-of-life improvement on a real API response, where the fields to keep are often a dozen. The braces can nest and interleave with full key-value pairs, so mixing kept-as-is fields with renamed ones is straightforward.

The fourth is map(f), which applies the filter f to each element of an input array and collects the results back into an array. Given a small users.json:

[
  { "name": "Ada", "role": "engineer" },
  { "name": "Bjorn", "role": "designer" },
  { "name": "Chen", "role": "engineer" }
]

Turning the list of user objects into a bare list of usernames is a one-liner:

# Command:
jq 'map(.name)' users.json

# Output:
[
  "Ada",
  "Bjorn",
  "Chen"
]

map(f) is shorthand for [.[] | f]: iterate the array, apply f, and wrap the stream back up in an array. It is worth writing out the long form once to see the machinery, then using the short form everywhere after.

These four moves — dot navigation, select, the {field} shorthand, and map — compose into almost anything a developer needs to do with a JSON document. The rest of this article is built on them.

Reshaping API responses

An API rarely returns only the fields a downstream script actually cares about. GitHub's issues endpoint returns dozens of fields per issue; a quick dashboard needs four. Whether the array came from curl or from a file on disk, the reshape pattern is the same. Reusing the same issues.json from earlier:

# Command:
jq '[.[] | {number, title, user: .user.login, state}]' issues.json

# Output:
[
  {
    "number": 42,
    "title": "Crash when parsing empty input",
    "user": "ada",
    "state": "open"
  },
  {
    "number": 47,
    "title": "Document the --slurp flag",
    "user": "bjorn",
    "state": "closed"
  },
  {
    "number": 51,
    "title": "Typo in README",
    "user": "ada",
    "state": "open"
  }
]

The outer [...] rebuilds an array from the stream of objects produced by the pipe. Each new object keeps number, title, and state as-is, and flattens user.login into a single user field. The output is a clean, narrow array ready to be piped into another tool or pretty-printed for a human.

Key renaming comes up often enough to deserve its own pattern. Given a single issue in one-issue.json:

{
  "number": 42,
  "title": "Crash when parsing empty input",
  "state": "open",
  "user": { "login": "ada" }
}

The one-off rename form is a direct rebinding inside the object literal:

# Command:
jq '{id: .number, headline: .title, who: .user.login}' one-issue.json

# Output:
{
  "id": 42,
  "headline": "Crash when parsing empty input",
  "who": "ada"
}

For bulk renaming across many keys, with_entries is the right tool. It turns an object into a list of {key, value} pairs, applies a transformation, and turns it back into an object. DJ Adams has a good walk-through of the pattern in a data-reshaping tutorial⁵. A common concrete example is converting snake-case keys to camel-case across a whole document. Given snaked.json:

{
  "first_name": "Ada",
  "last_name": "Lovelace",
  "years_active": 47
}

Apply the transformation:

# Command:
jq 'with_entries(.key |= gsub("_(?<c>.)"; .c | ascii_upcase))' snaked.json

# Output:
{
  "firstName": "Ada",
  "lastName": "Lovelace",
  "yearsActive": 47
}

Reading left to right: for each entry, update the key in place by substituting every underscore-followed-by-letter with the upper-cased letter. This is jq doing what shell scripting does badly and Python does verbosely, in one line.

When the target is a spreadsheet or a dashboard that chews through rows, not nested structures, jq has built-in string formatters for CSV and TSV. The Programming Historian lesson on reshaping JSON with jq shows the shape of it⁶. Reusing the issues.json from earlier:

# Command:
jq -r '.[] | [.number, .title, .state] | @csv' issues.json

# Output:
42,"Crash when parsing empty input","open"
47,"Document the --slurp flag","closed"
51,"Typo in README","open"

The -r flag switches jq into raw-output mode, which strips the surrounding JSON quotes from strings so the CSV is actually valid CSV. @csv takes an array and emits a comma-separated, correctly-quoted row — including handling commas, quotes, and embedded newlines inside field values. @tsv does the tab-separated equivalent, and is gentler on anything that needs to round-trip through a spreadsheet.

curl plus jq, the two-command API explorer

The pattern that earns jq its place in the toolbox is the two-command form: curl -s URL | jq 'filter'. The -s (silent) flag stops curl from writing progress bars to standard error, where they would not corrupt the pipe but would clutter the terminal. The same shape appears in walkthroughs from Contentful's engineering blog⁷ and GitLab's developer blog⁸, because it is a natural fit for poking at a REST API.

Capturing one value into a shell variable is the most common side of the pattern. Running a filter that selects a single string, with -r to strip the outer JSON quotes, is the trick:

{
  "id": 100,
  "login": "alice",
  "name": "Alice Example"
}

Against that response, the shell capture looks like this:

# Command:
user=$(curl -s https://api.example.com/profile | jq -r '.login')
echo "logged in as $user"

# Output (illustrative, trimmed):
logged in as alice

Without the -r, $user would include surrounding JSON quotes, and the next shell command that read it would get a nasty surprise. With -r, it holds the unadorned login name.

The inverse pattern — constructing a JSON body from shell variables and POSTing it — is where developers most often reach for printf or string concatenation and most often get burned. Shell quoting and JSON escaping are both hard on their own; composing them by hand is where production bugs live. jq's -n (null input) and --arg flags exist exactly for this:

# Input:
TITLE='Crash when parsing empty input'
BODY='Occurs when stdin is empty.'

# Command:
jq -n --arg title "$TITLE" --arg body "$BODY" \
  '{title: $title, body: $body}' \
  | curl -s -X POST -H 'Content-Type: application/json' \
      -d @- https://api.example.com/issues

# Output (illustrative, trimmed):
{
  "id": 100,
  "title": "Crash when parsing empty input",
  "body": "Occurs when stdin is empty."
}

-n tells jq not to wait for input — it runs the filter once against null. --arg name value binds $name inside the filter as a proper JSON string literal, correctly escaping quotes, backslashes, and whatever else the shell variable contained. The filter then constructs the object, pipes it to curl, and -d @- tells curl to read the body from standard input. Shell quoting and JSON escaping both become someone else's problem. GitLab's engineering blog demonstrates the same pattern in a CI-focused workflow⁸.

A second practical pattern uses jq to generate a batch of shell commands from a JSON listing. The same interpolation works whether the JSON came from curl or from a file. Given servers.json:

[
  { "host": "web-01", "user": "alice" },
  { "host": "web-02", "user": "alice" },
  { "host": "db-01", "user": "bob" }
]

Against that file, the one-liner looks like this:

# Command:
jq -r '.[] | "ssh \(.user)@\(.host)"' servers.json

# Output:
ssh alice@web-01
ssh alice@web-02
ssh bob@db-01

The \(expr) syntax is string interpolation inside a jq string literal. With -r, the output is a stream of plain lines suitable for piping into sh or an orchestration tool.

I have used this shape more times than I care to count, often against an internal service-registry API, and every single time it has felt faster than the Python alternative.

Things that bite

jq is not perfect, and a handful of papercuts are worth knowing about before the first production incident introduces them rudely.

The first is shell quoting. The manual's own advice is explicit: "it's best to always quote (with single-quote characters) the jq program"¹. Double quotes let the shell interpolate $ signs and backticks into the filter before jq ever sees it, which is sometimes what a developer wants and more often a subtle source of mystery errors. Default to single quotes; use --arg when a shell variable needs to reach into the filter.

The second follows from the first. Passing values via --arg name value binds $name inside the jq program as a JSON string, fully escaped. This is strictly better than sprintf-ing the shell variable into the filter text, because the former understands what a JSON string is and the latter does not. The GitLab post makes the same point in a CI context, where filter strings tend to be long-lived and shell variables tend to contain surprises⁸.

The third is null propagation. In jq 1.6, reaching into a key that is not present returns null, not an error:

# Command:
echo '{"a": 1}' | jq '.b.c'

# Output:
null

That null then flows downstream silently. A filter like select(.age > 18) applied to a record missing age evaluates null > 18, which is false, which quietly drops the record. The cautious habits are to use the ? suffix for genuinely optional paths (.maybe.here?), and // for a default value (.age // 0). Neither of these is hard; forgetting them is the bug.

The fourth is numbers. jq 1.6 represents JSON numbers as IEEE-754 double-precision floats. The wiki FAQ is explicit about the consequence: very large integers can lose precision during parsing and re-emission⁹. In most day-to-day use this is invisible. When it bites — 64-bit Twitter-style IDs, precise currency values — the bite is real, and the workaround is usually to keep the field as a string on its way through the pipeline.

When jq isn't the answer

jq is small and sharp, and a small sharp tool is by definition the wrong choice for some jobs.

The first is genuinely huge JSON values. jq's ordinary input mode processes a stream of JSON values one at a time, but a single enormous object or array can still force the useful part of the problem into memory before the filter can reshape it. jq has a --stream mode that emits a sequence of path-leaf pairs as the parser walks the document, bounding memory use at the cost of a more complicated filter. The project's manual documents the streaming parser, and the wiki cookbook has worked examples¹¹⁰. Streaming is the right reach for anything big enough to matter.

The second is interactive exploration when the shape of the document is genuinely unknown. jq is optimised for "I know roughly what I want and need to write it down once". For the earlier phase of "I have no idea what this JSON contains", tools like Anton Medvedev's fx (an interactive tree viewer) or Tom Hudson's gron (which flattens JSON into grep-friendly dotted paths) are often better first stops. They are complements to jq, not replacements.

The third is neighbouring formats. jq operates on JSON. For YAML, yq is the common reach; for XML, reach for a proper XML tool; for CSV at scale, csvkit or miller are the better fit. Trying to force jq onto a format it does not understand ends in pain.

The fourth is the deeper critique, and it is worth taking seriously. Steven McCanne, making the case for the Zed/zq project, argued that jq's real difficulty is not its syntax but its computational model: jq is a stateless-dataflow tool, so any operation that needs state across multiple inputs (joins, running totals, windowed aggregates) requires either --slurp acrobatics or building intermediate data structures by hand¹¹. For reshape-and-filter work — the bulk of what most developers do — this model is exactly right. For analytical workloads that want to behave like a small database, it can be a poor fit.

jq solves one problem well: take JSON in, shape JSON out, with filters that compose like shell pipes. For the pattern it was designed for — exploring APIs, wiring CLI tools together, one-off data reshaping — nothing else in the standard Unix toolbox comes close.

A concrete first step is the fastest way to build fluency. Find a REST API already in use somewhere in the developer's working day, curl -s it, pipe the response into jq '.' to see what it looks like, then add one | stage at a time until the output is the view that is actually wanted. Learning happens on a real payload with a real question; the manual is a reference to return to, not a document to read front-to-back.

JSON does not have to arrive as a wall of text.

---

Footnotes

1. Dolan, Stephen. (2018). 'jq 1.6 Manual.' stedolan.github.io. https://stedolan.github.io/jq/manual/v1.6/ ↩ ↩² ↩³

2. jq Project. (2018). 'NEWS' and 'AUTHORS' at tag jq-1.6.' GitHub source repository. https://raw.githubusercontent.com/jqlang/jq/jq-1.6/NEWS (AUTHORS: https://raw.githubusercontent.com/jqlang/jq/jq-1.6/AUTHORS) ↩ ↩² ↩³

3. jq Project. (2018). 'jq 1.6 release.' GitHub. https://github.com/jqlang/jq/releases/tag/jq-1.6 ↩ ↩²

4. Dolan, Stephen. (2018). 'jq.' stedolan.github.io. https://stedolan.github.io/jq/ ↩ ↩² ↩³

5. Adams, DJ. (2022). 'Reshaping data values using jq's with_entries.' qmacro.org. https://qmacro.org/blog/posts/2022/05/30/reshaping-data-values-using-jqs-with-entries/ ↩

6. Lincoln, Matthew. (2016). 'Reshaping JSON with jq.' Programming Historian. https://programminghistorian.org/en/lessons/json-and-jq ↩

7. Judis, Stefan. (2017). 'How to quickly access API responses with curl and jq.' Contentful. https://www.contentful.com/blog/how-to-quick-access-api-responses-with-curl-and-jq/ ↩

8. Friedrich, Michael. (2021). 'JSON formatting with jq and CI/CD linting automation.' GitLab. https://about.gitlab.com/blog/devops-workflows-json-format-jq-ci-cd-lint/ ↩ ↩² ↩³

9. jq Project. (2022). 'FAQ.' GitHub wiki. https://github.com/stedolan/jq/wiki/FAQ ↩

10. jq Project. (2022). 'Cookbook.' GitHub wiki. https://github.com/stedolan/jq/wiki/Cookbook ↩

11. McCanne, Steven. (2022). 'Introducing zq: an easier (and faster) alternative to jq.' Brim Data. https://www.brimdata.io/blog/introducing-zq/ ↩