JQ Cheatsheet

JQ Command-Line JSON Processor

JQ is a lightweight and flexible command-line JSON processor. It allows you to easily slice, filter, map, and transform structured data with the same power and expressiveness as `sed`, `awk`, and `grep` for JSON data. This cheatsheet provides practical examples for common JQ operations.

Basic Data Filtering with JQ

Here's a sample JSON structure we'll use for demonstrations:

{
  "images": [
    {
      "creation_date": "2019-01-03T14:52:34.514699+00:00",
      "default_bootscript": {
        "kernel": "http://169.254.10.254/kernel/arm64/1.2.1",
        "initrd": "http://169.254.42.254/initrd/initrd-Linux-arm64-v3.14.6.gz",
        "architecture": "arm64",
        "title": "arm64 1.2.1"
      },
      "arch": "arm64",
      "id": "x6cevb6f-fe18-475f-4b92-x8ce51f92051",
      "name": "ubuntu xenial"
    },
    {
      "creation_date": "2019-01-03T14:52:34.514699+00:00",
      "default_bootscript": {
        "kernel": "http://169.254.10.254/kernel/arm64/1.2.2",
        "initrd": "http://169.254.10.254/initrd/initrd-Linux-arm64-v3.14.6.gz",
        "architecture": "arm64",
        "title": "arm64 1.2.2"
      },
      "arch": "arm64",
      "id": "x3ce2buf-fe58-47bf-8b52-c8cv51nn2050",
      "name": "alpine"
    },
    {
      "creation_date": "2019-01-03T14:52:34.514699+00:00",
      "default_bootscript": {
        "kernel": "http://169.254.10.254/kernel/arm64/1.2.1",
        "initrd": "http://169.254.10.254/initrd/initrd-Linux-arm64-v3.14.6.gz",
        "architecture": "arm64",
        "title": "amd64 1.2.1"
      },
      "arch": "arm64",
      "id": "x3cf2bbf-fe08-947f-8bf2-c8ce51f92050",
      "name": "ubuntu xenial"
    }
  ]
}

Iterating Through Arrays

To return all objects within the 'images' array:

$ cat data.json | jq '.images[]'
  {
    "creation_date": "2019-01-03T14:52:34.514699+00:00",
    "default_bootscript": {
      "kernel": "http://169.254.10.254/kernel/arm64/1.2.1",
      "initrd": "http://169.254.42.254/initrd/initrd-Linux-arm64-v3.14.6.gz",
      "architecture": "arm64",
      "title": "arm64 1.2.1"
    },
    "arch": "arm64",
    "id": "x6cevb6f-fe18-475f-4b92-x8ce51f92051",
    "name": "ubuntu xenial"
  }
  {
 ...

Filtering Array Elements with `select`

To return only data relevant to the 'amd64' architecture:

$ cat data.json | jq '.images[] | select(.arch == "amd64")'
{
  "creation_date": "2019-01-03T14:52:34.514699+00:00",
  "default_bootscript": {
    "kernel": "http://169.254.10.254/kernel/amd64/1.2.1",
    "initrd": "http://169.254.10.254/initrd/initrd-Linux-amd64-v3.14.6.gz",
    "architecture": "amd64",
    "title": "amd64 1.2.1"
  },
  "arch": "amd64",
  "id": "x3cf2bbf-fe08-947f-8bf2-c8ce51f92050",
  "name": "ubuntu xenial"
}

Extracting Specific Fields

To return only the 'id' value from the filtered query:

$ cat data.json | jq '.images[] | select(.arch == "amd64") | .id'
"x3cf2bbf-fe08-947f-8bf2-c8ce51f92050"

String Matching Functions

Using `contains` and `startswith` for more flexible filtering:

$ cat data.json | jq '.images[] | select(.arch | contains("amd")) | .id'
$ cat data.json | jq '.images[] | select(.arch | startswith("md6")) | .id'

Looping and Transformation

Iterating through records and extracting data:

$ echo '{"Records": [{"Data": "abc", "PartitionKey": "123"}, {"Data": "def", "PartitionKey": "456"}]}' | jq -c '.Records[]'
{"Data":"abc","PartitionKey":"123"}
{"Data":"def","PartitionKey":"456"}

Looping and applying transformations like base64 encoding:

$ echo '{"Records": [{"Data": "abc", "PartitionKey": "123"}, {"Data": "def", "PartitionKey": "456"}]}' | jq -cr '.Records[].Data | @base64'
YWJj
ZGVm

Checking for Existence in Arrays

Checking if a string exists in a simple array:

$ echo '["superfast","fast","slow"]' | jq -c '.[] | select(. | contains("fast"))'
"superfast"
"fast"

Checking if a string exists in an array of dictionaries:

$ echo '[{"speed": "superfast"},{"speed": "fast"},{"speed": "slow"}]' | jq -c '.[] | select(.speed | contains("fast"))'
{"speed":"superfast"}
{"speed":"fast"}

Customizing Output Format

Manipulating the output by specifying fields and formatting strings:

echo '{"ListOfThings": [{"Name": "James", "Age": 23, "Country": "Italy"}]}' | jq -r '.ListOfThings[] | "name=\(.Name)\n age=\(.Age)\n"'
name=James
age=23

JQ Cheatsheet - Command-Line JSON Processor Examples