Well, my affinity for shell scripting shows no sign of slowing down. Today I found out some more cool stuff that is possible with the jq utility.

In addition to the easy handling of raw JSON, jq also provides classic functional operators like map, reduce and filter which can be chained together.

To facilitate, there’s a fantastic website called jqplay which I highly recommend!

Here’s a quick example of how to curl some API data and transform it into a format that can be easily bash’ed with.

Sample of input data from https://jsonplaceholder.typicode.com/:

(api returns 10 at a time, this is just one sample)

[
  {
    "id": 1,
    "name": "Leanne Graham",
    "username": "Bret",
    "email": "Sincere@april.biz",
    "address": {
      "street": "Kulas Light",
      "suite": "Apt. 556",
      "city": "Gwenborough",
      "zipcode": "92998-3874",
      "geo": {
        "lat": "-37.3159",
        "lng": "81.1496"
      }
    },
    "phone": "1-770-736-8031 x56442",
    "website": "hildegard.org",
    "company": {
      "name": "Romaguera-Crona",
      "catchPhrase": "Multi-layered client-server neural-net",
      "bs": "harness real-time e-markets"
    }
  }
]

The command:

curl -s "https://jsonplaceholder.typicode.com/users" \
  | jq 'map([.website, .id|tostring] | join(" => ")) | join("\n")'

Notes:

  • The curl request returns an array of objects
  • Each object is yielded to map
  • map returns an array for each object, composed of website and id|tostring (nice pipe function to turn int into string)
  • each array is flattened into a new string with => symbol in between
  • then the whole array is flattened into a string of elements separated by new line characters

Output:

hildegard.org => 1
anastasia.net => 2
ramiro.info => 3
kale.biz => 4
demarco.info => 5
ola.org => 6
elvis.io => 7
jacynthe.com => 8
conrad.com => 9
ambrose.net => 10

We can use the read/while loop described yesterday to loop over this data and do whatever we want with it. For example, we can use the original call to gather the elements needed to compose URLs, then we can build the URLs dynamically in the loop and fetch the data.