taizilongxu
4/6/2016 - 10:14 AM

jq Cheet Sheet

jq Cheet Sheet

Processing JSON using jq

jq is useful to slice, filter, map and transform structured json data.

Installing jq

On Mac OS

brew install jq

On AWS Linux

Not available as yum install on our current AMI. It should be on the latest AMI though: https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/

Installing from the source proved to be tricky.

Useful arguments

When running jq, the following arguments may become handy:

ArgumentDescription
--versionOutput the jq version and exit with zero.
--sort-keysOutput the fields of each object with the keys in sorted order.

Basic concepts

The syntax for jq is pretty coherent:

SyntaxDescription
,Filters separated by a comma will produce multiple independent outputs
?Will ignores error if the type is unexpected
[]Array construction
{}Object construction
+Concatenate or Add
-Difference of sets or Substract
lengthSize of selected element
|Pipes are used to chain commands in a similar fashion than bash

Dealing with json objects

DescriptionCommand
Display all keysjq 'keys'
Adds + 1 to all itemsjq 'map_values(.+1)'
Delete a keyjq 'del(.foo)'
Convert an object to arrayto_entries | map([.key, .value])

Dealing with fields

DescriptionCommand
Concatenate two fieldsfieldNew=.field1+' '+.field2

Dealing with json arrays

Slicing and Filtering

DescriptionCommand
Alljq .[]
Firstjq '.[0]'
Rangejq '.[2:4]'
First 3jq '.[:3]'
Last 2jq '.[-2:]'
Before Lastjq '.[-2]'
Select array of int by valuejq 'map(select(. >= 2))'
Select array of objects by value** jq '.[] | select(.id == "second")'**
Select by type** jq '.[] | numbers' ** with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars

Mapping and Transforming

DescriptionCommand
Add + 1 to all itemsjq 'map(.+1)'
Delete 2 itemsjq 'del(.[1, 2])'
Concatenate arraysjq 'add'
Flatten an arrayjq 'flatten'
Create a range of numbersjq '[range(2;4)]'
Display the type of each itemjq 'map(type)'
Sort an array of basic typejq 'sort'
Sort an array of objectsjq 'sort_by(.foo)'
Group by a key - opposite to flattenjq 'group_by(.foo)'
Minimun value of an arrayjq 'min' .See also min, max, min_by(path_exp), max_by(path_exp)
Remove duplicatesjq 'unique' or jq 'unique_by(.foo)' or jq 'unique_by(length)'
Reverse an arrayjq 'reverse'