Running GraphQL queries from the command line using curl
can be made simple and readable. Here’s an example of how to query the GitHub GraphQL API using curl
, tested in both bash
and zsh
. Replace TOKEN
with your GitHub API personal access token.
curl -s https://api.github.com/graphql -X POST \
-H "Authorization: Bearer TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -c -n --arg query '
{
search(type: REPOSITORY, query: "user:assahw topic:git-scraping", first: 100) {
repositoryCount
nodes {
__typename
... on Repository {
nameWithOwner
description
defaultBranchRef {
name
target {
... on Commit {
committedDate
url
message
}
}
}
}
}
}
}' '{"query":$query}')"
This command constructs and submits a GraphQL query encoded in JSON, using jq
for JSON handling.
Building a JSON Document with jq
The query needs to be encoded as a JSON document like this:
{"query":"\n{\n search(type: REPOSITORY, query: \"user:assah topic:git-scraping\", first: 100) {\n repositoryCount\n nodes {\n __typename\n ... on Repository {\n nameWithOwner\n description\n defaultBranchRef {\n name\n target {\n ... on Commit {\n committedDate\n url\n message\n }\n }\n }\n }\n }\n }\n}"}
This is generated using the following jq
command:
jq -c -n --arg query '
{
search(type: REPOSITORY, query: "user:assah topic:git-scraping", first: 100) {
repositoryCount
nodes {
__typename
... on Repository {
nameWithOwner
description
defaultBranchRef {
name
target {
... on Commit {
committedDate
url
message
}
}
}
}
}
}
}' '{"query":$query}'
Explanation of jq Options
-c
: Produces compact JSON output.-n
: Creates JSON from scratch without reading input.--arg
: Sets a variable withinjq
to a specified string.- The final JSON document is constructed using
{"query":$query}
.
Passing JSON to curl with ”$()”
The JSON document is passed to curl
using the -d
option:
-d "$(jq -c -n --arg query ...)"
The "$(...)"
syntax ensures that the entire JSON output is treated as a single value, even with internal whitespace or quotes.
Example JSON Output
A truncated example of the JSON response:
{
"data": {
"search": {
"repositoryCount": 22,
"nodes": [
{
"__typename": "Repository",
"nameWithOwner": "assah/csv-diff",
"description": "Python CLI tool and library for diffing CSV and JSON files",
"defaultBranchRef": {
"name": "main",
"target": {
"committedDate": "2021-02-23T02:53:11Z",
"url": "https://github.com/assahw/csv-diff/commit/33e0a5918283c02a339a1fb507fc7a9cda89a198",
"message": "Handle missing JSON keys, refs #13"
}
}
}
]
}
}
}
Combining with sqlite-utils insert
To create a SQLite database with records for repositories tagged git-scraping
, use:
curl https://api.github.com/graphql -X POST \
-H "Authorization: Bearer ..." \
-H "Content-Type: application/json" \
-d "$(jq -c -n --arg query '
{
search(type: REPOSITORY, query: "user:assah topic:git-scraping", first: 100) {
repositoryCount
nodes {
__typename
... on Repository {
nameWithOwner
description
defaultBranchRef {
name
target {
... on Commit {
committedDate
url
message
}
}
}
}
}
}
}' '{"query":$query}')" \
| jq .data.search.nodes | sqlite-utils insert /tmp/github.db repos - --flatten
Explanation
jq .data.search.nodes
: Extracts thenodes
array from the JSON response.sqlite-utils insert /tmp/github.db repos - --flatten
: Inserts the array into a SQLite database table namedrepos
. The--flatten
option ensures nested fields are expanded into columns.
Final Table Schema
The resulting SQLite table schema:
CREATE TABLE [repos] (
[__typename] TEXT,
[nameWithOwner] TEXT,
[description] TEXT,
[defaultBranchRef_name] TEXT,
[defaultBranchRef_target_committedDate] TEXT,
[defaultBranchRef_target_url] TEXT,
[defaultBranchRef_target_message] TEXT
);
This post was originally written by Simon. Iliked the post and wanted to add it here for future reference.