Scopus API in Unix Shell#
by Avery Fernandez
These recipe examples use the Elsevier Scopus API. Code was tested and sample data downloaded from the Scopus API on April 7, 2022 via http://api.elsevier.com and http://www.scopus.com. This tutorial content is intended to help facillitate academic research. Before continuing or reusing any of this code, please be aware of Elsevier’s API policies and appropiate use-cases. You will also need to register for an API key in order to use the Scopus API.
Setup#
Program requirements#
In order to run this code, you will need to first install curl, and jq. curl is used to request the data from the API and jq is used to parse the JSON data.
API Key Information#
We will start by setting up the API key. Save your key in a text file in your current directory and import your key as follows:
apiKey=$(cat "apikey.txt")
Set the url for the base API:
api="https://api.elsevier.com/content/search/scopus"
3. Get References via a Title Search#
Number of Title Match Records#
Search Scopus for all references containing’ ChemSpider’ in the record title
All the data will be stored into individual entry locations
query=$(curl "$api"$"?query=TITLE(ChemSpider)&apiKey=""$apiKey" | jq '.["search-results"]')
echo "$query" | jq '.["entry"][0]'
length=$(echo "$query" | jq '.["entry"] | length')
Repeat this in a loop to get number of Scopus records for each title search:
declare -a titles=("ChemSpider" "PubChem" "ChEMBL" "Reaxys" "SciFinder")
declare -A storage
for title in "${titles[@]}";
do
storage["$title"]=$(curl "$api"$"?query=TITLE(""$title"$")&apiKey=""$apiKey" | jq '.["search-results"]')
sleep 1
done
for title in "${!storage[@]}";
do
search=$(echo "${storage["$title"]}" | jq '.["opensearch:totalResults"]')
echo "$title"$": ""$search"
done
Output:
Reaxys: "8"
PubChem: "83"
SciFinder: "31"
ChemSpider: "7"
ChEMBL: "53"
Title Match Record Data#
Create a csv of selected metadata:
truncate -s 0 titles.csv
echo $"Title,DOI,Article,Date" >> titles.csv
for title in "${!storage[@]}";
do
length=$(echo "${storage["$title"]}" | jq '.["entry"] | length')
for (( i = 0 ; i < "$length" ; i++));
do
data=$(echo "${storage["$title"]}" | jq ".entry[$i]" )
doi=$(echo "$data" | jq '.["prism:doi"]')
articleTitle=$(echo "$data" | jq '.["dc:title"]')
date=$(echo "$data" | jq '.["prism:coverDate"]')
echo "$title"$",""$doi"$",""$articleTitle"$",""$date" >> titles.csv
done
done
Output:
Title,DOI,Article,Date
Reaxys,null,"Store unit files for bundling activities - Reaxys","2018-04-06"
Reaxys,null,"Hybrid Retrosynthesis: Organic Synthesis using Reaxys and SciFinder","2015-01-01"
Reaxys,null,"Comparisons of the most important chemistry databases - Scifinder program and reaxys database system","2014-01-30"
Reaxys,"10.1021/bk-2014-1164.ch008","The making of reaxys - Towards unobstructed access to relevant chemistry information","2014-01-01"
Reaxys,null,"A chemistry searcher compares CAS'S SciFinder and elsevier's reaxys","2013-09-01"
Reaxys,null,"Od beilsteina do reaxys","2012-04-30"
Reaxys,null,"Store unit files for bundling activities - Reaxys","2011-11-07"
Reaxys,"10.1002/nadc.201179450","Beilstein and Gmelin combined in Reaxys","2011-04-01"
PubChem,"10.1016/j.bioorg.2022.105648","Structure-based discovery of a specific SHP2 inhibitor with enhanced blood–brain barrier penetration from PubChem database","2022-04-01"