Scopus API in Mathematica

by Vishank Patel

These recipe examples use the Elsevier Scopus API and Mathematica’s inbuilt functionalities to manipulate and visualize data. Code was tested and sample data downloaded from the Scopus API on March 6, 2022 via http://api.elsevier.com and http://www.scopus.com. This tutorial content is intended to help facilitate academic research. Before continuing or reusing any of this code, please be aware of Elsevier’s API policies and appropriate use-cases. You will also need to register for an API key in order to use the Scopus API.

Setup

We will start by setting up the API key. If you do not have a key already, one can be obtained from the following link: https://dev.elsevier.com/apikey/manage.

Once obtained, save the key in a text file in the same directory as the current Mathematica notebook and import your key as follows:

myAPIKey = Import["INSERT PATH HERE"]; 

We will concatenate the key at the end of every URL using <>, the shortcut for StringJoin.

1. Get Author Data

Number of Records for Author

We will be using the import function instead of URLExecute as it allows us to specify the import format. Raw JSON is chosen as the format as it interprets the data as Mathematica associations that are much easier to work with compared to lists of lists.

rawAuthorSearch = Import["https://api.elsevier.com/content/search/scopus?query=AU-ID(55764087400)&apiKey="<> myAPIKey,"RawJSON"]; 
rawAuthorSearch // Short

As we can see, the total number of results is 21.

All the data lies under the key “entry”, hence,

authorSearch = rawAuthorSearch["search-results"]["entry"];

The Raw JSON file is converted into Mathematica associations, which can be queried using the keys listed below:

Keys[authorSearch][[1]]

{@_fa, link, prism:url, dc:identifier, eid, dc:title, dc:creator, prism:publicationName, 
 
>   prism:issn, prism:eIssn, prism:volume, prism:issueIdentifier, prism:pageRange, 
 
>   prism:coverDate, prism:coverDisplayDate, prism:doi, citedby-count, affiliation, 
 
>   prism:aggregationType, subtype, subtypeDescription, source-id, openaccess, 
 
>   openaccessFlag}

Extracting all the DOIs from the author,

authorSearch[[All,"prism:doi"]]

{10.1021/acs.jchemed.1c00904, 10.5860/crln.82.9.428, 10.1021/acs.iecr.8b02573, 
 
>   10.1021/acs.jchemed.6b00602, 10.5062/F4TD9VBX, 10.1021/acs.macromol.6b02005, 
 
>   10.1186/s13321-016-0181-z, 10.1021/acs.chemmater.5b04431, 
 
>   10.1021/acs.jchemed.5b00512, 10.1021/acs.jchemed.5b00375, 10.5860/crln.76.9.9384, 
 
>   10.5860/crln.76.2.9259, 10.1126/science.346.6214.1258, 10.1021/ed400887t, 
 
>   10.1016/j.acalib.2014.03.015, 10.5062/F4XS5SB9, 10.1021/ma300328u, 
 
>   10.1021/mz200108a, 10.1021/ma201170y, 10.1021/ma200184u, 10.1021/cm102374t}

The respective titles for the DOIs above:

authorSearch[[All,"dc:title"]]

{Using NCBI Entrez Direct (EDirect) for Small Molecule Chemical Information Searching in\
 
>    a Unix Terminal, Using the linux operating system full-time tips and experiences\
 
>    from a subject liaison librarian, 
 
>   Analysis of the Frequency and Diversity of 1,3-Dialkylimidazolium Ionic Liquids\
 
>    Appearing in the Literature, Rapid Access to Multicolor Three-Dimensional Printed\
 
>    Chemistry and Biochemistry Models Using Visualization and Three-Dimensional\
 
>    Printing Software Programs, Text analysis of chemistry thesis and dissertation\
 
>    titles, Phototunable Thermoplastic Elastomer Hydrogel Networks, 
 
>   Programmatic conversion of crystal structures into 3D printable files using Jmol, 
 
>   Dangling-End Double Networks: Tapping Hidden Toughness in Highly Swollen\
 
>    Thermoplastic Elastomer Hydrogels, 
 
>   Replacing the Traditional Graduate Chemistry Literature Seminar with a Chemical\
 
>    Research Literacy Course, 3D Printed Block Copolymer Nanostructures, 
 
>   Hypotheses in librarianship: Applying the scientific method, 
 
>   Recruiting students to campus: Creating tangible and digital products in the\
 
>    academic library, Finally free, 
 
>   3D printed molecules and extended solid models for teaching symmetry and point\
 
>    groups, Repurposing Space in a Science and Engineering Library: Considerations for\
 
>    a Successful Outcome, A model for managing 3D printing services in academic\
 
>    libraries, Morphological phase behavior of poly(RTIL)-containing diblock copolymer\
 
>    melts, Network formation in an orthogonally self-assembling system, 
 
>   Access to nanostructured hydrogel networks through photocured body-centered cubic\
 
>    block copolymer melts, Synthesis and ordered phase separation of imidazolium-based\
 
>    alkyl-ionic diblock copolymers made via ROMP, 
 
>   Thermally stable photocuring chemistry for selective morphological trapping in block\
 
>    copolymer melt systems}

---

Citation information:

citationList = authorSearch[[All,"citedby-count"]]

{0, 0, 16, 24, 4, 11, 20, 6, 10, 25, 0, 0, 0, 96, 6, 34, 39, 31, 18, 45, 11}

2. Author Data in a Loop

Number of Records for Author

authorList = Import["INSERT PATH HERE","TSV"]

{{Emy Decker, 36660678600}, {Lindsey Lowry, 57210944451}, {Karen Chapman, 35783926100}, 
 
>   {Kevin Walker, 56133961300}, {Sara Whitver, 57194760730}}

Finding the number of records for each author:

numRecords={};
For[i=1,i<=Length[authorList],i++,
    tempAuthorURL="https://api.elsevier.com/content/search/scopus?query=AU-ID("<>ToString[authorList[[i,2]]]<>")&apiKey="<>myAPIKey;
    tempAuthorData=Import[tempAuthorURL,"RawJSON"]["search-results"];
    AppendTo[numRecords,Flatten[{authorList[[i]],tempAuthorData["opensearch:totalResults"]}]];
    Pause[1]
];
numRecords

{{Emy Decker, 36660678600, 14}, {Lindsey Lowry, 57210944451, 4}, 
 
>   {Karen Chapman, 35783926100, 29}, {Kevin Walker, 56133961300, 8}, 
 
>   {Sara Whitver, 57194760730, 4}}

Download Record Data

Let’s say we want the DOIs and cited by counts in a list

cites={};
For[i=1,i<=Length[authorList],i++,
    tempAuthorURL="https://api.elsevier.com/content/search/scopus?query=AU-ID("<>ToString[authorList[[i,2]]]<>")&apiKey="<>myAPIKey;
    tempAuthorData=Import[tempAuthorURL,"RawJSON"]["search-results"]["entry"];
    AppendTo[cites,{authorList[[i,1]],tempAuthorData[[All,"prism:doi"]],tempAuthorData[[All,"citedby-count"]]}];
    Pause[1]
];

cites[[1;;2]]

{{Emy Decker, {10.1108/RSR-08-2021-0051, 10.1080/1072303X.2021.1929642, 
 
>     10.1080/15367967.2021.1900740, 10.1080/15367967.2020.1826951, 
 
>     10.1080/10691316.2020.1781725, 10.1145/3347709.3347805, 
 
>     10.4018/978-1-5225-5631-2.ch09, 10.1016/B978-0-08-102409-6.00007-9, 
 
>     10.1108/LM-10-2016-0078, 10.1016/B978-0-08-100775-4.00010-8, 
 
>     10.1108/S0732-067120160000036013, 10.4018/978-1-4666-8624-3, 
 
>     10.1108/S0065-2830(2013)0000037006, 10.1108/07378831011096268}, 
 
>    {0, 0, 7, 0, 0, 0, 3, 0, 6, 1, 2, 0, 0, 10}}, 
 
>   {Lindsey Lowry, {10.1080/1941126X.2021.1949153, 10.5860/lrts.65n1.4-13, 
 
>     10.1080/00987913.2020.1733173, 10.1080/1941126X.2019.1634951}, {1, 0, 1, 0}}}

Transpose[{cites[[1,2]],cites[[1,3]]}]

{{10.1108/RSR-08-2021-0051, 0}, {10.1080/1072303X.2021.1929642, 0}, 
 
>   {10.1080/15367967.2021.1900740, 7}, {10.1080/15367967.2020.1826951, 0}, 
 
>   {10.1080/10691316.2020.1781725, 0}, {10.1145/3347709.3347805, 0}, 
 
>   {10.4018/978-1-5225-5631-2.ch09, 3}, {10.1016/B978-0-08-102409-6.00007-9, 0}, 
 
>   {10.1108/LM-10-2016-0078, 6}, {10.1016/B978-0-08-100775-4.00010-8, 1}, 
 
>   {10.1108/S0732-067120160000036013, 2}, {10.4018/978-1-4666-8624-3, 0}, 
 
>   {10.1108/S0065-2830(2013)0000037006, 0}, {10.1108/07378831011096268, 10}}

citesFlat={};
For[i=1,i<=Length[cites],i++,
    For[j=1,j<=Length[cites[[i,2]]],j++,
       AppendTo[citesFlat, 
           Flatten[{cites[[i,1]],Transpose[{cites[[i,2]],cites[[i,3]]}][[j]]}]
        ]
    ]
]
citesFlat // Dataset

Save Record Data to a file

Exporting the results as a file:

For[i=1,i<=Length[authorList],i++,
    tempAuthorURL="https://api.elsevier.com/content/search/scopus?query=AU-ID("<>ToString[authorList[[i,2]]]<>")&apiKey="<>myAPIKey;
    tempAuthorData=Import[tempAuthorURL,"RawJSON"]["search-results"]["entry"];
    
    entries=Values[tempAuthorData][[All,;;9]];  (*Taking the first 9 columns of data*)
    keys=Keys[tempAuthorData[[1]]][[;;9]];
    csvDataset=PrependTo[entries,keys];
    
    Export[StringReplace[authorList[[i,1]]," "->"_"]<>"_"<>ToString[authorList[[i,2]]]<>"_"<>"ScopusData"<>".csv",csvDataset];
    Pause[2]
]

Importing one of the created files (first three elements):

author3Import=Import["/home/.../Lindsey_Lowry_57210944451_ScopusData.csv",
                    "Dataset","EmptyField"->" ",HeaderLines->1];
author3Import[[;;3]] // Short

Output not shown here

author3Import[[All,"dc:title"]] // Short

3. Get References via a Title Search

Number of Title Match Records

Search Scopus for all references containing’ ChemSpider’ in the record title

q2=Import["https://api.elsevier.com/content/search/scopus?query=TITLE(ChemSpider)&apiKey="<>myAPIKey,
        "RawJSON"];
q2["search-results"]["opensearch:totalResults"]

7

Repeat this in a loop to get number of Scopus records for each title search

titleWordList={"ChemSpider","PubChem","ChEMBL","Reaxys","SciFinder"};
numRecordsTitle={};

For[i=1,i<=Length[titleWordList],i++,
    qt=Import["https://api.elsevier.com/content/search/scopus?query=TITLE("<>titleWordList[[i]]<>")&apiKey="<>myAPIKey,"RawJSON"];
    numt=qt["search-results"]["opensearch:totalResults"];
    
    AppendTo[numRecordsTitle,{titleWordList[[i]],numt}];
    Pause[1]
]

numRecordsTitle

{{ChemSpider, 7}, {PubChem, 82}, {ChEMBL, 53}, {Reaxys, 8}, {SciFinder, 31}}

Download Title Match Record Data

Download records and create a list of selected metadata

titleWordList={"ChemSpider","PubChem","ChEMBL","Reaxys","SciFinder"};
scopusTitleData={};

For[i=1,i<=Length[titleWordList],i++,

    qt=Import["https://api.elsevier.com/content/search/scopus?query=TITLE("<>titleWordList[[i]]<>")&apiKey="<>myAPIKey,"RawJSON"]["search-results"]["entry"];
    doiList=qt[[All,"prism:doi"]];
    titleList=qt[[All,"dc:title"]];
    coverDateList=qt[[All,"prism:coverDate"]];
    
    AppendTo[scopusTitleData,{titleWordList[[i]],doiList,titleList,coverDateList}];
    Pause[1]
]

scopusTitleData // Dataset

Output not shown here

scopusTitleData[[1]]

Creating a flat list of scopusTitleData

scopusTitleDataFlat={};

For[i=1,i<=Length[titleWordList],i++,
    For[j=1,j<=Length[scopusTitleData[[i,2]]],j++,
        AppendTo[scopusTitleDataFlat,{titleWordList[[i]],scopusTitleData[[i,2,j]],scopusTitleData[[i,3,j]],scopusTitleData[[i,4,j]]}]
    ]
]

scopusTitleDataFlat // Dataset

scopusTitleDataFlatNamed =
    <|"titleWord"->scopusTitleDataFlat[[All,1]],"doi"->scopusTitleDataFlat[[All,2]],"title"->scopusTitleDataFlat[[All,3]],"coverDate"->scopusTitleDataFlat[[All,4]]|>;

Transpose[Dataset[scopusTitleDataFlatNamed]]