Back to SDG Research Home

Project deliverables: reports, data and code

Below you’ll find all the information about the deliverables of this project, open and freely available.

Our aim is to build a SDG research dashboard that demonstrates the societal relevance and societal impact of research of Aurora Universities. This dashboard shows the research contributions in these societal challenges, and how policymakers have used the research available to tackle these challenges. To reach that goal we want to upgrade the previously build SDG Dashboard to over come issues on completeness and  research written in multiple languages. 

The steps involved are depicted in the timeline below.

[1] Aurora SDG classification model Query Language robustness evaluation

Status: Finished

For this deliverable we have created a SDG classification model based on a query language (ASCM-QL), using keywords and advanced Boolean operators to search for research output that relate to each of the targets within the SDG’s. This has been done in consultation with the Aurora research community.

Using human feedback of 244 senior researchers, we were able to evaluate the accuracy of the ASCM-QL. The outcomes of the report must be taken into consideration when the results are used for strategic decision making.

Average Precision of all SDG’s is 70% (70% of the publications found in the SDG result sets, are related to that SDG, according to the respondents.)

Average Recall of all SDG’s is 14% (14% of the publications researchers suggested that should be in the SDG result set  in the survey, did appear in the SDG result set.)

SDG Queries:

Evaluation Data:

  • Vanderfeesten, Maurice, Spielberg, Eike, & Gunes, Yassin. (2020). Survey data of “Mapping Research Output to the Sustainable Development Goals (SDGs)“ (1.0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3813230 

  • Vanderfeesten, Maurice, Spielberg, Eike, & Hasse, Linda. (2020). Text Analyses of Survey Data on “Mapping Research Output to the Sustainable Development Goals (SDGs)” (1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3832090 

  • Vanderfeesten, Maurice. (2021). DOI’s with SDG labels on Target level | 1.4M research articles (2009-2020) related to Sustainable Development Goals (1.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5224005 

Evaluation Report:

  • Schmidt, Felix, & Vanderfeesten, Maurice. (2021). Evaluation on accuracy of mapping science to the United Nations’ Sustainable Development Goals (SDGs) of the Aurora SDG queries (v1.0.2). Zenodo. https://doi.org/10.5281/zenodo.4917107

Version controlled:  https://github.com/Aurora-Network-Global/sdg-queries-evaluation-report

Deliverable 1

[2] Aurora SDG classification model Machine Learning Multi-lingual Text Classifier

Status: Finished

In order to classify research output written in multiple European languages, and to be independent of the use of query language specifics from research index databases, and to get more relevant research papers that would have been left out using keywords; we aim to migrate the query language based model to a machine learning model processed with a state-of-art multi-language model.

Software

Models

  • Binary: the trained 169 models on SDG-Target level can be found here (2GB each):

    • Jaworek, Robert, & Vanderfeesten, Maurice. (2021). AI for mapping multi-lingual academic papers to the United Nations’ Sustainable Development Goals (SDGs). Zenodo. https://doi.org/10.5281/zenodo.5603019 

  • Binary: re-trained multi-language model, based on Elsevier SDG 2022 definitions data

Publication: 

  • Publication: explaining the model architecture, the data flow, the data used to train the model and the evaluation scores of the trained model for English language and other EU languages:

    • Jaworek, Robert, & Vanderfeesten, Maurice. (2021). AI for mapping multi-lingual academic papers to the United Nations’ Sustainable Development Goals (SDGs). Zenodo. https://doi.org/10.5281/zenodo.5603019

[3] Aurora SDG classification service

Status: Finished

We have put the trained model in a web service to classify our own papers, and will make this service available for public use based on FAIR use policy.

The web service will contain two parts, an API for developers, and a browser-friendly user interface (UI).

The API will process a text fragment, and return data; the confidence percentages of the text fragment to the SDG goals, and a link to an image one can use as an SDG-badge.

The UI offers a webform to enter a text fragment, or to upload a table. Using a text fragment the UI will show a diagram to what extent this text fragment relates to each SDG goal. 

Service:

Software:

Publication:

[4]  Aurora SDG classified research output

Status: Finished

Once we have the SDG text classifier up and running, we need to label all research output from aurora universities to the SDG goals and SDG targets. 

To help with this task we have to build a data collection and orchestration infrastructure that enables us to automatically collect, categorize and enrich publications with research & societal impact data from a multitude of sources and formats. This then will be unified and offered as an API data access point.

Data:

  • Collected from universities in separate files:

    • Maurice Vanderfeesten, Eike Spielberg, & Max Paulus. (2022). Aurora Publications 2022 | 700.000+ records | Enriched with data measuring Societal Relevance, Societal Impact and Scientific Impact, using SDG’s, Unpaywall, Altmetric, Scite and Scival (1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.6644947 

  • Harvested from University CRIS systems:

    • List of Aurora Repositories and CRIS systems

    • Manghi, Paolo, Atzori, Claudio, Bardi, Alessia, Baglioni, Miriam, Schirrwagen, Jochen, Dimitropoulos, Harry, La Bruzzo, Sandro, Foufoulas, Ioannis, Mannocci, Andrea, Horst, Marek, Czerniak, Andreas, Kiatropoulou, Katerina, Kokogiannaki, Argiro, De Bonis, Michele, Artini, Michele, Ottonello, Enrico, Lempesis, Antonis, Ioannidis, Alexandros, & Summan, Friedrich. (2022). OpenAIRE Research Graph: Dumps for research communities and initiatives (latest) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3974604 

      • Download: Aurora_Universities_Network.tar

Data Collection Service:

Instead of building your own data collection service,  we make use of OpenAIRE to harvest the metadata from our Repositories, CRIS systems and all publications affiliated to our Universities.

Software:

[5]  Aurora SDG research dashboard

Status: Finished

Once we have the enriched data, we want to visualise the data. Users will help us with the design process of the visual interactive reports they use in their daily work.

Service:

Software: 

  • Binary (stand-alone): you can find the files to run and build your stand-alone version of the SDG research dashboard  yourself here: 

Publication: 

[6] Guest Alliances and Universities

Status: Finished

Other EU University alliances are invited to use and re-use the services and software mentioned above.

  • Communication and Dissemination plan, including Letter that has been sent:
    • Vanderfeesten, M. (2023). Aurora SDG Research Dashboard and Classifier – Communication and Dissemination Plan and Material (v1.0). Aurora Universities. https://doi.org/10.5281/zenodo.10040762
  • Presentations given can be found here:

[7] Bibliometrics Expert & Support Centre

Status: Finished

We have set up a support center to answer questions of the aurora members and guest alliances on how to use the services and software mentioned above.

Services: