Introduction

“Publish or perish” a well-too-know axiom to academics involved in research throughout the world. In simple terms, researchers feel impelled to publish periodically to be recruited and to secure tenure or funding for their research. The fact that a high publication rate reflects academic success has led to a numbers game (^{Parnas, 2007}) that has not only caused an increase in the quantity of worldwide publications but has also encouraged a series of misconducts (^{Fanelli, 2009}; ^{Hvistendahl, 2015}). While in the year 2000 approximately 1.3 million publications were registered by Scopus, by the end of 2021 that number increased to approximately 3.7 million publications. Similarly, Web of Science registered approximately 1.2 million publications in the year 2000 versus 3.6 million in 2021. As the quantity of publications rises rapidly, there is a need to make one´s research visible to other academics.

Undoubtedly, the massification of the internet in the 1990s led to changes in the structure of scholarly communications. A turning point was the availability of the first graphical browser for the world-wide-web in 1994 (^{Haynes, 1999}). Journals, which used to print issues to communicate research outcomes, began embracing the use of the World Wide Web to disseminate knowledge by establishing webpages which provided a venue to access the information that was being published. At first, access to these documents worked as expected. Nonetheless, after some years, users began encountering error messages (such as “404 not found”) when trying to access the journal´s content, which occurred when journals changed their URL (^{Germain, 2000}). While this study established that, after a three-year period, 50% of URL citations from 31 journals could not be accessed (^{Germain, 2000}), a later work reported that almost 17% of the URL references from all the articles published by PLoS Medicine between 2005-2007 were non-operational (^{Nagajara et al., 2011}). Due to the dynamics of the web, several other studies also found a high proportion of URL citations that were not accessible through the web browsers (^{Sampath Kumar and Vinay Kumar, 2013}; ^{Vinay Kumar and Sampath Kumar, 2017}; ^{Manjunatha et al., 2020}; ^{Shanthakumari, 2021}). As the URL refers to the home address of a journal, any changes in its web architecture will affect the accessibility of a particular publication (^{Ansorge, 2023}).

The digital object identifier (commonly known as DOI), which originated in 1998, offered a way to overcome this limitation. The DOI was conceived as a persistent and unique identifier of various types of physical and electronic objects (^Paskin,2005; ^{Liu, 2021}). It is managed by the International DOI Foundation, a non-profit organization that governs the federation of registration agencies (^{Wang et al., 2018}). The DOI name consists of a case-insensitive string made up of a prefix beginning with “10.” that represents a unique naming authority and a suffix, separated by a slash, that indicates the specific object (^{Paskin, 2005}). Among its advantages are its capability to unambiguously identify an object, providing a simple and efficient system to track a journal article. This allows other web-based tools, such as Altmetric Explorer, to track journal articles and provide alternative metrics on the use and impact of the research at an article-level. While these metrics mainly focus on activity in online environments (such as Twitter, YouTube and Wikipedia), citation data is collected from the Dimensions database and patent data is collected from IFI Claims (https://help.altmetric.com/support/solutions/articles/6000236721-patents). The Altmetric Attention Score (AAS) is derived from these metrics, (https://help.alt-metric.com/support/solutions/articles/6000233311-how-is-the-altmetric-attention-score-calculated-). Though various studies have shown dissimilar results regarding a correlation between AAS and citations (^{Cintra et al., 2018}; ^{Breuer et al., 2022}; ^{Chen and Wang., 2022}; ^{Liu y Huang, 2022}), Altmetrics still provides a means to assess online conversations surrounding a research output.

Limitations in the use of traditional bibliometrics to assess disciplines related to social sciences, arts, and humanities have been widely reported. Because some research topics are generally locally or regionally orientated, the corresponding articles are published in the language of the region where the research was developed (^{Frandsen y Nicolaisen, 2008}; ^{Hammarfelt and Haddow, 2018}). Consequently, articles published in any other language besides English may not appear in search queries performed in either of the two main two databases -Web of Science and Scopus- since both index a significantly higher proportion of English-language journals (^{Hammarfelt and Haddow, 2018}). Furthermore, some researchers tend to publish monographs or books to communicate their findings, and these outputs are usually not indexed by these databases (^{Lariviere et al., 2006}; ^{Marx and Bornmann, 2015}). And contrary to what occurs in natural science, citation accumulation in social sciences, arts, and humanities tends to be slower over the years, causing a detrimental effect during curricular assessments of researchers (^{Pajic et al., 2019}). It is important to note that both databases tally citations based on the references registered by each database (^{Marx and Bornmann, 2015}).

While studying the online attention generated by articles published by Chilean researchers in journals included in the Social Sciences Citation Index and the Arts and Humanities Citation Index from Web of Science, we noticed that not only some documents lacked a DOI, but some journals exhibited an irregular use of DOIs throughout the 5-year period assessed. To verify whether the lack of DOIs was due to database processing errors or to the absence of DOIs in the articles, we repeated the query using Scopus as the source of information and later verified the information by accessing the website of each journal. Therefore, this study aimed to establish whether the DOIs associated to every article are being registered by the two main citation databases, using data from 40 journals that exhibited the highest degree of irregularity in DOI usage according to the Web of Science output. To our knowledge, only one previous study reported issues with the registration of DOIs by a citation database for documents published by two journals between 2005-2014 (^{Gorraiz et al., 2016}). It is important to note that while more recent studies have reported that a proportion of the records downloaded from these databases lacked DOIs (^{Khurana et al., 2022}; ^{Mugnaini et al., 2021}), none performed a DOI check in the journals’ websites to determine the origin of the problem.

Materials and methods

Bibliometric data for this study was retrieved from the Web of Science (WoS) database in December 2021. The WoS database was chosen for this study because the main Chilean research grant (Fondecyt) utilizes this database to assess all researchers as part of the evaluation process performed by the Chilean National Agency for Research and Development (ANID).

The search criteria (CU=Chile and PY=2015-2019) was used to extract publications registered by the Arts & Humanities Citation Index (A&HCI) and the Social Science Citation Index (SSCI) collections. This query resulted in 12,396 documents that registered at least one affiliation to a Chilean institution. The downloaded data was sorted and processed using Microsoft Excel. For each document we obtained the title, source, ISSN, publisher, document type and DOI.

The Scopus database was used to search for article-type documents for the 40 journals that exhibited the highest degree of irregularity in DOI usage according to the previous Web of Science query.

Furthermore, publisher names were normalized and classified into one of three categories (^{Krauskopf, 2021}):

Results

Among the 12,396 documents that were published between 2015-2019 by Chilean researchers, we observed that 3,085 (24.9%) were not associated to a DOI. These documents were published in 323 different journals. To assess the implementation of DOIs over the period 2015-2019, we selected all the journals that had published a minimum of 20 documents without DOIs (Figure 1). The purpose was to determine whether these journals were not employing DOIs or whether DOI usage was inconsistent throughout the selected period. This accounted for 40 journals (Table 1), which have been sorted by publisher type (commercial, non-university academic publisher or university publisher). Among the journals managed by commercial publishers, the Journal of Dental Education attracted attention because it was the only journal that appeared to not have implemented DOI usage during 2015 and 2016. As this seemed peculiar, we consulted the journal website (https://onlinelibrary.wiley.com/loi/19307837) to verify whether this was correct. To our surprise, we confirmed that all the papers published in 2015 and 2016 were associated with a DOI. Therefore, we questioned whether this inaccuracy extended to the other 39 journals. One way to solve this question was to download data from another database, Scopus, which indexed the same journals. However, as Web of Science and Scopus differ slightly in the type of documents registered, we focused our queries on article-type documents as these are recorded by both databases (^{Liu et al., 2021}).

Figure 1 Overview of the search query and journal eligibility process 

Table 2 shows the total number of article-type documents published by each journal between 2015 and 2019 and the proportion of these documents that registered a DOI, according to Web of Science and Scopus. The differences observed across data sources are concerning considering we have adjusted our queries to the same document type. Among 16 journals (Revista Argentina de Clínica Psicológica, RLA - Revista de Lingüística Teorica y Aplicada, ALPHA - Revista de Artes Letras y Filosofía, Arte Individuo y Sociedad, Convergencia - Revista de Ciencias Sociales, Estudios Filológicos, Historia Unisinos, Historia y Comunicación Social, Ideas y Valores, Pensamiento, Revista 180, Revista Chilena de Derecho, Revista Chilena de Literatura, Revista de Geografía Norte Grande, Revista Musical Chilena, Teología y Vida) we established a difference of more than 20% in the number of article-type documents registered by both databases. Furthermore, in the case of the journals Historia Unisinos and Historia y Comunicación Social this difference is over 100%. It is important to note that Scopus’s coverage of the journal Revista de Filosofia began in 2016, thus it was not included among these journals.

In addition, Table 2 shows that Web of Science registered less DOIs associated to article-type documents in comparison to Scopus. In fact, while Web of Science registered 100% of DOIs for only three journals (Economic Systems, Journal of the Royal Statistical Society Series, and Historia Unisinos), Scopus reported 100% for 17 journals. Since we had previously established a database processing error with the Journal of Dental Education, we accessed the websites of the 23 journals that exhibited less than 100% of article-type documents associated to a DOI to confirm whether it was due to the absence of DOI in the document or another database processing error (Table 3).

An analysis of each article-type document published by the 23 journals revealed that the DOIs were under recorded by Web of Science and Scopus for 20 publications (Table 3). In fact, these 20 journals had a DOI associated to the totality of article-type documents. It is alarming that for a journal such as Revista 180 Web of Science only registered 21% of DOIs and Scopus 39%, even though the journal had a DOI associated to every article-type document. A similar situation was observed for another journal, Revista Panamericana de Salud Publica, for which less than 50% of DOIs were registered by both databases.

Finally, special attention needs to be drawn to the journal Atenea. While all the article-type documents published in the five-year period included a DOI (Table 3) only 26% were associated to a DOI (Table 2) according to Web of Science. A closer look revealed that, for some unknown reason, Web of Science did not register the DOIs for all the article-type documents published in 2016 and those published in 2019 lacked DOIs altogether. Contrarily, Scopus registered DOIs for 63% of article-type documents published by the journal on every year assessed.

Discussion and conclusion

Databases play an important role during research assessment. Data extracted from Web of Science and Scopus, two high-cost subscription databases, is recurrently utilized to produce university rankings, evaluate tenure and promotion at universities, perform curricular assessments of researchers by grant funding agencies, etc. While these databases are perceived as accurate regarding the information registered, they are not exempt of errors (^{Krauskopf, 2019}; ^{Shuo et al., 2019}; ^{Liu, 2020}; ^{Purnell, 2022}; ^{Savchenko y Kosyakov, 2022}). Concerning errors in the DOI field, a few years ago ^{Franceschini et al. (2015)} established the existence of some DOIs that were shared between different publications indexed by Scopus, distorting the outcome of the bibliometric analysis. A later study from ^{Zhu et al. (2019)} reported 119 DOIs downloaded from Web of Science that were not found in the DOI System. Considering that the DOI was conceived as a unique and permanent identifier, the existence of such cases is perplexing.

In this study we report inconsistencies in DOI registration by Web of Science and Scopus. Even though most of the analyzed journals have associated a DOI to every article-type document (Table 3), both databases displayed problems registering this information. While ^{Gorraiz et al. (2016)} previously established that Web of Science had problems registering DOIs for documents published by two journals (out of 35), we determined this matter has not been solved for Latin American journals managed by non-university academic publishers and university presses in disciplines related to the Social Sciences, Arts, and Humanities. Likewise, ^{Sanz-Casado et al. (2021)} reported that DOI usage was higher (62%) for Norwegian, Finnish, and Spanish journals in the field of humanities, in comparison to journals managed by university presses (59%) and non-university academic publishers (29%). Besides the complications that may arise from citation tracking (^{Liu, 2021}; ^{Habermann, 2023}), the missing DOIs in the downloaded data prevents users from utilizing these identifiers to track the online attention of these articles through Altmetrics.

In a recent study using data extracted from Web of Science, ^{Khurana et al. (2022)} established that 16.1% of the documents published by 1000 journals were without a DOI. It is important to note that this study did not discriminate between document types, therefore we expected to find a minor proportion of documents (such as news items or corrections) not associated to a DOI. Conversely, the study published by ^{Mugnaini et al. (2021)} which only considered three document types (articles, reviews, and proceedings papers) written by Brazilian researchers between 2012-2016, established that 68% of them included a DOI.

Another important issue that needs to be considered is the fact that to obtain a DOI publishers need to pay a fee, which may be cost-prohibited for all document types. A study on Ecuadorian journals suggested that this might be the reason why only 78% complied with the use of DOI (^{Santiago-Padilla y Ramos-Gil, 2022}). Similarly, another study on Chinese journals reported that 80% used DOIs due to a lack of government support and lack of understanding about their importance. (^{Wang et al., 2018}).

While databases need to continually improve their procedure to annotate all the metadata provided by each journal to ensure the accuracy and consistency of the collected data, the editorial management team also share responsibility in this matter. By the time databases collect metadata from the journal´s website, all articles should have DOIs assigned. A delay in the process of assigning DOIs will result in databases registries lacking DOIs. It is important to note that, for a researcher, choosing a journal to submit a manuscript is not trivial as everyone hopes that their work will be visible to others. Thus, the implementation of DOIs by journals will not only assist citation tracking but will increase their visibility.