Methodology

A quantified and multi-level scale of authenticity

Compass to Publish uses a criteria-based evaluation to quantify the degree of authenticity of open access journals requiring or hiding article processing charges (APCs).

The tested journal's degree of authenticity is displayed dynamically on a quantified scale ranging from -20 (extremely predatory) to +20 (no suspicion of deceptive behavior or fake character). A journal's score can exceed the scale's boundaries but for practical reasons and ease of use, the actual score is not displayed beyond those boundaries.

The quantified scale comprises 7 shades of color, which are meant to categorize and explain the potential fake and/or deceptive character of tested journals:

Dark red
Your answers indicate that the journal is very likely to be predatory. It is best to avoid it altogether.
Medium red
This journal is highly suspicious. There are many signs indicating it may be a predatory journal. 
Light redThis journal looks risky.
YellowYour answers make it hard to determine whether or not this may be a predatory journal.
Light green
 The risk that this may be a predatory journal is fairly limited.
Medium green
This is most likely not a predatory journal. 
Dark greenRest assured! Everything seems to indicate that this journal is very unlikely to be predatory.

These shades of color are meant to provide a more nuanced picture of the predatory publishing landscape than the binary logic sustained by trusted lists and databases of predatory journals.

What are the criteria?

Compass to Publish uses an evaluation method based on 26 criteria which take the form of questions. These criteria and questions are the result of the critical and analytical work of the team behind Compass to Publish, who have examined the practices of a significant number of predatory journals and publishers. This examination was then followed by a qualitative survey and selection of criteria developed by trusted lists and directories, as well as checklists for the identification of predatory journals, including:

Looking at the full range of these criteria, we only retained those that are:

  • truly incriminating and easy to check to ensure user-friendliness;
  • sufficiently relevant and clear;
  • easy to use and check for users.

Some information regarding journal policies and procedures can be very hard and/or time-consuming to verify. We deliberately decided not to include this type of criteria in the evaluation process in an effort to ensure user-friendliness.

Weighing method

Every criterion can generate positive or negative points, following the answer given. The weighing of criteria may vary according to:

  • their degree of severity (severe, moderate, ...)
  • their subjective or objetive character

Details of points and criteria may be found below in the section "Criteria and Scoring".

Additional penalty points

Depending on the answers given, additional penalty points may apply to certain questions that are part of a specific and pre-determined subgroup of questions.

This pre-determined subgroup only contains questions whose answers may testify to clear misleading behaviors and/or a willingness to deceive. These specific questions are marked with a in the "Criteria and Scoring" section below.

Additional penalty points are applied if the test registers at least two answers to questions of this subgroup which indicate deceptive and/or miseleading behaviors. 

These additional penalty points are calculated as follows:

Condition
Additional penalty points

If  2 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;

-2
If  3 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;-4
If  4 registered answers to questions included in the subgroup indicate misleading and/or a deceptive behaviors;-6
If  5 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;-8
If  6 registered anwsers to questions included in the subgroup indicate misleading and/or deceptive behaviors;-10

These additional penalty points are meant to:

  • make sure that truly predatory journals stand out
  • put into perspective the potential inclusion of journals in trusted lists AND in databases of alleged predatory journals

Criteria and scoring


1. Trusted lists

Question Answers
Is the journal indexed in the DOAJ (Directory of Open Access Journals)? Check here. Yes (10) - No (0) - I don't know (0)
Is the journal hosted on one of the Open Acess platforms below? Check OpenEdition PoPuPS SciELO Yes (10) - No (0) - I don't know (0)
Is the publisher or the journal a member of OASPA (the Open Access Scholarly Publishers Associati... Yes (10) - No (0) - I don't know (0)
Is the publisher or the journal a member of COPE (the Committee on Publication Ethics)? Check here Yes (10) - No (0) - I don't know (0)
Is the journal published by a well-known publisher? Yes (10) - No (0) - I don't know (0)

2. Lists of alleged predatory journals and publishers (often referred to as blacklists)

Question Answers
Is the journal included the DOAJ's list of journals claiming to be indexed in DOAJ (Directory of ... Yes (-10) - No (0) - I don't know (0)
Is the journal or publisher included in one of the lists on the website Stop Predatory Journals?C... Yes (-5) - No (0) - I don't know (0)

3. Hijacked journals

Question Answers
Is the ISSN or eISSN included on the journal's website authentic? Check here. Yes (0) - No (-10) - I don't know (0)
Is the journal you are testing a hijacked journal? Check the Retraction Watch Hijacked Journal Ch... Yes (-10) - No (0) - I don't know (0)
Does the publisher have a strangely similar name to another publisher? Yes (-5) - No (0) - I don't know (0)

4. Indexing and metrics

Question Answers
Does the journal have the registered trademark "Impact Factor" (Clarivate Analytics TM)? Check here. Yes (10) - No (0) - I don't know (0)
Does the journal pretend to have an "Impact Factor", or does it use questionable metrics whose na... Yes (-5) - No (0) - I don't know (0)
Is the journal really included in the various databases mentioned on its website? Check on MIAR Yes (0) - No (-5) - I don't know (0)

5. Editorial board and peer review

Question Answers
Are the editorial board members mentioned on the website? Yes (1) - No (-5) - I don't know (0)
Do the members of the editorial board seem legitimate, especially the editor-in-chief? Yes (1) - No (-3) - I don't know (0)
Does the possibly announced peer review policy seem surprisingly rapid for your discipline(s)? Yes (-3) - No (1) - I don't know (0)

6. Content and presentation

Question Answers
Are the journal's articles really free and open for access? Yes (1) - No (-5) - I don't know (0)
Is the journal's website obviously author-oriented rather than reader-oriented? Yes (-3) - No (1) - I don't know (0)
If contact details of the journal / publisher can easily be identified, do they look legitimate? Yes (0) - No (-3) - I don't know (0)
Are the articles clearly related to the journal's aims and scope? Yes (1) - No (-3) - I don't know (0)
Does the journal and / or the publisher boast an international reputation or pretend to be a majo... Yes (-2) - No (0) - I don't know (0)

7. Communication strategies

Question Answers
Do you repeatedly get unsollicited email (spam) from the journal / publisher? Yes (-3) - No (0) - I don't know (0)
Do these unsollicited emails offer you to republish an already published or archived text? Yes (-5) - No (0) - I don't know (0)
Do these unsollicited emails offer you to become a reviewer or editorial board member for a journ... Yes (-3) - No (0) - I don't know (0)
Is the tone of the email(s) particularly generous in the praise of your work or of the journal? Yes (-3) - No (0) - I don't know (0)
Do these unsollicited emails offer you to publish in a journal whose field is not your area of ex... Yes (-3) - No (0) - I don't know (0)
FEEDBACK