Evidence level grading: how do we know research is trustworthy?
Introduction
With so much information and misinformation these days, how do we know what to trust – especially when it comes to the critical topic of health?
Defining what quality research is is not an easy task. In fact, it has been an important topic of discussion for scientists for years. While various grading systems exist to ensure the trustworthiness of research, no single system is accepted by all. In this article, we discuss what we consider trustworthy research and what you can look for in studies as a mark of quality.
Who Can Publish Research?
Anyone can write anything online, which is why we turn to research to get information based on science. But who writes the research? Essentially, anyone can publish research papers or scholarly articles as an independent researcher; however, this is more often found in humanities research than in scientific research.
Respectable journals have publication criteria, policies, and guidelines to ensure higher-quality research. An important aspect of quality research is ethical publishing, which includes originality, data transparency, disclosure of conflicts of interest, acknowledgment of sources, participant consent, etc. (1). After these studies are accepted, they are edited and often peer-reviewed to avoid possible errors.
It is naturally best if the author of the research paper has expertise in the given field. There are metrics to assess the impact of the academic work the author has produced. One of these metrics is the h-index, which shows how many papers the author has that have been cited by others as many times. For example, an h-index of 5 means the author has 5 works, each of which has been cited by other authors at least 5 times.
One way of deciding a journal’s quality is by examining its impact factor. A journal’s impact factor is the number of times a few chosen articles have been cited within the previous few years. Generally, the higher this number is, the more trustworthy the journal.
For example, a few journals with the highest impact factors in 2024 include the New England Journal of Medicine, British Medical Journal, Journal of American Medical Association (JAMA), and Nature (2).
What Can You Look For In Studies?
It is important to pay attention to the original aim of the study. You may notice in most studies a section called “Conflicts of Interest” or “Competing Interests.” Researchers may have actual, potential, or perceived conflicts of interest that can presume financial benefits or otherwise. For example, some research can be financed by organizations with a personal interest, making the results of these studies less reliable. These circumstances should be declared as a conflict of interest.
A simple factor in the quality of the research is its publication date. Naturally, a study carried out in the past century may be less reliable than a newer research paper that has examined the available information on the topic, including the older study, and added valuable information of its own.
Another aspect to look for is the research's limitations. While all studies have limitations, some can significantly affect research quality. For example, methodological limitations, such as small sample size or self-reported data, limit the quality, amount, and diversity of the data, while analytical limitations, such as measurement errors, can lower the accuracy and completeness of the outcome.
Overall, the quality of research is measured by qualitative and quantitative indicators. A qualitative indicator of a study is whether it’s peer-reviewed or not, while quantitative indicators are the number of times the author has been published, the author’s h-index, the journal’s impact factor, etc.
What Classifications Are There?
Research can be classified according to the method used or research design. Some methods provide more trustworthy information than others. The evidence pyramid is a well-known concept used to visualize how not all scientific information is equally reliable (3).
One way of measuring research outcomes is through the GRADE system. GRADE evaluates the certainty of evidence and gives a recommendation level to a given topic. To use GRADE, the authors pose a question and give an estimated effect. Then, they research all the available evidence on the topic and come to a conclusion (4).
| Certainty | Meaning |
| Very low | The true effect is probably significantly different from the assumed effect |
| Low | The true effect might be significantly different from the assumed effect, but the evidence is limited |
| Moderate | There is moderate evidence that the true effect is probably close to the assumed effect |
| High | There is a lot of evidence that the true effect is close to the assumed effect. |
GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) is endorsed by many respected scientific and health organizations, such as the World Health Organization (WHO) and the UK National Institute for Health and Care Excellence (NICE).
A modified version of the GRADE system suggests dividing recommendation and evidence grades. This version is used as a grading guide in our research articles, as well as the respectable clinical resource library UpToDate (5).
Recommendation grades:
| 1 | Strong recommendation | The benefits clearly outweigh the risks for most people. |
| 2 | Weak recommendation | The benefits and risks are similar or uncertain. |
The strength of the recommendation is mostly based on the benefit/risk and burden/cost ratios.
Evidence grades:
| A | High-quality evidence |
|
| B | Moderate-quality evidence |
|
| C | Low-quality evidence |
|
Observational studies are cohort studies, case-control studies, case reports, etc.
The evidence grades consider the research's quality, including the risk of bias, the accuracy of estimates, the consistency of the results, and the directness of the evidence (5).
For example, red meat is a Group 2A carcinogenic, while processed meat is a Group 1 carcinogenic (6).
Of course, this method of research classification also has its flaws. Mainly, GRADE is subjective and cannot be implemented automatically; different people may have different quality standards.
Resources
- https://www.elsevier.com/researcher/author/policies-and-guidelines
- https://www.researchgate.net/publication/381961222_JCR-Impact-Factor-2024
- https://libguides.winona.edu/ebptoolkit/Levels-Evidence
- https://bestpractice.bmj.com/info/toolkit/learn-ebm/what-is-grade/
- https://www.wolterskluwer.com/en/solutions/uptodate/policies-legal/grading-guide
- https://www.who.int/news-room/questions-and-answers/item/cancer-carcinogenicity-of-the-consumption-of-red-meat-and-processed-meat
