Skip to content

Add Evidence Synthesis Tools#4015

Open
aurumz-rgb wants to merge 2 commits into
sindresorhus:mainfrom
aurumz-rgb:main
Open

Add Evidence Synthesis Tools#4015
aurumz-rgb wants to merge 2 commits into
sindresorhus:mainfrom
aurumz-rgb:main

Conversation

@aurumz-rgb
Copy link
Copy Markdown

@aurumz-rgb aurumz-rgb commented Mar 19, 2026

https://github.com/evidencesynthesis-tools/awesome-evidence-synthesis#readme

This PR adds a curated list of open-source tools for systematic reviews, meta-analysis, and evidence synthesis. Derived from a comprehensive mapping study, it organizes tools by workflow stage (e.g., search, screening, analysis).

This list provides a vetted, centralized directory of transparent and reusable software. It promotes open science by helping researchers find free, non-proprietary solutions for every step of the evidence synthesis process.

By submitting this pull request I confirm I've read and complied with the below requirements 🖖

Please read it multiple times. I spent a lot of time on these guidelines and most people miss a lot.

Requirements for your pull request

  • Fully AI-generated pull requests are not accepted.
  • Don't open a Draft / WIP pull request while you work on the guidelines. A pull request should be 100% ready and should adhere to all the guidelines when you open it. Instead use #2242 for incubation visibility.
  • Don't waste my time. Do a good job, adhere to all the guidelines, and be responsive.
  • You have to review at least 4 other open pull requests.

Reviewed:
#4012
#4003
#3940
#4008
#4014
#3961
#4016
#4017

  • You have read and understood the instructions for creating a list.
  • This pull request has a title in the format Add Name of List. It should not contain the word Awesome.
    • Add Swift
    • Add Software Architecture
    • Update readme.md
    • Add Awesome Swift
    • Add swift
    • add Swift
    • Adding Swift
    • Added Swift
  • Your entry here should include a short description of the project/theme of the list. It should not describe the list itself. The first character should be uppercase and the description should end in a dot. It should be an objective description and not a tagline or marketing blurb. It should not contain the name of the list.
    • - [iOS](…) - Mobile operating system for Apple phones and tablets.
    • - [Framer](…) - Prototyping interactive UI designs.
    • - [iOS](…) - Resources and tools for iOS development.
    • - [Framer](…)
    • - [Framer](…) - prototyping interactive UI designs
  • Your entry should be added at the bottom of the appropriate category.
  • The title of your entry should be title-cased and the URL to your list should end in #readme.
    • Example: - [Software Architecture](https://github.com/simskij/awesome-software-architecture#readme) - The discipline of designing and building software.
  • No blockchain-related lists.
  • The suggested Awesome list complies with the below requirements.

Requirements for your Awesome list

  • Has been around for at least 30 days.
    That means 30 days from either the first real commit or when it was open-sourced. Whatever is most recent.
  • Is not AI-generated.
  • Run awesome-lint on your list and fix the reported issues. If there are false-positives or things that cannot/shouldn't be fixed, please report it.
  • The default branch should be named main, not master.
  • Includes a succinct description of the project/theme at the top of the readme. (Example)
    • Mobile operating system for Apple phones and tablets.
    • Prototyping interactive UI designs.
    • Resources and tools for iOS development.
    • Awesome Framer packages and tools.
  • It's the result of hard work and the best I could possibly produce.
    If you have not put in considerable effort into your list, your pull request will be immediately closed.
  • The repo name of your list should be in lowercase slug format: awesome-name-of-list.
    • awesome-swift
    • awesome-web-typography
    • awesome-Swift
    • AwesomeWebTypography
  • The heading title of your list should be in title case format: # Awesome Name of List.
    • # Awesome Swift
    • # Awesome Web Typography
    • # awesome-swift
    • # AwesomeSwift
  • Non-generated Markdown file in a GitHub repo.
  • The repo should have awesome-list & awesome as GitHub topics. I encourage you to add more relevant topics.
  • Not a duplicate. Please search for existing submissions.
  • Only has awesome items. Awesome lists are curations of the best, not everything.
  • Does not contain items that are unmaintained, has archived repo, deprecated, or missing docs. If you really need to include such items, they should be in a separate Markdown file.
  • Includes a project logo/illustration whenever possible.
    • Either centered, fullwidth, or placed at the top-right of the readme. (Example)
    • The image should link to the project website or any relevant website.
    • The image should be high-DPI. Set it to a maximum of half the width of the original image.
    • Don't include both a title saying Awesome X and a logo with Awesome X. You can put the header image in a # (Markdown header) or <h1>.
  • Entries have a description, unless the title is descriptive enough by itself. It rarely is though.
  • Includes the Awesome badge.
    • Should be placed on the right side of the readme heading.
      • Can be placed centered if the list has a centered graphics header.
    • Should link back to this list.
  • Has a Table of Contents section.
    • Should be named Contents, not Table of Contents.
    • Should be the first section in the list.
    • Should only have one level of nested lists, preferably none.
    • Must not feature Contributing or Footnotes sections.
  • Has an appropriate license.
    • We strongly recommend the CC0 license, but any Creative Commons license will work.
      • Tip: You can quickly add it to your repo by going to this URL: https://github.com/<user>/<repo>/community/license/new?branch=main&template=cc0-1.0 (replace <user> and <repo> accordingly).
    • A code license like MIT, BSD, Apache, GPL, etc, is not acceptable. Neither are WTFPL and Unlicense.
    • Place a file named license or LICENSE in the repo root with the license text.
    • Do not add the license name, text, or a Licence section to the readme. GitHub already shows the license name and link to the full text at the top of the repo.
    • To verify that you've read all the guidelines, please comment on your pull request with just the word unicorn.
  • Has contribution guidelines.
    • The file should be named contributing.md. The casing is up to you.
    • It can optionally be linked from the readme in a dedicated section titled Contributing, positioned at the top or bottom of the main content.
    • The section should not appear in the Table of Contents.
  • All non-important but necessary content (like extra copyright notices, hyperlinks to sources, pointers to expansive content, etc) should be grouped in a Footnotes section at the bottom of the readme. The section should not be present in the Table of Contents.
  • Has consistent formatting and proper spelling/grammar.
    • The link and description are separated by a dash.
      Example: - [AVA](…) - JavaScript test runner.
    • The description starts with an uppercase character and ends with a period.
    • Consistent and correct naming. For example, Node.js, not NodeJS or node.js.
  • Does not use hard-wrapping.
  • Does not include a CI (e.g. GitHub Actions) badge.
    You can still use a CI for linting, but the badge has no value in the readme.
  • Does not include an Inspired by awesome-foo or Inspired by the Awesome project kinda link at the top of the readme. The Awesome badge is enough.

Go to the top and read it again.

@sindresorhus
Copy link
Copy Markdown
Owner

Thanks for making an Awesome list! 🙌

It looks like you didn't read the guidelines closely enough. I noticed multiple things that are not followed. Try going through the list point for point to ensure you follow it. I spent a lot of time creating the guidelines so I wouldn't have to comment on common mistakes, and rather spend my time improving Awesome.

Copy link
Copy Markdown

@Leooo-Huang Leooo-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together! A few issues I noticed:

  1. Invisible character in the diff: The entry line starts with an invisible Unicode character (U+2060 Word Joiner) before the [ bracket: - ⁠[Evidence Synthesis Tools]. This will likely cause awesome-lint to fail or render incorrectly. It should be a plain ASCII dash-space-bracket: - [Evidence Synthesis Tools].

  2. Trailing blank line: The diff adds an extra blank line after the entry, before ## Events. The surrounding entries don't have double blank lines between them — this should be removed to match the existing formatting.

  3. Description length: The description is quite long (19 words). Most entries in the Miscellaneous section are under 12 words. Consider shortening to something like: "Tools for systematic reviews, meta-analysis, and evidence synthesis." — this removes the redundant second half that restates the same concept.

  4. Logo/illustration: The PR checklist claims a logo is included, but I checked the linked repo and the README uses only the Awesome badge with no project-specific logo or illustration. The guidelines say "Includes a project logo/illustration whenever possible."

Otherwise, the repo itself looks well-maintained with good categorization by workflow stage. Nice work!

@Leooo-Huang Leooo-Huang mentioned this pull request Mar 19, 2026
35 tasks
@aurumz-rgb
Copy link
Copy Markdown
Author

@Leo-Cyberautonomy @sindresorhus Thank you for the review.

I’ve fixed the issues mentioned as well as rechecked the guidelines to ensure full compliance. Please let me know if anything else needs adjustment.

@aurumz-rgb
Copy link
Copy Markdown
Author

unicorn

@aurumz-rgb
Copy link
Copy Markdown
Author

@alpha-one-index Thank you for the thorough review and for pointing out these structural and hygiene issues. I have addressed all the points you raised:

  • .DS_Store: I have removed the committed .DS_Store file and added it to .gitignore to prevent this from happening again.

  • Description: The description has been updated to describe the topic/theme ("Open-source software, libraries, and frameworks...") rather than the list itself.

  • Links & Meta Content: I have moved the Web Directory, Source Repo, and Research Paper links to a new Footnotes section at the bottom of the README. Additionally, the "Inclusion Criteria" and "External API Dependency Policy" sections have been moved to the Footnotes and removed from the Table of Contents to streamline navigation.

  • Commit Messages: I have noted your feedback regarding the commit history. I have ensured this specific commit is descriptive, and I will maintain this standard for all future contributions.

@wolffcatskyy
Copy link
Copy Markdown

Reviewed the repo:

  • Extra badges (Open Source, Maintenance, Open Science, DOI) — only the Awesome badge should be in the README per guidelines.
  • The "Source Project" section at the bottom ("derived from the Evidence Synthesis Tools Directory") reads like an "Inspired by" attribution, which the guidelines say to avoid.
  • The Inclusion Criteria and External API Dependency Policy sections are lengthy — might work better in a separate doc or condensed into Footnotes.
  • LICENSE.md — guidelines say the file should be named license or LICENSE without the .md extension.

@aurumz-rgb
Copy link
Copy Markdown
Author

@wolffcatskyy Thanks for the review, I’ve addressed all the points:

  • Removed extra badges, keeping only the Awesome badge
  • Revised the “Source Project” section to avoid “inspired by” attribution language
  • Moved and condensed the criteria and API policy into Footnotes
  • Renamed LICENSE.md to LICENSE

Appreciate the feedback!

Copy link
Copy Markdown

@Kirandcruz3 Kirandcruz3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

@yesw2000
Copy link
Copy Markdown

yesw2000 commented Apr 1, 2026

The link PROMETHEUS in the section Workflow & Automation is broken.

@aurumz-rgb
Copy link
Copy Markdown
Author

@yesw2000 Hi, it was a typo mistake, its 'PROMPTHEUS' and the typo has been fixed. Thank you for letting me know!

@yesw2000 yesw2000 mentioned this pull request Apr 1, 2026
31 tasks
@sindresorhus
Copy link
Copy Markdown
Owner

  • License not detected by GitHub. GitHub shows "NOASSERTION" for the license despite a LICENSE file existing. Re-add it via https://github.com/evidencesynthesis-tools/awesome-evidence-synthesis/community/license/new?branch=main&template=cc0-1.0 to ensure GitHub recognizes it.

  • Top description describes the list. > Open-source software, libraries, and frameworks designed to support the systematic review, meta-analysis, and evidence synthesis workflow. describes what the list collects ("Open-source software, libraries, and frameworks designed to support..."). Describe the subject, e.g.: "Systematic methods for identifying, evaluating, and integrating research evidence across studies."

  • Banner image not linked. ![Preview](assets/banner4.png) is a plain image with no wrapping <a> tag. Guidelines: "The image should link to the project website or any relevant website." Also, no explicit width is set for high-DPI compliance.

  • General-purpose tools with evidence-synthesis framing. At least 16 entries are general-purpose tools with domain-specific descriptions tacked on:

    • spaCy - general NLP library
    • scikit-learn - general ML library
    • transformers (Hugging Face) - general ML library
    • Gensim - general topic modeling library
    • Tesseract OCR - general OCR engine
    • OCRmyPDF - general PDF OCR tool
    • Tabula, Camelot, PDFPlumber - general PDF extraction tools
    • Nougat - general PDF-to-markdown converter
    • GROBID - general document extraction tool
    • Whoosh - general Python search library
    • Hypothes.is - general web annotation tool
    • Scholarly - general Google Scholar scraper

    These are useful tools but not evidence-synthesis-specific software. Appending "useful for evidence synthesis" doesn't make a general OCR engine a curated evidence synthesis resource.

  • Meta-analysis section is an exhaustive R package catalog, not a curation. The Meta-analysis and Statistics sections together list ~70 CRAN packages. Many are extremely niche (e.g., mc.heterogeneity, metaquant, heterometa, selectMeta, metansue, metapro). The awesome manifesto says: "Only awesome is awesome... You should rather leave stuff out than include too much." This reads as a CRAN search dump rather than a curated selection of the best tools.

  • Contributing section nested under Footnotes. The ### Contributing heading is inside the ## Footnotes section. The guidelines say Contributing should be "a dedicated section titled Contributing, positioned at the top or bottom of the main content", not buried inside another section.

  • Broken link. http://www.rtexttools.com (RTextTools) does not resolve.

  • 277 entries suggest a directory, not a curation. For a niche field like evidence synthesis tools, 277 entries is exhaustive rather than curated. The list appears to include every known tool rather than selecting the best.

  • Footnotes section has a ### Project Links sub-heading with source project attribution. "Web Directory: evidencesynthesis-tools.github.io / Source Repo: ... / Research Paper: ..." This reads like an "Inspired by" or source attribution, which the guidelines say to avoid.

@aurumz-rgb
Copy link
Copy Markdown
Author

@sindresorhus Thank you for the detailed and thoughtful feedback.

I’ve addressed the points you raised:

  • Re-added the license using the GitHub template so it is now properly recognized
  • Refined the repository description to better reflect the subject focus
  • Reviewed and pruned several general-purpose tools to better align with evidence-synthesis relevance
  • Fixed the banner image by linking it appropriately and ensuring proper formatting
  • Reduced and curated the all the sections to avoid exhaustive listings to ensure Awesomeness
  • Moved the Contributing section out of Footnotes into a dedicated standalone section
  • Removed/updated broken links (including RTextTools)
  • Cleaned up the Footnotes section and removed unnecessary project attribution formatting

I also made broader adjustments to improve overall curation quality and alignment with the Awesome list guidelines.

Please let me know if there are any remaining issues or further improvements you’d recommend.

Copy link
Copy Markdown

@omkar-foss omkar-foss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks great and the list is lovely. Some improvements:

  1. You can remove R: from revtools, metagear, mada, meta, etc. to make your list more readable. Because from their CRAN links it's understood those are all R packages.
  2. Dedupe (under Reference Management) should either be removed or updated to Dedupe python repo. Item link currently points to Dedupe.io which shut down on 31st Jan 2023. If you're looking for an actively maintained alternative, you could check out Zingg.
  3. Contributing section has an icon in header, might be an issue as this is non-standard as per awesome list guidelines.

@omkar-foss omkar-foss mentioned this pull request Apr 2, 2026
35 tasks
@aurumz-rgb
Copy link
Copy Markdown
Author

Hi @omkar-foss,

Thank you for your suggestions. I’ve implemented the following updates:

  1. Removed the “R:” prefix as recommended
  2. Updated the outdated link to the latest version: dedupe.io/en/latest/
  3. Regarding the icon in the footnote, since @sindresorhus did not raise any concerns about it, I assume it aligns with the Awesome list guidelines

Apologies for tagging you again, Sindre.

@stuffbucket
Copy link
Copy Markdown

The paragraph immediately after the tagline ("This list is derived from the comprehensive directory at evidencesynthesis-tools.github.io and features only the best, most impactful open-source tools...") is describing the list rather than the subject. The guidelines are pretty strict about keeping the top section focused on what the topic is, not how the list was made. Worth trimming or removing.

The depth of coverage here is impressive — evidence synthesis tooling is a real gap in the research software discovery space.

@aurumz-rgb
Copy link
Copy Markdown
Author

@stuffbucket Done. Thanks for pointing that out, really appreciate the feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants