Files in this directory comprise the GlobDB release 226. Descriptions of files and brief methods how the files were created are listed below. More information can be found on https://globdb.org/. ### File descriptions globdb_r226_anvio_dbs.tar.gz Anvi'o contigs databases for all GlobDB genomes, generated using the program anvi-gen-contigs-database from the fasta files provided in the "genome fasta". These anvi'o databases include the gene calling and annotation information of the genomes, and can be used to generate the protein fasta and gff files. Furthermore, the anvi'o databases can be used as a resource for phylogenomics, pangenomics, or other comparative analyses. Note that the anvi'o contigs databases are created with the development version of anvi'o and therefore can only be used with that version. globdb_r226_dictionaries.tar.gz Dictionaries of the genome identifiers in source datasets and the corresponding standardized GlobDB IDs. No dictionary is needed for GTDB, MGnify, and MRGM/HRGM2 as the genome identifiers in these datasets were already sufficiently standardized globdb_r226_genome_fasta.tar.gz Fasta files for all genomes in the GlobDB. Fasta headers have been renamed using the anvi'o script anvi-script-reformat-fasta to simplify the contig header lines and include the GlobDB genome ID at the start of every contig. globdb_r226_gff_cog.tar.gz Anvi'o exported files in GFF format for all GlobDB genomes with COG annotation as done using the anvi'o program anvi-run-ncbi-cogs. See "Methods" page for more details. globdb_r226_md5sum.txt md5sums of the downloadable files in the top level directory globdb_r226_protein_annotations.tar.gz Tab delimited files of KEGG/COG/Pfam/dbCAN2 annotations of all GlobDB genomes. globdb_r226_protein_fasta.tar.gz fasta files for all proteins in the GlobDB genomes annotated using prodigal as integrated in anvi'o, exported using the program anvi-get-sequences-for-gene-calls. Note that the headers of all proteins were renamed after export to include the GlobDB genome ID, and to be consistent with the IDs in the exported gff files. globdb_r226_taxonomy.tsv.gz Two column Tab delimited file containing GlobDB genome ID as well as 7 level taxonomy (assigned by GTDB-tk), each as separate field. globdb_r226_tax_plus_stats.tsv.gz Tab delimted file containing taxonomy, basic genome statistics, and contamination/completeness for all GlobDB genomes. pLM_embeddings Directory containing the files associated with the ProtT5 protein language model embeddings of 82,972,511 GlobDB protein cluster representatives pLM_embeddings/globdb_r226_linclust_2plus_annotations.tsv.gz Subset of the KEGG/COG/Pfam/dbCAN2 annotations, including only annotations for the protein cluster representatives of cluster size >= 2 in a single file pLM_embeddings/globdb_r226_linclust_2plus.faa.gz Protein fasta file including only the 82,972,511 protein cluster representatives of cluster size >= 2 pLM_embeddings/globdb_r226_linclust_2plus_IDs.txt.gz GlobDB IDs of the protein cluster representatives of cluster size >= 2. Can be used to efficiently access the pLM embeddings of specific GlobDB ids pLM_embeddings/globdb_r226_linclust_2plus_ProtT5_embeddings.h5 pLM embeddings of 82,972,511 GlobDB protein cluster representatives pLM_embeddings/globdb_r226_linclust_cluster_members.tsv.gz Tab delimited file listing the GlobDB protein IDs of the proteins included in each cluster pLM_embeddings/globdb_r226_linclust_md5sums.txt md5sums of the downloadable files in the pLM_embeddings directory taxonomic_profiling Directory containing the sylph database and singleM metapackage that can be used to generate taxonomic profiles of metagenomes using the GlobDB taxonomy taxonomic_profiling/globdb_r226_singleM_metapackage.tar.gz singleM metapackage for taxonomic profiling of metagenome datatsets using SingleM and the GlobDB taxonomy taxonomic_profiling/globdb_r226_sylph.syldb Sylph database or taxonomic profiling of metagenome datatsets using Sylph and the GlobDB taxonomy. Requires the globdb_r226_taxonomy.tsv.gz file taxonomic_profiling/globdb_r226_tax_profile_md5sums.txt md5sums of the downloadable files in the taxonomic_profiling directory ### Brief methods # Dereplication of the underlying databases 11 of the 14 data sources for the GlobDB release 226 (GTDB, mOTU, RBG, SPIRE, GEM, MGnify, GOMC, SMAG, MRGM/HRGM2, cFMD, SHGO, AMXMAG, GFS) provides a dereplicated set of genomes, using a 95 % average nucleotide identity (ANI) dereplication criterion. Since the underlying data for each database is distinct, there may be centroids between datasets that belong to the same species by the operational definition of 95 % ANi similarity, but between them represent a cluster of genomes that extends beyond that operational species boundary. Thus it is possible that the genomes that two centroids represent wouldn't all be classified to the same species if all data was processed together. The GlobDB therefore conservatively dereplicates the underlying (already dereplicated) datasets at 96 % ANI over 50 % aligned fractions (AF). We have chosen these cutoffs after inspection of a crossplot of ANI vs AF values, showing a clear ANI gap between the dereplicated datasets starting at approximately 96% ANI. Furthermore, the order of priority for inclusion in the GlobDB is as follows: - All GTDB species representatives are included. - The other datasets are sequentially dereplicated. This means that the MAGs of the second dataset that are not represented in the GTDB are added to the GlobDB, then the MAGs from the third dataset that are not represented in this new combined set are added to the GlobDB, and so on. - The order of sequential dereplication in release 226 is GTDB, mOTU, RBG, SPIRE, GEM, MGnify, GOMC, SMAG, MRGM/HRGM2, cFMD, SHGO, AMXMAG, GFS Dereplication of the data sources is done using fastANI, version 1.3.4, run with the 3000bp fragment length default. For convenience in later use, the names of the genomes in the all datasets except the GTDB, MGnify, and MRGM/HRGM2 are standardized to GlobDB IDs that represent their origin and are propagated throughout the GlobDB resources. Dictionary files to relate the GlobDB identifiers back to the identifiers from the original publications are available in the "Downloads" page. # Generation of anvi'o databases and annotation After dereplication, the resulting genome fasta files are turned into anvi'o databases and basic annotation is performed. First, contig names are standardized using anvi'o with anvi-script-reformat-fasta and the options: --seq-type NT # specifies the sequence type --simplify-names # standardizes the contig name format --overwrite-input # overwrites the source file instead of generating a new file --prefix #prepends the GlobDB ID to the contig name Then, contigs databases are created using anvi-gen-contigs-database, using the GlobDB ID as database name (-n flag) These contigs databases are then populated with annotations using the following commands: anvi-run-hmms # annotates rRNA genes, single copy marker genes, and tRNAs using the "--also-scan-trnas" flag anvi-run-kegg-kofams # annotates coding regions with kegg kofams, and a custom heuristic anvi-run-pfams # annotates coding regions with pfams anvi-run-cazymes # annotates coding regions with dbCAN2 CAzymes anvi-run-ncbi-cogs # annotates coding regions with COGs (2020 update) Next, protein fasta files and gff files for KEGG, COG and Pfam annotations are exported from the contigs databases These files can also be generated from the anni'o databases directly using the anvi-get-sequences-for-gene-calls command with different flags. --get-aa-sequences --wrap 0 --defline-format {contigs_db_project_name}___{gene_caller_id} # protein fasta --export-gff3 --annotation-source COG20_FUNCTION # gff with COG annotation # Clustering amino acid sequences and generating pLM embeddings The protein fastA files of all GlobDB genomes were concatenated into a single fastA file for clustering using Linclust as integrated in MMseqs2 (Release 17-b804f). Clustering was done at 40 % sequence identity (--min-seq-id 0.4) over 80 % of the length of both query and target sequence (--cov-mode 0 -c 0.8). Representatives of clusters consisting of at least two proteins were then used for protein language model (pLM) embedding using the ProtT5-XL-U50 protein large language model. A more detailed description of the embedding process, as well as the scripts used to generate the embeddings are available in this github repository. # Establishing the GlobDB Taxonomy De novo genome trees for bacterial and archaeal genomes from novel species (determined based on 95% ANI clustering) were generated using GTDBtk v2.4.0 de_novo_wf [Chaumeil et al. 2022. Novel clades were identified through a custom algorithm based on relative evolutionary divergence (RED), similar to that described in Supplementary Figure S1 from Chaumeil et al. 2020. Firstly, RED values were calculated by PhyloRank v0.1.12 (github.com/dparks1134/phyloRank) using the de novo genome trees from GTDB-tk. Next, the RED values and genome trees were processed using a custom script (see name_clades.py at [github]). Genomes were sequentially assigned to either pre-existing or novel clades, ordered by genome quality. For each genome, this involved traveling up the tree until a node was reached within the RED-bounds of a new taxa level (e.g. phylum, class, order, etc.) for which the genome has not yet been assigned. Parents and children of the node were checked to find if any were closer to the median RED of that taxa level (from GTDB R226) than the current node. If the current node was the closest node, it is annotated as a new clade, with the current genome being assigned as the genome representative. This continued until a node was reached that had an offspring from GTDB (or until reaching a previously labelled node). The remaining taxonomy was assigned based on the median taxonomy of the GTDB offspring from that node. # Matching genome IDs to culture availability The BacDive python API was queried to linked species representatives in the GlobDB to their respective isolates deposited in major culture collections. A search of BacDive was performed for genomes whose accession numbers began with the prefix "GC" and the entries associated with these genomes retrieved. These isolates, and their culture collection numbers, were then matched to the representatives in the GlobDB based on the genome accession. This match was performed on the numeric part of the genome accession, which remains invariant between GenBank and RefSeq. This ensured that isolates submitted to culture collections and originally linked to a GenBank accession could still be correctly associated, even if the genome was later upgraded to a RefSeq accession not reflected in BacDive.