I'm excited to release what I've been cooking up the past few months at @arcinstitute BINSEQ is a family of binary file formats for sequencing data built with paired records and parallel processing in mind with big performance gains (2x-40x) over gzip-fastq with similar storage
bioRxiv Bioinfo @biorxiv_bioinfo
Apr 15
BINSEQ: A Family of High-Performance Binary Formats for Nucleotide Sequences biorxiv.org/content/10.1101/… #biorxiv_bioinfo
7
41
6
273
BINSEQ and VBINSEQ are the only family members (for now). They are both built around two-bit encoded nucleotides and each support paired records natively. We provide rust libraries for IO, C and C++ bindings to BINSEQ, and a CLI tool to easily manipulate them.
1
5
Where these formats really shine is in their ability to scale. Because they don't require being parsed sequentially they continue being performant as sequential formats (FASTQ) plateau. Simple tasks like kmer-counting show gains as early as 8 threads.
1
1
6
Interestingly we also observe the behavior for more complex tasks like sequence alignment. We demonstrate that BINSEQ continues to scale linearly with thread usage with both minimap2 and STAR.

Apr 15, 2025 · 2:35 PM UTC

1
1
BINSEQ is very simple and is built for fixed length records (think single-cell sequencing data). Each record or record pair has true random access and the file can be memory mapped and processed in parallel. github.com/arcinstitute/bins…

GitHub - ArcInstitute/binseq: A high efficiency binary format for sequencing data

A high efficiency binary format for sequencing data - ArcInstitute/binseq

github.com
2
3
VBINSEQ is a flavor of BINSEQ with support for variable length records and quality scores. It is is built around independent record blocks. Each block stores two-bit encoded records and has random access (via an associated index). github.com/arcinstitute/vbin…

GitHub - ArcInstitute/vbinseq: [ARCHIVED] An extension of BINSEQ to handle variable-length records...

[ARCHIVED] An extension of BINSEQ to handle variable-length records - See main binseq repo - ArcInstitute/vbinseq

github.com
1
2
To make working with BINSEQ files easy we’ve also built a command-line tool to convert to and from the format - bqtools. This tool uses both above libraries and can handle various operations such as encoding, decoding, grepping, subsampling, etc. github.com/arcinstitute/bqto…

GitHub - ArcInstitute/bqtools: A command line utilty for working with BINSEQ files

A command line utilty for working with BINSEQ files - ArcInstitute/bqtools

github.com
1
2
We also show how easy it is to integrate BINSEQ and VBINSEQ into existing tools. Here is a CLI which makes use of the minimap2 rust bindings that accepts FASTQ, BINSEQ, and VBINSEQ inputs. github.com/arcinstitute/mmr

GitHub - ArcInstitute/mmr: A minimap2 implementation with binseq inputs

A minimap2 implementation with binseq inputs. Contribute to ArcInstitute/mmr development by creating an account on GitHub.

github.com
1
4
We’ve also built C and C++ bindings to BINSEQ (VBINSEQ upcoming!) so that you can use the libraries without needing to be a rustacean (yet) 🦀. github.com/arcinstitute/bins…

GitHub - ArcInstitute/binseq-bindings: C and C++ Bindings to the BINSEQ library

C and C++ Bindings to the BINSEQ library. Contribute to ArcInstitute/binseq-bindings development by creating an account on GitHub.

github.com
1
2
This work is the first step in a list of upcoming projects that @arcinstitute has for high-throughput genomics tools so be on the lookout over the next few months for some more exciting bioinformatics tools!
1
6
Here is a link to all the tools and a description of the work: arcinstitute.org/tools/binse…

BINSEQ | Arc Institute

Arc Institute is a independent nonprofit research organization headquartered in Palo Alto, California.

arcinstitute.org
3
2