Skip to content
@jennifer/ngs

Cheatsheet

Every exported name in the deck, by module. A function's return type is not declared in Jennifer, so it is given in the description.

For the reasoning behind these calls, start from the topic pages - this page is the index, not the explanation.

core

Shared primitives the rest of the deck is built on: error construction, Go-speed counting, tab-delimited field access.

Constants

constanttypevaluedescription
core.ERROR_KINDstring"ngs"Deck-wide error kind prefix, so a caller can match on $e.kind.
core.TABstring"\t"Tab, the field separator every text format in this deck uses.

Functions

functiondescription
core.fail(what as string, message as string)Raise a deck error with a stable kind.
core.countIn(haystack as string, needle as string)Count non-overlapping occurrences of needle in haystack.
core.parseInt(what as string, label as string, raw as string)Parse an integer field, reporting a malformed value as a deck error.
core.parseFloat(what as string, label as string, raw as string)Parse a float field, reporting a malformed value as a deck error.
core.reraise(what as string, line as string, e as Error)Re-raise a parse failure as a deck error naming the record it came from.
core.splitTab(line as string)Split a tab-delimited line into its fields.
core.field(fields as list of string, i as int, fallback as string)Field i of fields, or fallback when the column is absent.
core.intField(what as string, fields as list of string, i as int, fallback as int)Integer field i, or fallback when absent, empty or a placeholder dot.
core.floatField(what as string, fields as list of string, i as int, fallback as float)Float field i, or fallback when absent, empty or a placeholder dot.
core.isSkippable(line as string, marker as string)Whether a line should be skipped by a record reader: empty or a comment.
core.padNum(n as int, width as int)Left-pad an integer with zeros to width, for building sortable string keys.
core.shellQuote(s as string)Quote a string for safe inclusion in a POSIX sh -c command line.
core.reverseComplement(seq as string)Reverse complement of a DNA sequence.
core.complementBase(b as string)Complement of a single nucleotide character.
core.gcCount(seq as string)G+C count of a sequence, counted at Go speed.
core.nCount(seq as string)Ambiguous-base (N) count of a sequence, counted at Go speed.
core.hasTool(name as string)Whether an executable is on PATH.
core.sameFile(a as string, b as string)Whether two paths resolve to the same file.

source

Streaming line sources and sinks, with transparent gzip in both directions.

Types

typekinddescription
source.SourcestructAn open line source. Treat it as an opaque handle and pass it around by value: the fs.File and os.Process fields are registry handles, so copies share the same underlying stream.
source.SinkstructAn open line sink, the write-side counterpart of Source.

Constants

constanttypevaluedescription
source.GZIP_MAGIC1int0x1fFirst byte of the gzip magic number (RFC 1952).
source.GZIP_MAGIC2int0x8bSecond byte of the gzip magic number (RFC 1952).

Functions

functiondescription
source.isGzip(path as string)Whether a file starts with the gzip magic number.
source.open(path as string)Open a file for streaming line reads, decompressing gzip transparently.
source.openPlain(path as string)Open an uncompressed file for streaming line reads.
source.openStdin()Open standard input as a line source, so the deck composes with Unix pipes.
source.openGzip(path as string)Open a gzipped file for streaming line reads at constant memory.
source.openGzipBuffered(path as string)Open a gzipped file by decompressing it whole, for hosts without exec.
source.hasNext(s as Source)Whether another line can be read from the source.
source.nextLine(s as Source)Read the next line, with its line terminator stripped.
source.close(s as Source)Close a source and release its child process and temporary files.
source.closeStrict(s as Source)Close a fully-consumed source, raising if its decompressor failed.
source.create(path as string)Open a file for streaming line writes, gzip-compressing when the name ends in .gz.
source.createPlain(path as string)Open an uncompressed file for streaming line writes.
source.createGzip(path as string)Open a gzip-compressed file for streaming line writes at constant memory.
source.writeText(k as Sink, text as string)Write text to a sink verbatim, adding no line terminator.
source.writeLine(k as Sink, line as string)Write one line to a sink, appending a newline.
source.closeSink(k as Sink)Close a sink, flush it and wait for its compressor to finish.

fastq

FASTQ records, streaming reader and writer, per-record accessors.

Types

typekinddescription
fastq.ReadstructOne FASTQ record.
fastq.ReaderstructA streaming FASTQ cursor.

Constants

constanttypevaluedescription
fastq.PHRED33int33Sanger / Illumina 1.8+ quality offset; the modern default.
fastq.PHRED64int64Legacy Illumina 1.3-1.7 quality offset.

Functions

functiondescription
fastq.open(path as string)Open a FASTQ file for streaming, decompressing gzip transparently.
fastq.openStdin()Open standard input as a FASTQ stream, for use in Unix pipes.
fastq.hasNext(r as Reader)Whether another record is available.
fastq.next(r as Reader)Read the next record.
fastq.close(r as Reader)Close the reader and release its decompressor.
fastq.closeStrict(r as Reader)Close a fully-consumed reader, raising if its decompressor failed.
fastq.parse(header as string, seq as string, plus as string, qual as string)Build a record from its four raw lines, validating the FASTQ shape.
fastq.format(rec as Read)Render a record back to its four-line FASTQ text, newline-terminated.
fastq.toFasta(rec as Read)Render a record as two-line FASTA text, newline-terminated.
fastq.headerOf(rec as Read)The full header text of a record: id plus description when present.
fastq.write(k as source.Sink, rec as Read)Write a record to a sink as FASTQ.
fastq.length(rec as Read)Read length in bases.
fastq.qualities(rec as Read, offset as int)Decode the quality string to Phred scores.
fastq.meanQuality(rec as Read, offset as int)Mean Phred quality of a record.
fastq.minQuality(rec as Read, offset as int)Lowest Phred score in a record.
fastq.gcPercent(rec as Read)G+C content of a record as a percentage, counted at Go speed.
fastq.ambiguousCount(rec as Read)Number of ambiguous (N) bases in a record, counted at Go speed.
fastq.subseq(rec as Read, start as int, end as int)A sub-range of a record, slicing sequence and quality together.
fastq.reverseComplement(rec as Read)Reverse complement of a record, with the quality string reversed to match.

bed

Streaming BED reading and writing, straight onto the interval engine.

Types

typekinddescription
bed.ReaderstructA streaming BED cursor.
bed.NextenumThe result of advancing a BED cursor: either a feature or end-of-stream.

Constants

constanttypevaluedescription
bed.MISSINGstring"."Placeholder written for an absent name or strand.

Functions

functiondescription
bed.open(path as string)Open a BED file for streaming, decompressing gzip transparently.
bed.openStdin()Open standard input as a BED stream.
bed.next(r as Reader)Advance the cursor to the next feature, skipping blank, comment, track and browser lines.
bed.close(r as Reader)Close the cursor and release its decompressor.
bed.closeStrict(r as Reader)Close a fully-consumed cursor, raising if its decompressor failed.
bed.parse(line as string)Parse one BED line into an interval.
bed.format(iv as intervals.Interval)Render an interval as a six-column BED line.
bed.readAll(path as string)Read a whole BED file into memory.
bed.writeAll(path as string, ivs as list of intervals.Interval)Write intervals to a BED file, gzipping when the name ends in .gz.
bed.write(k as source.Sink, iv as intervals.Interval)Write one interval to an already-open sink.

gff

Streaming GFF3 and GTF reading and writing, with attribute parsing in both styles.

Types

typekinddescription
gff.FeaturestructOne GFF3 or GTF feature.
gff.ReaderstructA streaming GFF/GTF cursor.
gff.NextenumThe result of advancing a GFF cursor.

Constants

constanttypevaluedescription
gff.MISSINGstring"."Placeholder used for an absent column.
gff.STYLE_GFF3string"gff3"Attribute style: key=value; pairs, percent-escaped.
gff.STYLE_GTFstring"gtf"Attribute style: key "value"; pairs.

Functions

functiondescription
gff.open(path as string)Open a GFF3 or GTF file for streaming, decompressing gzip transparently.
gff.openStdin()Open standard input as a GFF/GTF stream.
gff.next(r as Reader)Advance the cursor to the next feature.
gff.close(r as Reader)Close the cursor and release its decompressor.
gff.closeStrict(r as Reader)Close a fully-consumed cursor, raising if its decompressor failed.
gff.parse(line as string)Parse one GFF3 or GTF record.
gff.parseAttributes(text as string)Parse a ninth-column attribute string in either GFF3 or GTF style.
gff.styleOf(text as string)Which attribute style a ninth column uses.
gff.attribute(f as Feature, key as string, fallback as string)Read an attribute, or a fallback when it is absent.
gff.identifier(f as Feature)The feature's identifier: GFF3 ID, else GTF transcript_id, else gene_id.
gff.name(f as Feature)The feature's display name: GFF3 Name, else gene_name, else identifier.
gff.toInterval(f as Feature)Convert a feature to a half-open interval for the interval engine.
gff.fromInterval(iv as intervals.Interval, origin as string, kind as string)Build a feature from a half-open interval, converting back to 1-based.
gff.format(f as Feature, style as string)Render a feature as a nine-column record.
gff.formatAttributes(attrs as map of string to string, style as string)Render an attribute map in GFF3 or GTF style.
gff.readAll(path as string)Read a whole annotation file into memory.
gff.readIntervals(path as string, kind as string)Read an annotation file directly as half-open intervals.

sam

Streaming SAM reading and writing, FLAG and CIGAR handling.

Types

typekinddescription
sam.RecordstructOne SAM alignment record.
sam.CigarOpstructOne CIGAR operation.
sam.ReaderstructA streaming SAM cursor, carrying the file's header block.
sam.NextenumThe result of advancing a SAM cursor.

Constants

constanttypevaluedescription
sam.FLAG_PAIREDint0x1Read is paired in sequencing.
sam.FLAG_PROPER_PAIRint0x2Each segment is properly aligned according to the aligner.
sam.FLAG_UNMAPPEDint0x4This segment is unmapped.
sam.FLAG_MATE_UNMAPPEDint0x8The mate segment is unmapped.
sam.FLAG_REVERSEint0x10This segment is on the reverse strand.
sam.FLAG_MATE_REVERSEint0x20The mate segment is on the reverse strand.
sam.FLAG_FIRSTint0x40This is the first segment of the template.
sam.FLAG_LASTint0x80This is the last segment of the template.
sam.FLAG_SECONDARYint0x100This is a secondary alignment.
sam.FLAG_QC_FAILint0x200The read failed platform or vendor quality checks.
sam.FLAG_DUPLICATEint0x400The read is a PCR or optical duplicate.
sam.FLAG_SUPPLEMENTARYint0x800This is a supplementary alignment.
sam.MISSINGstring"*"Placeholder used for an absent column.

Functions

functiondescription
sam.open(path as string)Open a SAM file for streaming, capturing its header block.
sam.openStdin()Open standard input as a SAM stream.
sam.next(r as Reader)Advance the cursor to the next alignment record.
sam.close(r as Reader)Close the cursor and release its decompressor.
sam.closeStrict(r as Reader)Close a fully-consumed cursor, raising if its decompressor failed.
sam.parse(line as string)Parse one SAM alignment line.
sam.format(rec as Record)Render an alignment back to a SAM record line.
sam.hasFlag(rec as Record, bit as int)Whether a FLAG bit is set on a record.
sam.isUnmapped(rec as Record)Whether the record is unmapped.
sam.isReverse(rec as Record)Whether the record is mapped to the reverse strand.
sam.isSecondary(rec as Record)Whether the record is a secondary alignment.
sam.isSupplementary(rec as Record)Whether the record is a supplementary alignment.
sam.isDuplicate(rec as Record)Whether the record is flagged as a PCR or optical duplicate.
sam.isPrimary(rec as Record)Whether the record is a primary alignment: mapped, not secondary, not supplementary.
sam.strandOf(rec as Record)The strand a record aligns to.
sam.parseCigar(cigar as string)Split a CIGAR string into its operations.
sam.referenceLength(cigar as string)Number of reference bases a CIGAR string consumes.
sam.queryLength(cigar as string)Number of query bases a CIGAR string consumes.
sam.toInterval(rec as Record)The reference interval an alignment covers, as a half-open interval.
sam.tagValue(rec as Record, tagName as string, fallback as string)Read the value of an optional tag.
sam.referenceSizes(r as Reader)The reference sequences declared in the header, as chromosome sizes.
sam.headerOf(path as string)Read just the @ header block of a SAM file.

vcf

Streaming VCF reading and writing, INFO and per-sample FORMAT access.

Types

typekinddescription
vcf.VariantstructOne VCF variant record.
vcf.ReaderstructA streaming VCF cursor, carrying the file's header.
vcf.NextenumThe result of advancing a VCF cursor.

Constants

constanttypevaluedescription
vcf.MISSINGstring"."Placeholder used for an absent column.
vcf.FIXED_COLUMNSint9Number of fixed columns before the per-sample genotype columns.

Functions

functiondescription
vcf.open(path as string)Open a VCF file for streaming, capturing its header.
vcf.openStdin()Open standard input as a VCF stream.
vcf.next(r as Reader)Advance the cursor to the next variant record.
vcf.close(r as Reader)Close the cursor and release its decompressor.
vcf.closeStrict(r as Reader)Close a fully-consumed cursor, raising if its decompressor failed.
vcf.headerOf(path as string)Read just the header lines of a VCF file.
vcf.parse(line as string)Parse one VCF variant line.
vcf.parseInfo(text as string)Parse a VCF INFO column into a map.
vcf.info(v as Variant, key as string, fallback as string)Read an INFO field, or a fallback when it is absent.
vcf.hasInfo(v as Variant, key as string)Whether an INFO flag is present.
vcf.passed(v as Variant)Whether the variant passed every filter.
vcf.isSnv(v as Variant)Whether the variant is a single-nucleotide substitution.
vcf.isIndel(v as Variant)Whether the variant is an insertion or deletion.
vcf.sampleValue(v as Variant, sampleIndex as int, key as string, fallback as string)The per-sample value of one FORMAT key.
vcf.genotype(v as Variant, sampleIndex as int)The genotype call of one sample.
vcf.toInterval(v as Variant)The reference span of a variant, as a half-open interval.
vcf.format(v as Variant)Render a variant back to a VCF record line.
vcf.formatInfo(entries as map of string to string)Render an INFO map back to its column text.

intervals

Interval arithmetic over genomic features - the bedtools-lite layer.

Types

typekinddescription
intervals.IntervalstructA half-open genomic interval.

Constants

constanttypevaluedescription
intervals.KEY_WIDTHint12Width used when zero-padding coordinates into sort keys.
intervals.MAX_COORDINATEint1000000000000One past the largest coordinate an interval may carry.

Functions

functiondescription
intervals.make(chrom as string, start as int, end as int)Build an unnamed, unstranded interval.
`intervals.named(chrom as string,

start as int, end as int, name as string, score as float, strand as string) | Build a fully specified interval. | | intervals.length(iv as Interval) | Width of an interval in bases. | | intervals.overlaps(a as Interval, b as Interval) | Whether two intervals share at least one base. | | intervals.overlapLength(a as Interval, b as Interval) | Number of bases two intervals share. | | intervals.contains(outer as Interval, inner as Interval) | Whether outer fully contains inner. | | intervals.intersection(a as Interval, b as Interval) | The shared region of two overlapping intervals. | | intervals.sortKey(iv as Interval) | Sort key for one interval: chromosome, then zero-padded start and end. | | intervals.sort(ivs as list of Interval) | Sort intervals by chromosome then start then end. | | intervals.merge(ivs as list of Interval, maxGap as int) | Merge overlapping and nearby intervals, per chromosome. | | intervals.totalLength(ivs as list of Interval) | Total number of bases covered, counting overlaps once. | | intervals.intersect(a as list of Interval, b as list of Interval) | Pairwise intersection regions between two interval sets. | | intervals.countOverlaps(a as list of Interval, b as list of Interval) | Count, for each query interval, how many target intervals it overlaps. | | intervals.coverage(a as list of Interval, b as list of Interval) | Bases of each query interval covered by the target set. | | intervals.subtract(a as list of Interval, b as list of Interval) | Remove from each query interval every base covered by the target set. | | intervals.complement(ivs as list of Interval, sizes as map of string to int) | The gaps between intervals, across whole chromosomes. | | intervals.slop(iv as Interval, left as int, right as int, limit as int) | Grow an interval on both sides, clamped at zero and an optional chromosome end. | | intervals.byChromosome(ivs as list of Interval) | Group intervals by chromosome. | | intervals.toBed3(iv as Interval)` | Render an interval as a three-column BED line. |

qc

One-pass streaming read QC - the FastQC-lite layer.

Types

typekinddescription
qc.OptionsstructHow to run an analysis.
qc.ReportstructEverything one pass measured.

Constants

constanttypevaluedescription
qc.MAX_PHREDint94Highest Phred score tracked; covers the whole printable ASCII range.
qc.Q30int30Quality score at or above which a base is counted towards Q30.
qc.DEFAULT_BATCHint2000Records buffered before each batch of Go-speed counting.
qc.DEFAULT_BATCH_BASESint4000000Bases buffered before a batch is counted, whichever limit is reached first.
qc.ADAPTER_TRUSEQstring"AGATCGGAAGAGC"Illumina TruSeq / NEBNext universal adapter prefix.
qc.ADAPTER_NEXTERAstring"CTGTCTCTTATACACATCT"Nextera / Illumina transposase adapter prefix.
qc.ADAPTER_SMALL_RNAstring"TGGAATTCTCGG"Illumina small-RNA 3' adapter prefix.
qc.ARTEFACT_POLY_Gstring"GGGGGGGGGG"Poly-G run, the two-colour-chemistry artefact seen on NovaSeq / NextSeq.

Functions

functiondescription
qc.options()Default analysis options: every read, no per-position quality, Phred+33.
qc.defaultAdapters()The adapters and artefacts counted when Options.adapters is empty.
qc.analyze(path as string, opts as Options)Run a single streaming pass over a FASTQ file.
qc.meanQuality(rep as Report)Mean Phred quality across every sampled base.
qc.q30Percent(rep as Report)Percentage of sampled bases at Q30 or better.
qc.gcPercent(rep as Report)G+C content of the sampled bases as a percentage.
qc.meanLength(rep as Report)Mean sampled read length.
qc.perBaseMean(rep as Report)Mean quality at each read position.
qc.summary(rep as Report)Render a report as a human-readable text summary.

trim

Trimming, filtering and subsampling - the fastp-lite layer.

Types

typekinddescription
trim.OptionsstructHow to trim and filter.
trim.StatsstructWhat a run did.

Constants

constanttypevaluedescription
trim.DEFAULT_WINDOWint4Default sliding window width for quality trimming.
trim.DEFAULT_OVERLAPint6Shortest partial adapter match accepted at a read's 3' end.

Functions

functiondescription
trim.options()Default options: adapter clipping on, sliding-window quality trimming at Q20, and reads shorter than 25 bases discarded.
trim.run(inPath as string, outPath as string, opts as Options)Stream a FASTQ file through trimming and filtering into a new file.
trim.trimRead(rec as fastq.Read, opts as Options)Apply every configured trimming step to one record.
trim.clipFixed(rec as fastq.Read, front as int, tail as int)Remove a fixed number of bases from each end.
trim.clipAdapters(rec as fastq.Read, adapters as list of string, minOverlap as int)Clip the read at the first adapter occurrence, including a partial adapter running off the 3' end.
trim.adapterStart(seq as string, adapters as list of string, minOverlap as int)The earliest position at which an adapter starts in a sequence.
trim.clipQuality(rec as fastq.Read, threshold as int, window as int, offset as int)Trim the 3' end where a sliding window's mean quality falls below a threshold.
trim.qualityKeepLength(qual as string, threshold as int, window as int, offset as int)How many bases survive sliding-window quality trimming.
trim.truncate(rec as fastq.Read, maxLength as int)Cap a record's length, keeping the 5' end.
trim.keeps(rec as fastq.Read, opts as Options)Whether a trimmed record passes the configured filters.
trim.subsampleEvery(inPath as string, outPath as string, everyNth as int, maxReads as int)Copy every Nth record of a FASTQ file to a new file.
trim.subsampleFraction(inPath as string, outPath as string, fraction as float)Copy a random fraction of a FASTQ file to a new file.

pipeline

Run, skip, check and fan out external tools.

Types

typekinddescription
pipeline.JobstructOne step of a pipeline.
pipeline.ResultstructWhat running a job produced.

Functions

functiondescription
pipeline.job(name as string, argv as list of string)Build a job from a name and an argv.
pipeline.shell(name as string, cmdline as string)Build a job that runs a shell command line.
pipeline.needing(j as Job, inputs as list of string)A copy of the job that declares the files it reads.
pipeline.producing(j as Job, outputs as list of string)A copy of the job that declares the files it produces.
pipeline.feeding(j as Job, text as string)A copy of the job with text to feed to its standard input.
pipeline.tolerant(j as Job)A copy of the job whose failure is reported but not fatal.
pipeline.needsRun(j as Job)Whether a job still needs to run.
pipeline.run(j as Job)Run one job, skipping it when its outputs are already up to date.
pipeline.runAll(jobs as list of Job)Run jobs one after another, stopping at the first real failure.
pipeline.runParallel(jobs as list of Job, width as int)Run jobs concurrently, at most width at a time.
pipeline.check(res as Result)Raise when a result represents a real failure.
pipeline.checkAll(results as list of Result)Raise on the first real failure in a batch of results.
pipeline.allOk(results as list of Result)Whether every result succeeded or was tolerated.
pipeline.hasTool(name as string)Whether an executable is on PATH.
pipeline.requireTools(names as list of string)Raise unless every named tool is available.
pipeline.workDir(tag as string)A scratch directory that the caller is responsible for removing.
pipeline.cleanUp(dir as string)Remove a directory and everything under it.
pipeline.summary(results as list of Result)Render results as one status line per job.

ngs

The deck entry module: identity and host capabilities.

Types

typekinddescription
ngs.CapabilitiesstructWhat this host can and cannot do with the deck.

Constants

constanttypevaluedescription
ngs.VERSIONstring"0.1.0"The deck's version.
ngs.NAMEstring"@jennifer/ngs"The deck's canonical package name.

Functions

functiondescription
ngs.capabilities()What the current interpreter and host support.
ngs.describe()A one-line description of the deck and what this host supports.
ngs.requireFullHost()Raise unless the host can stream gzip and drive external tools.

244 exported names across 12 modules.