Home >
Community >
What is the index fastq file (sample_I*.fastq.gz) generated when demultiplexing Illumina paired-end runs?
Upvote
26
Downvote
+ Bioinformatics
+ Illumina
+ Sequencing
Posted by
Pamela Smith
What is the index fastq file (sample_I*.fastq.gz) generated when demultiplexing Illumina paired-end runs?
tldr - The I*.fastq.gz file contains the read index sequences.
long explanation
Illumina uses a program called bcl2fastq to demultiplex sequencing runs.
This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (i5 and i7 indices, see page 6 for HiSeq).
Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index, CAGCCCAC can easily have read errors in the A sandwiched between many Cs: CAGCCCAC -> CAGCCCCC.
Instead of throwing out all of the reads with indexing sequencing errors, the bcl2fastq program includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.
In the example provided above, the four indices for mysample were: ACATTACT, CAGCCCAC, GGCAATGG, TTTGGGTA. If we look at all of the index sequences in the Mysample_I1_001.fastq.gz file, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.
bioawk -cfastx '{print($seq)}' Mysample_I1_001.fastq.gz | \
sort | uniq -c | sort -k1 -nr | head
count index source
41362311 CAGCCCAC True index
37209190 GGCAATGG True index
36863213 ACATTACT True index
33674467 TTTGGGTA True index
1140358 NAGCCCAC CAGCCCAC
1026099 NGCAATGG GGCAATGG
1016754 NCATTACT ACATTACT
933342 NTTGGGTA TTTGGGTA
119626 TTTGGGGA TTTGGGTA
98657 GTTGGGTA TTTGGGTA
96625 GGCAATGA GGCAATGG
tldr - The I*.fastq.gz file contains the read index sequences.
long explanation
Illumina uses a program called bcl2fastq to demultiplex sequencing runs.
This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (i5 and i7 indices, see page 6 for HiSeq).
Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index, CAGCCCAC can easily have read errors in the A sandwiched between many Cs: CAGCCCAC -> CAGCCCCC.
Instead of throwing out all of the reads with indexing sequencing errors, the bcl2fastq program includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.
In the example provided above, the four indices for mysample were: ACATTACT, CAGCCCAC, GGCAATGG, TTTGGGTA. If we look at all of the index sequences in the Mysample_I1_001.fastq.gz file, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.
I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library. These four indexes were separate from the individual oligos used to mark single cells. For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data. There is a brief explanation here: https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3
I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library. These four indexes were separate from the individual oligos used to mark single cells. For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data. There is a brief explanation here: https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3
It took me a while to figure out that the "index" is the same thing as the "barcode" that says which sample each sequence is from on a multiplexed run.
If your data is not demultiplexed (single R1.fastq and R2.fastq files contain the information for multiple samples), then this I1.fastq file is what you use to assign each sequence to a sample (ie to "demultiplex"). The spreadsheet listing which barcode/index is which sample is often called a "mapping file."
If your data is already demultiplexed (you have separate R1.fastq and R2.fastq files for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien "Quality filtering of Illumina index reads mitigates sample cross-talk": https://doi.org/10.1186/s12864-016-3217-x
It took me a while to figure out that the "index" is the same thing as the "barcode" that says which sample each sequence is from on a multiplexed run.
If your data is not demultiplexed (single R1.fastq and R2.fastq files contain the information for multiple samples), then this I1.fastq file is what you use to assign each sequence to a sample (ie to "demultiplex"). The spreadsheet listing which barcode/index is which sample is often called a "mapping file."
If your data is already demultiplexed (you have separate R1.fastq and R2.fastq files for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien "Quality filtering of Illumina index reads mitigates sample cross-talk": https://doi.org/10.1186/s12864-016-3217-x
The term index is almost always used to refer to the 8-bp sequence that the Illumina machine reads to identify the library-of-origin of the individual read, while the term barcode could also mean some sequence within R1 or R2 itself that the user added to label provenance - such as being from a specific cell or specific amplicon.More
I received 3 files that exactly match the file names listed by the original poster, except that my data was clearly not demultiplexed yet. My index file referred to what my mapping file listed as the "SampleBarcode". More
For perspective, my data is from a multiplexed 16S rRNA gene amplicon miSeq run. My barcodes identified which sample each sequence was from, and they were a 12 bp sequence attached to the forward primer (so it was not dual-indexed). This is the protocol used by the Earth Microbiome Project.More
Yes, in the context of a 10xGenomics single cell run, the "barcode" means the cell barcode, which is not at all the same thing as the sample index.More
tldr - The I*.fastq.gz file contains the read index sequences.
long explanation
Illumina uses a program called
bcl2fastqto demultiplex sequencing runs.This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (i5 and i7 indices, see page 6 for HiSeq).
Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index,
CAGCCCACcan easily have read errors in theAsandwiched between manyCs:CAGCCCAC -> CAGCCCCC.Instead of throwing out all of the reads with indexing sequencing errors, the
bcl2fastqprogram includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.In the example provided above, the four indices for
mysamplewere:ACATTACT,CAGCCCAC,GGCAATGG,TTTGGGTA. If we look at all of the index sequences in theMysample_I1_001.fastq.gzfile, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.tldr - The I*.fastq.gz file contains the read index sequences.
long explanation
Illumina uses a program called
bcl2fastqto demultiplex sequencing runs.This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (i5 and i7 indices, see page 6 for HiSeq).
Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index,
CAGCCCACcan easily have read errors in theAsandwiched between manyCs:CAGCCCAC -> CAGCCCCC.Instead of throwing out all of the reads with indexing sequencing errors, the
bcl2fastqprogram includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.In the example provided above, the four indices for
mysamplewere:ACATTACT,CAGCCCAC,GGCAATGG,TTTGGGTA. If we look at all of the index sequences in theMysample_I1_001.fastq.gzfile, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.More
VOTE
I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library. These four indexes were separate from the individual oligos used to mark single cells. For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data. There is a brief explanation here: https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3
I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library. These four indexes were separate from the individual oligos used to mark single cells. For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data. There is a brief explanation here: https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3
More
VOTE
It took me a while to figure out that the "index" is the same thing as the "barcode" that says which sample each sequence is from on a multiplexed run.
If your data is not demultiplexed (single
R1.fastqandR2.fastqfiles contain the information for multiple samples), then thisI1.fastqfile is what you use to assign each sequence to a sample (ie to "demultiplex"). The spreadsheet listing which barcode/index is which sample is often called a "mapping file."If your data is already demultiplexed (you have separate
R1.fastqandR2.fastqfiles for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien "Quality filtering of Illumina index reads mitigates sample cross-talk": https://doi.org/10.1186/s12864-016-3217-xIt took me a while to figure out that the "index" is the same thing as the "barcode" that says which sample each sequence is from on a multiplexed run.
If your data is not demultiplexed (single
R1.fastqandR2.fastqfiles contain the information for multiple samples), then thisI1.fastqfile is what you use to assign each sequence to a sample (ie to "demultiplex"). The spreadsheet listing which barcode/index is which sample is often called a "mapping file."If your data is already demultiplexed (you have separate
R1.fastqandR2.fastqfiles for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien "Quality filtering of Illumina index reads mitigates sample cross-talk": https://doi.org/10.1186/s12864-016-3217-xMore
VOTE
VOTE
VOTE
VOTE
VOTE
VOTE