### Pipeline run code and environment:
* Command: `/home/mjs5kd/.local/bin/refgenie build hg18/hisat2_index`
* Compute host: udc-ba25-18
* Working dir: /sfs/qumulo/qproject/shefflab/deploy/rg.databio.org_full/rg.databio.org
* Outfolder: /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/_refgenie_build/
* Pipeline started at: (04-07 18:44:21) elapsed: 0.0 _TIME_
### Version log:
* Python version: 3.6.6
* Pypiper dir: `/sfs/qumulo/qhome/mjs5kd/.local/lib/python3.6/site-packages/pypiper`
* Pypiper version: 0.12.1
* Pipeline dir: `/sfs/qumulo/qhome/mjs5kd/.local/bin`
* Pipeline version: None
### Arguments passed to pipeline:
* `asset_registry_paths`: `['hg18/hisat2_index']`
* `assets`: `None`
* `command`: `build`
* `config_file`: `refgenie.yaml`
* `docker`: `False`
* `files`: `None`
* `genome`: `None`
* `genome_config`: `None`
* `genome_description`: `None`
* `logdev`: `False`
* `new_start`: `False`
* `outfolder`: `/project/shefflab/deploy/rg.databio.org_full/genomes/data`
* `params`: `None`
* `recipe`: `None`
* `recover`: `False`
* `requirements`: `False`
* `silent`: `False`
* `skip_read_lock`: `False`
* `tag_description`: `None`
* `verbosity`: `None`
* `volumes`: `None`
----------------------------------------
Target to produce: `/project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/_refgenie_build/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01_hisat2_index__default.flag`
> `hisat2-build /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/fasta/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.fa /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01` (28434)
[33mWARNING:[0m Skipping mount /opt/singularity/3.5.2/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
Settings:
Output files: "/project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.*.ht2"
Line rate: 6 (line is 64 bytes)
Lines per side: 1 (side is 64 bytes)
Offset rate: 4 (one in 16)
FTable chars: 10
Strings: unpacked
Local offset rate: 3 (one in 8)
Local fTable chars: 6
Local sequence length: 57344
Local sequence overlap between two consecutive indexes: 1024
Endianness: little
Actual local endianness: little
Sanity checking: disabled
Assertions: disabled
Random seed: 0
Sizeofs: void*:8, int:4, long:8, size_t:8
Input files DNA, FASTA:
/project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/fasta/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.fa
Reading reference sizes
Time reading reference sizes: 00:00:27
Calculating joined length
Writing header
Reserving space for joined string
Joining reference sequences
Time to join reference sequences: 00:00:16
Time to read SNPs and splice sites: 00:00:00
Using parameters --bmax 540266568 --dcv 1024
Doing ahead-of-time memory usage test
Passed! Constructing with these parameters: --bmax 540266568 --dcv 1024
Constructing suffix-array element generator
Building DifferenceCoverSample
Building sPrime
Building sPrimeOrder
V-Sorting samples
V-Sorting samples time: 00:01:48
Allocating rank array
Ranking v-sort output
Ranking v-sort output time: 00:00:29
Invoking Larsson-Sadakane on ranks
Invoking Larsson-Sadakane on ranks time: 00:00:49
Sanity-checking and returning
Building samples
Reserving space for 12 sample suffixes
Generating random suffixes
QSorting 12 sample offsets, eliminating duplicates
QSorting sample offsets, eliminating duplicates time: 00:00:00
Multikey QSorting 12 samples
(Using difference cover)
Multikey QSorting samples time: 00:00:00
Calculating bucket sizes
Splitting and merging
Splitting and merging time: 00:00:00
Avg bucket size: 3.60178e+08 (target: 540266567)
Converting suffix-array elements to index image
Allocating ftab, absorbFtab
Entering GFM loop
Getting block 1 of 8
Reserving size (540266568) for bucket 1
Calculating Z arrays for bucket 1
Entering block accumulator loop for bucket 1:
bucket 1: 10%
bucket 1: 20%
bucket 1: 30%
bucket 1: 40%
bucket 1: 50%
bucket 1: 60%
bucket 1: 70%
bucket 1: 80%
bucket 1: 90%
bucket 1: 100%
Sorting block of length 447150746 for bucket 1
(Using difference cover)
Sorting block time: 00:06:54
Returning block of 447150747 for bucket 1
Getting block 2 of 8
Reserving size (540266568) for bucket 2
Calculating Z arrays for bucket 2
Entering block accumulator loop for bucket 2:
bucket 2: 10%
bucket 2: 20%
bucket 2: 30%
bucket 2: 40%
bucket 2: 50%
bucket 2: 60%
bucket 2: 70%
bucket 2: 80%
bucket 2: 90%
bucket 2: 100%
Sorting block of length 457899971 for bucket 2
(Using difference cover)
Sorting block time: 00:07:03
Returning block of 457899972 for bucket 2
Getting block 3 of 8
Reserving size (540266568) for bucket 3
Calculating Z arrays for bucket 3
Entering block accumulator loop for bucket 3:
bucket 3: 10%
bucket 3: 20%
bucket 3: 30%
bucket 3: 40%
bucket 3: 50%
bucket 3: 60%
bucket 3: 70%
bucket 3: 80%
bucket 3: 90%
bucket 3: 100%
Sorting block of length 353936055 for bucket 3
(Using difference cover)
Sorting block time: 00:05:53
Returning block of 353936056 for bucket 3
Getting block 4 of 8
Reserving size (540266568) for bucket 4
Calculating Z arrays for bucket 4
Entering block accumulator loop for bucket 4:
bucket 4: 10%
bucket 4: 20%
bucket 4: 30%
bucket 4: 40%
bucket 4: 50%
bucket 4: 60%
bucket 4: 70%
bucket 4: 80%
bucket 4: 90%
bucket 4: 100%
Sorting block of length 216139682 for bucket 4
(Using difference cover)
Sorting block time: 00:03:28
Returning block of 216139683 for bucket 4
Getting block 5 of 8
Reserving size (540266568) for bucket 5
Calculating Z arrays for bucket 5
Entering block accumulator loop for bucket 5:
bucket 5: 10%
bucket 5: 20%
bucket 5: 30%
bucket 5: 40%
bucket 5: 50%
bucket 5: 60%
bucket 5: 70%
bucket 5: 80%
bucket 5: 90%
bucket 5: 100%
Sorting block of length 460006621 for bucket 5
(Using difference cover)
Sorting block time: 00:07:41
Returning block of 460006622 for bucket 5
Getting block 6 of 8
Reserving size (540266568) for bucket 6
Calculating Z arrays for bucket 6
Entering block accumulator loop for bucket 6:
bucket 6: 10%
bucket 6: 20%
bucket 6: 30%
bucket 6: 40%
bucket 6: 50%
bucket 6: 60%
bucket 6: 70%
bucket 6: 80%
bucket 6: 90%
bucket 6: 100%
Sorting block of length 280879014 for bucket 6
(Using difference cover)
Sorting block time: 00:04:17
Returning block of 280879015 for bucket 6
Getting block 7 of 8
Reserving size (540266568) for bucket 7
Calculating Z arrays for bucket 7
Entering block accumulator loop for bucket 7:
bucket 7: 10%
bucket 7: 20%
bucket 7: 30%
bucket 7: 40%
bucket 7: 50%
bucket 7: 60%
bucket 7: 70%
bucket 7: 80%
bucket 7: 90%
bucket 7: 100%
Sorting block of length 342869993 for bucket 7
(Using difference cover)
Sorting block time: 00:05:34
Returning block of 342869994 for bucket 7
Getting block 8 of 8
Reserving size (540266568) for bucket 8
Calculating Z arrays for bucket 8
Entering block accumulator loop for bucket 8:
bucket 8: 10%
bucket 8: 20%
bucket 8: 30%
bucket 8: 40%
bucket 8: 50%
bucket 8: 60%
bucket 8: 70%
bucket 8: 80%
bucket 8: 90%
bucket 8: 100%
Sorting block of length 322539607 for bucket 8
(Using difference cover)
Sorting block time: 00:05:08
Returning block of 322539608 for bucket 8
Exited GFM loop
fchr[A]: 0
fchr[C]: 850568906
fchr[G]: 1439951472
fchr[T]: 2029686830
fchr[$]: 2881421696
Exiting GFM::buildToDisk()
Returning from initFromVector
Wrote 964771868 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.1.ht2
Wrote 720355432 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.2.ht2
Re-opening _in1 and _in2 as input streams
Returning from GFM constructor
Returning from initFromVector
Wrote 1266926365 bytes to primary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.5.ht2
Wrote 733549128 bytes to secondary GFM file: /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01.6.ht2
Re-opening _in5 and _in5 as input streams
Returning from HGFM constructor
Headers:
len: 2881421696
gbwtLen: 2881421697
nodes: 2881421697
sz: 720355424
gbwtSz: 720355425
lineRate: 6
offRate: 4
offMask: 0xfffffff0
ftabChars: 10
eftabLen: 0
eftabSz: 0
ftabLen: 1048577
ftabSz: 4194308
offsLen: 180088857
offsSz: 720355428
lineSz: 64
sideSz: 64
sideGbwtSz: 48
sideGbwtLen: 192
numSides: 15007405
numLines: 15007405
gbwtTotLen: 960473920
gbwtTotSz: 960473920
reverse: 0
linearFM: Yes
Total time for call to driver() for forward index: 01:18:14
Command completed. Elapsed time: 1:18:16. Running peak memory: 5.208GB.
PID: 28434; Command: hisat2-build; Return code: 0; Memory used: 5.208GB
> `touch /project/shefflab/deploy/rg.databio.org_full/genomes/data/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index/default/_refgenie_build/9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01_hisat2_index__default.flag` (10263)
Command completed. Elapsed time: 0:00:00. Running peak memory: 5.208GB.
PID: 10263; Command: touch; Return code: 0; Memory used: 0.001GB
Asset digest: 0ac7d3e119d2672b2c9181044f3f7733
Default tag for '9b7fd88cae1bd01bf53760fb426e2aeb299150cc64af2b01/hisat2_index' set to: default
### Pipeline completed. Epilogue
* Elapsed time (this run): 1:18:27
* Total elapsed time (all runs): 1:18:16
* Peak memory (this run): 5.2081 GB
* Pipeline completed time: 2021-04-07 20:02:48
Finished building 'hisat2_index' asset
Created alias directories:
- /project/shefflab/deploy/rg.databio.org_full/genomes/alias/hg18/hisat2_index/default