Dependence of BAM to FastQ run-time on number of concurrently running instances on compute farm nodes. For running Picard we have adjusted the amount of memory available by setting the -Xmx switch of the virtual machine and chosen the serial garbage collector switch -XX: Competing interests The authors declare that they have no competing interests. We use this machine type as an example for a large scale compute farm type system. This was sufficient to not let run Picard out of memory on any data set.


Uploader: Kagatilar
Date Added: 1 June 2018
File Size: 27.85 Mb
Operating Systems: Windows NT/2000/XP/2003/2003/7/8/10 MacOS 10/X
Downloads: 58800
Price: Free* [*Free Regsitration Required]

biobambam: tools for read pair collation based algorithms on BAM files

In addition the current code also considers single ended reads and orphans as duplicates if they map to the same coordinate as one end of a mapped pair. Author information Article notes Copyright and License information Disclaimer. The memory usage of bamtofastq can be lowered to 33 MB using non standard settings. These tests were generally run with multiple identical instances in parallel, which reduces the total amount of wall clock time required to obtain the at least ten run time values we measured for each setting to compute a mean run time and standard deviation.

Implementation The code presented in this paper is split into two parts. We have run ten jobs of the same kind parallel.


None of bamtofastq and SamToFastQ gained significantly by using more main memory. We have reduced it to 8 instead of This involved a reduction of the number of concurrently running instances from 10 to 3, where we increased the amount of available memory per process to 64 from 16 GB as mentioned above Picard duplicate marking fails when given too much memory.



In addition to our front end programs we also make the collation algorithm directly accessible as an API for use in other projects in the libmaus project. In these plots we observe sharp increases of the bampe2fqworphans program run-time when going from 24 to 32 concurrently running instances.

Both lists are sorted by coordinates, where the sorting of the list L P is lexicographic in the coordinates of the two ends i. Picard suffers much more heavily from increased concurrency.

biobambam: tools for read pair collation based algorithms on BAM files

The contained tools bamtofastq and bammark duplicates2 are more efficient in terms of run-time and memory requirements then equivalent tools in the widely used Picard suite see [ 12 ]. The pointers to the alignment starting positions in the byte array are filled in from the back of the array. Pseudo code for pair and fragment list generation in duplicate marking algorithm.

Apparently hyper threading offers less of the required parallelism in this case as for the BAM to FastQ conversion case. The alignment data is filled in at the front end of the array. If the size of L has reached a given threshold, then we sort Lmove newly discovered pairs to the output list Owrite the remaining unmatched alignments to a new temporary file and erase L. Published online Viobambam Comparisons were performed with the current biobambsm of the programs when blobambam benchmarking for this paper started.


biovambam Alignments are inserted from the start of the array. Performance comparison for bammarkduplicates2 on server blades We have used the same data sets for comparing bammarkduplicates2 on server blades as we have used for the comparisons concerning bamtofastq.


The actual median we observe in the file is Characteristics of test files used for compute farm test. In the second case a straight-forward but often inefficient way is to first sort the input BAM file by query name using tools like SAMtools or Picard and then resort to a conversion as employed in the first case, as a BAM file sorted by query name will have biibambam alignments collated by biboambam name.

In the following we will compare the performance of our new tools bamtofastq and bammarkduplicates2 to available programs offering the same functionality. Choosing s L in O n both of these become O n. Before starting the timing tests on the PC type system we have checked the capability of the programs to handle our test data sets biboambam a machine equipped with GB of memory.

We used the corrected version of the bampe2fqworphans program as posted by Richard Finney on the SEQanswers forum on the 24th of September Run-time comparison of bammarkduplicates2 and alternatives on compute farm nodes part a. However the difference between the two programs is small for the two programs at a maximum of 5. On the compute farm nodes the systematic variance in run-time seen on the Xeon E system does not occur as only full CPU cores are used.

Most of this memory usage consists of program code. If P [ h ] is used, then the currently stored alignment for h needs to be moved biobamban L and erased from H.