Sending one datastream to a unix command is made by piping (e.g. zcat seq1.fq.gz | seqs). These examples demonstrate how to pipe multiple data streams to one *nix command.

Credit and additional examples:

• Thunks 'n things
• Linuxjournal
• http://en.wikipedia.org/wiki/Process_substitution

Process substitution

This works in bash and zsh, but not csh shells.

 seqs -file1 <(zcat seq1.fa.gz) -file2 <(zcat seq2.fa.gz)

Limitation: program using seek() fail, due to piping. In zsh, this can be overcome via creation of temporary files, using =(...) instead of <(...), but this will incur an I/O penalty for writing to disk

 seqs -file1 =(zcat seq1.fa.gz) -file2 =(zcat seq2.fa.gz)

Similarly, we can provide an arbitrary number of items, here cast to a variable used in a one-liner bash script for loop rather than a file pipe

 for file in $(ls *.fasta); do qsub submitBlastJob.sh $file; done

Use mkfifo to create named pipes

Under the hood, process substitution of files makes temporary named pipes. We can also do this explicitly, if we would like the pipes to be persistent.

 mkfifo pipe1 pipe2
 zcat seq1.fq.gz > pipe1 &
 zcat seq2.fq.gz > pipe2 &
 seqs -file1 pipe1 -file2 pipe2
 rm pipe1 pipe2 

Multiple pipes on program output using tee

Example downloads a file, and computes the sha1 hash and md5 hash at the same time

wget -O - http://example.com/dvd.iso \
  | tee >(sha1sum > dvd.sha1) \
        >(md5sum > dvd.md5) \
  > dvd.iso