I came across a simple one line command allowing take files in a folder one by one and pipe them into another command, but I cannot find this thread. Can someone help? Say, decompress hundreds of gziped files in a folder after downloading a database, or pipe to another process.
Seqanswers Leaderboard Ad
Collapse
Announcement
Collapse
No announcement yet.
X
-
Originally posted by yaximik View PostI came across a simple one line command allowing take files in a folder one by one and pipe them into another command, but I cannot find this thread. Can someone help? Say, decompress hundreds of gziped files in a folder after downloading a database, or pipe to another process.
To uncompress all the gzip files in the current directory this may suffice:
Code:gunzip *.gz ## Use gunzip -r to descent in subdirectories
Code:for gz in `find mydir/ -name '*.gz'` do gunzip -c $gz | head -n 15 > ${gz}.head done
-
This should do it: http://www.cyberciti.biz/faq/bash-loop-over-file/
There will be analogous commands for tcsh, if that is your favorite.
Comment
-
Originally posted by dariober View PostTo uncompress all the gzip files in the current directory this may suffice:
Code:gunzip *.gz ## Use gunzip -r to descent in subdirectories
Code:for gz in `find mydir/ -name '*.gz'` do gunzip -c $gz | head -n 15 > ${gz}.head done
And to get the results in the same file, you can do
Code:for gz in *.gz ; do less $gz | head -15 ; done > outfile
Comment
-
Thanks everyone. I realized I did not express the question clearly. Gzip is just one of applications, i rather asked about general looping over files:
Code:while/for/if [I]something is in the folder[/I] do [I]something to each file[/I] until [I]all are processed[/I] done
Comment
-
In fact, you can also do for loop one-liners with ";" or even with & to fork them to background
For example looping over Fasta files (*.fa) and counting the headers (i.e. number of sequences) could be done on one line like so:
Code:for f in *.fa; do grep -c ">"; done
Code:for f in *.gz; do gunzip ${f} & done; wait
I learned the hard way: While those one-liners are nice and quick for scripting on a terminal, never use them in longer bash-scripts because a) you will at some point have a typo and then will have a very hard time finding it and b) there is essentially no need in saving space/shortening out commands in a bash script.
Best
Simon
Comment
-
Thanks a lot, everyone who provided inputs. That was very educational in general as I get a better idea how I can use simple loops. I wonder, would it be useful to have something like WiKiBits (or a better name) as a collection of little ingenious solutions?
Comment
Latest Articles
Collapse
-
by seqadmin
The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...-
Channel: Articles
04-22-2024, 07:01 AM -
-
by seqadmin
Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...-
Channel: Articles
04-04-2024, 04:25 PM -
ad_right_rmr
Collapse
News
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by seqadmin, Yesterday, 11:49 AM
|
0 responses
15 views
0 likes
|
Last Post
by seqadmin
Yesterday, 11:49 AM
|
||
Started by seqadmin, 04-24-2024, 08:47 AM
|
0 responses
16 views
0 likes
|
Last Post
by seqadmin
04-24-2024, 08:47 AM
|
||
Started by seqadmin, 04-11-2024, 12:08 PM
|
0 responses
62 views
0 likes
|
Last Post
by seqadmin
04-11-2024, 12:08 PM
|
||
Started by seqadmin, 04-10-2024, 10:19 PM
|
0 responses
60 views
0 likes
|
Last Post
by seqadmin
04-10-2024, 10:19 PM
|
Comment