Tab delimited file with 3d column consisting of comma-separated integers.
Goal: using bash script
pick all records matching pattern=PATTERN
count all occurrences of each integer in the file
sort desc by occurrences
Highlights:
>tr -s ',' '\n' ==or== sed 's/,/\n/g'
>sort -r -n -b ----- reverse, numeric, omit trailing blanks
>uniq ------- for counting pre-sorted records
> | -------- piping output from one process into another (all interim files are not necessary, everything could be piped through!
#!/bin/bash
grep PATTERN | sed 's/\t*PATTERN\t*,/\t/g' > matching_records.csv
cat matching_records.csv | awk -F '\t' '{print $2}' > csv_column.csv
cat csv_column.csv | tr -s ',' '\n' | sort -n | uniq -c | sort -r -n -b
Saturday, April 6, 2013
Linux / unix tips: paste
>paste
[-s] [-d delim-list] [--serial] [--delimiters=delim-list] [--help] [--version]
[file...]
http://linux.about.com/library/cmd/blcmdl_paste.htm
paste prints lines consisting of sequentially corresponding lines of each specified file. In the output the original lines are separated by TABs. The output line is terminated with a newline.
Example - convert tab-delimited file into csv:
http://askubuntu.com/questions/174008/write-bash-script-which-takes-input-from-pipe
http://linux.about.com/library/cmd/blcmdl_paste.htm
paste prints lines consisting of sequentially corresponding lines of each specified file. In the output the original lines are separated by TABs. The output line is terminated with a newline.
Example - convert tab-delimited file into csv:
http://askubuntu.com/questions/174008/write-bash-script-which-takes-input-from-pipe
cat some_list_of_elements.txt | awk '{print "\047"$1"\047"}' | paste -d, -sThursday, April 4, 2013
Linux tips
Copy shell script from inside
cp $0 destination
Remove file with name starting with dash
rm -- -file-with-weird-prefix
cp $0 destination
Remove file with name starting with dash
rm -- -file-with-weird-prefix
Sunday, March 10, 2013
Lamb soup and more
This soup is a compilation of many traditional Uzbekistan soups.
Of course, you need lamb and some special sour plums. Cherry plums? When was the last time you've seen these plums? They are out of season in produce section of my supermarket for the past 20+ years.
Anyway, for best results do the following.
Buy a lamb rack - a smaller one is better, it will have less fat - Costco is your best bet price-wise.
You'll also need salt, water, 3 onions, 5 potatoes, 1 of cup rice. Garlic, black pepper and cilantro are optional.
Put lamb rack whole into wide pot, cover with enough water to make soup and bring to boil. Add salt and simmer at low. Cube 3 medium onions (I don't know exactly how to cube onions, but this is one of the lines from one of the recipes, so I followed this line literally). Add to soup. Add 3 medium-size tomatoes whole. Peel 5 small-to-medium size potatoes of perfect shape and similar size. Add them to soup, bring to boil. In 20 minutes take out the rack and cut along the ribs - you'll get 7-8 bones with meat on. Return them to pot. Take out tomatoes, squish in colander above the pot, remove skin and whatever else you don't like from colander and return the rest of tomatoes to the pot. Add 2-3 chopped garlic cloves and simmer for another few minutes.
To serve on Day One
Each portion will have 1 rib, 1 potato and broth.
Top with chopped cilantro or something else green if you can get this time of the year.
To serve on Day Two
Put generous amount of boiled rice into the soup plate. Add broth remaining from day 1 to the plate. Heat in micro.
If you are especially lucky, you'll get a bone too.
Tastes great.
It's a different dish with a different name tough, I am not even sure it's called soup.
Please note that you can feed a family of 5 for 2 days with 1 lamb rack.
Of course, you need lamb and some special sour plums. Cherry plums? When was the last time you've seen these plums? They are out of season in produce section of my supermarket for the past 20+ years.
Anyway, for best results do the following.
Buy a lamb rack - a smaller one is better, it will have less fat - Costco is your best bet price-wise.
You'll also need salt, water, 3 onions, 5 potatoes, 1 of cup rice. Garlic, black pepper and cilantro are optional.
Put lamb rack whole into wide pot, cover with enough water to make soup and bring to boil. Add salt and simmer at low. Cube 3 medium onions (I don't know exactly how to cube onions, but this is one of the lines from one of the recipes, so I followed this line literally). Add to soup. Add 3 medium-size tomatoes whole. Peel 5 small-to-medium size potatoes of perfect shape and similar size. Add them to soup, bring to boil. In 20 minutes take out the rack and cut along the ribs - you'll get 7-8 bones with meat on. Return them to pot. Take out tomatoes, squish in colander above the pot, remove skin and whatever else you don't like from colander and return the rest of tomatoes to the pot. Add 2-3 chopped garlic cloves and simmer for another few minutes.
To serve on Day One
Each portion will have 1 rib, 1 potato and broth.
Top with chopped cilantro or something else green if you can get this time of the year.
To serve on Day Two
Put generous amount of boiled rice into the soup plate. Add broth remaining from day 1 to the plate. Heat in micro.
If you are especially lucky, you'll get a bone too.
Tastes great.
It's a different dish with a different name tough, I am not even sure it's called soup.
Please note that you can feed a family of 5 for 2 days with 1 lamb rack.
Saturday, March 9, 2013
Resolving to port 80 instead of 8080 (AWS, Linux, Tomcat)
Launch AWS EC2 instance with Linux, securely connect to it
>yum -y install tomcat7
....
Compile and deploy your webapp / mobile app
Let's make it simple - for example, deploy helloworld.jsp
....
>service tomcat7 status
tomcat7 (pid <...>) is running...
....
Go to web browser / g.g. Google Chrome
http://<IP address or domain name>/helloworld.jsp
For example
http://hastasiempre-ismyfavourite.com
Oops! Google Chrome could not connect to <...>
My humble contribution in the form of sed script to switch from port 8080 to 80:
#!/bin/bash
tomcat=tomcat7
tomcat_conf=/usr/share/$tomcat/conf
cd $tomcat_conf
sed -e 's/="8080/="80/g' -e 's/="8443/="443/g' -i $tomcat_conf/server.xml
sed '/Connector port="80"/i\<!-- change user tomcat to root in tomcat7.conf, change ports 8080 and 8443 to 80 and 443 respectively -->/' -i $tomcat_conf/server.xml
sed -e '/TOMCAT_USER=/i\# Switch user from tomcat to root' -e 's/TOMCAT_USER="tomcat"/TOMCAT_USER="root"/' -e 's/="8080/="80/' -i $tomcat_conf/$tomcat.conf
>yum -y install tomcat7
....
Compile and deploy your webapp / mobile app
Let's make it simple - for example, deploy helloworld.jsp
....
>service tomcat7 status
tomcat7 (pid <...>) is running...
....
Go to web browser / g.g. Google Chrome
http://<IP address or domain name>/helloworld.jsp
For example
http://hastasiempre-ismyfavourite.com
Oops! Google Chrome could not connect to <...>
Darn! I mean, WTF?
Browse some forums. Figure that default port for Tomcat7 to listen on is 8080.
http://<IP address or domain name>:8080/helloworld.jsp
Hell of a world!
Go to .../tomcat7/conf. Edit server.xml with your editor of choice to change ports 8080 and 8443 to 80 and 443 respectively.
http://<IP address or domain name>/helloworld.jsp
Oops! Google Chrome could not connect to <...>
Browse even more forums. Figure out that you need to run tomcat as root to be allowed to use port under 1024. Edit tomcat7.conf with your editor of choice to change TOMCAT_USER="tomcat" to TOMCAT_USER="root".
http://<IP address or domain name>/helloworld.jsp
Hell of a world!
Bingo!
My humble contribution in the form of sed script to switch from port 8080 to 80:
#!/bin/bash
tomcat=tomcat7
tomcat_conf=/usr/share/$tomcat/conf
cd $tomcat_conf
sed -e 's/="8080/="80/g' -e 's/="8443/="443/g' -i $tomcat_conf/server.xml
sed '/Connector port="80"/i\<!-- change user tomcat to root in tomcat7.conf, change ports 8080 and 8443 to 80 and 443 respectively -->/' -i $tomcat_conf/server.xml
sed -e '/TOMCAT_USER=/i\# Switch user from tomcat to root' -e 's/TOMCAT_USER="tomcat"/TOMCAT_USER="root"/' -e 's/="8080/="80/' -i $tomcat_conf/$tomcat.conf
Subscribe to:
Posts (Atom)