Extract Last Field

You can use 'cut' to extract the last field of a line if you know how many fields there are, eg:
field=`cut -d: -f8 file`

But if you don't know the maximum number of fields or the number of fields per line are not consistent, awk can come to the rescue. awk has the inbuilt variable NF for the number of fields. By using this variable we can use it to extract the last field by using:

field=`awk -F: '{print $NF}'`

or you can use calculations to retrieve any field relative to the last field. For example to retrieve the second last field, use:

field=`awk -F: '{print $(NF-1)}'`

Making current directory available on HTTP

python -m SimpleHTTPServer

Serve current directory tree at http://$HOSTNAME:8000/

COMMANDLINEFU.COM

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Grep Tips(color=always)

Do the following while using grep in future.

cat httpd.conf | grep --color=always httpd

Apache Adding Login Site Protection

http://www.yolinux.com/TUTORIALS/LinuxTutorialApacheAddingLoginSiteProtection.html

Download a website

Here is a simple effective way to get the files downloaded recursively from a website without actually visiting each and every link to the sub pages.

wget -r -p -k -E http://www.linuxdriver.co.il/ldd3/

...where

• -r is for recursive download of pages
• -p is for linking pages locally so that user can browse them easily once the download is completed.
• -k is to create the directory structure, and
• -E is to create .html extensions to the type XHTML or text files.
  

Kill all process instances at one go

Sometimes you might need to urgently kill all the java processes. Use the following command to do so:

ps -ef | grep java | xargs kill -9 awk '{print $2}' > /dev/null 2>&1

Find Quickly

find directory-path -name "*" -mtime -file-age -print

Example. Suppose you want to find the files which are created in last 5 days in /tmp directory your command will be.

find /tmp/ -name "*" -mtime -5 -print

organizational-battle-ubuntu-vs-gentoo-vs-rhel-fedora

http://oz.blogial.com/2008/07/29/organizational-battle-ubuntu-vs-gentoo-vs-rhel-fedora-3/

sed tips

Great one liner sed tips and tricks.

http://sed.sourceforge.net/sed1line.txt

Now its easy to deploy

You lead a busy life. You don’t have the time to waste logging in and logging out of your servers all day, running tasks, installing software, keeping machines in sync, and trying to make sure it all happens correctly.

Let Capistrano do the heavy lifting for you. It is designed with repeatability in mind, letting you easily and reliably automate tasks that used to require login after login and a small army of custom shell scripts.

And if you still find it difficult then you have the Webistrano interface for the same.

copy large number of small files over the network - the fastest method

find /source/path -print |cpio -o -Hnewc |ssh -C 192.168.32.31 "cd /path/to/destgnation/; cpio -idvum" &>/dev/null

Getting login password from Apache authentication using PHP

$headers = apache_request_headers();

$auth= $headers['Authorization'];

$autha = explode(' ',$auth);

$pair = base64_decode($autha[1]);

$authb = explode(':',$pair);

echo $login = $authb[0];

echo '\n';

echo $pass = $authb[1];

include this in php tags :)

Apache-PHP User Authentication

http://www.weberdev.com/ViewArticle/User-Authentication-With-Apache-and-PHP

OFC standards

You might have some confusion about the OFC cable standards, like LC, SC etc. The following link will help you to understand all these standards

http://www.thefoa.org/tech/connID.htm
http://en.wikipedia.org/wiki/Optical_fiber_connector

Redirect Using Iptables

To redirect port 80 to 8080 use:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080

HowTo get a small sample dataset from a mysql database using mysqldump

Here is a quick tip that will show how you can get a small sample dataset from a mysql database using mysqldump. We frequently need to get a small snapshot from a very big production database to import it into a development or staging database that will not need all the original data; let’s say we need 1,000,000 records from all the tables in the database; we will just use the option –where=”true LIMIT X”, with X the number of records we want mysqldump to stop after.

Simply we will run something like (add whatever other options you need to mysqldump):

 mysqldump --opt --where="true LIMIT 1000000" mydb > mydb1M.sql

and this will get 1M records from each of the tables in the database. If you want this for a single table you would use something like this:

 mysqldump --opt --where="true LIMIT 1000000" mydb mytable > mydb_mytable_1M.sql

To restore this, you would use the same as on a regular dump:

 mysql -p mydb_stage < mydb1M.sql

This will give you a small number of records that you can use for development, testing, etc. whatever you would need.

To find all the files of a particular size.

Let’s assume you are searching for all the files of exactly 6579 bytes size inside the home directory. You will just have to run something like:

 find /home/ -type f -size 6579c -exec ls {} \;

As units you can use:

   * b - for 512-byte blocks (this is the default if no suffix is used)
  * c - for bytes
  * w - for two-byte words
  * k - for Kilobytes (units of 1024 bytes)
  * M - for Megabytes (units of 1048576 bytes)
  * G - for Gigabytes (units of 1073741824 bytes)

You can search for exact file size, or just for bigger (+) or smaller (-) files. For example all bigger than 512k files would be found with something like:

 find /home/ -type f -size +512k -exec ls -lh {} \;

I have added here -lh to the ls output so it will actually show the files with their sizes in a nice human readable format. Similar for smaller files you would use -size -512k.

Change password in single command

echo redhat | passwd --stdin test

This will set the password redhat for user test.

Type Casting column type in Postgres

Consider you have a filled up table in postgres, and one of the column is defined as 'character varying'. Now, if you want to change the column type to 'integer' then the normal alter command will not work.

If you will do this :

alter table tablename alter columnname type integer

you will get this error:

ERROR: column "column_name" cannot be cast to type "pg_catalog.int4"

So, you need to do this:

alter TABLE tablename alter column columnname type integer using columnname::integer;

Mysql Password Reset

Edit my.cnf and add the line "skip-grant-tables" in "[mysqld] section and restart the mysql. You can login to mysql without password.

To change the password execute following command @ mysql prompt.

mysql> update mysql.user set Password=PASSWORD('<>') where user='root';

Vi Tricks

http://www.kingcomputerservices.com/

http://www.eng.hawaii.edu/Tutor/vi.html

http://thomer.com/vi/vi.html

This time, I'll show you how to make a Web server running Apache and Linux survive heavy loads.

http://rudd-o.com/en/linux-and-free-software/tuning-an-apache-server-in-5-minutes

Tuning Primer

Tuning the performance of MySQL can be a really hard job to do.There are many thinks to consider and no two servers are identical so there is no universal solution. Tuning Primer is a script that will help you tune your mysql installation by providing very healthy recommendations based on past mysql records.

http://day32.com/MySQL/tuning-primer.sh

MySQLTuner

MySQLTuner is a script written in Perl that will assist you with your MySQL configuration and make recommendations for increased performance and stability. Within seconds, it will display statistics about your MySQL installation and the areas where it can be improved.

http://tools.assembla.com/svn/mysqltuner/

Tune Mysql

MySQL Performance site, published tips about tuning your MySQL server Performance.

From the article:
My favorite question during Interview for people to work as MySQL DBAs or be involved with MySQL Performance in some way is to ask them what should be tuned in MySQL Server straight after installation, assuming it was installed with default settings.

I'm surprised how many people fail to provide any reasonable answer to this question, and how many servers are where in wild which are running with default settings.

http://www.linuxweblog.com/tune-my.cnf

http://www.mysqlperformanceblog.com/2006/09/29/what-to-tune-in-mysql-server-after-installation/

Optimizing the database regularly will also help you run the server fast! Put the following line in cron.

mysqlcheck -o --all-databases

AWK Magic

Suppose you have the two files with following content

ip.txt

bmdes001;172.16.33.100
bmdes002;172.16.33.101
bmdes003;172.16.33.102
bmdes004;172.16.33.103
bmdes005;172.16.33.104

dhcp.txt

00:1A:4D:A6:B5:4C;bmdes001;
00:1D:09:25:7C:B2;bmdes002;
00:1D:09:19:82:EF;bmdes003;
00:1D:09:26:45:46;bmdes004;
00:1D:09:19:62:79;bmdes005;

And you want to combine the two files where 1st column of ip.txt file matches with 2nd column of dhcp.txt then following is the command for you.

awk 'BEGIN{FS=";"}FILENAME ~ /ip.txt/{a[$1]=$2}FILENAME ~ /dhcp.txt/{print $1,$2,a[$2]}' ip.txt dhcp.txt

00:1A:4D:A6:B5:4C bmdes001 172.16.33.100
00:1D:09:25:7C:B2 bmdes002 172.16.33.101
00:1D:09:19:82:EF bmdes003 172.16.33.102
00:1D:09:26:45:46 bmdes004 172.16.33.103
00:1D:09:19:62:79 bmdes005 172.16.33.104

Find and Replace several files

If you need to change the same thing in hundreds of files then following script is for you. It will create the backup of every file with .bak extension.

perl -pi.bak -e 's/your-find-string/your-replace-string/g' filename

Example:

perl -pi.bak -e 's/abc/xyz/g' *.php 

This will replace abc with xyz in all the files in current directory which are with .php extension.

Basic Interview Questions

1. How many arguments shell script can take
   Ans: execute "getconf ARG_MAX" the output will tell you how many argument shellscript can take.
   
   
2. How do you capture variable in shell script
   
   Ans: Using $1,$2 etc.
   
   
   
3. What is difference between tomcat and jboss
   
   Ans: Tomcat is a pure servlet engine, with no EJB. JBoss supports EJB and JMS related stuff. JBoss is a full J2EE application server.. it supports the bulk of the J2EE spec. It is more common to refer to J2EE as a more complete reference of what an application server can and should include, of which Tomcat only implements a part, while the complete download of JBoss (including Tomcat) conforms to the full J2EE specification, and hence is amore "complete" application server.
   
   
   
4. What is difference between sh file-name and ./file-name
   
   Ans: ./ can be executed only if the file has execute permission
   
   
   
5. What is the command to check the free ports
   
   Ans: use “nmap”
   
   
   
6. How do you reset password in mysql
   
   Ans: Start mysql with “mysql-safe –skip-grant-tables”
   
   
   
7. What is the Default flag for "Kill"
   
   Ans: Default flag for kill is "TERM/-15" 
   
   
   
8. I create a user, the user logs in but can't create files in his dir. Why? The dir permissions are ok.
   
   Ans: check quota, check if disk is full
   
   
   
9. df and du show different values, why ?
   
   Ans: http://sysunconfig.net/aixtips/df_du_diff_out.txt
   
   
   
10. How to implement round-robin load-balancing in bind (dns) ?
    
    Ans: Add two A records for the same hosts with different IP and set TTL as zero for them.
    
    
    
11. Write a shell script to generate a HTML.
    
    
    
12. What is the difference between hard link and soft links ? Atleast 2 differences.
    
    Ans:
    
    1. About inode: In a hard link, inode nos will be same for both files, but will have different names. Soft link is a file which links to the original file (equivalent to shortcut in Windoze)
    
    
    
    2. About filesystem location: Hard links cannot reside on different partitions/filesystems (why? bcos same inode), soft links can point to files in any location.
    
    
    
13. What are the types of files in Linux ? How do u know a file type ?
    
    Ans:
    
    d = directory
    
    l = symbolic link
    
    s = socket
    
    p = named pipe
    
    - = regular file
    
    c= character (unbuffered) device file special
    
    b=block (buffered) device file special
    
    
    
14. How to Replicate different databases to different slaves?
    
    Ans: either one of the following (Ref: http://www.databasejournal.com/features/mysql/article.php/10897_3355201_2)
    
    1. Do the following settings in master (my.cnf)
    
        binlog-ignore-db=test
    
        binlog-ignore-db=scratch
    
        binlog-do-db=catalog
    
        binlog-do-db=users
    
        binlog-do-db=sessions
    
    2. Do the following settings in slave (my.cnf)
    
        replicate-do-db=catalog
    
        replicate-ignore-db=test
    
        replicate-ignore-db=scratch
    
        replicate-do-db=test
    
        replicate-do-db=sessions
    
    
    
15. What is SOA in DNS
    
    Ans: Statement Of Authority
    
    
    
16. What is Exit code for Ctrl + C
    
    Ans: 130
    
    
    
17. What to do if i am getting error "passwd: Authentication token manipulation error" while updating the root password
    
    Ans: chattr -i /etc/shadow
    
    
    
18. What Could happen if /sys directory is not there?
    
    Ans: Your system might say "Couldnt open //dev/tty3" n/w wont be start. And system also gives error like "comp init: Id "c2" respawning too fast: disabled for 5 minutes"
    
    
    
19. What to do if dh -h gives "df: cannot read table of mounted file systems: Is a directory"
    
    Ans: /etc/mtab is the file. check if there if something wrong with it. This error mostly come when /etc/mtab is the directory not the file. Delete the /etc/mtab and do "mount -a" this will resolve your issue.
    
    
    
20. What is the difference between "/etc/bashrc" and "/etc/profile"
    
    Ans: The global /etc/profile and your own local  ~/.bash_profile are read when the shell is invoked as an interactive login shell, for example when you open a remote terminal session to someone else's machine, or when you log into your own machine without X Windows running, or when you hit ctrl-alt-F1 from X Windows to start a virtual terminal session.
    
    
    
    The global /etc/bashrc (if it exists) and your own local  ~/.bashrc is read when the shell is invoked as an interactive non-login shell, for example when you open up a terminal window in Gnome or KDE.
    
    
    
21. How do I print the line in between 25 to 50 from abc.txt which is of 100 lines in it.
    
    Ans: tail -n +25 abc.txt | head -n 25