Creating extract custom function to extract files

As there are different file formats present for archiving and compressing files using different utilities. So, to make a uniform function for extracting those files can be helpful. Add the below piece of code in your .bash_profile or .profile file present in home directory to extract the data.

extract () {

        if [ -f $1 ] ; then

           case $1 in

                *.tar.bz2) tar xvjf $1 ;;

                *.tar.gz) tar xvzf $1 ;;

                *.bz2) bunzip2 $1 ;;

                *.rar) rar x $1 ;;

                *.gz) gunzip $1 ;;

                *.tar) tar xvf $1 ;;

                *.tbz2) tar xvjf $1 ;;

                *.tgz) tar xvzf $1 ;;

                *.zip) unzip $1 ;;

                *.Z) uncompress $1 ;;

                *.7z) 7z x $1 ;;

                *) echo "don't know how to extract '$1′…" ;;

            esac

        else

                echo "'$1′ is not a valid file!"

        fi

}

 

Compare two files for duplicate entries using bash and php

Hi Friends,

Need to compare two files and searching for different entries in them. Here is the solution.

Create a new file named cmp.sh and put it in a folder.

[code]
#!/bin/bash
uniq temp1 > sort_temp1
sort sort_temp1 > sort1.txt
uniq temp2 > sort_temp2
sort sort_temp2 >> sort1.txt
sort sort1.txt > sort_dup.txt
cat sort_dup.txt | uniq > sort.txt
echo "Elements present in file temp1 not in file temp2"
php index.php 1
echo "Elements present in file temp1 not in file temp2"
php index.php 2
[code]
Save it.

Now, we need to create a php file named index.php in same folder.
[php]
<?php
$handle = @fopen("sort.txt", "r");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);

//======================================
if($argv[1] == 1)
{
$handle1 = @fopen("sort_temp1", "r");
if ($handle1) {
$test = 1;
while (!feof($handle1)) {
$buffer1 = fgets($handle1, 4096);
if ($buffer==$buffer1)
$test = 0;
}
if ($test == 1)
echo $buffer;
}
fclose($handle1);
}

if($argv[1] == 2)
{
$handle2 = @fopen("sort_temp2", "r");
if ($handle2) {
$test = 1;
while (!feof($handle2)) {
$buffer2 = fgets($handle2, 4096);
if ($buffer==$buffer2)
$test = 0;
}
if ($test == 1)
echo $buffer;
}
fclose($handle2);
}
//========================================

    }
    fclose($handle);
}
?>
[/php]
Save the file in same folder. Add your both files need to compare, with the name temp1 and temp2.

Run the cmd.sh script and it will give you the desired result.

Hope it helps.

Reduce image size

Sometimes, you need to fill an application form, but the size of your image is

too large, as per the mentioned size on the application. So, you can use imagemagick

utility of ubuntu, and get the desired size for your image.

To reduce the image size, you can run either of the below command.

$ convert  -resize 50% source.png dest.jpg and 

$ convert  -resize 1024X768  source.png dest.jpg

RAINBOW ROSE in shell script

To gift your friend, you can send him/her a rainbow rose by using below shell script

#!/bin/bash
RED='\033[1;31m'
GREEN='\033[1;32m'
CYAN='\033[0;36m'
YELLOW='\033[1;33m'

printf "${RED}    .--.---.\n"
sleep 1;
printf "   ( ${CYAN}\'--'/${RED} )\n"
sleep 1;
printf "  ( ${CYAN}'..-...'${RED} )\n"
sleep 1;
printf "   '.${CYAN}'-._.'${RED}.'\n"
sleep 1;
printf "    <'${CYAN}-.,.${RED}->\n"
sleep 1;
printf "${GREEN}       \`\\(  ${YELLOW}  _\n"
sleep 1;
printf "${GREEN}        < \ ${YELLOW} / \ \n"
sleep 1;
printf "       ${YELLOW}__${GREEN} \\\\\\ $YELLOW|_/\n"
sleep 1;
printf "      /  \ ${GREEN}\\\\\\ \n"
sleep 1;
printf "      ${YELLOW}'--'.${GREEN}\\\\\\ ${YELLOW}>\n"
sleep 1;
printf "${GREEN}            \\\\\\   \n"
printf "\n"
printf "${CYAN}RAI${RED}NBOW${GREEN} ${RED}ROSE${RED} FOR ${CYAN}YOU${CYAN} ${GREEN}MY${YELLOW} FRIEND\n"

And it will look like this:

Understanding Exec

At times, you have noticed that there is a usage of exec in shell scripts. What it actually does, and where to use it exactly?

The command works on the theory of, is "never coming back"

exec command is shell builtin, and other members of the commands are execve, execvp, execv.

1. exec command starts the new process, without forking, so once it starts the new process, the old process get finished.

Case 1:
bash:~$ ps -ef | grep -i bash
csblog  13062 12371  1 15:19 pts/5    00:00:00 bash
ksh$ ksh
ksh$ ps -ef | grep -i ksh
csblog  13093 13062  0 15:19 pts/5    00:00:00 ksh
ksh$ ps -ef | grep -i bash
csblog  13062 12371  0 15:19 pts/5    00:00:00 bash

Case 2:
bash:~$ exec ksh
ksh$ ps -ef | grep -i bash
ksh$ ps -ef | grep -i ksh
csblog  13062 12371  0 15:19 pts/5    00:00:00 ksh

You can see, in case 1, i have started the process, without using exec, so the new shell, is subshell for the old one. But in case of case2, a totally new process is started. In this manner, your resources are saved.

2. Sometimes, you don't need user to access the shell, you can directly change in /etc/passwd, but you need to load the environment, so you can add the exec command, at the end of .profile.

3. exec, also used with file descriptor. To open, close, read and write.

exec 3< thisfile          # open "thisfile" for reading on file descriptor 3
exec 4> thatfile          # open "thatfile" for writing on file descriptor 4
exec 8<> tother           # open "tother" for reading and writing on fd 8
exec 6>> other            # open "other" for appending on file descriptor 6
exec 5<&0                 # copy read file descriptor 0 onto file descriptor 5
exec 7>&4                 # copy write file descriptor 4 onto 7
exec 3<&-                 # close the read file descriptor 3
exec 6>&-                 # close the write file descriptor 6

read <&3
echo stuff >&4


Source: Stackoverflow.com

How to log your mysql queries in Ubuntu 16.04

At times, you need to log your mysql queries.

The first you need to locate your configuration file of mysql. You can use either locate command or the find command for it. The name of the configuration file for mysql is "my.cnf", and in ubuntu its usually present in your /etc directory.

$ find /etc -name "my.cnf"

or

$ locate my.cnf

In my system, it's located in /etc/mysql/my.cnf

Once the file is located, open it using your favorite text editor.
There are chances that the file is not writable by your user, so you need to use sudo to switch your user to root to edit the file.

Open the file and add the lines,

[mysqld]
general_log = on
general_log_file = /var/log/mysql/mysql_query.log

It will add the group as mysqld, and start the logging. The logs will be save in file:

/var/log/mysql/mysql_query.log

Save it and close it.

Then you need to restart your mysql server, to enable the option. Once the mysql service is started, the logging will start.

$ sudo service mysqld restart

Sample log file:

$ cat /var/log/mysql/mysql_query.log
/usr/sbin/mysqld, Version: 5.7.15-0ubuntu0.16.04.1-log ((Ubuntu)). started with:
Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
Time                                                  Id Command    Argument
2016-10-13T14:02:57.300324Z    3 Connect myuser@localhost on  using Socket
2016-10-13T14:02:57.300563Z    3 Query select @@version_comment limit 1
2016-10-13T14:03:02.895987Z    3 Query SELECT DATABASE()
2016-10-13T14:03:02.896302Z    3 Init DB test
2016-10-13T14:03:02.898415Z    3 Query show databases
2016-10-13T14:03:02.902113Z    3 Query show tables

Difference bewteen pipe, xargs, and exec

 xargs, pipes and exec:

First off, exec is nothing like the first two. exec does a variety of things from executing a command without returning(the shell is replaced by the new command), as well as opening/closing files.

pipes transfer the output of one program into the input of another. Without the pipe or redirection, they'd be trying to read from your keyboard and trying to print to your screen, but you can feed them data from any source and send their output anywhere you like. Shell utilities are intended to be flexible that way.

Code:

# useless example
echo asdf | cat

'echo' prints to standard output, 'cat' reads from standard input, the pipe joins them together so 'asdf' is read by cat and printed.

But what about this:

Code:

# This won't print anything
echo something | echo

echo does not read from standard input, it takes commandline parameters and nothing but parameters. So how do we translate from pipe into that?

This is what xargs is for.

Code:

echo a b c d | xargs command

is equivalent to command a b c d. Whenever you need to translate from a pipe or file into commandline arguments, xargs serves.

Code:

# This prints 'something' into xargs' input, causing xargs to run 'echo something'.
echo something | xargs echo 

Let's consider an example of exec:

$ find . -name "*.txt" -exec ls -l {} \;

The above command will create multiple process and generate the result. Suppose there are 3 text files named file1.txt, file2.txt, & file3.txt. The above command will start below 3 commands:

ls -l file1.txt
ls -l file2.txt
ls -l file3.txt

To restrict to 1 thread or 1 process, you can use xargs or +symbol. For example:

$ find . -name "*.txt" | xargs ls -l
or
$ find . -name "*.txt" -exec ls -l {} \+

However, you can mention the parallelism in xargs with -P option. The above command will result in:

ls -l file1.txt file2.txt file3.txt

Only 1 thread.


Credit: unix.com

One Liner to delete all the files with names present in the txt file using xargs bash

There is simple one liner, that will delete all the files, you listed in a text file containing their names.

For Ex:

Create a file with name "filenames" contains the list of all the files  that you want to delete, then execute below command on the terminal.

xargs -a filenames -d'\n' rm

xargs -  to build and execute command lines from
-a - to use for providing input from text file instead of stdin
filenames - the file you just created
-d - delimiter
rm - command to pass the list

You can also use "-i" with rm for interactive mode.

Cheers.