Wednesday, September 9, 2009

Yepa
Regards,

Jeronimo Calvo

Unix Xargs Piping Toolkit Utility

Xargs constructs an argument list for an arbitrary Unix command using
standard input and executes this command. xargs [options] [command]

The xargs command creates an argument list for command from standard
input. It is typically used with a pipe getting its input from
commands like ls and find The latter is probably the most common
source of xargs input and is covered in examples below.

One of the most common xargs applications in pipes is to execute a
command once for each piped record:

find . -name '*050106*' print | xargs -n2 grep 'From: Ralph'
cat iplist | xargs -n1 nmap -sV
The find command searches the entire directory structure for filenames
that contain 050106 . The xargs command executes grep command for each
two argument (so that filename was visible). In the second example cat
command supplies the list of IPs for map to scan.

In many Unix shells there is a limit to the number of arguments
allowed on a single command line. This is often a problem when you
analyze spam blocked by a spam filter. Here xargs can help: if the
argument list read by xargs is larger than the maximum allowed by the
shell, xargs will bundle the arguments into smaller groups and execute
command separately for each argument bundle.

If no command is specified, xargs works similar to the echo command
and prints the argument list to standard output.

Options
Option Description
-n# Execute command once for every # argument. For example, -n2
bundles arguments into groups of two or less and executes command on
each argument bundle.
-l# Execute command once for every # lines of input. For example, -l1
creates a bundle of arguments for every one line of input and executes
command on each argument bundle.
-i Normally xargs places input arguments at the end of command. Used
with the -i option, xargs will replace all instances of {} with input
arguments. You need to put them in single brackets or use a backslash
(\) before each bracket to keep the shell from interpreting the
special characters.
-t Echo each command before executing. Nice for debugging
-p Prompts the user before executing each command. Useful for debugging.

Examples
To use a command on files whose names are listed in a file, enter:
xargs lint -a < cfiles
If the cfiles file contains the following text:

main.c readit.c
gettoken.c
putobj.c
the xargs command constructs and runs the following command:

lint -a main.c readit.c gettoken.c putobj.c
If the cfiles file contains more file names than fit on a single shell
command line (up to LINE_MAX), the xargs command runs the lint command
with the file names that fit. It then constructs and runs another lint
command using the remaining file names. Depending on the names listed
in the cfiles file, the commands might look like the following:

lint -a main.c readit.c gettoken.c . . .
lint -a getisx.c getprp.c getpid.c . . .
lint -a fltadd.c fltmult.c fltdiv.c . . .

This command sequence is not quite the same as running the lint
command once with all the file names. The lint command checks
cross-references between files. However, in this example, it cannot
check between the main.c and the fltadd.c files, or between any two
files listed on separate command lines.

For this reason you may want to run the command only if all the file
names fit on one line. To specify this to the xargs command use the -x
flag by entering:
xargs -x lint -a <cfiles
If all the file names in the cfiles file do not fit on one command
line, the xargs command displays an error message.


To construct commands that contain a certain number of file names, enter:
xargs -t -n 2 diff <<EOF
starting chap1 concepts chap2 writing
chap3
EOF
This command sequence constructs and runs diff commands that contain
two file names each (-n 2):
diff starting chap1
diff concepts chap2
diff writing chap3
The -t flag causes the xargs command to display each command before
running it, so you can see what is happening. The <<EOF and EOF
pattern-matching characters define a here document, which uses the
text entered before the end line as standard input for the xargs
command.


To insert file names into the middle of command lines, enter:
ls | xargs -t -I {} mv {} {}.old
This command sequence renames all files in the current directory by
adding .old to the end of each name. The -I flag tells the xargs
command to insert each line of the ls directory listing where {}
(braces) appear. If the current directory contains the files chap1,
chap2, and chap3, this constructs the following commands:
mv chap1 chap1.old
mv chap2 chap2.old
mv chap3 chap3.old

To run a command on files that you select individually, enter:
ls | xargs -p -n 1 ar r lib.a
This command sequence allows you to select files to add to the lib.a
library. The -p flag tells the xargs command to display each ar
command it constructs and to ask if you want to run it. Enter y to run
the command. Press the any other key if you do not want to run the
command.
Something similar to the following displays:

ar r lib.a chap1 ?...
ar r lib.a chap2 ?...
ar r lib.a chap3 ?...

To construct a command that contains a specific number of arguments
and to insert those arguments into the middle of a command line,
enter:
ls | xargs -n6 | xargs -I{} echo {} - some files in the directory
If the current directory contains files chap1 through chap10, the
output constructed will be the following:

chap1 chap2 chap3 chap4 chap5 chap6 - some files in the directory
chap7 chap8 chap9 chap10 - some file in the directory
Typically arguments are lists of filenames passed to xargs via a pipe.
Please compare:

$ ls 050106*
$ ls 050106* | xargs -n2 grep "From: Ralph"
In the first example list of files that starts with 050106 is printed.
In the second for each two such files grep is executed.
Additional Examples
John Meister's UNIX Notes

Change permissions on all regular files in a directory subtree to mode
444, and permissions on all directories to 555:

find -type f -print | xargs chmod 444

$ ls * | xargs -n2 head -10
line 1 of f1 line 2 of f1 line 3 of f1

ls * } xarg -n1 wc -1

(date +%D ; du -s ~) | xargs >> log

ls *.txt | xargs -i basename \{\} .ascii \
| xargs -i mv \{\}.ascii \{\}.ask

(Note that the backslash usage)


Another example. Let's "cat" the Contents of Files Listed in a File,
in That Order.

$ cat file_of_files
file1
file2

$ cat file1
This is the data in file1

$ cat file 2
This is the data in file2
So there are 3 files here "file_of_files" which contains the name of
other files. In this case "file1" and "file2". And the contents of
file1" and "file2" is shown above.


$ cat file_of_files | xargs cat
This is the data in file1
This is the data in file2

What if you want to find a string in all finds in the current
directory and below. Well the following script will do it.

#!/bin/sh
SrchStr=$1
shift
for i in $*; do
find . -name "$i" -type f -print | xargs egrep -n "$SrchStr"/dev/null
done
Another quite nice thing, used for updating CVS/Root files on a Zaurus:
find . -name Root | xargs cp newRoot
Just copies the contents of newRoot into every Root file. I think this
works too:

find . -name Root | xargs 'echo user@machine.dom:/dir/root >'

as long as the quote are used to avoid the initial interpretation of the >.

These pieces of randomness will look for all .sh files in PWD and
print the 41st line of each - don't ask me why I wanted to know.
Thanks to Brian R for these.

for f in *.sh; do sed -n '41p' $f; done
or

ls *.sh | xargs -l sed -n '41p' Remove all the files in otherdir that
exist in thisdir.
ls -1d ./* | xargs -i rm otherdir/{}

Thursday, September 3, 2009

Tech Tip: Get Notifications from Your Scripts with notify-send

Notify-send is a great application for notifying you when an event has occurred. An event such as a script running to completion.

If notify-send is not installed on your machine already, install the package "libnotify1" (or possibly just "libnotify") from your repositories.

Once installed you can simply type the following, at the command line, to display a pop-up message near your system tray:

notify-send "hello"

By default the message will be displayed for 5 seconds. To change how long a message stays displayed use the "-t" switch. This will change, in milliseconds, how long the message is displayed. Enter "-t 0" to leave the message up until the user closes it.

notify-send "This message will be displayed for 3 seconds" -t 3000
notify-send "Click me to close me." -t 0

You can even add a title and an icon to the notification.

notify-send "This is the Title" \
"Check out the cool icon" \
-i /usr/share/pixmaps/gnome-terminal.png

When used in a script you could set it to notify you periodically by placing the command in a loop:

#!/bin/bash

while [ 1 ]; do
notify-send "Up Time" "`uptime`"
sleep 5m
done
__________________________

UBUNTU LINKS

Ubuntu-related links:

Community Support
GetDeb - Ubuntu Linux
Paid Support
Planet Ubuntu
Psychocats Resources
Report A Problem
ShipIt
The Fridge
Ubuntu
Ubuntu Archives
Ubuntu brainstorm
Ubuntu Documentation
Ubuntu Download
Ubuntu Forums
Ubuntu Hardware Support
Ubuntu News
Ubuntu Spotlight
Ubuntu Training
Ubuntu Tutorials
Ubuntu Wiki
Using APT
What Is Ubuntu?

Extra links useful in Ubuntu:
Gnome Themes
Nautilus Scripts

Wednesday, September 2, 2009

Finding Files On The Command Line

One of the things I like about Linux is the command line. I have used nautilus, gnome-commander, konqueror, kommander, dolphin and thunar to manage files in Linux and these file managers are great for what they do. But there are times when one simply wants to find a file when working on the command line without having to open a GUI application.

From the find man page:

GNU find searches the directory tree rooted at each given file name by evaluating the given expression from left to right, according to the rules of precedence until the outcome is known at which point find moves on to the next file name.

Find empty directories:

find /path -depth -type d -empty

Find empty files:

find /path -depth -type f -empty

Find a file with a specific name:

find /path -name name_of_file

Find a files with specific extensions:

find /path -name "*.given_extension"

Find files with specific permissions which have a ".txt. file extension:

find /path -name '*.txt' -perm 644

Find files with some given permissions:

find /path -perm -permision_bits

Find files with a given name and any extension:

find /path -name 'given_name.*'

Find files modified in the latest blocks of 24 hours:

Find files that were accessed in the latest blocks of 24 hours:

find -atime n

Where n is:

  • 0 for the last 24 hours
  • 1 for the last 48 hours
  • 2 for the last 72 hours

Find files according to owner:

find /path -user root

One can also pipe find commands to the xargs command to execute commands on files.

Find and delete files:

find /path -name mytestfile | xargs rm

See man find and man xargs for more information about these powerful commands.

Many new Linux users are intimidated by the command line and this feeling should be overcome from the onset because the command line can be faster and more powerful than most GUI applications.


An alternative way to find and delete files using only find.

find /path -name mytestfile -exec rm '{}' \;

Everything between -exec and \; get executed per file, and '{}' is replaced with the name of the file found.

This can be used to do just about anything with files. So for example, find and delete all CVS folders in a project:

find /path -name CVS -type d -exec rm -r '{}' \;

Create and md5 hash of all files in a folder

find /home/williamb/ -type f -exec md5sum '{}' \;