Command line tips

The intention of this post is to collect a list of my favorite command line tips. Feel free to leave yours in the comment section.

• Instead of using clear you can just press ctrl-r.
• The command units can be used for any unit conversion you can imagine. See man page.
• You can use multitail to monitor multiple log files on a terminal.

Sources:

Give me that one command you wish you knew years ago

Mark Stosberg's bash tips
TechBar: Linux shell tips and tricks

Extract decimals from text using grep

Let's say you have a log file that looks like this:

I/Profiling: TimeInterval1:  3271.72 ms
I/Profiling: TimeInterval1:  3379.15 ms
I/Profiling: TimeInterval1:  3645.32 ms
...

and you want to extract all decimal numbers so that you can get the average, or plot them. Here's how you would use grep to do this:

grep -o ' [0-9]\+\.[0-9]\+' mylogfile.log

The -o option is for just printing the matching text and not the whole line.

Elapsed Time for Shell Scripts

A lot of times you want to calculate the time it takes for a part of a shell script to run. Here is one of the ways you can do this:

T1=$(date +%s)
# Do something here
T2=$(date +%s)
diffsec="$(expr $T2 - $T1)"
echo | awk -v D=$diffsec '{printf "Elapsed time: %02d:%02d:%02d\n",D/(60*60),D%(60*60)/60,D%60}'

This will tell you the elapsed time between setting T1 and setting T2. The output is formatted as HH:MM:SS and printed to the console. You can easily redirect to a file for logging.

Using FFMPEG

FFMPEG is a complete, cross-platform solution to record, convert and stream audio and video. It includes libavcodec; the leading audio/video codec library. FFMPEG is a free software project and it is published under the GNU Lesser General Public License 2.1+ or GNU General Public License 2+ (depending on which options are enabled).

To install FFMPEG in Ubuntu just type:

sudo apt-get install ffmpeg

This post contains a collection of useful examples of using FFMPEG from the command line.

Video Information
Get information on the video (codec, frame rate, etc.).

ffmpeg -i myvideo.avi

Video Encoding
To encode the video using a different codec we specify it using the -vcodec option.

ffmpeg -i myvideo.avi -vcodec mpg4 video_out.avi

Now let's say we want a better quality compression and a different frame rate from the original. We specify 1Mb/s bitrate using the -b option (default is 200kb/s) and we change the frame rate to 12 fps using the -r option.

ffmpeg -i myvideo.avi -vcodec mpg4 -b 1M -r 12 video_out.avi

Other values for the -vcodec option include:

copy:	Copy raw codec data as is (do not reencode)
rawvideo:	Raw video (uncompressed)
xvid:	XviD (MPEG-4 Part2)
wmv2:	Windows Video Media

Extracting Video Frames
The following example extracts the frames from the video and saves them as frame_001.png, frame_002.png,...

ffmpeg -i myvideo.avi -f image2 frame_%03d.png

Cropping a video to a specific time interval
The following example extracts a scene from the input video that start at 1 minute 45 seconds and has a duration of 30 seconds.

ffmpeg -i myvideo.avi -ss 00:01:45 -t 00:00:45 myscene.avi

Slow down or speed up a video
Let's say we need to slow down a video now. Unfortunately, decreasing the frame rate with the -r option will not do, since ffmpeg will drop frames to keep the duration of the video the same. We need to use mjpegtools to do that. To install mjpegtools type:

sudo apt-get install mjpegtools

For this example the original video has a frame rate of 30fps and we want to make it play at 1/3 of the speed. So we have to trick ffmpeg to believe that the original video had a 10fps frame rate. The yuvfps tool does exactly that. Make sure that the input frame rate (-s option) and the output frame rate (-r option) are the same.

ffmpeg -i myVideo.avi -f yuv4mpegpipe - | yuvfps -s 10:1 -r 10:1 | ffmpeg -f yuv4mpegpipe -i - -vcodec mpeg4 -b 12M  mySlowVideo.avi

Note that this will not work unless your video has one of the yuv pixel formats. If your pixel format is not compatible (rgba for example) you would have to convert it like this:

ffmpeg -i myVideo.avi -pix_fmt yuv420p myVideo2.avi

How to add users on Ubuntu

First, add the new user:

sudo useradd -c “John Smith” -m -s “/bin/bash” jsmith

and then create a password for the new user:

sudo passwd jsmith

The -c options lets you give a full name for the user, -m option creates a home folder, -s specifies the default shell.

Install Ubuntu Tweak on Ubuntu 10.10

Open a terminal and type:

sudo add-apt-repository ppa:tualatrix/ppa
sudo apt-get update
sudo apt-get install ubuntu-tweak

Install and Configure Tomcat6 on Ubuntu Linux

Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies.

The Java Servlet and JavaServer Pages specifications are developed under the Java Community Process.

Part 1
First, install tomcat6:

sudo apt-get install tomcat6 tomcat6-docs tomcat6-examples tomcat6-admin

Then configure a user to be manager and admin by adding the following lines

  <role rolename="admin"/>
  <role rolename="manager"/>
  <user username="YOURUSERNAME" password="YOURPASSWORD" roles="admin,manager"/>

in your /var/lib/tomcat6/conf/tomcat-users.xml. Make sure you put it between the <tomcat-users> and </tomcat-users> tags and don't forget to replace YOURUSERNAME with your actual username.
Now go to a browser and hit:

http://localhost:8080/

and you should see the welcome page.

Part 2
To restart tomcat you just need to say:

sudo /etc/init.d/tomcat6 restart

or if you want to start/stop/get status you just replace restart with start/stop/status.
To make this easier go to a terminal and type:

gedit tomcat

then add the following lines of text:

#!/bin/bash

/etc/init.d/tomcat6 $1

then save and close and type:

chmod u+x tomcat
sudo mv tomcat /usr/local/bin

so if you want to stop tomcat you can now say:

sudo tomcat stop

Let's add auto-completion to our new command now. Open a terminal and type:

gedit tomcat

then copy paste the following

_tomcat() 
{
    local cur prev opts
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    prev="${COMP_WORDS[COMP_CWORD-1]}"
    opts="start stop restart status"

    if [[ ${cur} == * ]] ; then
        COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
        return 0
    fi
}
complete -F _tomcat tomcat

save and close gedit. Now type:

sudo mv tomcat /etc/bash_completion.d

You need to restart for this to take effect, but if you want to test it right away type:

. /etc/bash_completion.d/tomcat

Then as an example, type:

sudo tomcat re

and hit tab a couple of times. It should complete the word restart for you since it's the only valid option that begins with "re".

How to delete all .svn directories

If you are using subversion, at some point you probably tried to take some folders from one repository and try to commit then to another. Since each folder has a hidden .svn folder, subversion will complain that "they are already under version control". Here is how to recursively delete all .svn folders:

find . -name ".svn" -type d -exec rm -rf {} \; -prune -print

Command Line Shortcuts

I was reading this post today on how to become a command line ninja and here is some interesting stuff I learned:

• Ctrl+U: This clears the entire line so you can type in a completely new command.
• Ctrl+K: This deletes the line from the position of the cursor to the end of the line.
• Ctrl+W: This deletes the word before the cursor only.
• Ctrl+R: This lets you search your command history for something specific. For example, if you wanted to search for the recent commands that included nano, you would hit Ctrl+R and type nano. It would show your most recent  command, and you could use the up and down arrows to cycle through your history of commands using nano in them.