Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
command-line_ninjary [2013/04/01 14:40]
c.axen [Navigation]
command-line_ninjary [2013/04/01 14:59] (current)
c.axen [Navigation]
Line 1: Line 1:
 +====== Command-line Ninjary GSPS 2013 ======
  
 +This is meant to be a quick reference guide to command-line usage for those of us 
 +who have the basics down, but aren't yet true ninjas. ​ Please add your own tips
 +or quick examples! ​ We spend a surprising amount of our lives inside a terminal,
 +so learn to love it and use it well.  Note: this page was written for ''​bash'',​
 +but much of this stuff should translate to ''​csh''​ or your shell of choice.  ​
 +
 +------------------------------------
 +
 +==== Navigation ====
 +General notes about navigation commands: First, "​Ctrl"​ commands are the same as in the text editor emacs, so you may find that other emacs line navigation commands work as well. Second, if you are on a Unix machine, note that the line navigation commands work almost anywhere you can edit text - you will learn that accidentally if you start using these commands compulsively. Third, if you use "​Ctrl"​ commands often, you may consider setting the "Caps Lock" key as an additional "​Control"​ key - this is just a computer setting for Unix (e.g. System Preferences > Keyboard > Modifier Keys) but requires slightly more work on Windows (e.g. the AutoHotkey program).
 +
 +
 +  * ''​cntrl-a'':​ go to beginning of line
 +  * ''​cntrl-e'':​ go to end of line
 +  * ''​cntrl-k'':​ delete everything to right
 +  * ''​cntrl-u'':​ delete everything to left
 +  * ''​cntrl-w'':​ delete word to left
 +  * (''​option''​ or ''​cntrl''​)-(''​left''​ or ''​right''​):​ move left/right. ''​cntrl''​ for linux, ''​option''​ for mac.
 +  * Ctrl-f: move forward one character
 +  * Ctrl-b: move backward one character
 +  * use backslash ''​\''​ to literally interpret any command characters
 +  ​
 +  * ''​cd'' ​ : go home
 +  * ''​cd -''​ : go back to previous working directory
 +  * ''​~''​ : stand-in for home directory
 +      * e.g.: ''​ls ~/​Documents''​
 +  ​
 +  * ''​!!'':​ repeat previous command
 +  * ''​!xxxx'':​ repeat most recent command that starts with //xxxx//
 +  * ''​cntrl-r xxxx'':​ search recursively through history for match to //xxxx//
 +      * keep typing to refine search==
 +      * ''​cntrl-r''​ again to cycle through matches
 +      * and hit ''​enter''​ to execute
 +  * ''​^this^that'':​ rerun most recent command, but replace //this// with //that//
 +  * ''​history'':​ shows history of recent commands
 +      * ''​history | grep xxxx'':​ search for command //xxxx// in history
 +      * ''​!xxxx'':​ repeat item number //xxxx// in history
 +  ​
 +  * ''​cntrl-x-e'':​ invoke your editor to compose a hairy command
 +      * executes command upon exit
 +      * can be invoked in the middle of composing command
 +  ​
 +  * curly braces perform parameter expansion, inserting a copied string with contents of braces swapped
 +      * ''​cp file.{txt,​txt.backup}'':​ same as ''​cp file.txt file.txt.backup''​
 +      * ''​cp file_{,​copied}one.txt'':​ same as ''​cp file_one.txt file_copied_one.txt''​
 +      * ''​rm file_{a,​b,​d,​g}.txt'':​ same as ''​rm file_a.txt file_b.txt file_d.txt file_g.txt''​
 +
 +
 +
 +------------------------------------
 +
 +
 +
 +==== Really Useful Built-ins ====
 +
 +  * ''​clear'':​ clear the terminal screen
 +  * ''​cntrl-z'':​ shunt something to the background
 +      * ''​fg'':​ to get it back
 +      * really helpful when in some other program like a text editor or python/IRAF terminal
 +  ​
 +  * ''​man xxxx'':​ display manual for //xxxx//
 +  * (''​which''​ or ''​whereis''​) ''​xxxx'':​ display path for //xxxx//
 +  ​
 +  * ''​wc xxxx'':​ character, word, and line count on file //xxxx//
 +      * ''​wc -l'':​ return only line count
 +  ​
 +  * ''​cat xxxx'':​ display contents of file //xxxx//
 +  * ''​head -n 10 xxxx'':​ display first 10 lines of file //xxxx//
 +  * ''​tail -n 10 xxxx'':​ display last 10 lines of file //xxxx//
 +  ​
 +  * ''​grep xxxx yyyy'':​ search for //xxxx// in file //yyyy//
 +      * //xxxx// can be simple string or regular expression
 +  * ''​find path xxxx'':​ search for files with names matching //xxxx//
 +      * //path// is optional - defaults to working directory
 +      * //xxxx// can be simple string or regular expression
 +  ​
 +  * ''​kill xxxx'':​ kill (end) process with process id //xxxx//
 +  * ''​top'':​ display running processes and system info
 +      * ''​q'':​ quit
 +      * ''​k xxxx'':​ in linux, kills process with id //xxxx//
 +      * sadly, no equivalent to above on most macs
 +  * ''​ps -e'':​ show all running processes
 +      * ''​ps -e | grep xxxx'':​ search for //xxxx// amongst running processes
 +  ​
 +  * ''​date'':​ print date & time
 +      * can choose output format - i.e.: ''​date +%H:​%M:​%S''​
 +  * ''​crontab'':​ fully-functional command scheduler
 +      * good examples [[http://​www.thegeekstuff.com/​2009/​06/​15-practical-crontab-examples/​|here]]
 +  * ''​at'':​ a simple one-off scheduler
 +      * ''​cmd | at time''​ : schedule //cmd// to run at //time//
 +      * i.e.: ''​ls ~/* > ~/​home_dir.list | at 3:​05pm''​
 +      * understands pretty broad range of //time// definitions
 +  ​
 +  * ''​screen'':​ like VNC but only for the terminal, lets you start a session and then disconnect and reconnect at will
 +      * ''​screen''​ to connect
 +      * ''​screen -d''​ to disconnect
 +      * ''​screen -r''​ to reconnect
 +      * ''​screen -list''​ to show current sessions
 +      * ''​screen -r xxx''​ to reconnect to session with PID //xxx//
 +      * great for ''​ssh''​ sessions
 +      * full guide [[http://​www.cyberciti.biz/​tips/​how-to-use-screen-command-under-linux.html|here]]
 +
 +==  Get A Package Manager ==
 +
 +Best way to install and manage command-line tools. Native for all linux builds; have a couple good choices for macs.
 +
 +  * [[http://​mxcl.github.com/​homebrew/​|Homebrew]]
 +  * [[http://​www.macports.org/​|MacPorts]]
 +
 +
 +
 +------------------------------------
 +
 +
 +
 +==== Pipes & Redirects ====
 +
 +=== Invoking multiple commands ===
 +  * ''​cmd &''​ : run //cmd// in subshell
 +  * ''​cmd1 ; cmd2''​ : run //cmd1// then run //cmd2//
 +  * ''​cmd1 && cmd2''​ : run //cmd2// after //cmd1// only if //cmd1// is successful
 +  * ''​cmd1 || cmd2''​ : run //cmd2// after //cmd1// only if //cmd1// is not successful
 +
 +=== Piping stuff around ===
 +  * ''​cmd > file''​ : sends output of //cmd// to //file//
 +      * overwrites //file//
 +      * ''​%%cmd >> file%%''​ : appends output of //cmd// to //file//
 +  ​
 +  * ''​cmd1 | cmd2''​ : connects stdout of //cmd1// to stdin of //cmd2//
 +  ​
 +  * ''​cmd 2> file'':​ sends error of //cmd// to //file//
 +      * ''​%%cmd 2>> file%%''​ : appends error of //cmd// to //file//
 +      * e.g.: ''​ls not.a.file* 2> output; cat output''​
 +  * ''​cmd > output 2> errors''​ : save output and errors of //cmd// separately, in files //output// and //errors//
 +      * e.g.: ''​ls * not.a.file* > success.out 2> failure.err''​
 +  ​
 +  * ''​cmd < file''​ : sends //file// to //cmd//
 +
 +
 +== Useful tools when piping ==
 +
 +  * ''​sort'':​ sorts stuff alphanumerically
 +      * ''​ls | sort''​
 +  ​
 +  * ''​xargs cmd''​ : a piping glue - runs //cmd// with standard input as arguments
 +      * can be used with a pipe or with a file
 +      * use it to translate between lines in a file or pipe and arguments to a function
 +      * best to understand by example
 +      * ''​echo one two three > test.txt; xargs echo < test.txt''​
 +      * ''​echo -e '​one\ntwo\nthree'​ > test.txt; xargs echo < test.txt''​
 +      * ''​ls | sort | xargs echo''​
 +  ​
 +  * ''​awk'':​ programming language to edit pipes on the fly
 +      * basic usages are powerful, learn more [[http://​www.gnu.org/​software/​gawk/​manual/​gawk.html|here]]
 +      * ''​awk '​{print $1}'':​ print the first item of each line, items split by spaces/tabs
 +      * ''​awk -F, '​{print $2}'':​ print the second item of each line, items split by '',''​
 +  * Example: search for all running python instances, find their //PID//, and kill them
 +      * ''​ps -e | grep python | awk '​{print $1}' | xargs kill''​
 +  * Example: count and display all the items in current working directory
 +      * ''​ls | awk '​{sum+=1;​ print sum ": " $1}'''​
 +  * Example: replace every occurrence of ''​NA''​ with the number ''​0''​ in a file
 +      * ''​cat file.txt | awk '​{gsub("​NA",​ "​0"​);​ print $0}' > clean.file.txt''​
 +
 +------------------------------------
 +
 +
 +==== Loops & Functions ====
 +
 +=== Variables ===
 +Variables that have already been declared are prefaced by a ''​$'',​ but you don't need
 +one when first declaring a variable. ​ Any variables declared normally (''​x=4''​) are considered strings,
 +to declare a number use the ''​let''​ keyword: ''​let x=4''​. ​ You can also declare a variable to be a command by 
 +wrapping it in back-ticks or parentheses. ​ I.E. ''​LIST=(ls);​ $LIST''​.
 +There are some system-wide variables that should always be available, like ''​$SHELL'',​ ''​$PATH'',​ and ''​$HOME''​.
 +Note that though convention says system-wide variables should be all caps, they don't need to be.
 +
 +=== Simple Loops ===
 +There are several ways to write loops, the most useful of which are below.
 +Note that in bash an ''​$''​ always indicates a variable, and that all variables are,
 +by default, treated as strings. ​ The actions to be repeated must be wrapped in 
 +''​do/​done''​ statements, and the semicolon marks an end-of-statement (equivalent to a line return).
 +A few examples:
 +
 + for i in {1..20}
 +   do
 +     echo $i
 +   done
 +
 + for i in {a,b,d,g}; do echo $i; done
 +
 + for i in a 2 b 6;
 +   do
 +     echo $i;
 +   done
 +
 + for file in *filePattern*;​
 +   do
 +     echo file;
 +   done
 +
 +
 +=== Simple Tests ===
 +
 +The shell can do arithmetic and logic tests, but I think if you're doing
 +anything very complex you should move into ''​Python''​ or ''​Perl''​. ​ Logic comparisons
 +should be put in square brackets (spacing is important), ''​if''​ statements must
 +be closed with a ''​fi'',​ and the operators you have at your access are:
 +
 +  * ''​=='':​ string equality
 +  * ''​!='':​ string inequality
 +  * ''​-eq'':​ number equality
 +  * ''​-ne'':​ number inequality
 +  * ''​-gt'':​ greater than
 +  * ''​-lt'':​ less than
 +  * ''​-ge'':​ greater than or equal
 +  * ''​-le'':​ less than or equal
 +  * ''&&'':​ and
 +  * ''​||'':​ or
 +
 +
 +== Use more aliases! ==
 +Use aliases to make hard-to-remember commands less terrible.
 +
 +Example:
 +http://​xkcd.com/​1168/:​
 +
 +  * ''​alias maketar='​tar -czvf '''​
 +  * ''​alias untar='​tar -xzvf '''​
 +
 +
 +== Defining your own functions ==
 +
 +You can declare functions and use them at any time.  The most common way
 +to interact with functions is through required arguments, as shown below.
 +Adding in flags (like ''​-v''​) for optional arguments takes some more work
 +(look up ''​getopts''​).
 +
 +  * ''​$#'':​ number of parameters passed
 +  * ''​$@'':​ array of all parameters
 +  * ''​$N'':​ the Nth parameter passed to the function, with $0 being the function name
 +
 +Some examples:
 +
 + function speak () {
 +   echo here are all $# arguments: $@
 +   echo here is argument 1: $1
 +   echo here is argument 3: $3
 + }
 +
 + function killme () {
 +   ps -e | grep $1 | awk '​{print $1;}' | xargs kill
 +   echo $1 is dead
 + }
 +
 + function cls () {
 +   cd $1
 +   ls
 + }
 +
 +
 +== .profile or .bashrc ==
 +
 +There are a set of system files in your home directory that are executed every time
 +a terminal is opened - use those to define aliases, functions, or variables
 +you want always available.