Differences

This shows you the differences between two versions of the page.

Link to this comparison view

miriad_in_python [2008/09/23 21:42]
miriad_in_python [2008/09/23 21:42] (current)
Line 1: Line 1:
 +This is documentation on my system for interacting with MIRIAD data via the Python language.
 +
 +==== Berkeley Quickstart ====
 +
 +If you're at Berkeley and want to use Miriad-Python scripts, hopefully all you need to do is:
 +
 +<​code>​
 +source /​cosmic1/​pkwill/​usemir.sh
 +</​code>​
 +
 +If you're unlucky, the above commands will tell you that there'​s isn't a build for your machine yet. Peter will happily attempt to build for new configurations though it can take a lot of effort to get all of the necessarily dependencies configured correctly. (If there'​s no build for your computer and you really need to run the scripts, you can SSH into ''​cosmic''​ and run them there -- this will always work, with a hefty performance hit if your data don't live on ''​cosmic''​.)
 +
 +Peter'​s scripts (which serve as examples of how to use Miriad-Python) live at the [[https://​svn.hcro.org/​mmm/​|HCRO MMM SVN repository]] in [[https://​svn.hcro.org/​mmm/​pwilliams/​fancy|this directory]] for scripts and  [[https://​svn.hcro.org/​mmm/​pwilliams/​pylib|this directory]] for modules.
 +
 +
 +
 +
 +
 +
 +==== Relevant Systems ====
 +
 +Here are some links for the relevant software:
 +
 +  * We use [[http://​carma.astro.umd.edu/​miriad/​|MIRIAD]] data. Familiarity with this is assumed.
 +  * We want to use the Python language ([[http://​www.python.org/​]]). ((I can't make a link with the landing text "​Python"​ because DokuWiki freaks out in some way. Weird. And annoying.)) Documentation for Python is all around; try [[http://​docs.python.org/​|the official documentation site]] (especially the [[http://​docs.python.org/​tut/​tut.html|tutorial]]) or a google search if you have a specific question. You will need to get the hang of what Python code looks like, but should be able to get away without knowing all of its nuances.
 +  * We'll interact with code and data using the nice [[http://​ipython.scipy.org/​|IPython]] shell, which provides an IDL-style interactive programming interface. IPython has [[http://​ipython.scipy.org/​moin/​Documentation|a documentation site]].
 +  * Python itself doesn'​t have IDL-style numerical and array capabilities. These are added to the language with the [[http://​numpy.scipy.org/​|Numpy]] module. There is also [[http://​www.scipy.org/​Documentation|documentation for Numpy]] available. The documentation site mentions a Guide to Numpy book; I've bought an electronic copy of this book, and can lend it to you if you want.
 +  * The rest is my software, which generally isn't too well-documented. The modules I use include ''​miriad'',​ ''​mirtask'',​ ''​mirexec'',​ and ''​omega''​.
 +
 +
 +
 +
 +
 +==== Exploring with IPython ====
 +
 +Here are some generals tips for using IPython. These may not make much sense until you get a basic sense of the elements of the Python language.
 +
 +  * **Use the built-in help**! At the prompt, you can type ''​blah ?''​ to get information on "​blah",​ whether it's a function, class, variable, or whatever. Sometimes there'​s no documentation,​ but usually there is, and it should be enough to get you oriented.
 +  * **Use tab-completion**! If you're typing a line, you can hit the Tab key in the middle of a word to have IPython automatically finish it for you. This is kind of fun in itself, but what's more important is that if there'​s not a unique valid completion to what you've typed, a second press of Tab will print out the possibilities. This is a great way to explore the structure of unfamiliar code. If you're working with a Spectrum object called ''​spec''​ and don't remember what you can do with it, typing ''​spec . <TAB> <​TAB>''​ will print out the various methods you can call and fields you can access. Or ''​% <TAB> <​TAB>''​ will print out the various "​magic"​ functions built into IPython.
 +  * **Use the source**. One useful magic function is ''​%pfile blah'',​ which will let you view the file associated with the ''​blah'',​ whether blah is a variable, a class, a function, a module, etc. ''​%edit blah''​ will open up the file in an editor and then reload the definition when you close the editor.
 +  * There are a lot more fun features in IPython: logging, saving and restoration of variables, interaction with the Unix shell, and more. Check out the magic functions mentioned above and read their documentation.
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +==== Numpy Tips ====
 +
 +To actually work with the data, you'll need to know a little bit of the things that Numpy supports. The main tool provided by Numpy is its N-dimensional array, or ''​ndarray''​. The array objects that you'll be working with are instances of this class, and it pays to be familiar with the tools it offers you. The ''​numpy''​ module (often named as just ''​N''​) also has a bunch of routines that are useful.
 +
 +Numpy ''​ndarrays''​ support IDL-style math:
 +
 +<code python>
 +import numpy as N
 +
 +a = N.linspace (1, 100, 30) # create an array of 30 elements spanning from 1 to 100 linearly
 +a = a + 1 # now '​a'​ spans from 2 to 101 linearly
 +a += 1 # shorthand for the above
 +a = a / 3 # now '​a'​ spans from 1 to 34 linearly
 +a *= 3 # now '​a'​ is back to 3 to 102.
 +a = a**2 # square every element of a
 +
 +b = N.logspace (0, 3, 30) # create an array of 30 elements spanning from 10**0 to 10**3 logarithmically
 +
 +a = a / b # divide each element of '​a'​ by the corresponding element of '​b'​ and store the results in '​a'​.
 +
 +print a.ndim # print the number of dimensions of '​a':​ 1
 +print a.size # print the number of elemens in '​a':​ 30
 +print a.shape # print the size of each dimension of '​a':​ (30, )
 +print a.dtype # print the kind of data the '​a'​ contains: floating-point
 +</​code>​
 +
 +It's important to remember that, unlike in IDL, you can do fun things with array objects directly:
 +
 +<code python>
 +print a.mean () # compute the mean of the elements of '​a'​
 +print a.std () # standard deviation of the above
 +print a.min (), a.max (), a.sum () # and so on 
 +a.sort () # sort '​a'​ in-place (i.e., '​a'​ itself is modified)
 +if a.any (): print '​something in a is not zero'
 +if a.all (): print '​everything in a is not zero!'
 +</​code>​
 +
 +Numpy also has many complex ways that you can index into arrays:
 +
 +<code python>
 +a[0] # first element of a
 +a[-1] # last element of a
 +a[-2] # second-to-last element of a
 +a[1:3] # a subarray starting at a[1] and going up to, BUT NOT INCLUDING, a[3]
 +a[p:q] # more generally: a[p:q] has q - p elements and starts at a[p]
 +a[1:-1] # every element of '​a'​ except the first and last ones
 +a[0:​1000000] # if you go off the edge, you'll just get everything up until the end.
 +a[-1:3] # this gives an empty array, since we're trying to go from the end to the beginning
 +
 +b = N.zeros ((20, 10)) # create a 2-dimensional 20-by-10 array of zeros
 +b.shape # = (20, 10)
 +b.size # = 200
 +b.ndims # = 2
 +
 +b[0] # the first row of '​b',​ a 10-element array
 +b[0,0] # the first element of the first row of '​b'​
 +b[:,0] # the first column of '​b',​ a 20-element 1D array
 +b[:,0] += 3 # add 3 to each element in the first column of b
 +b[1:5,2:8] # a 4-by-6 subarray of b
 +
 +c = N.ones ((4, 3, 2, 6, 2)) # create a 5-dimensional 4-by-3-by-2-by-6-by-2 array of ones ...
 +</​code>​
 +
 +Numpy supports IDL-style where statements, with a similar implementation in fact:
 +
 +<code python>
 +import numpy as N
 +x = N.linspace (0, 10, 100)
 +
 +x[N.where (x < 4)] = 4 # set every element of x that is less than four, to four.
 +
 +y = x > 8 # y is an array of 100 booleans, True where x > 8, false where not.
 +x[N.where(y)] = -1
 +
 +condition = N.logical_and (x < 4, x > 2)
 +x[N.where (condition)] = -17
 +
 +a = N.linspace (0, 10, 100)
 +b = N.linspace (0, 100, 100)
 +w = N.where (a > 5) # you can save the result of a '​where'​ and use it several times
 +a[w] = 0
 +b[w] = -1
 +print w[0] # shows the indices where the condition was true
 +</​code>​
 +
 +Finally, a rundown of the interesting functions in the ''​numpy''​ modules:
 +
 +<code python>
 +import numpy as N
 +
 +print N.pi, N.e # constants.
 +
 +x = N.linspace (0, 10, 100) # 100-element array going from 0 to 10
 +
 +N.sin (x) # returns 100-element array of sin values
 +N.cos (x) # similar. N.tan (x), N.abs (x), N.log (x), etc.
 +
 +counts, edges = N.histogram (x) # compute a histogram of x: see the builtin help on N.histogram
 +
 +b = N.zeros_like (x) # create an array of zeros in the same shape as x
 +</​code>​