Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| carma_fluxcal [2012/09/28 20:45] – [Record of Past Servitude] amberb | carma_fluxcal [2022/05/19 21:41] (current) – removed nathan_sandford | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== CARMA FluxCal ====== | ||
| - | This page is for those unfortunate Berkeley grad students that have been tasked with maintaining fluxcal in the absence of a Berkeley CARMA postdoc. | ||
| - | |||
| - | [[fluxcal_notes|Meeting Notes]] | ||
| - | |||
| - | [[fluxes_script|Fluxes.py Script Notes]] | ||
| - | |||
| - | [[http:// | ||
| - | ===== To Do / Current Issues ===== | ||
| - | * **We should check how our list of accepted calibrators compares to the full list of calibrators in FluxSource.cat - see [[fluxes_script|Fluxes.py Script Notes]]** | ||
| - | |||
| - | |||
| - | |||
| - | ===== Sign Up / Future Servitude ===== | ||
| - | |||
| - | The general strategy is to catch up each week on Fridays: | ||
| - | ~order is Amber Statia Chat Katey | ||
| - | |||
| - | Upcoming out-of-town times: | ||
| - | |||
| - | **Future Servitude (now 2012!):** | ||
| - | | ??? | ??? | | ||
| - | |||
| - | ===== Record of Past Servitude ===== | ||
| - | |||
| - | ^ Date caught up to ^ Peon ^ Last Track ^ Notes ^ | ||
| - | | Feb 17 | Frank | cx319.1E_90MACS04.4.mir | no more Frank =( | | ||
| - | | Mar 17 | Amber | cx319.4D_90Abell2.1.mir | not set up for CARMA23 - just flag ant(16-23) for now | | ||
| - | | Mar 29 | Statia | ||
| - | | Apr 1 | Amber | c0722.1D_230SgrA.53.mir | | | ||
| - | | Apr 8 | Amber | c0674.1D_115M31.17.mir | | | ||
| - | | Apr 15 | Statia | c0713.1C_115NGC278.6.mir| only 13 tracks to run? | | ||
| - | | Apr 22 | Chat | c0690.2C_88NGC126.2.mir | | ||
| - | | Apr 29 | Amber | c0699.1C_115OrionO.14.mir | 19 tracks run | | ||
| - | | May 6 | Katey | c0753.3C_104HATLAS.4.mir | 28 tracks | | ||
| - | | May 13 | Statia | c0685.2D_113MCG022.1.mir |10 tracks (bad weather at hs)| | ||
| - | | May 20 | Chat | c0672.2D_110DMTau.1.mir | | | ||
| - | | May 27 | Amber | c0672.3D_110UXTau.1.mir | | | ||
| - | | Jun 3 | Katey | --- | didn't run Jun 3 - split with Amber Jun 10 | | ||
| - | | Jun 10 | Amber | c0685.7D_113CGCG46.7.mir | split May 27 - Jun 10 with Katey | | ||
| - | | June 17 | Statia | c0688.1D_100015028.5.mir| | ||
| - | | Jun 24 | Chat | c0685.7D_113CGCG46.13.mir | | | ||
| - | | Jul 1 | Katey | c0712.1D_115NGC545.7.mir | ~30 tracks | | ||
| - | | Jul 8 | Amber | c0657.2E_91HS0810.3.mir | | | ||
| - | | Jul 15 | Statia | c0732.1E_245Epsilo.2.mir | lots of tracks... | ||
| - | | Jul 22 | Chat | c0684.18SL_30SAUS.1.mir| Actually 7/26...Did half of the next person' | ||
| - | | Aug 6 | Katey | c0849.9SH_31RXJ175.3.mir | did two weeks worth | | ||
| - | | Aug 12 | Amber | c0768.6E_107G12H34.6.mir | | | ||
| - | | Aug 19 | Statia | c0857.5E_95PTF5.3.mir | | | ||
| - | | Aug 26 | Chat | c0768.3D_96G12H89.2.mir | My God that took a long time. | | ||
| - | | Sep 2 | Amber | c0834.12D_88C2.2.mir | didn't take too long (addLastN=65), | ||
| - | | Sep 9 | Statia | cx327.1SH_31DQFlex.2.mir | | | ||
| - | | Sep 16 | Chat | c0758.3D_114HE2149.6.mir | The new era of fluxcal has begun...or at least, we're running old fluxcal in a new directory! | | ||
| - | | Sep 23 | Katey | |Didn' | ||
| - | | Sep 30 | Statia | c0817.14D_90MACSJ1.1.mir | Split last two weeks with Katey| | ||
| - | | Oct 7 | Chat | c0843.5SH_90A1319.7.mir | | | ||
| - | | Oct 14 | Amber | c0845.1SH_30Field1.12.mir | | | ||
| - | | Oct 21 | Katey | c0834.13D_89C3.10.mir | | | ||
| - | | Oct 28 | Amber | c0852.2D_230NGC545.5.mir | kept notes on what passed/ | ||
| - | | Nov 4 | Katey | c0785.1D_86G03539.9.mir | | | ||
| - | | Nov 11 | Chat | cx329.1D_90S140.4.mir | Kept notes on what passed/ | ||
| - | |Nov 18| Statia | c0857.1A_95PTF1.2.mir | made up last Chat track (listed one is from Nov 7)| | ||
| - | |Nov 25 | Amber | c0857.1A_95PTF1.9.mir | very few tracks with A array move and bad weather; also bootflux crash issue with MARS - bring up at telecon | | ||
| - | |Dec 2| Statia | c0811.1A_30NGC452.PACS.11.mir | almost all tracks failed at bootflux (mars, uranus and neptune| | ||
| - | |Dec 9| Chat| c0811.1A_30NGC452.PACS.19.mir | Didn't run 1mm tracks. | ||
| - | | Dec 16 | Katey | c0773.2B_95GRB2.1.mir | | | ||
| - | | Dec 23 | Amber | c0794.2B_30Mol160.PACS.5.mir | after running out of space issue on Dec 23rd, cleared out some space and able to get tracks on Jan 3, 2012; fluxcal done on Jan 3 - bootflux crashed on MARS, 1mm and 3mm in B array | | ||
| - | | Dec 30 | Chat | cx334.1B_30GRB111.PACS.6.mir | Space cleared, Amber put tracks in directory Jan 3, 2012 | | ||
| - | |Jan 6| Statia | c0811.1B_230NGC452.13.mir |blah | | ||
| - | | Jan 13 | Katey | c0769Z.3SL_30MACSJ0.6.mir | | | ||
| - | | Jan 20 | Amber | c0820.3C_86115049.3.mir | | | ||
| - | | Jan 27 | Statia | c0820.3C_86115049.6.mir | | | ||
| - | | Feb 3 | Amber | c0843.6SL_30A1622.1.mir | C12 low amplitude issue - I just flagged C12 in all 1 and 3mm tracks | | ||
| - | | Feb 10 | Katey | c0834I.1C_97B1.2.mir | | | ||
| - | | Feb 17 | Chat | c0834M.2C_96B2.3.mir | | | ||
| - | | Feb 24 | Statia | c0859.2C_108G12H29.1.mir| | | ||
| - | | Mar 2 | Chat | c0866.2C_100NGC624.4.mir | | | ||
| - | | Mar 9 | Katey | c0888C.15C_230Ser1.1.mir | Was out of town, had to do fluxcal on Monday | | ||
| - | | Mar 16 | Amber | c0933.29SL_30C58114.8.mir | | | ||
| - | | Mar 23 | Statia | c0932.12SL_30RXJ223.3.mir | Bad things happened when I was getting tracks | | ||
| - | | Mar 30 | Chat | c0933.26SL_30C57507.1.mir | Bad things happened when I was getting tracks | | ||
| - | | Apr 6 | Katey | c0885.1D_91XMMM5.2.mir | Didn't have any problems | | ||
| - | | Apr 13 | Amber | c0884.1SL_31MOO002.3.mir | | | ||
| - | | Apr 20 | Statia |c0957.7D_115M31BRK.4.mir | ||
| - | | Apr 27 | Chat | c0922.1D_98GC.2.mir | ||
| - | | May 4 | Katey | c0915.1D_110Taurus.1.mir | | | ||
| - | | May 11 | Amber | c0924V.3D_88Serpen.4.mir | | | ||
| - | | May 25 | Amber | c0961.7SL_30PX03.1.mir | OLD FLUXCAL IS DONE! Running 2 weeks of tracks with the new script! | | ||
| - | | Jun 5 | Amber | c0911.5D_90IRAS4F.3.mir | new fluxcal at Illinois | | ||
| - | | Jun 14 | Amber | c0849Z.10SH_95Abell1.11.mir | new fluxcal at high site pending fix of issue at Illinois | | ||
| - | | Jun 22 | Amber | c0933.2SH_30C13062.1.mir | at high site from ~/ | ||
| - | | Jul 3 | Amber | c0884.3SH_31MOO004.1.mir | at high site; some weird Exception: 1 and 8...I don't know what this is coming from | | ||
| - | | Jul 6 | Amber | c0879.1E_90MS0451.2.mir | at high site | | ||
| - | | Jul 16 | Amber | c0883.1SL_30ACTJ03.12.mir | at high site | | ||
| - | | Aug 6 | Amber | c0892.12E_100MACSJ0.2.mir | at high site | | ||
| - | | Aug 13 | Amber | c0973.30E_100HeLMS5.2.mir | at high site | | ||
| - | | Aug 21 | Amber | cx340.1E_115gulf.4.mir | at high site | | ||
| - | | Aug 31 | Amber | c1017.7E_100MACSJ0.1.mir | at high site | | ||
| - | | Sep 28 | Amber | c1009.9SH_90ABELL1.7.mir | at high site | | ||
| - | |||
| - | ===== Why? ===== | ||
| - | |||
| - | To monitor fluxes of common calibrators...etc... | ||
| - | |||
| - | Wiki page for fluxcal activities status stuff (from Nikolaus, Nov 4 2011) [[http:// | ||
| - | |||
| - | **Other fluxcal stuff:** | ||
| - | * Woojin' | ||
| - | * NOT done on science tracks | ||
| - | * results are [[http:// | ||
| - | * John Carpenter' | ||
| - | * reduction script is / | ||
| - | * results are posted to [[http:// | ||
| - | * Tom Plagge' | ||
| - | * results posted [[http:// | ||
| - | * uses [[http:// | ||
| - | * the only calibration instructions given on the public CARMA site is a link to the current [[http:// | ||
| - | * also these fluxes are available through the xplore tool | ||
| - | * information on flux monitoring on internal CARMA site is **woefully** out-of-date ([[http:// | ||
| - | * I know Tom C wants to get this updated on the wiki somewhere, but I couldn' | ||
| - | |||
| - | |||
| - | ===== Who? ===== | ||
| - | |||
| - | Amber\\ | ||
| - | Statia\\ | ||
| - | Chat\\ | ||
| - | Katey | ||
| - | |||
| - | ===== How? ===== | ||
| - | |||
| - | |||
| - | ---- | ||
| - | **Update Sept 14,2011:** due to network issues (at the request of Rick and Tom), we need to do our fluxcal analysis in a different directory. Specifically, | ||
| - | ---- | ||
| - | |||
| - | |||
| - | The day-to-day grunt work: fluxcal should be done ~ once a week to keep up with all the incoming tracks. The idea is to switch off among those of us involved in some way so that we each only have to do a week's worth of tracks once a month or so. | ||
| - | |||
| - | Full instructions on how to do all this can be found {{: | ||
| - | |||
| - | - Log into the highsite computers: | ||
| - | * ssh -Y obs@ssh.mmarray.org (-Y for Mac's, -X for Linux) | ||
| - | * type ' | ||
| - | * cd into your working directory | ||
| - | - Figure out how many tracks there are since the last time fluxcal was done: | ||
| - | * ./ | ||
| - | * this will output the tracks since the last track run and tell you what to use for addLastN | ||
| - | - Type ' | ||
| - | * import GetNewCalFlux | ||
| - | - GetNewCalFlux.getTracksWithFlux(addLastN=10) | ||
| - | * put whatever getLastN told you to here instead of 10 | ||
| - | * this will copy all the files you need to look at into your directory and create FluxTable.txt (a list of all the tracks you need to run doFluxCal on, listing the calibrators in each track. The calibrator list is not always accurate though, so you should check it with uvindex - see note below) | ||
| - | - Figure out the appropriate GetNewCalFlux.doFluxCal commands to run for each track using the writeCommands.csh program: | ||
| - | * writeCommands.csh reads the .mir files in your directory and finds the calibrators observed in each track (and their ' | ||
| - | * run writeCommands.csh | ||
| - | * this will create the file fluxcalCommands.txt, | ||
| - | - For each track, run | ||
| - | * GetNewCalFlux.doFluxCal(track name, primary (ie Neptune), secondary, passband) | ||
| - | * ie. GetNewCalFlux.doFluxCal(' | ||
| - | * NOTE: make sure your primary calibrator (MARS, NEPTUNE, etc.) and the " | ||
| - | * At the end of the script, you will be prompted for whether or not you want to add this value to science_fluxes.txt - if the data look good, then type ' | ||
| - | * If the track seemed good (ie. you added the secondary calibrator flux to science_values.txt), | ||
| - | - When you're done, science_fluxes.txt should contain all the fluxes you chose to record. You first need to copy science_fluxes.txt to your machine where you have CVS set up. Then, you simply append these fluxes to the end of FluxSource_newadd.cat in your miriad install and commit it to CVS through the following 3 steps: | ||
| - | * make sure your copy is up-to-date: < | ||
| - | * add the fluxes in science fluxes.txt to the end of FluxSource_newadd.cat using the text editor of your choice | ||
| - | * commit your new version to the official repository, with your initials in the comment line, ie.: < | ||
| - | |||
| - | |||
| - | **A few suggestions on general workflow: | ||
| - | * It is good to have two terminals logged in to the highsite: a shell to run MIRIAD commands in, and a Python or iPython session (avoid running fluxcal in a SAC, where you run the risk of colliding with observing tasks). | ||
| - | * After you have created FluxTable.txt, | ||
| - | * The doFluxCal program will pop up several plots for you to check the quality of the data. If you see obvious problems (like an antenna is bad or a time chunk is bad), stop the script, flag the data and restart the script. However, you should not spend too long on a given track, so if the flagging is going to be involved or the data seem suspect for some other reason, just ignore that track and move on. | ||
| - | |||
| - | ===== Getting Set-Up ===== | ||
| - | |||
| - | - ...on the high-site machines | ||
| - | * Make your own directory (e.g. amberb/) | ||
| - | * Copy necessary files from ~/amberb/: | ||
| - | * getLastN.csh | ||
| - | * writeCommands.csh | ||
| - | * all .py files: | ||
| - | * fileIOPython.py | ||
| - | * GetNewCalFlux.py | ||
| - | * shortCuts.py | ||
| - | * FindSource.py | ||
| - | * pymiriad.py | ||
| - | * shortNew.py | ||
| - | * Edit GetNewCalFlux.py to change directory (' | ||
| - | 42: fluxFile = '/ | ||
| - | 94: os.system(" | ||
| - | 261: | ||
| - | - CVS on your local machine (according to Amber' | ||
| - | - If you have a real install of Miriad from CVS, you may already be set up | ||
| - | * Look at the end of the {{: | ||
| - | * Contact Peter Teuben if you have problems | ||
| - | - If you have Peter W's Miriad MacPorts install or if you don't have Miriad, then you can just do a CVS checkout of the catalogs folder without getting all of Miriad: | ||
| - | * First, put the appropriate CVSROOT command in your .cshrc or .bashrc file. For .cshrc, \\ < | ||
| - | setenv CVSROOT : | ||
| - | setenv CVSEDITOR emacs | ||
| - | </ | ||
| - | export CVSROOT=: | ||
| - | export CVSEDITOR=emacs | ||
| - | </ | ||
| - | * Then type < | ||
| - | * Finally, you can check out the current version of the catalogs directory of Miriad. Simply go to the directory where you want to put it (e.g. ~/ | ||
| - | ===== Trouble-Shooting ===== | ||
| - | |||
| - | * Script crashes: | ||
| - | * Check that your primary flux calibrator name (MARS, NEPTUNE, etc.) is in all caps...don' | ||
| - | * Check that you have no spaces in your GetNewCalFlux.doFluxCal call... python seems to hate spaces? | ||
