Computer System User Guide for Observers (NcsUG)
Title: New Control System - Computer System User Guide Identifier - Master URL: https://mrt-doc.iram.es/mainWiki/NcsUG Revision: v 0.27.1 Date: 2015-06-22 Expiration Date: none Supersedes: Author: brunswig(at)iram.es Contributors: sievers(at)iram.es, ungerechts(at)iram.es Audience: observers, operators, AODs Publisher: IRAM, Granada, Spain Subject and Keywords: About this document: Gives information related to the computer system important for observers
Title: New Control System - Computer System User Guide
Identifier - Master URL: https://mrt-doc.iram.es/mainWiki/NcsUG
Revision: v 0.27.1
Expiration Date: none
Contributors: sievers(at)iram.es, ungerechts(at)iram.es
Audience: observers, operators, AODs
Publisher: IRAM, Granada, Spain
Subject and Keywords:
About this document: Gives information related to the computer system important for observers
Computer System User Guide for Observers (NcsUG)
- Related Documents and Web-,Wiki-Pages
- Computer Maintenance
- Computer Account for Projects
- Which Computer to Use
- File Access Policy
- PaKo observing script templates and catalogs
- Observation Data
- Bring your own device (byod)
- Skype for communication between operator, (remote-)observer, and (Granada-)AOD
- Special Observing Modes
- Page Info
The document (from v 0.9 on) uses aliases related to their functions for computer names. See also https://gra-lx1.iram.es/mainWiki/HostNames :
mrt-lx1: observe mrt-doc mrt-remote mrt-lx2: mrt-astro projectArchive mrt-enter mrt-lx3: odp projects pool tapas
For local astronomers: information about tools, etc. is given in the NcsOG
This guide is under development. We appreciate very much any comment and suggestion.
gives special information about the computer systems for observers.
does not give information about how to do and analyze observations. See below for links.
Related Documents and Web-,Wiki-Pages
computers at observer's desk: http://www.iram.es/IRAMES/mainWiki/InformationObservations
NCS documents in http://www.iram.es/IRAMES/ncs30m/ :
Computer System User Guide https://gra-lx1.iram.es/mainWiki/SystemUG
Astronomical Software see: https://gra-lx1.iram.es/mainWiki/SwAstronomical
NCS Monitors: see https://mrt-doc.iram.es/mainWiki/NcsMonitors
NCS Internals: see https://mrt-doc.iram.es/mainWiki/NcsInternals
Remote Observing Guides (DocSet): see https://mrt-doc.iram.es/mainWiki/RemoteObservingDocSet
Monitoring Plots of Telescope Data: see https://gra-lx1.iram.es/munin/iram.es/telescopeStatus/
Computers To Use: ComputerToUse
The server computer at the observatory will normally be stopped for maintenance on Tuesdays between 11:00 and 12:00 !
Computer Account for Projects
For each observing project:
creation: we create a computer account with the same name. The account should normally be available two weeks before the project is scheduled. The PI will be informed per -email and get instructions on how to access from Internet.
availability: the account will be available until the end of the next scheduling period (ex: *-13 until 2015-05). However, your observation data will be kept longer, see below section Observation Data Archive.
technical projects: accounts are only created upon demand.
ReadMeIRAM.txt: check this file in the home directory of the project
How to access from project accounts from Internet ?
You can use:
ssh <project>@mrt-enter.iram.es (slogin, sftp) to have a terminal session, or
x2goclient <project>@mrt-enter.iram.es to have an X-Windows environment (VNC)
Where is my "home" directory
The files of the account are physically on projects:/vis/<project> until about two weeks after the end of the observing session (projects=mrt-lx3).
After that, all files of the project will be moved to the ProjectArchive and the "home directory" will be changed to the ProjectArchive (/prArch/prArch<yy>/closed/nnn-yy/ for project nnn-yy or /prArch/prArch99/closed/<project> for permanent projects). These directories are physically on mrt-astro1(=mrt-lx2).
- You can check your home directory with command:
Note on Projects reloaded from the ProjectArchive
Reloaded Project Accounts: projects that are rescheduled for observation will be copied from the ProjectArchive to the /vis partition. The home directory of rescheduled projects will be set to /mrt-lx3/vis/<project>. However, imbfits files and *.30m are not copied back to /vis/<project>. You can access those files on the ProjectArchive:
PR=<project> YY=<scheduling year> ssh $PR@mrt-astro1; cd /prArch$YY/closed/$PR # example: PR=000-13 YY=13 ssh $PR@mrt-astro1; cd /prArch$YY/closed/$PR
Important: avoid creating files with identical path/name on /vis and in the ProjectArchive. Files on /vis might be overwritten in this case !
Which Computer to Use
for scheduled projects:
preparing scripts,use PaKo,...: slogin <project>@observe
observing slogin <project>@observe
for other projects: slogin <project>@[gra,mrt]-astro
computer monitors that have a sticker terminal:
- you can login into those systems with your project account, but
you should connect to a server (mrt-astro, project, observe,...) to use astronomical software SwAstronomical
- check if your projects is a "scheduled" project with
echo $HOMEIf the "home" directory is [/mrt-lx3]/vis, the project is a scheduled projects (including those that were observed recently).
a list of computer to use is ComputerToUse
We have defined aliases for computers that have a special functionality:
use observe to do observations, also for preparing observations with program PaKo
use mrt-astro to do data analysis (fallback mrt-astro2)
We also have defined:
mrt-remote: use for remote observing, see RemoteObserving
mrt-enter: access from Internet (e.g., with ssh)
gra-astro: data-analysis, astronomical software
See also the "Computer System User Guide" (see #Related).
File Access Policy
Note 13-11-06: We have started to modify projects based on the policy below. You can check this:
ls -ld /mrt-lx3/vis/03*-13 drwxr-xr-x 57 034-13 034-13 4096 Sep 30 17:55 034-13 # old policy drwx--x--- 58 038-13 cosgroup 4096 Nov 6 10:31 038-13 # new policy
Below we refer to cos-group which means the computer-, operator-, and scienceSupport-group and their computer accounts.
- For project accounts that use the new policy (group: cosgroup), the following rules applies:
The files of a project can be accessed:
By persons knowing the password to log into the account. During account creation, the password is sent to the PI:
The PI is entitled to forward the password to co-investigators of the project. Please make sure you do it in a safe way (ex; forward the messages that were sent to the PI).
- You can change the password (not recommended though).
- The members of the cos-group also know the initial password.
The members of the cos-group can read files:
- in the subdirectory observationData
in the subdirectory PaKo
- Members of the cos-group, also might login into the account to help observers or check the account.
We suggest you do private work in a specific subdirectory priv with proteccion u:rwx, g: o: (700). Such a directory could only be accessed from the privileged account root (system adminitration). Feel free to use encryption in addition:
cd ~ mkdir priv chmod -v 700 priv
See also https://gra-lx1.iram.es/mainWiki/SystemUG for the general System User-Guide.
PaKo observing script templates and catalogs
Where is the data stored ?
Please note: Under PaKo, the user shall define the observing project with set project .... The project specified under PaKo shall normally be the same as the computer account used, except for pool observation. The project specified under PaKo is used to record the observation header in TAPAS as the observing project. This means that if the project is not defined under PaKo, you will not find the observation in TAPAS.
The observation data is stored in /vis/<compAcc>/observationData/ (<compAcc> = computer account where PaKo is running!)
<yyyymmdd>/ - scan directories, one subdirectory per scan, see https://mrt-doc.iram.es/mainWiki/FileOrganizationScans
imbfits/ - contains IMBFits files, see https://mrt-doc.iram.es/mainWiki/FileOrganizationIMBFits
mira/ - calibrated data (using MIRA)
nika/, gismo/ - data using those receivers
# enter you project account # calibrated spectra: ls ~/observationData/mira # imbfits files: ls ~/observationData/imbfits
Note for pool:
The IMBFits files are written in the imbfits directory of the computer account of the pool.
If a different project is specified under PaKo, then the .30m files created by ODP (MIRA) are written to /vis/<project>/observationData/mira
Note for TAPAS:
The project specified under PaKo is used to record the
observation header in TAPAS as the observing project. This means that if the project is not defined under PaKo, you will not find the observation in TAPAS.
compAcc = computer account under which Pako is used project = specified under PaKo with ''set project'' command <compAcc> and <project> should be the same except for pools TAPAS: scan information is written using <project>, enter user <project> to get list of scans ODP/MIRA .30m: written to /vis/<project>/observationData/mira/ if directory exists otherwise in /vis/<compAcc>/observationData/mira/ other observationData: written to /vis/<compAcc>/observationData/ ...
How to copy my data to a USB memory or USB disk ?
- you can use
- mrt-vis* computers connected with 100Mb/s or
- mrt-cg (=mrt-lx5 in office AOD), connected with 1Gb/s
- enter your project or the hello account
- connect the device (should be mounted automatically)
- and use for example:
PR=<projectID> BW=10000 rsync -avH --bwlimit $BW $PR@mrt-lx3:/$PR/observationData/mira
BW=10000 (KB/s) would transfer about 36GB/hr, for higher rates contact the systemOperation@iram.es
- consider to run rsync several times (e.g., after each session)
- the computers also support exFat file system that shall be used for Apple devices
devices that have a FAT system cannot store large files (>2GB, quite normal for *FTS*.30m)
How to send data to your "home" computer
If you want us to send you the data, follow instruction given https://mrt-doc.iram.es/mainWiki/VisitorBackupForm .
If you want to copy data via Internet:
PR=000-14 # example, use your project account name instead # use rsync, please limit bandwidth to 1MB/s (~3GB/hr, 10% of our bandwidth to Internet) # from your Internet: rsync -v --bwlimit 1000 $PR@mrt-enter.iram.es:observationData/mira/*.30m <path on your system> # from iram.es LAN rsync -v --bwlimit 1000 ~/observationData/mira/*.30m <path to your system>
- If you have a laptop with your at the observatory, you can also use rsync as shown above
- Note also that you can use rsync several times, e.g., after each observing session. rsync will only transfer new data
- Note also that the files on mrt-enter (mrt-lx2) are "rsynced" every hour from mrt-lx3:/vis .
Observation Data Archive
Policy (v. 2013-08-20)
observations since 2009:
IRAM keeps the final result files of an observation:
in case of calibrated data: CLASS files *.30m
for other receivers: IMBFits files
during the lifetime of the observatory (ProjectArchive).
In case of the calibrated data, the IMBFits files the calibration is based upon will also be available for two years to allow recalibration.
Other files of a project: will normally also be kept for two years, but IRAM might remove files, e.g., copies of large data files, after informing the PI.
observations before 2009: we are in the process of recovering older observations
Details of the Project Archive can be found https://mrt-doc.iram.es/mainWiki/ProjectArchive .
How to recalibrate observations from Internet
For recalibration, log into computer projectArchive.iram.es
that we allow only registered users to access projectArchive.iram.es from Internet.
In case, you cannot access the computer, contact ncsOperator@iram.es .
- You can transfer data then via Internet or contact us to get the data sent to you, see also above.
Scenario: How to access the ProjectArchive from Internet ?
Your project account will normally be available during one more year. Connect to the project archive:
# replace 000-13 by your project ID # connect to .iram.es gateway PR=<your project account> ssh $PR@mrt-enter.iram.es # use data transfer tools (sftp, rsync) to send data to you
For more information about the Project-Archive see https://mrt-doc.iram.es/mainWiki/ProjectArchive.
Use different versions
May13 is now default at the 30-m from 16.07.2013 Use:
- gag_may12, gag_apr11,gag_apr10 - to go back to an older version
- gag_hd - to go forward (gives you a more recent Version (Currently beginning of July 2013))
- gag_may13 - to go back
- PDF manuals on $GAG_ROOT_DIR/doc/pdf
- HTML manuals on $GAG_ROOT_DIR/doc/html
e.g. konqueror $GAG_ROOT_DIR/doc/html/class-html/index.html
displays the CLASS manual.
Bring your own device (byod)
Please note that the use of wlan is not allowed at the observatory. We have an ethernet adapter for Apple devices. Ask the operator.
- We encourage you to bring storage devices (USB disks, laptops, ...) to copy your data. In case of problems, contact the computer group.
See VisitorBackup for more information
Skype for communication between operator, (remote-)observer, and (Granada-)AOD
The Skype software can be used to communicate between the operator, observer, and AOD: We have several skype users defined:
mrt-iram.es: should be started by the operator and allows remote observers, ... to talk to the operator
aod-iram.es: should be started by the AOD (remote or at the observatory)
observer-iram.es: should be started by an observer (remote or at the observatory)
Ask the operator for the password. (issue: http://issues:8000/system/ticket/1055)
In case the AOD is not at the telescope, you should check with the operator how to contact the AOD. You can:
use program x11vnc to allow the remote AOD to monitor or control your desktop (ask the operator how to start x11vnc)
use skype for communication
You can use the following tools under the project account on computer observe:
Plots of az/el tracking are shown on standard monitors. You can also look at the tracking plots of previous scans, example:
# show last scan showTracking # show a previous scan showTracking 1002s1 # scan 1 of 2nd of October # show a specific imbfits file showTracking /ncsServer/mrt/ncs/data/imbfits/het/iram30m-bbc-20140104s60-imb.fits
Besides the standard plot, a second plot which shows the tracking per subscan is created. Enter <return> in the terminal window to get the second plot.
You can see the jobs of your observation queue using command:
remove jobs (using move to trash and then confirm delete)
see its contents, although right now only as xml, by selecting a job and then hitting <space>
You can see the last ODP results with command
Special Observing Modes
Calibration without Sky
Calibrations can be done without moving the antenna. This is useful for tests and receiver tuning. Note: The conditions for observations are checked independent of moving or not moving the antenna. This means a source has to be specified which could be observed (e.g.: >10d elevation).
Please note that we do not create e-mail boxes for project accounts! Our firewall is configured to only forward e-mail sent from our mail server. You cannot send e-mail from a mail server running on your laptop.
See the https://gra-lx1.iram.es/mainWiki/SystemUG (Computer System User Guide).