How to transport observation data

This page is intended to inform observers on how to take their observation data back home (last review: 14-09-08).

Overview

During an observing run at the IRAM 30mRT, considerable amount of data may be produced, in particular when the FTS backend is used. You can check the disk space:

PR=000-14
# log into your project 
slogin $PR@mrt-lx3
# complete project:
du -sh 
# complete observation data (incl. IMBFits files):
du -sh observationData
# calibrated data:
du -sh observationData/mira

In order to carry the data, we have the following alternatives:

Frequently Asked Questions

Which data shall I copy ?

In your project account you find:

The imbfits directory (2) might contain much more data than the rest and the data is only needed if you plan to recalibrate the data. In case the data does not fit on your media, do not copy the raw data using option "--exclude "*imb.fits" with rsync command.

How to Copy Data to USB Memory, USB Disk, or Laptop (Linux)

You can connect external memory stick or disk to the desktop computers available to visitors (gra-lx14, gra-lx3, mrt-lx7, mrt-* in visitor office). These computers are prepared to allow users to mount external devices. We recommend that you use the rsync utility to transfer files from the project directories to the external memory, example:

# set your project
PR=000-13
# cd to the directory where you want to copy the data to
# 
# Example copying the complete project directory tree including 30m files, but not copying IMB-Fits raw data:
rsync -rtvlH --safe-links --bwlimit 2000 --exclude '*imb\.fits*' $PR@mrt-lx3:/vis/$PR/* ./
# Example copying the complete project directory tree including 30m and IMB-Fits raw data:
rsync -rtvlH --safe-links --bwlimit 2000 $PR@mrt-lx3:/vis/$PR/* ./

Note: you can do this as often as you want, for example after each observing session. (see man rsync)

Which filesystem should I use on an external disk ?

We recommend ext-3 or higher and NTFS.

Can I get a USB disk or memory from IRAM ?

Ask the operator and send it back as soon as possible.

Can IRAM send me the data ?

Yes we can. Fill out the VisitorBackupForm (ask the operator) or contact ncsOperator@iram.es .

How much space for my data do I need ?

Calculate the space you might according to the formula

space per hour of observing time = Backend-Channels * 4Bytes * spectra/sec * 3600

I do not have access rights for my ext* disk

In case of ext* disks, you could ask the operator to create a directory (with your project name) in the root directory of the disk:

# under root@localhost do:
DISK=[disk]
PR=xxx-yy
mkdir /$DISK/$PR
chown $PR /$DISK/$PR
# caution: the line below would allow any user (o=others) to write to this directory
chmod o=rwx /$DISK/$PR

How to Copy data via Internet

You can copy your data via Internet. Please note that our bandwidth is about 100M bits/s. You should limit your transfer speed, e.g., using rsync with option --bwlimit=KBPS limit I/O bandwidth; KBytes per second, example:

# from your home institute:
PR=<your project>
rsync -avH --bwlimit 1000 $PR@enter.iram.es:observationData/* ./$PR 

Page info

VisitorBackup (last edited 2014-09-08 14:42:10 by WalterBrunswig)