Page History

...

wget – retrieves the contents of an Internet URL
cp – copies directories or files located on any local file system
scp – copies directories or files to/from a remote system
rsync – copies directories or files on either local or remote systems

(Read more about Copying files and directories)

TACC storage areas and Linux commands to access data
(all commands to be executed at TACC except
laptop-to-TACC copies, which must be executed on your laptop)

...

Well, you don't have a desktop at TACC to "Save as" to, so what to do with a link? The wget program knows how to access web URLs such as http, https and ftp. Anchorwget yeast datawget yeast data

Get ready to run wget from the directory where you want to put the data.

...

Code Block

language	bash
title	Single file copy with cp

mkdir -p $SCRATCH/data/test1
cp $CORENGS/misc/small.fq  $SCRATCH/data/test1/
ls $SCRATCH/data/test1

# or..
cds
mkdir -p data/test1
cd data/test1
cp $CORENGS/misc/small.fq .

# or..
mkdir -p ~/scratch/data/test1   # use the symbolic link in your Home directory
cd ~/scratch/data/test1
cp $CORENGS/misc/small.fq  .
ls

Notice the different ways of referring to a directory (Read more about using Absolute or Relative pathname syntax.)

Now copy an entire directory to your Scratch area. The -r option says "recursive" recursive.

Code Block

language	bash
title	Directory copy with cp

mkdir -p $SCRATCH/data 
cds
cd data
cp -r $CORENGS/general/ general/

...

Tip

title	Always add a trailing slash ( / ) after directory names

The trailing slash ( / ) on the source and destination directories are very important for rsync( – and for other Linux copy commands also)!

rsync will create the last directory level for you, but earlier levels must already exist.

...

Tip

The bash shell has several convenient line editing features:

use the Up arrow to scroll back through the command line history; Down arrow goes forward
use Ctrl-a to move the cursor to the beginning of a line; Ctrl-e to the enduse Backspace to remove text before the cursor; Delete to remove text after the cursor
Ctrl-k ("kill") to delete all text on your command line after the cursor
Ctrl-y ("yank") to copy the last killed text to where the cursor is

Once the cursor is positioned where you want it:

Just type in any additional text you want
To delete text after the cursor, use:
- Delete key on Windows
- Function-Delete keys on Macintosh
To delete text before the cursor, use:
- Backspace key on Windows
- Delete key on Macintosh

(Read more about Command line history and editing)

Copy from a remote computer - scp or rsync

...

rsync can be run just like before, but using the remote-host syntax. Here we use two tricks:

The tilde ( ~ ) at the start of the path means "relative to my home Home directory"
We use the tilde ( ~ ) in the destination to traverse the ~/scratch symbolic link created in your home directory.

Code Block

language	bash
title	rsync (remote directory)

cat $CORENGS/tacc/dragonfly_access.txt
rsync -avrW corengstools@dragonfly.icmb.utexas.edu:~/custom_tracks/ ~/scratch/data/custom_tracks/

...

Page tree

Versions Compared

Old Version 41

New Version Current

Key

Copy from a remote computer - scp or rsync