Conversation
… are not equivalent in object storage!)
|
Branch is now ready for review! This is a workflow that we don't plan on supporting indefinitely, but giving it a place to live for future documentation. |
| } | ||
| PUT_DATA_TO_CLOUDIAN(sync_names, params.cloudian_prefix, params.cloudian_config) | ||
|
|
||
| } No newline at end of file |
There was a problem hiding this comment.
file does not end in a newline
|
|
||
| # There are batch files in the folder containing data to process. | ||
| # Batch files are titled batch_#.txt | ||
| BATCH_START=1 |
There was a problem hiding this comment.
BATCH_START doesn't appear to be used elsewhere in the file
gbeane
left a comment
gbeane
left a comment
There was a problem hiding this comment.
I don't know much about Nextflow but LGTM
|
As a note on this branch: I'm currently running stuff that looks at it. No more reviews are necessary (will merge when jobs are complete). |
| // Cloudian will not collapse slashes (it is TRUE object storage). | ||
| // This prefix assumes that the file starts with a slash. | ||
| // Also separated alias and prefix to allow quoting of paths (allows spaces) | ||
| cloudian_config = "/projects/kumar-lab/meta/secrets/tokens/rclone_cloudian.conf" |
There was a problem hiding this comment.
This needs to be updated to new config.
|
|
||
| params { | ||
| input_batch = "" | ||
| dropbox_config = "/projects/kumar-lab/meta/secrets/tokens/rclone_dropbox.conf" |
There was a problem hiding this comment.
This needs to be updated to new config.
3 participants
This adds the effort related to compressing data from one cloud resource and placing it on another.
Rough workflow follows 3 steps: