From 806b06d1d94d06c2ffa1db66a4ce6757b9886861 Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Mon, 11 Aug 2025 15:46:53 +0300 Subject: [PATCH 01/10] Roihu migration guide From 782575358fcad999a7a52dee0478b53240279364 Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg <43936697+rkronberg@users.noreply.github.com> Date: Mon, 19 Jan 2026 13:35:19 +0200 Subject: [PATCH 02/10] SSH CA (#2810) * SSH CA * headers * add mycsc instructions * add instructions * fix linkl * fix link * extra step * instructions * SSH agent * computing index page * fix link * use snippets * roihu fingerprints for pilot phase * updates * authentication agent for windows * agent forwarding * document python version of cert helper * quickstart guide * add some link --- .../assets/snippets/graphical-connection.md | 4 + .../assets/snippets/ssh-agent-forwarding.md | 4 + csc-overrides/assets/snippets/ssh-ca.md | 8 + .../assets/snippets/using-ssh-keys.md | 8 + docs/computing/connecting/index.md | 60 +++++--- docs/computing/connecting/ssh-keys.md | 128 +++++++++++++++- docs/computing/connecting/ssh-unix.md | 117 +++++++++----- docs/computing/connecting/ssh-windows.md | 143 ++++++++++++------ docs/computing/index.md | 46 +++--- docs/data/moving/disk_mount.md | 2 +- docs/data/moving/rsync.md | 2 +- docs/data/moving/scp.md | 2 +- docs/data/moving/tar_ssh.md | 2 +- docs/support/tutorials/index.md | 5 + docs/support/tutorials/ml-guide.md | 2 +- .../support/tutorials/roihu/data-migration.md | 3 + docs/support/tutorials/roihu/index.md | 94 ++++++++++++ 17 files changed, 503 insertions(+), 127 deletions(-) create mode 100644 csc-overrides/assets/snippets/graphical-connection.md create mode 100644 csc-overrides/assets/snippets/ssh-agent-forwarding.md create mode 100644 csc-overrides/assets/snippets/ssh-ca.md create mode 100644 csc-overrides/assets/snippets/using-ssh-keys.md create mode 100644 docs/support/tutorials/roihu/data-migration.md create mode 100644 docs/support/tutorials/roihu/index.md diff --git a/csc-overrides/assets/snippets/graphical-connection.md b/csc-overrides/assets/snippets/graphical-connection.md new file mode 100644 index 0000000000..d6da1032fd --- /dev/null +++ b/csc-overrides/assets/snippets/graphical-connection.md @@ -0,0 +1,4 @@ +!!! info "Note" + For performance reasons, we generally recommend using the + [HPC web interfaces](/computing/webinterface/index.md) to run applications which + require displaying graphics. diff --git a/csc-overrides/assets/snippets/ssh-agent-forwarding.md b/csc-overrides/assets/snippets/ssh-agent-forwarding.md new file mode 100644 index 0000000000..4637bafd4e --- /dev/null +++ b/csc-overrides/assets/snippets/ssh-agent-forwarding.md @@ -0,0 +1,4 @@ +!!! warning "Note" + You should only forward your SSH agent to remote servers that you trust and + only when you really need it. Forwarding your SSH agent by default to any + server you connect to is considered insecure. diff --git a/csc-overrides/assets/snippets/ssh-ca.md b/csc-overrides/assets/snippets/ssh-ca.md new file mode 100644 index 0000000000..b9e847ee20 --- /dev/null +++ b/csc-overrides/assets/snippets/ssh-ca.md @@ -0,0 +1,8 @@ +!!! warning "SSH certificates are required to connect to Roihu over SSH" + + To connect to Roihu, users must sign their public key in MyCSC to obtain a + time-based SSH certificate. Each certificate is valid for 24 hours, and + once it expires, a new one must be generated by signing the public key + again. + + [Read the detailed instructions on signing your public key](/computing/connecting/ssh-keys.md#signing-public-key). diff --git a/csc-overrides/assets/snippets/using-ssh-keys.md b/csc-overrides/assets/snippets/using-ssh-keys.md new file mode 100644 index 0000000000..0e8975fda5 --- /dev/null +++ b/csc-overrides/assets/snippets/using-ssh-keys.md @@ -0,0 +1,8 @@ +!!! info "Using SSH keys" + See the page on [setting up SSH keys](/computing/connecting/ssh-keys.md) + for general information about using SSH keys and certificates for + authentication. Please note that it is mandatory to add your public key to + MyCSC – copying it directly to a CSC supercomputer does not work! + + Supported key types are Ed25519 and RSA 4096 through 16384. **We strongly + recommend Ed25519**. diff --git a/docs/computing/connecting/index.md b/docs/computing/connecting/index.md index d0086365f4..cbb424e743 100644 --- a/docs/computing/connecting/index.md +++ b/docs/computing/connecting/index.md @@ -1,6 +1,6 @@ # Connecting to CSC supercomputers ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" There are two main ways of connecting to CSC supercomputers. @@ -22,8 +22,7 @@ For instructions on connecting to the LUMI supercomputer, please see the ## Using the web interface The [web interface](../webinterface/index.md) is a good platform -for using graphical applications on the Puhti and Mahti supercomputers. -It hosts +for using graphical applications on CSC supercomputers. It hosts [interactive applications for select programs](../webinterface/apps.md) like Jupyter and RStudio, and for other GUI programs you can use the [remote desktop](../webinterface/desktop.md) interface. @@ -34,15 +33,21 @@ will keep running even if you close your browser or lose your internet connection. The shell applications are especially convenient for users whose workstation has a Windows operating system, since Windows does not typically come with a pre-installed SSH client. See the instructions for -[connecting to Puhti and Mahti web interfaces](../webinterface/connecting.md). +[connecting to HPC web interfaces](../webinterface/connecting.md). ## Using an SSH client -Logging in to Puhti and Mahti using an SSH client requires that you have -[set up SSH keys](ssh-keys.md) and -[added your public key to MyCSC](ssh-keys.md#adding-public-key-in-mycsc). -Traditional password-based authentication and public keys stored in your -personal `~/.ssh/authorized_keys` file will **not** work. +Logging in to CSC supercomputers using an SSH client requires that you have + +1. [set up SSH keys](ssh-keys.md), +2. [added your public key to MyCSC](ssh-keys.md#adding-public-key-in-mycsc), + and +3. [signed your public key](ssh-keys.md#signing-public-key) to + obtain a time-based SSH certificate. **This step is only required when + connecting to Roihu**. + +Please note that traditional password-based authentication and public keys +stored in your personal `~/.ssh/authorized_keys` file will **not** work. Unix-based systems like macOS and Linux typically come with a pre-installed terminal program called simply *Terminal*. The instructions for using an @@ -54,12 +59,13 @@ over SSH, there are multiple programs that can be used for this. The instructions for using an [SSH client on Windows](ssh-windows.md) lists a few popular options. -Once you have set up SSH keys and added your public key to MyCSC, use a -command like below to connect over SSH: +Once you have set up SSH keys, added your public key to MyCSC, and signed it to +generate an SSH certificate (only required for Roihu), use a command like below +to connect over SSH: ```bash # Replace with the name of your CSC user account and -# with "puhti" or "mahti" +# with "puhti", "mahti", "roihu-cpu" or "roihu-gpu" ssh @.csc.fi ``` @@ -106,6 +112,22 @@ should again verify the new key against fingerprints provided by CSC. | WC9Lb5tmKDzUJqsQjaZLvp9T7LTs3aMUYSIy2OCdtgg | ssh_host_ecdsa_key.pub (ECDSA) | | tE+1jA4Et1enbbat1V3dMRWlLtJgA8t7ZrkyIkU4ooo | ssh_host_ed25519_key.pub (ED25519) | | 0CxM3ECpD2LhAnMfHnm3YaXresvHrhW4cevvcPb+HNw | ssh_host_rsa_key.pub (RSA) | +=== "Roihu (pilot phase)" + | SHA256 checksum | Key | + |---------------------------------------------|------------------------------------| + | NnNuy5xLxXDhDyBTVCtRbGNSMmTTKdnH6dlomerCg14 | ssh_host_ecdsa_key.pub (ECDSA) | + | mAkMF6xpb4wc1eq+vPc4q4mo7YvcL4GHxe8XauPqGas | ssh_host_ed25519_key.pub (ED25519) | + | IHUo4GZOYH8V9qlcv155iP3w/83SdlS6E2jOb/z01hE | ssh_host_rsa_key.pub (RSA) | +=== "Roihu (general availability)" + | SHA256 checksum | Key | + |---------------------------------------------|------------------------------------| + | h3YVzmNucpxTXcxag8D2TaC21jH8/6LGNNCCOgRDaTU | ssh_host_ecdsa_key.pub (ECDSA) | + | YNdesHbXhxN0hKD4mWvYGQONebjRqY+CGXDqPiZyByQ | ssh_host_ed25519_key.pub (ED25519) | + | cXJ5h3Z9fgu0wVpC2kDIpjdsrFsJF/bfyWegQXsfQpU | ssh_host_rsa_key.pub (RSA) | + +!!! info "Note" + For security reasons, Roihu host keys will be changed after the pilot + phase. ### Graphical connection @@ -125,17 +147,17 @@ the login nodes on the system. However, you can also use your SSH client to connect to a specific login node: ```bash -ssh @-login.csc.fi # e.g. 'puhti-login11.csc.fi' +ssh @-login.csc.fi # e.g. 'roihu-gpu-login1.csc.fi' ``` The available login nodes are: -| Puhti | Mahti | -|-|-| -| `puhti-login11` | `mahti-login11` | -| `puhti-login12` | `mahti-login12` | -| `puhti-login14` | `mahti-login14` | -| `puhti-login15` | `mahti-login15` | +| Puhti | Mahti | Roihu CPU | Roihu GPU | +|-|-|-|-| +| `puhti-login11` | `mahti-login11` | `roihu-cpu-login1` | `roihu-gpu-login1` | +| `puhti-login12` | `mahti-login12` | `roihu-cpu-login2` | `roihu-gpu-login2` | +| `puhti-login14` | `mahti-login14` | `roihu-cpu-login3` | | +| `puhti-login15` | `mahti-login15` | `roihu-cpu-login4` | | This also applies to compute nodes, although just the ones where you have a job running. Use the `squeue` command to see which node(s) your job is on, and diff --git a/docs/computing/connecting/ssh-keys.md b/docs/computing/connecting/ssh-keys.md index 9acb9fd7f1..8e46880409 100644 --- a/docs/computing/connecting/ssh-keys.md +++ b/docs/computing/connecting/ssh-keys.md @@ -1,10 +1,11 @@ # Setting up SSH keys ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" [SSH keys](https://www.ssh.com/academy/ssh-keys) provide more convenient and -secure authentication. Setting them up is a two-step process, and is required -to be able to connect to CSC supercomputers using an SSH client. +secure authentication. SSH keys are required to be able to connect to CSC +supercomputers using an SSH client. Connecting to Roihu requires also that you +sign your public key in order to obtain a time-based SSH certificate. 1. [Generate SSH keys on your local workstation](#generating-ssh-keys). - SSH keys are always generated in pairs consisting of one _public key_ and @@ -16,6 +17,12 @@ to be able to connect to CSC supercomputers using an SSH client. the _public key_ to MyCSC. **Do not copy the private key.** Note that copying the public key directly to CSC supercomputers using tools such as `ssh-copy-id` will not work. +3. [Sign the public key in MyCSC and download SSH certificate](#signing-public-key) (required for Roihu only). + - To connect to Roihu, sign your public key in MyCSC to generate a + time-based SSH certficate that is used for authentication. SSH + certificates have a finite lifetime of 24 hours, which significantly + improves the security of the system. After the SSH certificate expires, a + new one must be generated by signing the public key in MyCSC again. For more information about SSH keys, see: @@ -103,7 +110,122 @@ cat /var/lib/acco/sshkeys/${USER}/${USER}.pub If you have added multiple keys to MyCSC, they should all be visible in the same `${USER}.pub` file. +## Signing public key + +!!! info "The following is a requirement for connecting to Roihu only" + +To connect to Roihu using SSH, you must sign your public key to get a so called +**SSH certificate**. SSH certificates significantly improve the security of the +system by introducing an additional authentication factor for SSH logins. + +**SSH certficates are valid for 24 hours at a time**. Once your certificate +expires, a new one must be signed following either of the processes below. + +### Option 1: Certificate helper tool + +The certificate helper is a tool developed by CSC to simplify the process of +signing and downloading SSH certificates. It is a Python script, so please +ensure that you have Python installed on your computer. Installation +instructions are available in the +[Python Beginners Guide](https://wiki.python.org/moin/BeginnersGuide/Download). +Contact your local IT-support if you need assistance. If Python for some reason +cannot be installed on your computer, fall back to [Option 2](#option-2-mycsc) +instead. + +1. Download the certificate helper tool here. +2. Run the tool: + + === "Linux & macOS" + + 1. Open terminal and execute: + + ```bash + # Replace with your CSC user name and + # with the path to your SSH public key + + python3 csc-cert.py -u + ``` + + 2. If you have an earlier certificate which is still valid, the tool + exits. + 3. If signing is needed, a login URL is displayed. Follow the link and + authenticate. + 4. Copy the 6-digit code displayed into your terminal and enter your + SSH key passphrase. The signed certificate is then downloaded and + added to your SSH agent. The signed certificate is saved as + `-cert.pub` (e.g., `~/.ssh/id_ed25519-cert.pub`). + 5. Each SSH certificate is valid for 24 hours. The expiration time can + be checked by running the tool again. + + === "Windows" + + 1. Optional, but helpful: + [Install WinSCP](https://winscp.net/eng/docs/installation) and + [start the Pageant authentication agent](https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter9.html#pageant) + that comes bundled with PuTTY to automatically add SSH key and + certificate to SSH agent. + 2. Open PowerShell and execute: + + ```bash + # Replace with your CSC user name and + # with the path to your SSH public key + + python3 csc-cert.py -u + ``` + + 3. If you have an earlier certificate which is still valid, the tool + exits. + 4. If signing is needed, a login URL is displayed. Follow the link and + authenticate. + 5. Copy the displayed 6-digit code into PowerShell and enter your SSH + key passphrase. The signed certificate is then downloaded (and added + to your SSH agent if you have WinSCP installed and Pageant running). + The signed certificate is saved as `-cert.pub` (e.g., + `C:\Users\\.ssh\id_ed25519-cert.pub`). + 6. Each SSH certificate is valid for 24 hours. The expiration time can + be checked by running the tool again. + +### Option 2: MyCSC + +1. Log in to MyCSC with your CSC or Haka/Virtu credentials. +2. Select _Profile_ from the left-hand navigation or the dropdown menu in the + top-right corner. +3. Locate _SSH PUBLIC KEYS_ section and click the three vertical dots next to + the public key you want to sign. +4. Click _Sign SSH key_. As a security measure, you are asked to log in again. + + ![Sign SSH key](https://a3s.fi/docs-files/sign-ssh-key.png 'Sign SSH key') + +5. Download the certificate by clicking the three vertical dots next to your + public key and selecting _Download SSH certificate_. + + !!! info "Where to store the SSH certificate?" + We **strongly** advice saving the certificate in the default folder for + SSH-related files (e.g. `~/.ssh`). Specifically, storing the + certificate in the same directory as your SSH private key **and** + naming it as `-cert.pub` will simplify connecting, working with + SSH agent, etc. + + For example, if you've stored your SSH private key in + `~/.ssh/id_ed25519`, please save your SSH certificate as + `~/.ssh/id_ed25519-cert.pub` + + ![Download SSH certificate](https://a3s.fi/docs-files/download-ssh-cert.png 'Download SSH certificate') + +6. Each SSH certificate is valid for 24 hours. The expiration time can be + checked with command (Linux, macOS, MobaXterm terminal, PowerShell): + + ```bash + ssh-keygen -L -f | grep "Valid" + ``` + + If using PuTTY or MobaXterm GUI, the validity period can be checked by + [adding the certificate to your private key in PuTTYgen or MobaKeyGen](ssh-windows.md#authentication-agent) + (look for _Certificate info_). + ## More information - [Tutorial on setting up SSH keys at CSC](https://csc-training.github.io/csc-env-eff/hands-on/connecting/ssh-keys.html) - [Troubleshooting issues with SSH keys](../../support/faq/ssh-keys-not-working.md) +- [Connecting to CSC supercomputers with SSH on Linux and macOS](ssh-unix.md) +- [Connecting to CSC supercomputers with SSH on Windows](ssh-windows.md) diff --git a/docs/computing/connecting/ssh-unix.md b/docs/computing/connecting/ssh-unix.md index 2d4a81f1da..78e69473ce 100644 --- a/docs/computing/connecting/ssh-unix.md +++ b/docs/computing/connecting/ssh-unix.md @@ -1,6 +1,6 @@ # SSH client on macOS and Linux ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" On Unix-based systems like macOS and Linux, it is recommended to connect to CSC supercomputers using the pre-installed terminal program. The OpenSSH client @@ -8,14 +8,7 @@ typically comes pre-installed on macOS and Linux systems. ## Generating SSH keys -!!! info "Using SSH keys" - See the page on [setting up SSH keys](ssh-keys.md) for general - information about using SSH keys for authentication. Please note that it is - mandatory to add your public key to MyCSC – copying it directly to a CSC - supercomputer does not work! - - Supported key types are Ed25519 and RSA 4096 through 16384. **We strongly - recommend Ed25519**. +--8<-- "using-ssh-keys.md" Connecting to CSC supercomputers using an SSH client requires setting up SSH keys. On macOS and Linux, you can use the `ssh-keygen` command-line utility for @@ -38,7 +31,7 @@ Overwrite (y/n)? Generally, you do not want to overwrite existing keys, so enter `n`, run `ssh-keygen` again and enter a different file name when prompted. See also the section on -[SSH key files with non-default name or location](#ssh-key-file-with-non-default-name-or-location). +[SSH key files with non-default name or location](#ssh-key-or-certificate-file-with-non-default-name-or-location). Next, you will be asked for a passphrase. Please choose a secure passphrase. It should be at least 8 characters long and contain numbers, @@ -47,48 +40,72 @@ generating an SSH key pair!** After you have generated an SSH key pair, you need to add the **public key** to the MyCSC portal. -[Read the instructions here](ssh-keys.md#adding-public-key-in-mycsc). +[Read the instructions here](ssh-keys.md#adding-public-key-in-mycsc). To +connect to Roihu, you must also +[sign your public key](ssh-keys.md#signing-public-key) to obtain a time-based +SSH certificate which is required for authentication. You may also wish to configure [authentication agent](#authentication-agent) to make using SSH keys more convenient. ## Basic usage -After setting up SSH keys and adding your public key to MyCSC, you can create a -remote SSH connection by opening the terminal and running: +After setting up SSH keys, adding your public key to MyCSC, and downloading an +SSH certificate (**required for Roihu only**), you can create a remote SSH +connection by opening the terminal and running: ```bash # Replace with the name of your CSC user account and -# with "puhti" or "mahti" +# with "puhti", "mahti", "roihu-cpu" or "roihu-gpu" ssh @.csc.fi ``` -### SSH key file with non-default name or location +This assumes that the SSH keys (and certificate for Roihu) are saved in a standard +location using standard naming: + +- Private key: `~/.ssh/id_` +- Public key: `~/.ssh/id_.pub` +- Certificate: `~/.ssh/id_-cert.pub` + +where `` is either `ed25519` or `rsa`. -If you have stored your SSH key file with a non-default name or in a -non-default location (somewhere else than `~/.ssh/id_`), you must -tell the `ssh` command where to look for the key. Use option `-i` as follows: +### SSH key or certificate file with non-default name or location + +If you have stored your SSH key and/or certificate file with a non-default name +or in a non-default location, you must tell the `ssh` command where to look for +these files. Use option `-i` as follows: ```bash # Replace with the name of your CSC user account, -# with "puhti" or "mahti" and -# with the path to your SSH private key +# with "puhti", "mahti", "roihu-cpu" or "roihu-gpu", +# with the path to your SSH private key and +# with the path to your SSH certificate file (Roihu only) -ssh @.csc.fi -i +ssh @.csc.fi -i -i ``` Alternatively, you may specify the key location in the `~/.ssh/config` file: ```bash -Host .csc.fi +Host HostName .csc.fi User IdentityFile + CertificateFile +``` + +The `~/.ssh/config` file above would allow you to log in to `` simply +using: + +```bash +ssh ``` ## Graphical connection +--8<-- "graphical-connection.md" + Displaying graphics, such as GUIs and plots, over an SSH connection requires a window system. Linux systems have a server program for the X window system (X11) installed by default. On macOS you need to install one separately, for @@ -125,27 +142,52 @@ Assuming your SSH private key is stored in `~/.ssh/id_ed25519`, add it to the authentication agent by running: ```bash -ssh-add ~/.ssh/id_ed25519 +$ ssh-add ~/.ssh/id_ed25519 +Enter passphrase for ~/.ssh/id_ed25519: # enter key passphrase here +Identity added: ~/.ssh/id_ed25519 +Certificate added: ~/.ssh/id_ed25519-cert.pub ``` +!!! info "Note regarding authentication agent and SSH certificates" + Please observe that your SSH certificate is only added to the agent if it + is stored in the same directory as your private key **and** named as + `-cert.pub`. In this case, `ssh-add` will output: + + ```bash + Certificate added: ~/.ssh/id_ed25519-cert.pub + ``` + + If the certificate is stored and/or named in any other way, it **cannot** + be added to the authentication agent because OpenSSH uses hard-coded naming + conventions. + + This is not an issue if you specify the custom path to the SSH certificate + [as outlined above](#ssh-key-or-certificate-file-with-non-default-name-or-location). + However, if you intend to connect to Roihu via a jump host (e.g. another + CSC supercomputer), also the SSH certificate must be added to the agent so + that it can be properly forwarded. [Read more below](#ssh-agent-forwarding). + +If you are using +[CSC's certificate helper tool](ssh-keys.md#option-1-certificate-helper-tool) +to sign and download SSH certificates, all of the above will be taken care of +automatically. + For more information about `ssh-agent`, see the [relevant SSH Academy tutorial](https://www.ssh.com/academy/ssh/agent). ### SSH agent forwarding -!!! warning "Note" - You should only forward your SSH agent to remote servers that you trust and - only when you really need it. Forwarding your SSH agent by default to any - server you connect to is considered insecure. +--8<-- "ssh-agent-forwarding.md" Agent forwarding is a useful mechanism where the SSH client is configured to allow an SSH server to use your local `ssh-agent` on the server as if it was local there. This means in practice that you can, for example, connect directly -from Puhti to Mahti using the SSH keys you have set up on your local machine, -i.e. you do not need to create a new set of SSH keys on CSC supercomputers. +between CSC supercomputers using the SSH keys (and certificates) you have on +your local machine, i.e. you do not need to create a new set of SSH keys on CSC +supercomputers. -Agent forwarding is also very handy if you need to copy data between Puhti and -Mahti, or, for example, push to a private Git repository from CSC +Agent forwarding is also very handy if you need to copy data directly between +CSC supercomputers, or, for example, push to a private Git repository from CSC supercomputers. To enable agent forwarding, include the `-A` flag to your `ssh` command: @@ -157,8 +199,15 @@ ssh -A @.csc.fi Once connected, you may verify that SSH agent forwarding worked by running: ```bash -ssh-add -l +$ ssh-add -l ``` -If you see the fingerprint(s) of your SSH key(s) listed, agent forwarding is -working. +If you see the fingerprint(s) of your SSH key(s) and certificate(s) listed, +agent forwarding is working. Associated SSH keys and certificates in the +authentication agent have the same fingerprints and are annotated with +`` and `-CERT`, respectively. For example: + +```text +256 SHA256:ZXG7TvhDAWOv8VveFAlt/UYarsO9Nx5md4owX+FE5/M optional_comment (ED25519) +256 SHA256:ZXG7TvhDAWOv8VveFAlt/UYarsO9Nx5md4owX+FE5/M optional_comment (ED25519-CERT) +``` diff --git a/docs/computing/connecting/ssh-windows.md b/docs/computing/connecting/ssh-windows.md index 2fd454f0d8..2d3870e54e 100644 --- a/docs/computing/connecting/ssh-windows.md +++ b/docs/computing/connecting/ssh-windows.md @@ -1,6 +1,6 @@ # SSH client on Windows ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" There are various programs that can be used for creating a remote SSH connection on a Windows system. This page provides instructions for three @@ -8,14 +8,7 @@ popular alternatives: MobaXterm, PuTTY and PowerShell. ## Generating SSH keys -!!! info "Using SSH keys" - See the page on [setting up SSH keys](ssh-keys.md) for general - information about using SSH keys for authentication. Please note that it is - mandatory to add your public key to MyCSC – copying it directly to a CSC - supercomputer does not work! - - Supported key types are Ed25519 and RSA 4096 through 16384. **We strongly - recommend Ed25519**. +--8<-- "using-ssh-keys.md" === "MobaXterm" @@ -43,7 +36,7 @@ popular alternatives: MobaXterm, PuTTY and PowerShell. Generally, you do not want to overwrite existing keys, so enter `n`, run `ssh-keygen` again and enter a different file name when prompted. See also the section on - [SSH key files with non-default name or location](#ssh-key-file-with-non-default-name-or-location). + [SSH key files with non-default name or location](#ssh-key-or-certificate-file-with-non-default-name-or-location). Next, you will be asked for a passphrase. Please choose a secure passphrase. It should be at least 8 characters long and contain numbers, @@ -104,7 +97,7 @@ popular alternatives: MobaXterm, PuTTY and PowerShell. Generally, you do not want to overwrite existing keys, so enter `n`, run `ssh-keygen` again and enter a different file name when prompted. See also the section on - [SSH key files with non-default name or location](#ssh-key-file-with-non-default-name-or-location). + [SSH key files with non-default name or location](#ssh-key-or-certificate-file-with-non-default-name-or-location). Next, you will be asked for a passphrase. Please choose a secure passphrase. It should be at least 8 characters long and contain numbers, @@ -115,16 +108,19 @@ popular alternatives: MobaXterm, PuTTY and PowerShell. After you have generated an SSH key pair, you need to add the **public key** to the MyCSC portal. -[Read the instructions here](ssh-keys.md#adding-public-key-in-mycsc). +[Read the instructions here](ssh-keys.md#adding-public-key-in-mycsc). To +connect to Roihu, you must also +[sign your public key](ssh-keys.md#signing-public-key) to obtain a time-based +SSH certificate which is required for authentication. -You may also wish to configure -[authentication agent](#authentication-agent) to make using SSH keys -more convenient. +You may also wish to configure [authentication agent](#authentication-agent) to +make using SSH keys more convenient. ## Basic usage -After setting up SSH keys and adding your public key to MyCSC, you can connect -to a CSC supercomputer. +After setting up SSH keys, adding your public key to MyCSC and downloading an +SSH certificate (**required for Roihu only**) you can connect to a CSC +supercomputer. === "MobaXterm" @@ -132,11 +128,20 @@ to a CSC supercomputer. ```bash # Replace with the name of your CSC user account and - # with "puhti" or "mahti" + # with "puhti", "mahti", "roihu-cpu" or "roihu-gpu" ssh @.csc.fi ``` + This assumes that the SSH keys (and certificate for Roihu) are saved in a standard + location using standard naming: + + - Private key: `~/.ssh/id_` + - Public key: `~/.ssh/id_.pub` + - Certificate: `~/.ssh/id_-cert.pub` + + where `` is either `ed25519` or `rsa`. + Alternatively, you may [connect using the GUI following this tutorial](https://csc-training.github.io/csc-env-eff/hands-on/connecting/ssh-puhti.html#connecting-from-windows). @@ -151,13 +156,13 @@ to a CSC supercomputer. | **Port** | `22` | | **Connection type** | `SSH` | - When creating a remote connection using PuTTY, select the private key file - under `Connection --> SSH --> Auth --> Credentials`. If you want the private - key to be used each time you connect, save your session to store your choice. - Finally, click `Open` and enter your CSC username and SSH key passphrase. + When creating a remote connection using PuTTY, select the private key and + certificate file (**only if connecting to Roihu**) under + `Connection --> SSH --> Auth --> Credentials`. Finally, click `Open` and + enter your CSC username and SSH key passphrase. - If you are connecting for the first time, PuTTY will ask if you trust the host. - Click `Accept`. + If you are connecting for the first time, PuTTY will ask if you trust the + host. Click `Accept`. === "PowerShell" @@ -165,11 +170,20 @@ to a CSC supercomputer. ```bash # Replace with the name of your CSC user account and - # with "puhti" or "mahti" + # with "puhti", "mahti", "roihu-cpu" or "roihu-gpu" ssh @.csc.fi ``` + This assumes that the SSH keys (and certificate for Roihu) are saved in a standard + location using standard naming: + + - Private key: `~/.ssh/id_` + - Public key: `~/.ssh/id_.pub` + - Certificate: `~/.ssh/id_-cert.pub` + + where `` is either `ed25519` or `rsa`. + !!! warning "Corrupted MAC on input" When connecting using the OpenSSH client software on Windows, you might encounter an error stating "Corrupted MAC on input". This is a known @@ -179,27 +193,25 @@ to a CSC supercomputer. --- -### SSH key file with non-default name or location +### SSH key or certificate file with non-default name or location If you are connecting via the MobaXterm terminal or PowerShell, and have stored -your SSH key file with a non-default name or in a non-default location -(somewhere else than `~/.ssh/id_`), you must tell the `ssh` command -where to look for the key. Use option `-i` as follows: +your SSH key and/or certificate file with a non-default name or in a +non-default location (somewhere else than `~/.ssh/id_`), you must +tell the `ssh` command where to look for these files. Use option `-i` as +follows: ```bash # Replace with the name of your CSC user account, # with "puhti" or "mahti" and # with the path to your SSH private key -ssh @.csc.fi -i +ssh @.csc.fi -i -i ``` ## Graphical connection -!!! info "Note" - For performance reasons, we generally recommend using the - [HPC web interfaces](../webinterface/index.md) to run applications which - require displaying graphics. +--8<-- "graphical-connection.md" === "MobaXterm" @@ -251,12 +263,31 @@ ssh @.csc.fi -i 1. Toggle the option `Use internal SSH agent "MobAgent"`. 2. Click the `+` button and select the private key you want to load at - MobAgent startup. + MobAgent startup. 3. Click `OK` and restart MobaXterm. You'll be prompted to enter your key passphrase. 4. You may now connect to CSC supercomputers without having to type your passphrase again. + **To connect to Roihu**, you may also with add your SSH certificate to the + agent. In this case, you must first "combine" the certificate and the SSH + private key. + + 1. Open MobaKeyGen from the Tools tab. + 2. Load your private key (`File --> Load private key`). + 3. Add a valid certificate to the key (`Key --> Add certificate to key`). + The validity period can be checked by selecting `Certificate info`. + 4. Save the private key and restart MobaXterm. + 5. Your private key including the certificate is now loaded into the agent + and you can sign in to Roihu either using the local terminal or the GUI + without having to type your SSH passphrase. + + If you're using the local terminal instead of the MobaXterm GUI, you can + also simply use the OpenSSH commands to add your keys and certificates to + the authentication agent. In this case, start the SSH agent by running + `eval "$(ssh-agent -s)"` and follow the + [instructions for Linux](ssh-unix.md#authentication-agent). + === "PuTTY" To avoid having to type your passphrase every time you connect, you can use @@ -276,6 +307,20 @@ ssh @.csc.fi -i Pageant, and use it to authenticate. You may now open as many PuTTY sessions as you like without having to type your passphrase again. + **To connect to Roihu**, you may also with add your SSH certificate to the + agent. In this case, you must first "combine" the certificate and the SSH + private key. + + 1. Open PuTTYgen. + 2. Load your private key (`File --> Load private key`). + 3. Add a valid certificate to the key (`Key --> Add certificate to key`). + The validity period can be checked by selecting `Certificate info`. + 4. Save the private key and add it to Pageant following the steps above. A + successfully combined key and certificate will show up as `Ed25519 cert` + in Pageant. + 5. You can now sign in to Roihu using PuTTY without having to type your SSH + passphrase. + === "PowerShell" To avoid having to type your passphrase every time you connect, @@ -287,19 +332,17 @@ ssh @.csc.fi -i ### SSH agent forwarding -!!! warning "Note" - You should only forward your SSH agent to remote servers that you trust and - only when you really need it. Forwarding your SSH agent by default to any - server you connect to is considered insecure. +--8<-- "ssh-agent-forwarding.md" Agent forwarding is a useful mechanism where the SSH client is configured to allow an SSH server to use your local `ssh-agent` on the server as if it was local there. This means in practice that you can, for example, connect directly -from Puhti to Mahti using the SSH keys you have set up on your local machine, -i.e. you do not need to create a new set of SSH keys on CSC supercomputers. +between CSC supercomputers using the SSH keys (and certificates) you have on +your local machine, i.e. you do not need to create a new set of SSH keys on CSC +supercomputers. -Agent forwarding is also very handy if you need to copy data between Puhti and -Mahti, or, for example, push to a private Git repository from CSC +Agent forwarding is also very handy if you need to copy data directly between +CSC supercomputers, or, for example, push to a private Git repository from CSC supercomputers. === "MobaXterm" @@ -338,5 +381,15 @@ Once connected, you may verify that SSH agent forwarding worked by running: ssh-add -l ``` -If you see the fingerprint(s) of your SSH key(s) listed, agent forwarding is -working. +If you see the fingerprint(s) of your SSH key(s) and certificate(s) listed, +agent forwarding is working. Associated SSH keys and certificates in the +authentication agent have the same fingerprints and are annotated with +`` and `-CERT`, respectively. For example: + +```text +256 SHA256:ZXG7TvhDAWOv8VveFAlt/UYarsO9Nx5md4owX+FE5/M optional_comment (ED25519) +256 SHA256:ZXG7TvhDAWOv8VveFAlt/UYarsO9Nx5md4owX+FE5/M optional_comment (ED25519-CERT) +``` + +If you're using a combined SSH key and certificate file (PuTTYgen and +MobaKeyGen methods), you should only see the `` line. diff --git a/docs/computing/index.md b/docs/computing/index.md index f02e2004e6..6b073095a7 100644 --- a/docs/computing/index.md +++ b/docs/computing/index.md @@ -12,7 +12,7 @@ [Learn more about Roihu :material-arrow-right:](systems-roihu.md) -Puhti and Mahti are CSC's supercomputers. Puhti has been available for CSC users +Puhti, Mahti and Roihu are CSC's supercomputers. Puhti has been available for CSC users since 2 September 2019 and Mahti has been available since 26 August 2020. LUMI is one of the pan-European pre-exascale supercomputers, located in CSC's data center in Kajaani. The CPU partition of LUMI (LUMI-C) has been available since @@ -35,8 +35,7 @@ basics of [Linux command line usage](../support/tutorials/env-guide/index.md) be For an overview of the LUMI supercomputer, see [the LUMI documentation](https://docs.lumi-supercomputer.eu/hardware/). - -## Accessing Puhti and Mahti +## Accessing CSC supercomputers To be able to use CSC's supercomputers, you need to have a CSC user account that belongs to a computing project which has access to the respective supercomputers. @@ -52,41 +51,46 @@ of this user guide. ## Connecting to the supercomputers ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" Connect using an SSH client: ```bash -ssh yourcscusername@puhti.csc.fi +ssh username@puhti.csc.fi +# or +ssh username@mahti.csc.fi ``` -or +Roihu has separate login nodes for the CPU and GPU partitions: ```bash -ssh yourcscusername@mahti.csc.fi +ssh username@roihu-cpu.csc.fi +# or +ssh username@roihu-gpu.csc.fi ``` This will connect you to one of the login nodes. If you need to connect to a specific login node, use the command: ```bash -ssh yourcscusername@puhti-login[11-12,14-15].csc.fi -``` - -or - -```bash -ssh yourcscusername@mahti-login[11-12,14-15].csc.fi +ssh username@puhti-login[11-12,14-15].csc.fi +# or +ssh username@mahti-login[11-12,14-15].csc.fi +# or +ssh username@roihu-cpu-login[1-4].csc.fi +# or +ssh username@roihu-gpu-login[1-2].csc.fi ``` -Where `yourcscusername` is the username you get from CSC. +Where `username` is the username you get from CSC. For more details, see the [connecting](connecting/index.md) page. -Puhti and Mahti can also be accessed via their respective +Puhti, Mahti and Roihu can also be accessed via their respective [web interfaces](webinterface/index.md) available at -[www.puhti.csc.fi](https://www.puhti.csc.fi) and -[www.mahti.csc.fi](https://www.mahti.csc.fi). +[www.puhti.csc.fi](https://www.puhti.csc.fi), +[www.mahti.csc.fi](https://www.mahti.csc.fi) and +[www.roihu.csc.fi](https://www.mahti.csc.fi). ### Scalability @@ -144,13 +148,13 @@ The [disk areas](disk.md) of your projects can be checked with the command: csc-workspaces ``` -## Using Puhti and Mahti +## Using CSC supercomputers * [Systems](available-systems.md): What computational resources are available * [Usage policy](usage-policy.md): Usage policy of CSC supercomputers * [Connecting](connecting/index.md): How to connect to CSC supercomputers -* [Puhti web interface](webinterface/index.md): How to connect to Puhti using the web - interface +* [Web interfaces](webinterface/index.md): How to connect to CSC supercomputers + using the web interfaces * [Disk areas](disk.md): What places are there for storing data on CSC supercomputers * [Modules](modules.md): How to find the programs you need diff --git a/docs/data/moving/disk_mount.md b/docs/data/moving/disk_mount.md index 247bf73a5c..2aec86b981 100644 --- a/docs/data/moving/disk_mount.md +++ b/docs/data/moving/disk_mount.md @@ -1,6 +1,6 @@ # Remote disk mounts ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" With remote disk mounts you can access your CSC directories in a way that resembles the usage of an external disk or USB memory stick. Using this diff --git a/docs/data/moving/rsync.md b/docs/data/moving/rsync.md index 9ebe2260bb..853b87367d 100644 --- a/docs/data/moving/rsync.md +++ b/docs/data/moving/rsync.md @@ -1,6 +1,6 @@ # Using rsync for data transfer and synchronization ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" **Rsync** is a data transfer tool that can be used much like the `scp` command. When transferring data, `rsync` checks the difference between the source and diff --git a/docs/data/moving/scp.md b/docs/data/moving/scp.md index 04cb316d3b..dda890b154 100644 --- a/docs/data/moving/scp.md +++ b/docs/data/moving/scp.md @@ -1,6 +1,6 @@ # Copying files using scp ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" Copying files between different Linux, macOS and Windows machines can be done with the `scp` command. Thus, you can use `scp` to transport data between CSC diff --git a/docs/data/moving/tar_ssh.md b/docs/data/moving/tar_ssh.md index f3ec1f5daa..0d4509cdb1 100644 --- a/docs/data/moving/tar_ssh.md +++ b/docs/data/moving/tar_ssh.md @@ -1,6 +1,6 @@ # Using Tar over SSH to move many files ---8<-- "auth-update-ssh.md" +--8<-- "ssh-ca.md" Linux tools such as `scp` and `rsync` are commonly used to transfer files between a remote server and a local machine. However, these tools are not diff --git a/docs/support/tutorials/index.md b/docs/support/tutorials/index.md index a94ac9f7aa..6022ce0036 100644 --- a/docs/support/tutorials/index.md +++ b/docs/support/tutorials/index.md @@ -17,6 +17,11 @@ * [Using Python on CSC supercomputers](python-usage-guide.md) * [Setting up SSH keys at CSC](https://csc-training.github.io/csc-env-eff/hands-on/connecting/ssh-keys.html) +## Roihu + +* [Getting started with Roihu](roihu/index.md) +* [Roihu data migration guide](roihu/data-migration.md) + ## Installation of tools on supercomputers * [Installing software with Spack](user-spack.md) diff --git a/docs/support/tutorials/ml-guide.md b/docs/support/tutorials/ml-guide.md index fa1a3901e9..15e4240374 100644 --- a/docs/support/tutorials/ml-guide.md +++ b/docs/support/tutorials/ml-guide.md @@ -65,7 +65,7 @@ European LUMI supercomputer. If you are [unsure which supercomputer to choose, read the discussion here](gpu-ml.md#puhti-mahti-or-lumi). If you are a new user, please read [how to access Puhti and -Mahti](../../computing/index.md#accessing-puhti-and-mahti), and [how +Mahti](../../computing/index.md#accessing-csc-supercomputers), and [how to submit computing jobs](../../computing/running/getting-started.md). If you have opted for LUMI read the [LUMI Get Started diff --git a/docs/support/tutorials/roihu/data-migration.md b/docs/support/tutorials/roihu/data-migration.md new file mode 100644 index 0000000000..df2987de92 --- /dev/null +++ b/docs/support/tutorials/roihu/data-migration.md @@ -0,0 +1,3 @@ +# Roihu data migration guide + +TBA \ No newline at end of file diff --git a/docs/support/tutorials/roihu/index.md b/docs/support/tutorials/roihu/index.md new file mode 100644 index 0000000000..2ebe957819 --- /dev/null +++ b/docs/support/tutorials/roihu/index.md @@ -0,0 +1,94 @@ +# Getting started with Roihu + +This is a quickstart guide for Roihu users. It is assumed that you have +previously used CSC supercomputing resources like Puhti, Mahti or LUMI. If not, +you can start by looking at our general +[getting started with supercomputing guide](../hpc-quick.md). We also recommend +checking the +[CSC Computing Environment self-learning course materials](https://csc-training.github.io/csc-env-eff/). + +To access Roihu, you need a CSC user account and project that has Roihu service +enabled. [Read more here](../../../accounts/index.md). + +[TOC] + +## Connecting + +Connect to Roihu using either: + +* [SSH client](#ssh-client) +* [Roihu web interface](#roihu-web-interface) + +### SSH client + +Connecting to Roihu using an SSH client requires that you have: + +1. Set up SSH keys and added your public key to MyCSC (like on Puhti & Mahti). +2. **New:** _Signed_ your public key and downloaded a _certificate_ that allows + authenticating. + * Each certificate is valid for 24 hours, after which a new one must be + generated. + +Read the detailed +[instructions for managing SSH keys and certificates](../../../computing/connecting/ssh-keys.md). + +Once you have set up SSH keys and obtained a valid SSH certificate, connect +using an SSH client: + +* [Instructions for Linux/macOS](../../../computing/connecting/ssh-unix.md). +* [Instructions for Windows](../../../computing/connecting/ssh-windows.md). + +!!! info "Note: Roihu has separate login nodes for CPU and GPU partitions" + Roihu has + [different CPU architectures on the CPU and GPU nodes](../../../computing/systems-roihu.md#compute). + Hence, there are separate login nodes for building and running programs on + the respective nodes: `roihu-cpu.csc.fi` and `roihu-gpu.csc.fi`. For + example, connect to one of the CPU login nodes using a command-line SSH + client like this: + + ```bash + # Replace with the name of your CSC user account. + + ssh @roihu-cpu.csc.fi + ``` + + Please observe that software built on `roihu-cpu.csc.fi` can only be run on + the CPU nodes, while software built on `roihu-gpu.csc.fi` can only be run + on the GPU nodes. Importantly, this applies also to Python environments. + +### Roihu web interface + +The simplest way to connect to Roihu is to use the web interface. + +1. Go to [www.roihu.csc.fi](https://www.roihu.csc.fi). +2. Log in using your Haka, Virtu or CSC user account. + [Multi-factor authentication (MFA)](../../../accounts/mfa.md) is required. + +## Migrating research data + +If you need to transfer research data from Puhti or Mahti to Roihu, we +recommend that you: + +1. Carefully review your data before transferring it – **only move what you + really need!** Notably, previous extended disk quotas on Puhti or Mahti will + not be automatically moved to Roihu. Quota extensions on Roihu must be + separately applied for and properly motivated. +2. Move your data _directly_ from Puhti or Mahti to Roihu for performance and + capacity management reasons. + +[Read our separate Roihu data migration guide here](data-migration.md). + +## Installing software + +Document: compilers, MPI, HPC libraries, pre-installed applications, CPU/GPU +architecture considerations + +## Running your first job + +Document: module system, Slurm queues, performance considerations, link to +examples + +## More information + +* [Roihu system overview](../../../computing/systems-roihu.md) +* [CSC Computing Environment self-learning materials](https://csc-training.github.io/csc-env-eff/) From 509a339921887302671e1d4792fd650e041c22b8 Mon Sep 17 00:00:00 2001 From: Kylli Ek Date: Mon, 19 Jan 2026 19:06:49 +0200 Subject: [PATCH 03/10] Add graphics how SSH login works Just a try to add something graphical. Would need at least some text around etc, but remove if not needed. --- docs/computing/connecting/index.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/computing/connecting/index.md b/docs/computing/connecting/index.md index cbb424e743..3dc57ffa60 100644 --- a/docs/computing/connecting/index.md +++ b/docs/computing/connecting/index.md @@ -80,6 +80,18 @@ working on the Linux command-line can be found in our [Linux basics tutorial for CSC](../../support/tutorials/env-guide/index.md). You can have several connections to CSC supercomputers open at the same time. +```mermaid +%%{init: {'theme': 'default', }}%% +flowchart TD + A(Set up SSH keys, + before first usage) + A -->| | B(Get SSH certificate, + **every day**) + B -->| |C(Connect to supercomputer: + - Linux/Max + - Windows) +``` + ### First connection When connecting to a given supercomputer for the first time, the SSH client From 1690ff7ee70f513c712fb8e0c18a1ed37d86a36e Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Wed, 21 Jan 2026 11:38:16 +0200 Subject: [PATCH 04/10] implement suggestions --- docs/computing/connecting/index.md | 29 +++--- docs/computing/connecting/ssh-keys.md | 93 +++++++++++-------- docs/computing/connecting/ssh-windows.md | 2 +- docs/computing/index.md | 4 +- docs/support/tutorials/index.md | 4 +- .../data-migration.md => roihu-data.md} | 0 .../tutorials/{roihu/index.md => roihu.md} | 0 7 files changed, 78 insertions(+), 54 deletions(-) rename docs/support/tutorials/{roihu/data-migration.md => roihu-data.md} (100%) rename docs/support/tutorials/{roihu/index.md => roihu.md} (100%) diff --git a/docs/computing/connecting/index.md b/docs/computing/connecting/index.md index 3dc57ffa60..625e4b6cbd 100644 --- a/docs/computing/connecting/index.md +++ b/docs/computing/connecting/index.md @@ -42,9 +42,10 @@ Logging in to CSC supercomputers using an SSH client requires that you have 1. [set up SSH keys](ssh-keys.md), 2. [added your public key to MyCSC](ssh-keys.md#adding-public-key-in-mycsc), and -3. [signed your public key](ssh-keys.md#signing-public-key) to - obtain a time-based SSH certificate. **This step is only required when - connecting to Roihu**. +3. [signed your public key](ssh-keys.md#signing-public-key) to obtain a + time-based SSH certificate. + * The final step is only required when connecting to Roihu and must be + repeated every 24 hours. Please note that traditional password-based authentication and public keys stored in your personal `~/.ssh/authorized_keys` file will **not** work. @@ -80,16 +81,20 @@ working on the Linux command-line can be found in our [Linux basics tutorial for CSC](../../support/tutorials/env-guide/index.md). You can have several connections to CSC supercomputers open at the same time. +In summary: + ```mermaid -%%{init: {'theme': 'default', }}%% -flowchart TD - A(Set up SSH keys, - before first usage) - A -->| | B(Get SSH certificate, - **every day**) - B -->| |C(Connect to supercomputer: - - Linux/Max - - Windows) +flowchart LR + A(**Before first connection:** + Set up SSH keys) + A --> B{Connecting + to Roihu?} + B -->|yes| C(**Once every 24 hours:** + Get a new SSH certificate) + C --> D(SSH with Linux/macOS + or + SSH with Windows) + B -->|no| D ``` ### First connection diff --git a/docs/computing/connecting/ssh-keys.md b/docs/computing/connecting/ssh-keys.md index 8e46880409..56fa5d34ba 100644 --- a/docs/computing/connecting/ssh-keys.md +++ b/docs/computing/connecting/ssh-keys.md @@ -17,9 +17,9 @@ sign your public key in order to obtain a time-based SSH certificate. the _public key_ to MyCSC. **Do not copy the private key.** Note that copying the public key directly to CSC supercomputers using tools such as `ssh-copy-id` will not work. -3. [Sign the public key in MyCSC and download SSH certificate](#signing-public-key) (required for Roihu only). +3. [Sign the public key in MyCSC and download SSH certificate](#signing-public-key) (**required for Roihu only**). - To connect to Roihu, sign your public key in MyCSC to generate a - time-based SSH certficate that is used for authentication. SSH + time-based SSH certificate that is used for authentication. SSH certificates have a finite lifetime of 24 hours, which significantly improves the security of the system. After the SSH certificate expires, a new one must be generated by signing the public key in MyCSC again. @@ -123,17 +123,19 @@ expires, a new one must be signed following either of the processes below. ### Option 1: Certificate helper tool -The certificate helper is a tool developed by CSC to simplify the process of -signing and downloading SSH certificates. It is a Python script, so please -ensure that you have Python installed on your computer. Installation -instructions are available in the -[Python Beginners Guide](https://wiki.python.org/moin/BeginnersGuide/Download). -Contact your local IT-support if you need assistance. If Python for some reason -cannot be installed on your computer, fall back to [Option 2](#option-2-mycsc) -instead. +The certificate helper is a Python tool developed by CSC to simplify the +process of signing and downloading SSH certificates. A detailed documentation +of the tool is available in the [source repository](https://github.com/CSCfi/). +The following instructions illustrate only basic usage. -1. Download the certificate helper tool here. -2. Run the tool: +1. Ensure that you have Python installed on your computer. + - Instructions are available in the + [Python Beginners Guide](https://wiki.python.org/moin/BeginnersGuide/Download). + Contact your local IT-support if you need assistance. + - If Python for some reason cannot be installed on your computer, fall + back to [Option 2](#option-2-mycsc) instead. +2. [Download the certificate helper tool here](https://github.com/CSCfi/). +3. Run the tool: === "Linux & macOS" @@ -147,41 +149,48 @@ instead. ``` 2. If you have an earlier certificate which is still valid, the tool - exits. + exits. 3. If signing is needed, a login URL is displayed. Follow the link and - authenticate. + authenticate. 4. Copy the 6-digit code displayed into your terminal and enter your - SSH key passphrase. The signed certificate is then downloaded and - added to your SSH agent. The signed certificate is saved as - `-cert.pub` (e.g., `~/.ssh/id_ed25519-cert.pub`). + SSH key passphrase. + - The signed certificate is automatically downloaded and added to + your SSH agent. The signed certificate is saved as + `-cert.pub` (e.g., `~/.ssh/id_ed25519-cert.pub`). 5. Each SSH certificate is valid for 24 hours. The expiration time can be checked by running the tool again. === "Windows" 1. Optional, but helpful: - [Install WinSCP](https://winscp.net/eng/docs/installation) and - [start the Pageant authentication agent](https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter9.html#pageant) - that comes bundled with PuTTY to automatically add SSH key and - certificate to SSH agent. + [Install WinSCP](https://winscp.net/eng/docs/installation) and + [start the Pageant authentication agent](https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter9.html#pageant) + that comes bundled with PuTTY to automatically add SSH key and + certificate to SSH agent. 2. Open PowerShell and execute: ```bash # Replace with your CSC user name and # with the path to your SSH public key - + python3 csc-cert.py -u ``` - 3. If you have an earlier certificate which is still valid, the tool - exits. - 4. If signing is needed, a login URL is displayed. Follow the link and - authenticate. + !!! info "Note" + PowerShell is just needed to run the certificate + helper script. You can still connect to Roihu using your + [favorite SSH client](ssh-windows.md#basic-usage). + + 3. If you have an earlier certificate which is still valid, the tool + exits. + 4. If signing is needed, a login URL is displayed. Follow the link and + authenticate. 5. Copy the displayed 6-digit code into PowerShell and enter your SSH - key passphrase. The signed certificate is then downloaded (and added - to your SSH agent if you have WinSCP installed and Pageant running). - The signed certificate is saved as `-cert.pub` (e.g., - `C:\Users\\.ssh\id_ed25519-cert.pub`). + key passphrase. + - The signed certificate is automatically downloaded and added to + your SSH agent (if you have WinSCP installed and Pageant + running). The signed certificate is saved as `-cert.pub` + (e.g., `C:\Users\\.ssh\id_ed25519-cert.pub`). 6. Each SSH certificate is valid for 24 hours. The expiration time can be checked by running the tool again. @@ -213,15 +222,25 @@ instead. ![Download SSH certificate](https://a3s.fi/docs-files/download-ssh-cert.png 'Download SSH certificate') 6. Each SSH certificate is valid for 24 hours. The expiration time can be - checked with command (Linux, macOS, MobaXterm terminal, PowerShell): + checked as follows: - ```bash - ssh-keygen -L -f | grep "Valid" - ``` + === "Terminal (Linux, macOS, PowerShell, MobaXterm)" + + 1. Open a terminal client. + 2. Run command: + + ```bash + ssh-keygen -L -f | grep "Valid" + ``` + + === "GUI (PuTTY, MobaXterm)" + + 3. Open PuTTYgen / MobaKeyGen. + 4. Load your private key: _File_ :material-arrow-right: _Load private key_. + 5. Add a certificate to the key: _Key_ :material-arrow-right: _Add certificate to key_. + 6. Select _Certificate info_ to see the validity period among other info. - If using PuTTY or MobaXterm GUI, the validity period can be checked by - [adding the certificate to your private key in PuTTYgen or MobaKeyGen](ssh-windows.md#authentication-agent) - (look for _Certificate info_). + --- ## More information diff --git a/docs/computing/connecting/ssh-windows.md b/docs/computing/connecting/ssh-windows.md index 2d3870e54e..60935e4772 100644 --- a/docs/computing/connecting/ssh-windows.md +++ b/docs/computing/connecting/ssh-windows.md @@ -392,4 +392,4 @@ authentication agent have the same fingerprints and are annotated with ``` If you're using a combined SSH key and certificate file (PuTTYgen and -MobaKeyGen methods), you should only see the `` line. +MobaKeyGen methods), you should only see the `-CERT` line. diff --git a/docs/computing/index.md b/docs/computing/index.md index 6b073095a7..2d18bddfd2 100644 --- a/docs/computing/index.md +++ b/docs/computing/index.md @@ -69,8 +69,8 @@ ssh username@roihu-cpu.csc.fi ssh username@roihu-gpu.csc.fi ``` -This will connect you to one of the login nodes. If you need to connect -to a specific login node, use the command: +These commands will connect you to one of the login nodes. If you need to +connect to a specific login node, use the commands: ```bash ssh username@puhti-login[11-12,14-15].csc.fi diff --git a/docs/support/tutorials/index.md b/docs/support/tutorials/index.md index 6022ce0036..9027d551d5 100644 --- a/docs/support/tutorials/index.md +++ b/docs/support/tutorials/index.md @@ -19,8 +19,8 @@ ## Roihu -* [Getting started with Roihu](roihu/index.md) -* [Roihu data migration guide](roihu/data-migration.md) +* [Getting started with Roihu](roihu.md) +* [Roihu data migration guide](roihu-data.md) ## Installation of tools on supercomputers diff --git a/docs/support/tutorials/roihu/data-migration.md b/docs/support/tutorials/roihu-data.md similarity index 100% rename from docs/support/tutorials/roihu/data-migration.md rename to docs/support/tutorials/roihu-data.md diff --git a/docs/support/tutorials/roihu/index.md b/docs/support/tutorials/roihu.md similarity index 100% rename from docs/support/tutorials/roihu/index.md rename to docs/support/tutorials/roihu.md From 81c0952f5a2dd69d0013935c138d1f7a13469754 Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Wed, 21 Jan 2026 11:43:47 +0200 Subject: [PATCH 05/10] fix links --- docs/computing/connecting/ssh-keys.md | 2 ++ docs/support/tutorials/roihu.md | 14 +++++++------- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/computing/connecting/ssh-keys.md b/docs/computing/connecting/ssh-keys.md index 56fa5d34ba..c6a56ab667 100644 --- a/docs/computing/connecting/ssh-keys.md +++ b/docs/computing/connecting/ssh-keys.md @@ -194,6 +194,8 @@ The following instructions illustrate only basic usage. 6. Each SSH certificate is valid for 24 hours. The expiration time can be checked by running the tool again. +--- + ### Option 2: MyCSC 1. Log in to MyCSC with your CSC or Haka/Virtu credentials. diff --git a/docs/support/tutorials/roihu.md b/docs/support/tutorials/roihu.md index 2ebe957819..a0873cb900 100644 --- a/docs/support/tutorials/roihu.md +++ b/docs/support/tutorials/roihu.md @@ -8,7 +8,7 @@ checking the [CSC Computing Environment self-learning course materials](https://csc-training.github.io/csc-env-eff/). To access Roihu, you need a CSC user account and project that has Roihu service -enabled. [Read more here](../../../accounts/index.md). +enabled. [Read more here](../../accounts/index.md). [TOC] @@ -30,17 +30,17 @@ Connecting to Roihu using an SSH client requires that you have: generated. Read the detailed -[instructions for managing SSH keys and certificates](../../../computing/connecting/ssh-keys.md). +[instructions for managing SSH keys and certificates](../../computing/connecting/ssh-keys.md). Once you have set up SSH keys and obtained a valid SSH certificate, connect using an SSH client: -* [Instructions for Linux/macOS](../../../computing/connecting/ssh-unix.md). -* [Instructions for Windows](../../../computing/connecting/ssh-windows.md). +* [Instructions for Linux/macOS](../../computing/connecting/ssh-unix.md). +* [Instructions for Windows](../../computing/connecting/ssh-windows.md). !!! info "Note: Roihu has separate login nodes for CPU and GPU partitions" Roihu has - [different CPU architectures on the CPU and GPU nodes](../../../computing/systems-roihu.md#compute). + [different CPU architectures on the CPU and GPU nodes](../../computing/systems-roihu.md#compute). Hence, there are separate login nodes for building and running programs on the respective nodes: `roihu-cpu.csc.fi` and `roihu-gpu.csc.fi`. For example, connect to one of the CPU login nodes using a command-line SSH @@ -62,7 +62,7 @@ The simplest way to connect to Roihu is to use the web interface. 1. Go to [www.roihu.csc.fi](https://www.roihu.csc.fi). 2. Log in using your Haka, Virtu or CSC user account. - [Multi-factor authentication (MFA)](../../../accounts/mfa.md) is required. + [Multi-factor authentication (MFA)](../../accounts/mfa.md) is required. ## Migrating research data @@ -90,5 +90,5 @@ examples ## More information -* [Roihu system overview](../../../computing/systems-roihu.md) +* [Roihu system overview](../../computing/systems-roihu.md) * [CSC Computing Environment self-learning materials](https://csc-training.github.io/csc-env-eff/) From 20083ff63c2d927cfd6faa0b42012ae03883e0ea Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Wed, 21 Jan 2026 11:48:44 +0200 Subject: [PATCH 06/10] fix links --- docs/support/tutorials/roihu.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/support/tutorials/roihu.md b/docs/support/tutorials/roihu.md index a0873cb900..f32d28d0cf 100644 --- a/docs/support/tutorials/roihu.md +++ b/docs/support/tutorials/roihu.md @@ -3,7 +3,7 @@ This is a quickstart guide for Roihu users. It is assumed that you have previously used CSC supercomputing resources like Puhti, Mahti or LUMI. If not, you can start by looking at our general -[getting started with supercomputing guide](../hpc-quick.md). We also recommend +[getting started with supercomputing guide](hpc-quick.md). We also recommend checking the [CSC Computing Environment self-learning course materials](https://csc-training.github.io/csc-env-eff/). @@ -76,7 +76,7 @@ recommend that you: 2. Move your data _directly_ from Puhti or Mahti to Roihu for performance and capacity management reasons. -[Read our separate Roihu data migration guide here](data-migration.md). +[Read our separate Roihu data migration guide here](roihu-data.md). ## Installing software From 61e974019c101661c600c48228fc3fe7d6f5e983 Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Wed, 21 Jan 2026 12:24:15 +0200 Subject: [PATCH 07/10] move diagram up --- docs/computing/connecting/index.md | 42 +++++++++++++----------------- 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/docs/computing/connecting/index.md b/docs/computing/connecting/index.md index 625e4b6cbd..421f8bc316 100644 --- a/docs/computing/connecting/index.md +++ b/docs/computing/connecting/index.md @@ -44,9 +44,23 @@ Logging in to CSC supercomputers using an SSH client requires that you have and 3. [signed your public key](ssh-keys.md#signing-public-key) to obtain a time-based SSH certificate. - * The final step is only required when connecting to Roihu and must be + * Step 3. is only required when connecting to Roihu and must be repeated every 24 hours. +```mermaid +flowchart LR + A(**Before first connection:** + Set up SSH keys) + A --> B{Connecting + to Roihu?} + B -->|yes| C(**Once every 24 hours:** + Get a new SSH certificate) + C --> D(SSH with Linux/macOS + or + SSH with Windows) + B -->|no| D +``` + Please note that traditional password-based authentication and public keys stored in your personal `~/.ssh/authorized_keys` file will **not** work. @@ -81,22 +95,6 @@ working on the Linux command-line can be found in our [Linux basics tutorial for CSC](../../support/tutorials/env-guide/index.md). You can have several connections to CSC supercomputers open at the same time. -In summary: - -```mermaid -flowchart LR - A(**Before first connection:** - Set up SSH keys) - A --> B{Connecting - to Roihu?} - B -->|yes| C(**Once every 24 hours:** - Get a new SSH certificate) - C --> D(SSH with Linux/macOS - or - SSH with Windows) - B -->|no| D -``` - ### First connection When connecting to a given supercomputer for the first time, the SSH client @@ -203,9 +201,11 @@ supercomputers in an [SSH config file](https://www.ssh.com/academy/ssh/config) (e.g. `~/.ssh/config`). ```bash -Host # e.g. "puhti" +Host # e.g. "roihu-cpu" HostName .csc.fi User + IdentityFile + CertificateFile # Required for Roihu only ``` Now you can connect to the host simply by running: @@ -213,9 +213,3 @@ Now you can connect to the host simply by running: ```bash ssh ``` - -#### Remote development - -Some editors like Visual Studio Code and Notepad++ can be used to -[work on files remotely](../../support/tutorials/remote-dev.md) -using an appropriate plugin. **However, this is not recommended.** From aa4062cc4053f3660878d5d63cbb1a0bcf0fb0ff Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Wed, 28 Jan 2026 15:59:24 +0200 Subject: [PATCH 08/10] migration guide: general notes --- docs/support/tutorials/roihu-data.md | 41 +++++++++++++++++++++++++++- docs/support/tutorials/roihu.md | 11 ++++---- 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/support/tutorials/roihu-data.md b/docs/support/tutorials/roihu-data.md index df2987de92..c179f221e2 100644 --- a/docs/support/tutorials/roihu-data.md +++ b/docs/support/tutorials/roihu-data.md @@ -1,3 +1,42 @@ # Roihu data migration guide -TBA \ No newline at end of file +## General guidelines + +1. **Review and clean up your data.** + * Like on Puhti and Mahti, Roihu scratch disk is not intended for long-term + data storage, but should only be used for data that is in active use. + Thus, only move data that you truly need. + * Roihu will implement a similar disk cleaning policy as Puhti, meaning + that files that have not been accessed in 180 days will be deleted. + +2. **Ensure that you have enough disk space available on Roihu.** + * After you have identified the data you need to transfer, check that it + fits within the default disk quotas on Roihu: + + | Disk area | Default size | Maximum size | Default file number limit | Maximum file number limit | + |-----------|-------------:|-------------:|--------------------------:|--------------------------:| + | home | 15 GiB | 15 GiB | 150k | 150k | + | projappl | 15 GiB | 250 GiB (< 100 GiB) | 150k | 2.5M (< 1M) | + | scratch  | 1 TiB | 100 TiB (< 20 TiB) | 1M | 10M (< 5M) | + + Values is parentheses indicate automatically approved limits. + + * Please note that existing quota extensions on Puhti/Mahti will not + automatically carry over to Roihu, so you must separately apply for + increased disk quota via MyCSC beforehand if your data does not fit + within the default limits. + +3. **Please transfer your data directly from Puhti and/or Mahti to Roihu.** + * For performance and capacity management reasons, it is best to avoid + transferring data to Roihu via Allas or your local workstation. Instead, + CSC recommends using command-line based tools such as `rsync` or `rclone` + to directly push or pull data from Puhti/Mahti to Roihu. + * Moving data directly from Puhti/Mahti to Roihu requires forwarding your + SSH agent including a valid SSH certificate to the system on which the + data transfer process is initiated. + +## Small data + +## Medium data + +## Large data \ No newline at end of file diff --git a/docs/support/tutorials/roihu.md b/docs/support/tutorials/roihu.md index f32d28d0cf..f281dc0b18 100644 --- a/docs/support/tutorials/roihu.md +++ b/docs/support/tutorials/roihu.md @@ -66,13 +66,14 @@ The simplest way to connect to Roihu is to use the web interface. ## Migrating research data -If you need to transfer research data from Puhti or Mahti to Roihu, we -recommend that you: +If you need to transfer research data from Puhti or Mahti to Roihu, we require +that you: 1. Carefully review your data before transferring it – **only move what you - really need!** Notably, previous extended disk quotas on Puhti or Mahti will - not be automatically moved to Roihu. Quota extensions on Roihu must be - separately applied for and properly motivated. + really need and check that you have enough space available on Roihu!** + Notably, previous extended disk quotas on Puhti or Mahti will not be + automatically moved to Roihu. Quota extensions on Roihu must be separately + applied for and properly motivated. 2. Move your data _directly_ from Puhti or Mahti to Roihu for performance and capacity management reasons. From 0ac39c35f3661d767a67d20a1270d477b099b4bc Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Fri, 6 Feb 2026 15:07:57 +0200 Subject: [PATCH 09/10] adjust auto-approved limits --- docs/support/tutorials/roihu-data.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/support/tutorials/roihu-data.md b/docs/support/tutorials/roihu-data.md index c179f221e2..14ffbb00a4 100644 --- a/docs/support/tutorials/roihu-data.md +++ b/docs/support/tutorials/roihu-data.md @@ -13,11 +13,11 @@ * After you have identified the data you need to transfer, check that it fits within the default disk quotas on Roihu: - | Disk area | Default size | Maximum size | Default file number limit | Maximum file number limit | - |-----------|-------------:|-------------:|--------------------------:|--------------------------:| - | home | 15 GiB | 15 GiB | 150k | 150k | - | projappl | 15 GiB | 250 GiB (< 100 GiB) | 150k | 2.5M (< 1M) | - | scratch  | 1 TiB | 100 TiB (< 20 TiB) | 1M | 10M (< 5M) | + | Disk area | Default size | Maximum size | Default file number limit | Maximum file number limit | + |-----------|-------------:|--------------------:|--------------------------:|--------------------------:| + | home | 15 GiB | 15 GiB | 150k | 150k | + | projappl | 15 GiB | 250 GiB (< 100 GiB) | 150k | 2.5M (< 1M) | + | scratch  | 1 TiB | 100 TiB (< 10 TiB) | 1M | 10M (< 5M) | Values is parentheses indicate automatically approved limits. @@ -39,4 +39,4 @@ ## Medium data -## Large data \ No newline at end of file +## Large data From 5430e543fb123ad288cfd538bf525189fcd1fd19 Mon Sep 17 00:00:00 2001 From: Rasmus Kronberg Date: Fri, 20 Feb 2026 17:10:09 +0200 Subject: [PATCH 10/10] updates --- docs/computing/connecting/ssh-unix.md | 10 +- docs/support/tutorials/roihu-data.md | 181 ++++++++++++++++++++------ 2 files changed, 146 insertions(+), 45 deletions(-) diff --git a/docs/computing/connecting/ssh-unix.md b/docs/computing/connecting/ssh-unix.md index 78e69473ce..492e253f5e 100644 --- a/docs/computing/connecting/ssh-unix.md +++ b/docs/computing/connecting/ssh-unix.md @@ -148,7 +148,7 @@ Identity added: ~/.ssh/id_ed25519 Certificate added: ~/.ssh/id_ed25519-cert.pub ``` -!!! info "Note regarding authentication agent and SSH certificates" +!!! info "Authentication agent and SSH certificates" Please observe that your SSH certificate is only added to the agent if it is stored in the same directory as your private key **and** named as `-cert.pub`. In this case, `ssh-add` will output: @@ -167,10 +167,10 @@ Certificate added: ~/.ssh/id_ed25519-cert.pub CSC supercomputer), also the SSH certificate must be added to the agent so that it can be properly forwarded. [Read more below](#ssh-agent-forwarding). -If you are using -[CSC's certificate helper tool](ssh-keys.md#option-1-certificate-helper-tool) -to sign and download SSH certificates, all of the above will be taken care of -automatically. + **Note! If you are using + [CSC's certificate helper tool](ssh-keys.md#option-1-certificate-helper-tool) + to sign and download SSH certificates, all of the above will be taken care of + automatically.** For more information about `ssh-agent`, see the [relevant SSH Academy tutorial](https://www.ssh.com/academy/ssh/agent). diff --git a/docs/support/tutorials/roihu-data.md b/docs/support/tutorials/roihu-data.md index 14ffbb00a4..5cdfbeea99 100644 --- a/docs/support/tutorials/roihu-data.md +++ b/docs/support/tutorials/roihu-data.md @@ -1,42 +1,143 @@ # Roihu data migration guide -## General guidelines - -1. **Review and clean up your data.** - * Like on Puhti and Mahti, Roihu scratch disk is not intended for long-term - data storage, but should only be used for data that is in active use. - Thus, only move data that you truly need. - * Roihu will implement a similar disk cleaning policy as Puhti, meaning - that files that have not been accessed in 180 days will be deleted. - -2. **Ensure that you have enough disk space available on Roihu.** - * After you have identified the data you need to transfer, check that it - fits within the default disk quotas on Roihu: - - | Disk area | Default size | Maximum size | Default file number limit | Maximum file number limit | - |-----------|-------------:|--------------------:|--------------------------:|--------------------------:| - | home | 15 GiB | 15 GiB | 150k | 150k | - | projappl | 15 GiB | 250 GiB (< 100 GiB) | 150k | 2.5M (< 1M) | - | scratch  | 1 TiB | 100 TiB (< 10 TiB) | 1M | 10M (< 5M) | - - Values is parentheses indicate automatically approved limits. - - * Please note that existing quota extensions on Puhti/Mahti will not - automatically carry over to Roihu, so you must separately apply for - increased disk quota via MyCSC beforehand if your data does not fit - within the default limits. - -3. **Please transfer your data directly from Puhti and/or Mahti to Roihu.** - * For performance and capacity management reasons, it is best to avoid - transferring data to Roihu via Allas or your local workstation. Instead, - CSC recommends using command-line based tools such as `rsync` or `rclone` - to directly push or pull data from Puhti/Mahti to Roihu. - * Moving data directly from Puhti/Mahti to Roihu requires forwarding your - SSH agent including a valid SSH certificate to the system on which the - data transfer process is initiated. - -## Small data - -## Medium data - -## Large data +!!! info "About this guide" + This guide is divided into three main parts: + + 1. [General guidelines and prerequisites](#1-general-guidelines-and-prerequisites) + 2. [Basic data migration](#2-basic-data-migration) + 3. [Advanced cases](#3-advanced-cases) + + If your data transfer needs are small and simple, reading the + [Basic data migration](#2-basic-data-migration) section may be enough. If + you have **a lot** of data or other special requirements, please check the + [Advanced cases](#3-advanced-cases). In any case, please read at least the + [General guidelines and prerequisites](#1-general-guidelines-and-prerequisites) + section before migrating any data to Roihu. + +## 1. General guidelines and prerequisites + +### 1.1 Review and clean up your data before migration + +* Like on Puhti and Mahti, Roihu scratch disk is not intended for long-term + data storage, but should only be used for data that is in active use. + Thus, **only move data that you truly need**. +* Good data hygiene reduces transfer time and load on the file system, as well + as eliminates the risk of moving redundant or duplicate data. +* Roihu will implement a similar disk cleaning policy as Puhti, meaning + that files that have not been accessed in 180 days will be deleted. +* We recommend using the [LUE tool](lue.md) to identify where you have lots of + data. Avoid using other tools such as `du` or `find` as they may cause a lot + of load on the file system. Simple usage example (run `lue -h` for other + options): + + ```bash + module load lue + lue + ``` + +* Other tips: + * Remove or exclude temporary files (cached data, intermediate results, + logs, unused checkpoint files, core dumps, etc.). + * It is best to re-build your applications on Roihu, so do not move + compiled programs or pre-built environments, unless they are + containerized. Note that all applications, including containers, that + target Roihu's ARM-based GPU nodes must be re-built regardless. + +### 1.2 Ensure that you have enough disk space on Roihu + +* Once you have identified the data you need to transfer, check that it + fits within the default disk quotas on Roihu: + + | Disk area | Path | Default size | Max. size [^1] | Default file number limit | Max. file number limit [^1] | + |-----------|-----------------------|-------------:|--------------------:|--------------------------:|----------------------------:| + | home | `/users/$USER` | 15 GiB | 15 GiB | 150k | 150k | + | projappl | `/projappl/` | 15 GiB | 250 GiB (< 100 GiB) | 150k | 2.5M (< 1M) | + | scratch  | `/scratch/` | 1 TiB | 100 TiB (< 10 TiB) | 1M | 10M (< 5M) | + + [^1]: Values in parentheses indicate automatically approved limits. + +* Please note that existing quota extensions on Puhti/Mahti will not + automatically carry over to Roihu, so you must separately + [apply for increased disk quota](../../accounts/how-to-increase-disk-quotas.md) + via [MyCSC](https://my.csc.fi) beforehand if your data does not fit + within the default limits. + +### 1.3. Add Roihu service access to your CSC project + +* Like any other CSC service, access to Roihu must be enabled for your project + via [MyCSC](https://my.csc.fi). +* Also note that each user must have at least a **medium** level of identity + assurance (LoA) to be able to access Roihu. You can check your LoA on your + [profile page in MyCSC](https://my.csc.fi/profile), and + [elevate it if needed following these instructions](../../accounts/strong-identification.md). + +### 1.4 Transfer your data directly from Puhti/Mahti to Roihu + +* For performance and capacity management reasons, it is not recommended to + transfer data to Roihu via Allas or your local workstation. Instead, CSC + recommends using command-line based tools such as + [`rsync`](../../data/moving/rsync.md) to directly push or pull data from + Puhti/Mahti to Roihu. + +### 1.5 Authenticating to Roihu over SSH + +* In addition to SSH keys, a signed SSH certificate is required to authenticate + to Roihu. Each certificate is valid for 24 hours. + * [Read the instructions here](../../computing/connecting/ssh-keys.md#signing-public-key). +* To transfer data directly from Puhti/Mahti to Roihu, you must make sure to + **forward your SSH agent** when connecting to the system on which the data + transfer process is initiated. + * If a data transfer process is launched on Puhti/Mahti (i.e., you _push_ + data from Puhti/Mahti), your forwarded SSH agent must hold your SSH keys + **and** a valid SSH certificate so that a connection to Roihu can be + formed. + * If a data transfer process is launched on Roihu (i.e., you _pull_ data + from Puhti/Mahti), it is enough that your forwarded SSH agent only holds + your SSH keys. However, you still need an SSH certificate to login to + Roihu in the first place (while useful, it does not have to be added to + your SSH agent). + * [SSH agent instructions for Linux/macOS](../../computing/connecting/ssh-unix.md#authentication-agent). + * [SSH agent instructions for Windows](../../computing/connecting/ssh-windows.md#authentication-agent). + +## 2. Basic data migration + +The following tools are suitable if: + +* Your data volume is relatively small (a few tens of GB). +* Number of files is small (a few hundred). +* There are no special requirements related to, for example, preservation of + file permissions or timestamps. + +### 2.1 `rsync` + +### 2.2 `scp` + +[`scp`](../../data/moving/scp.md) is useful only in the most simple cases. +Assuming you've connected to Puhti/Mahti and forwarded your SSH agent +(including SSH keys and a valid SSH certificate), run `scp` for example like +this: + +```bash +scp -r /scratch/project_2001234/folder-to-migrate/ $USER@roihu-cpu.csc.fi:/scratch/project_2001234/ +``` + +* Option `-r` means _recursive_, i.e. the directory `folder-to-migrate` and all + of its contents will be transferred to `/scratch/project_2001234/` on Roihu. + If you're just moving a single file, the `-r` can be left out. + +Assuming you're on Roihu and you want to pull data from, for example, Puhti to +your current working directory on Roihu, run `scp` like this: + +```bash +scp -r $USER@puhti.csc.fi:/scratch/project_2001234/folder-to-migrate/ . +``` + +!!! warning "Note" + Do **not** use `scp` if your data contains a lot of small files. `scp` is + extremely slow for transferring a large number of small files. + +## 3. Advanced cases + +Read more here. + +## 4. Final remarks