CESGA Technical Documentation

• What’s new at CESGA?
  • April 10, 2023: Checkpoint client for VPN connection
  • March 15, 2023: Shutdown of FinisTerrae II
  • March 10, 2023: Announce of 2023 CESGA User Workshop
  • March 9, 2023: cesga/system environment
  • March 6, 2023: Job requeue
  • March 1, 2023: AMD EPYC Nodes
  • December 2, 2022: Intel Cascade Lake nodes

• FinisTerrae III User Guide
  • Overview
    • Node groups description
  • First steps
    • Creating an account
    • Connecting with CESGA
  • HPC Glossary
  • How to connect
    • Installation and configuration of Checkpoint on Windows
    • Installation and configuration of Checkpoint on MacOS
    • Installation and configuration of Checkpoint on Linux
    • FinisTerrae III fingerprints
  • System use
  • Remote desktops and visualization tools
    • Remote desktops
    • Desktops CLI
    • Jupyter notebook
  • Data transfer
    • MobaXterm
    • SCP and SFTP clients
    • DTN nodes
  • Permanent storage
    • Filesystem Quotas
  • Temporary storage (Scratch)
  • Parallelization
    • Single-node parallelism (OpenMP)
    • Multi-node parallelism (MPI)
    • Hybrid MPI/OpenMP
  • Batch system: Submitting jobs with SLURM
    • Memory and time
    • Partitions
    • QOS
      • User Account Limits
      • Assoc-Group Limits
    • Basic commands
      • sbatch
      • srun
      • sinfo
      • squeue
      • scancel
      • scontrol
      • sacct
      • seff
      • sqstat
    • Sample Job Scripts
      • Using srun
      • Using sbatch
      • Using sbatch and GPUs
      • OpenMP job submission
      • MPI job submission
      • Hybrid MPI/OpenMP programs
    • Job array
    • Multiple tasks on a job
      • Running multiple programs with --multi-prog option
      • Running multistep programs and -r, –relative=<n>
      • Running multiple programs within a job using GNU Parallel and srun
    • Tasks on specific cores
      • 1. Task level affinity (Pure MPI)
      • 2. Thread level affinity (OpenMP and MPI/OpenMP jobs)
    • Job states
    • Output files
    • Email notifications
  • Binding: tasks on specific cores
  • Redundant cron
  • Job signals
    • Clean-up function
  • Job Requeue
    • Summary and Generalization
    • Use case
  • Applications, compilers and development tools
  • Environment modules (Lmod)
    • Conda environments
    • Module command use
  • High Memory nodes (fat nodes)
  • AI nodes (GPU nodes)
    • NVIDIA A100
    • Tesla T4
  • High Troughput nodes
    • Intel Cascade Lake nodes
    • AMD EPYC nodes
  • F.A.Q.
    • Connection
    • Use of FinisTerrae III
    • Storage
    • Remote Desktops
  • Dissemination and publicity
  • Further assistance

• CLOUD User Guide
  • Hardware architecture
    • Intel nodes
    • AMD nodes
  • Glossary
  • First steps
  • SSH key pair
    • Create an SSH key pair in OpenStack
    • Create an SSH key pair by yourself
    • Import SSH public key
    • Adding more public keys to authorized_keys
  • Security groups
    • Managing security groups
    • Adding security groups to your VM
  • Volumes
    • Create and attach a volume
    • Snapshots
  • Deploying a Virtual Machine
    • Launch an instance
    • Accesing the VM via SSH
  • Further assintance

• CESGA DTN User Guide
  • Introduction
  • Globus
    • Installation
      • Globus Connect Personal
    • Usage
      • Remote Transfers
  • SCP/SFTP
    • Usage
  • Aspera
    • Installation
    • Usage
  • Known Issues
    • tcllib error when using Anaconda
  • Want to know more

• BD|CESGA Hadoop 3 User Guide
  • Overview
  • Quickstart
  • What’s new
  • Known Issues
  • How to connect
  • BD|CESGA WebUI
  • HUE: A nice graphical interface to Hadoop
  • Migrating Data
  • Filesystem Quotas
  • How to upload data
  • YARN: The Hadoop Batch System
  • HDFS: The Hadoop File System
  • Spark
  • Sparklyr
  • Jupyter Notebooks
  • Hive
  • Impala
  • Sqoop
  • Modules: Additional Software
  • Want to know more
    • Tutorials
    • CDH Documentation
    • Reference Documentation
  • ANNEX I: VPN
    • How to install the VPN software in Windows
    • How to install the VPN sofware in MacOS
    • How to install the VPN software in Linux

• Quantum Random Number Generator (QRNG)
  • Use cases
  • Management
  • Access protocol
  • Access allocation
  • Final report
  • Dissemination and publicity
  • Usage fees

• Quantum Learning Machine (QLM)
  • QLM User Guide
    • System description
    • System connection
    • Data transfer
    • Storage, directories and file system
    • System use: Remote usage (recommended)
    • System use: Local usage
    • Useful commands: qlmaas_prompt.py
    • Useful commands: qlmaas_test.py
  • Annex I: Switch to certificate-based authentication for QLMAAS

• Shutdown systems: FinisTerrae II
  • Overview
  • Users Portal
  • How to connect
  • Data transfer
  • Storage
  • System use
  • Batch system
    • Basic commands
    • Partitions
    • QOS
    • Examples
    • Memory
    • Jobs monitoring
    • Standart output and error
    • Email warning
  • Useful Options
  • Jobs states
  • Compilers and development tools
  • Environment modules (Lmod)
  • Scratch dirs
  • Fat node
  • GPU nodes
  • Want to know more
  • FAQs
  • Annex I: System description
  • Annex II: Infiniband network
  • Annex III: Job arrays
  • Annex IV: Job dependencies
  • Annex V: Multiple tasks on a job
  • Annex VI: Binding: tasks on specific cores
  • Annex VII: Lustre
  • Annex VIII: Drop caches
  • Annex IX: Containers
  • Annex X: Redundant cron
  • Annex XI: Remote desktop on a job
  • Annex XII: Job signals
  • Annex XIII: Data transfer
  • Annex XIV: Hybrid MPI/OpenMP programs execution in batch

Indices and tables

• Index

• Search Page