Environment modules (Lmod)

The module command is the central command that allows the configuration and use of the applications provided by CESGA. It can be use as module or in its short form ml. It sets the appropriate environment variables for a given application independent of the user shell. The available applications are hierarchical and are tied to the preload of a particular combination of dependencies:

  • Compiler: with which the application was compiled.

  • MPI: for MPI applications, the version of MPI used in its compilation.

This tries to minimize incompatibility problems between the different combination of compiler/MPI libraries. Initially, the only modules available are those belonging to the “Core” section where compilers and applications whose use is independent of the compiler/MPI combination are located (applications compiled with the default gcc/glibc version used as the basis for the rest of the compilers and therefore compatible with each other).

In the “ORGS” section, the organizations appear (trees of independent modules to which the user has access). By default the available organizations are:

ORGS

Compiler/default glibc

cesga/2020

Gentoo Prefix: GCC 10.1.0 glibc 2.31

cesga/system

Gentoo Prefix: GCC 10.1.0 glibc 2.31
Cesga/2020 contains the following packages:

  • gcc/system

  • openmpi/4.0.5

  • gcccore/system

  • impi/2021.2.0

  • impi/2021.3.0

  • openmpi/4.1.1_ft3

  • intel/2021.3.0

  • intel/2021.2.0

Cesga/system contains the same packages as cesga/2020 but this new environment has been created to load miniconda3 and prevent compatibility issues with the Gentoo prefix in Cesga/2020. It is strongly recommended to load the cesga/system environment instead of cesga/2020 to ensure optimal performance if you will be using miniconda.

Please refer to the full list of available modules. Kindly note that this list is regularly updated, hence the module you are searching for may have been recently installed. To confirm whether a specific module is on our servers, please use the command module spider <module_name> to verify it.

Conda environments

IMPORTANT: If you need to deploy any conda environment or package, please do not make your own installation of anaconda, miniconda, mamba, mambaforge, etc., as modules serving miniconda3 with both conda and mamba package managers are already installed in our system. You can list all the miniconda3 available modules with module spider miniconda3.

You should bear in mind that this type of installation generates a large number of files that are saved in the $HOME directory by default. They often end up filling up the quota of this directory, so they should NOT be installed there. The appropriate directory to save them is the $STORE directory, so if you are going to need the installation of any conda environment/package, you should use this directory. If the environments and packages are already installed, the most effective way is to move them from $HOME to $STORE as follows:

  • Create a directory in $STORE: mkdir $STORE/.conda

  • Move the envs and pkgs directories from $HOME to $STORE: mv .conda/envs .conda/pkgs $STORE/.conda

  • Edit the .conda/environments.txt file to put the appropriate path to the $STORE (as it currently has the path to $HOME).

  • If you need to clean up conda environments, the command to use is conda clean -a

If you have already worked with Conda environments in your account but without appplying these changes, it is very likely to already have a .conda folder in $HOME. If so, you should move that folder to $STORE with mv $HOME\.conda $STORE\.conda and then make a symbolic link from the new location to the old one with ln -s $STORE\.conda $HOME\.conda. This way Conda still thinks that your .conda folder is in $HOME, but it physically resides in $STORE, which cancels the risk of exhausting your $HOME quotas with large Conda environments (it can still happen in $STORE, but both space and files number quotas are larger there).

If you still have any doubts regarding this matter, you cant contact the Applications Department to help you.

Module command use

-To see all available applications/libraries regardless of the compilers/MPI they depend on, use the command module spider.

-It is possible to search using this command if an application is available with module spider <module_name>. It also contains a brief description and use guide of the corresponding module.

-To load a module to configure the environment for an application use module load intel. When loading the module corresponding to the intel compilers, the applications section corresponding to these compilers is available for loading. The available MPI modules appear. If an MPI module is loaded, for example impi, the available applications section for the compilers (intel) / MPI (impi) combination becomes available.

-Unload module/s: module unload package1 package2 …

-Unload all loaded modules and reset everything to original state: module purge

-Currently loaded modules: module list

-Change the compilers in use: module swap intel gcc. With this command all compiler dependent libraries/applications will be changed. If the applications for the compiler do not exist, the corresponding modules are marked as inactive, disabling their use until the switch back to the original compiler is made.

-For help on using the module command itself: module help

-Modules available under the currently loaded toolchain: module avail