Skip to content

User Spack#

Introduction#

SCITAS provides its software stack using Spack. However, advanced users may want to build additional software.

It is possible for users to roll their own Spack environment that integrates with the existing Spack provided by SCITAS. The advantage of this approach is that users can leverage the highly optimized stack provided by SCITAS for each cluster, and augment it with additional Spack recipies to add additional software on top. This additional software will then be able to use the required dependencies that are already provided in the SCITAS spack-powered software stack.

To achieve this, the user can install spack in their home directory, and configure the SCITAS stack for a specific cluster as an upstream. This is called chaining, or upstreams in Spack terms.

User spack setup#

First, it is recommended to use the same Spack version that SCITAS is using in their upstream spack. If you were to use a more modern Spack version, it might still work, but any keywords that aren't available in the SCITAS spack version may lead to errors.

The following is a table that details the Spack version for each cluster's stack. Please note that on Kuma there are two software stacks, given the different compute capabilities the GPU (90 for H100 and 89 for L40s).

Cluster Spack Version Stack
Kuma - H100 releases/v0.22 /ssoft/spack/kuma-h100_stable
Kuma - L40s releases/v0.22 /ssoft/spack/kuma-l40s_stable
Jed releases/v0.22 /ssoft/spack/jed_stable
Izar v0.18.1 /ssoft/spack/izar_stable
Helvetios (NA) v0.18.1

When you install Spack, make sure to use the right version. For instance, if you want to use one of the v0.22 stacks:

git clone -b releases/v0.22 https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh

More than one Spack version

Should you need to use Spack with different versions, we recommend working on separate Spack repos/environments instead of switching branches within the same Spack repo back and forth.

Edit upstreams#

You can then proceed to edit the upstreams.yaml file. You can just do:

spack config --scope site edit upstreams

The contents of this file should point to the SCITAS spack path.

upstreams:
  jed:
    install_tree: /ssoft/spack/jed_stable/opt/spack

Check#

To check that the config is working, you can check whether spack considers the packages that are already present in the SCITAS stack or tries to install them:

$ spack spec -Il zlib-ng
Input spec
--------------------------------
 -   zlib-ng

Concretized
--------------------------------
[^]  7wmxgay  zlib-ng@2.1.6%gcc@11.4.1+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rhel9-x86_64_v3
[^]  lh6u3al      ^gcc-runtime@11.4.1%gcc@11.4.1 build_system=generic arch=linux-rhel9-x86_64_v3
[e]  w74nltl      ^glibc@2.34%gcc@11.4.1 build_system=autotools arch=linux-rhel9-x86_64_v3
[^]  xhpr67w      ^gmake@4.4.1%gcc@11.4.1~guile build_system=generic arch=linux-rhel9-x86_64_v3

The [^] is what you should be seeing here, which means that Spack has found an upstream (i.e. the SCITAS-provided) zlib package. If you see - (or [+]) instead, it means that Spack would either build a new (or has found an existing, respectively) zlib instead of using the SCITAS-provided one, which is probably not what you want, and meaning something in your Spack setup probably went wrong.