Skip to content

Security#

Security Update - SCITAS Infrastructure Status

We would like to inform you that a critical Linux kernel vulnerability (CVE-2026-31431) has recently been disclosed. This vulnerability allows any user with access to a system to gain root privileges with minimal effort. Such a flaw carries significant risks, including potential data compromise affecting users and laboratories, identity impersonation, and the installation of malicious software.

At the EPFL level, no unified policy has been enforced, and each unit has been asked to assess and manage the risk independently. Within SCITAS, we have decided to execute an emergency procedure by temporarily restricting access to the infrastructure while preserving running jobs. This approach allows us to conduct a thorough analysis of system logs and maintain full control over the situation.

We are fully aware that this unexpected disruption has a strong impact on your work. However, we have prioritized the security and integrity of both data and infrastructure.

At this stage, we have no indication of any system compromise. A complete verification of the software stack binaries has been carried out, and running jobs continue to execute without issue. In addition, all passwords and selected certificates for critical services have been rotated as a precautionary measure.

We are now progressively restoring access by restarting the frontends and compute nodes. We expect to be able to partially restore the service before noon.

We will continue to keep you informed as the situation evolves.

Progressive Restoration of SCITAS Clusters

We would like to inform you that we have started bringing patched compute nodes back into production. The frontends of both Kuma and Jed are now accessible to all users.

Users with running jobs on compute nodes that have not yet been patched will not be able to connect to these nodes via SSH.

Throughout this intervention, no running jobs were stopped, and we have not identified any evidence of data leakage.

We have made every effort to strike a balance between security and service availability. We sincerely apologize for this unexpected disruption, especially during such an important period for research activities.

Please do not hesitate to report any anomalies or issues you may encounter.

Partial Access Restoration - Data Retrieval on SCITAS Clusters

We would like to inform you that the frontends of the Jed and Kuma clusters have now been reopened in order to allow users to retrieve their data.

At this stage, users with running jobs on a given cluster will not be able to connect to its corresponding frontend. This is because the compute nodes hosting these jobs have not yet been redeployed and therefore remain vulnerable.

As a temporary workaround, users running jobs on Kuma may retrieve their data via the Jed frontend, and vice versa, as long as the data are not located on scratch.

Please note that the host keys of the frontends have been changed. When connecting, you should expect new SSH fingerprints. The expected fingerprints are available on the following pages:

During our investigation, we identified 193 private SSH keys stored in user home directories that could grant access to the clusters. To mitigate any risk of identity impersonation, access using these keys has been disabled. As a consequence, affected users will need to log in using their Gaspar password and then deploy a new, clean public SSH key.

If you have an urgent need to access a specific cluster where you currently have running jobs, you may contact us by opening a support ticket to request the termination of your jobs.

We appreciate your understanding and cooperation as we continue to restore the service safely.

Security Incident - Preventive Measures on SCITAS Clusters

We would like to inform you that, following the disclosure of a critical Linux kernel vulnerability (CVE-2026-31431), we have taken immediate preventive measures across the SCITAS infrastructure.

As of this morning: - All login frontends have been temporarily disabled to prevent new interactive access. - Running jobs are currently continuing on compute nodes. - We are actively analyzing system logs to assess any potential impact.

At this stage, there is no confirmed evidence of data compromise. However, investigations are ongoing.

We are currently awaiting the availability of official security patches from the operating system vendors. As soon as they are released, we will proceed with their deployment, which will require a coordinated restart of the clusters.

Depending on the outcome of our analysis, additional measures may be taken, including partial or full reinstallation of affected systems if deemed necessary.

We understand the impact this situation may have on your work and appreciate your understanding as we prioritize the security and integrity of the infrastructure.

We will keep you informed as the situation evolves.

Downfall vulnerability

The downfall vulnerability, identified as CVE-2022-40982, enables a user to access and steal data from other users who share the same computer. It is found in most Intel CPUs starting from the 6th generation (Skylake) up to the 11th generation (Tiger Lake) included. For instance, a malicious app obtained from an app store could use the Downfall attack to steal sensitive information like passwords, encryption keys, and private data such as banking details, personal emails, and messages.