Skip to content

How to use the file systems#

As detailed in the file systems documentation, there are six main file systems that you can use for different purposes: /home, /work, /export, /transfer, /archive and /scratch.

The following sections illustrate how to manage your data in different scenarios.

Storing data#

Numerical simulations often lead to some substantial amount of data results that need to be stored on different time scales. For example, you may need to store large restart files for a few days in order the continue a simulation or you may need to archive simulation results for decades in order to publish your results. For these different scenarios, we provide access to different file systems.

Note

Before going onward, please read carefully the file systems documentation to make sure you understand each file system and its properties. Using the wrong file system may result in unwanted loss of data.

During computation(s) (max 30 days)#

For your data used by your jobs during their run, you should use the /scratch file system.

  • The /scratch file system is a personal space that is mainly used to temporarily store your simulation results. Because of its high-performance storage capacities, your simulations should used it as an I/O space. It is not charged and has no quota, but a retention policy of 30 days. All the files older than 30 days will be deleted without notice.

Best Practices for data storage during computation

  • Avoid /home and /work: These directories are not optimized for high performance.
  • Use /scratch: It’s faster and designed for data processing.
  • Move data after your job: Transfer from /scratch to /home or /work when done.

Happy computing! 😊🚀

Permanent Storage#

For data that need to be kept a longer period or be shared with your colleagues, you can use either your /home or the /work file systems.

  • The /home file system is your personal space with a free quota of 100GB that is backed up by default. It is mainly used to store small files such as source code or input files.
  • The /work file system is a collaborative space that you can use to store the large files and share them with your colleagues. There is no quota as it is a pay-per-use system (you will be billed for the space you actually use) and it can easily enable backup services by creating a .backup file in the main directory. (How to ask for a pay-per-use account)
  • The /exportfile system is similar to /work as it's a collaborative space on a pay-per-use system. The main difference is that /exportcan be access from the clusters and also from the EPFL network. You can use the protocols SMB, S3 of NFS.

Note

Note that apart from the size of your data, you must also consider the time frame of your storage. Indeed, your /home will only be kept until 6 months after you left EPFL. If you need to store data for a longer period, we strongly advise that you use the /work file system. The data will be kept until 6 months after the end of payment.

Note

Note that the first two spaces, /home and /scratch, are personal. You must not change the access rights in order to open them to your colleagues. If you need to share your files, please use the /work file system.

Archiving#

In the future, we will also activate the /archive file system that allows you to store your data on magnetic tapes. The main advantages are that the price per TB is lower than hard drives and magnetic tapes can store your data for decades.

Storing data workflow

Transferring Data#

Please refer to our page Transferring Data