One of the boot disks was failing and in the process of trying to resolve the issue (reseating the disk), its mirror suddenly went offline as well. After restoring that half, the server is running and we are busy getting a replacement part. In theory the server should not have to go down when the disk is replaced.
The fileserver peck crashed this early morning. After a reboot, the machine ran fine again. Users of cluster nodes that use nfs exported volumes from peck might still experience problems as nfs (network file system) does not seem to be recovering well.
Introduction In our Faculty, different types of data storage are offered: Storage class Description Risk for data loss Home directories Small (several GB), reliable, backed up storage for individuals low Network shares Larger (1-200 TB), reliable, backed up storage for individuals or groups low Local storage Not backed up storage on desktop computers and cluster nodes, likely to be lost in case of a hardware problem or when the machine gets reinstalled high Home directories Your Science login comes with a home directory of 5GB at no costs. This is a safe place to store your work related documents. Home directories are stored on reliable hardware and backed up automatically. If you need more than 5GB of storage, you can enlarge it on DIY. Before doing so, you may want to clean up some data. ...
If you use C&CZ storage, you now have the new option to access these via network protocols other than SMB (for Windows and macOS) and NFS (for Linux), namely: sftp: a secure variant of FTP https: a (read-only) view of the share for browsers webdavs: a (read-write) view of the share for browsers and webdav clients s3: a read/write view using the Simple Storage Service (S3) Each of these may be active at the same time, and optionally can have an end date, so you can “open up” a share for writes from another organisation which then automatically “closes” at the end date. For sftp, authentication is done via an SSH key. The other protocols are authenticated with a username and password. These credentials are not tied to any person and are easily revoked or re-generated if the need arises. ...
A new storage option has been developed by C&CZ. This page will show the technical details and properties of this storage, which will hopefully help you make a decision if this is something you want. Any storage we handout we call a “volume”. For these ZFS volumes: Size Storage size can be as large as you want, but smaller than the amount we can put in a single machine. Currently (2024+) we can buy machines with roughly 140-200 TB storage. Snapshots On this we enable snapshots, so that you can retrieve (deleted) files your self. Backup Each volume will be back upped to local S3 storage. This should be thought of as disaster recovery, and restoring from the cloud should (hopefully) be rarely needed. All of these volumes are visible in DIY in the “Storage” section if you have access to them. How to order such storage can be seen here. ...
You might have received an email that your personal or group network disk is almost full. This articles describes some tools to identify large files/directories and cleanup disk space. Finding large files / directories with WinDirStat (graphical, Windows) WinDirStat is a disk cleanup tool for Windows that provides a visual representation of disk space usage. It helps you to quickly identify and understand which files and folders are consuming the most space. The program offers detailed information about each file, including type, last modified date, and file path. In addition to visualization, WinDirStat includes built-in cleanup options, enabling you to directly delete or move files within the program. ...
Network shares with ZFS as file system have the ability (which C&CZ switches on by default) to make snapshots. A snapshot is a read-only copy of the share, made at a specific point in time. Such snapshots allow you to find previous versions of your files. On most shares, snapshots go back 45 days. After that period they are removed. Tip ...
What path to mount? The storage page describes the paths for your home directory and network shares. Once you know what network share to mount, follow the steps for your operating system below. Info ...
Just before 23:09 quite a lot of ceph storage nodes became unreachable. This seems to be due to one of the redundant links between two datacenter locations failing for about 4 seconds. This triggered a whole slew of ceph osd processes being killed off and not starting again. A generic configuration change made for all our servers generated an extra interface, which confused some of the osd processes (depending on interface ordering) when starting up. We are reasonably confident we can avoid this from happening in the future. ...
Often people want to share data, choosing the right method can be complex. This page hopes to help you choose the right solution. Options: https (previously ftp) e-mail surfdrive surf filesender physical transfer custom solution Microsoft Teams Situations: one to few people one to many people share with public co-authoring moving datasets between institutions GDPR (Dutch: AVG) Whenever you are sharing documents or files, be aware that restrictions apply when sharing personal information and all involved parties must explicitly accept responsibility for keeping within the legal boundaries. ...