Data storage   howto

Introduction In our Faculty, different types of data storage are offered: Storage class Description Risk for data loss Home directories Small (several GB), reliable, backed up storage for individuals low Network shares Larger (< 1TB), reliable, backed up storage for individuals or groups, with ZFS > 1TB is possible low Ceph storage Large scale, reliable, snapshotted, not backed up storage for groups medium Local storage Not backed up storage on desktop computers and cluster nodes, likely to be lost in case of a hardware problem or when the machine gets reinstalled high Home directories Your Science login comes with a home directory of 5GB at no costs....

January 12, 2023 · updated February 7, 2024 ·  Bram

Ceph storage interruption   cpk

Just before 23:09 quite a lot of ceph storage nodes became unreachable. This seems to be due to one of the redundant links between two datacenter locations failing for about 4 seconds. This triggered a whole slew of ceph osd processes being killed off and not starting again. A generic configuration change made for all our servers generated an extra interface, which confused some of the osd processes (depending on interface ordering) when starting up....

October 24, 2022 · updated February 7, 2024 ·  Simon

Ceph problem   cpk

During a routine upgrade of ceph, a bug in the latest version manifested itself and made the ceph manager unreachable. After aborting the upgrade and with help from the ceph-users mailinglist, everything became available again using a workaround.

March 24, 2021 · updated February 7, 2024 · 

Backup   howto

If accidentally one or more files are lost from a home-directory (U:-disk) or a /vol-directory or some other place on a C&CZ server, probably the files can be restored. To be present on backup, a file must of course have existed during a (nightly) backup. Different independent backup schemes are used : Daily backups to Ceph-storage. Monthly backups to tape. Yearly backups to tape. Daily backup All server disk-partitions except temporary space like /tmp, /var/tmp, /scratch, are backed-up after each working day on Ceph storage....

December 17, 2020 · updated February 7, 2024 ·  Peter

New data storage: Ceph   news

Recently C&CZ presented an extra choice for data storage, next to the traditional RAID storage on individual fileservers: Ceph. The main advantages of Ceph are, that is scales almost without limits and does not have a single point of failure, but instead is “self-healing” and “self-managing”. Within Ceph storage, choices can be made, the more expensive ones have no problem when a complete datacenter goes down. The Ceph storage can also be used as S3-storage....

December 17, 2019 · updated February 7, 2024 ·  Simon