C&CZ Problem Kard

Ceph filesystem storage is experiencing reduced performance, because one of the storage nodes is currently offline. A combination of factors causes this to affect performance, fullness, uneven distribution of data over the storage nodes. We expect this to be resolved when the node is back in the cluster. update 7 Oct 2023 The cluster is mostly complete again for a while already, but there’s a lot of remaining issues to be resolved by the cluster....

The Science newsletter is sent from an @ru.nl mail-account. Recently, the @ru.nl mailservice was changed by adding Exchange Online Protection (EOP). EOP is the successor for Radboud University to the Proofpoint e-mail security filter. EOP added a mail header line to the newsletter List-Unsubscribe: https://c8ce19ec62454d21b73b7d5a25559d8f.svc.dynamics.com/t/lu/7... which made the mail appear to be spam to the Science spamfilter SpamAssassin 1.1 URI_HEX URI: URI hostname has long hexadecimal sequence When we noticed this, we fixed it by welcome listing (allowlisting/passlisting) the sender address communications-science@ru....

After the Monday morning reboot, the NFS server on home1 refused to start properly. We are investigating why a manual restart of nfs was needed.

RU mail management let us know that yesterday the forwarding to external (non-RU) mail addresses has been stopped as announced earlier. Unfortunately, mail for several dozens of Science users was/is not forwarded to the Science mailservers. These mails can still be found in MS365 (RU mail), either in the Inbox or in the Deleted Items. RU mail management promised that the forwarding will be corrected tomorrow.

Announcement of maintenance, Wednesday afternoon we are going to replace the cpu of one of our main vmhost servers, meaning vms gitlab9 (pep) slurm22 pep3 jitsivm poliep indicoimapp2vm pep4 mariavm01 smtp2 will be down for up to 1 hour. several services depend on the mariavm01 (websites, slurm), so they will be affected too.

Apologies for the short notice, we are now going to replace the motherboard of one of our main vmhost servers, meaning vms gitlab9 (pep) slurm22 pep3 jitsivm poliep indicoimapp2vm pep4 mariavm01 smtp2 will be down for up to 1 hour. several services depend on the mariavm01 (websites, slurm), so they are affected too.

Our daily backup system relies on cephfs storage, which is currently offline, see CPK#1337. This means that as of July 22nd we are unable to perform or restore daily backups. When the cephfs problems are resolved the daily backups should also be OK and restorable again. NB, this has no effect on the Monthly backups, which continue to work normally.

After the power down of the Huygens building we are experiencing a problem with bringing Ceph file system back online. We currently do not know when the Ceph cluster is operational again. Update 2023-08-01 10:30 Ceph is working again. This CPK is now closed. CPK#1338 is also closed. Update 2023-07-31 12:30 After some more support from 42on, we managed to restart the cephfs, we cannot be sure all files are there, but almost all files are....

The VPNsec service will be moved to a new server. This move will cause downtime and existing VPN connections will be destroyed. Downtime is expected not to exceed several minutes.