A combination of small failures in configuration and monitoring resulted in Daily backups not being run on peck between September 1st and last Friday. Backups are now running again, but other than the Monthly backups, there are no intermediate Daily backups for the volumes on server peck. Updates in monitoring/testing are in progress to prevent this kind of oversight in the future.

After moving the elabftw site to a new server, it was impossible to send e-mail from the server, like when a user wants to reset their password. The changes that were necessary are in between the host and the docker container running elabftw, while figuring out what was needed also caused some general issues reaching the elabftw webserver.

To make the network less complicated and more easily manageable, we need to make vlans for the subnets 131.174.8.0/24 and 131.174.9.0/24, this will involve some servers getting a new ip address. This affects firewall rules as well. We must coordinate this with ILS networking and the changes will cause disruptions between a few minutes up to a quarter of an hour. List of sites: alice.science.ru.nl alicecontrol.science.ru.nl coursetimetable.science.ru.nl dmarcreport.science.ru.nl edutools.science.ru.nl eduview.science.ru.nl ...

The certificate of the radius server has been replaced this morning, users who use their science account for authenticating to the Eduroam network will not have been able to connect until the certificate was fixed. You may need to ’trust-on-first-use’ or other setting in your device to properly reconnect and trust the certificates, if that still fails.

Every half year we do clusternode maintenance, with at least a package ugprade and a reboot, but sometimes other maintenance can happen, such as changes in filesystems or network configurations. The upcoming date for this maintenance is February 6th, 2026 (Friday) The maintenance is done.

A brief service interruption occurred this morning when one of our LDAP servers refused to start. This affected services that depend on LDAP for user authentication. Services such as Eduroam (using science login for authentication) and GitLab experienced authentication failures during this time. We are investigating the source of the failure to prevent recurrence.

Recently we have experienced some issues with nfs filesystems one of the main suspects was the ZFS filesystem having a lack of cache memory on the machine. In order to improve the situation we will install more memory in the servers. This will cause a downtime of about 10-15 minutes per server. We plan to do this before 10:00 or after 16:00 on Monday Oct 27, 2025. Update 27 Oct 2025 The memory upgrade was concluded around 09:15 on Monday 27 Oct.

The ElabFTW server will be upgraded and also moved to another host on Monday October 27th, this will be a little bit more involved than a normal version upgrade, and you can expect some downtime during the move and upgrade. Update: upgrading was done on the old server, moving to the new server ran into some unexpected issues, we will plan a new date for that. Update: move was done on Sunday November 2nd, some users report problems logging in, but that appears to be temporary due to elab running on a different ip address and machine.

On Thursday 23rd we will move the host of the rstudio website and another server to another location. This will cause some downtime, starting from about 9 O’clock and it should be done within 1 hour.

The GitLab registry used by gitlab.science.ru.nl and gitlab.pep.cs.ru.nl is currently unavailable due to a disk failure in the storage backend. The faulty drive is being replaced, after which the registry should be available again.