C&CZ Problem Card

In the process of migrating our server networks to new and faster stuff, we are moving the gateways to our (C&CZ) managed firewalls. These firewalls already handle the traffic to our 25Gbit networks, while the older 1Gbit/s connected nodes had the gateway in a temporary router, which came to replace the old DR-HUYG router a few years ago. While we don’t expect too much trouble, a hiccup may occur for up to a minute or so, while the network changes are being applied. ...

We are working to upgrade the mail servers, due to caching in various parts, you may get messages about incorrect certificates or other problems, we expect this to resolve itself once we finish the updates. Update, we apparently still had problems with the certificates, but somehow missed it after last week’s fix. There’s a temporary fix in place, so we can work further on it Tomorrow.

A change in the configuration management code has broken the certificate on the mailservers, we are trying to fix this properly, but until fixed, mail clients will complain about the server certificate. Fixed by restarting sendmail

We picked August 5th to have a planned downtime/reboot of all clusternodes in the Science Cluster. This will involve at least package updates and a reboot, if other maintenance can be included we will try to fit this in as well.

We will be moving servers from the serverroom ak008 (Huygens A-2.008) to other locations. A lot of services and virtual machines will be offline for a few hours while we disconnect, move and reconnect the hardware. Assuming everything goes well, the services should be up after turning the machines back on again. This CPK will be updated with more servers and information about affected services in the weeks before the move on August 7th 2025. ...

There are servers unreachable due to an unknown problem with the 25Gbit network switch in room ak008. We don’t know the root cause yet. Update: rebooting the affected switch has resolved the problem. DOWN amanda22.science.ru.nl DOWN cephgrafana.science.ru.nl DOWN cephgw3.science.ru.nl DOWN cephgw4.science.ru.nl DOWN cephmon2.science.ru.nl DOWN cephosd07.science.ru.nl DOWN cephosd08.science.ru.nl DOWN cephosd09.science.ru.nl DOWN cephosd10.science.ru.nl DOWN cephosd11.science.ru.nl DOWN cephosd12.science.ru.nl DOWN cephosd13.science.ru.nl DOWN cephosd23.science.ru.nl DOWN cephosd26.science.ru.nl DOWN cephosd27.science.ru.nl DOWN chemotionvm.science.ru.nl DOWN containervm02.science.ru.nl DOWN dockervm01.science.ru.nl DOWN dockervm02.science.ru.nl DOWN dockervm03.science.ru.nl DOWN ds3-huyg.net.science.ru.nl DOWN elabrolabvm.science.ru.nl DOWN filemakervm.science.ru.nl DOWN home3.science.ru.nl DOWN jupytervm.science.ru.nl DOWN mariavm02.science.ru.nl DOWN mossvm.vm.science.ru.nl DOWN rootcell.science.ru.nl DOWN vmhost03.science.ru.nl DOWN vmhost04.science.ru.nl DOWN vmhost07.science.ru.nl DOWN zammad3.vm.science.ru.nl

Services on networks that are currently behind our old picos switches will be transferred to be behind our firewalls. This may take a few minutes of downtime due to the changes needed for moving the gateway functionality. If all goes well, it will be one period of a few minutes, if we need to roll back and fix things, there may be a repeat downtime. All went well, there was a few minutes of interruption on the cncz homepage due to ARP caching issues.

Following the recent update of our LDAP servers certificates, multiple users have reported authentication failures when attempting to log in via RADIUS. The issue appears to affect only users with RADIUS-based authentication, while LDAP-based authentication continues to function successfully. Affected users typically include HFML technicians, guest console users, and Science logins with Wi-Fi access.

In the transition from the older 25Gbit switches to newer switches, two blocks of 4 connections were temporarily unavailable due to a peculiarity of the switches where ports are operated in blocks of 4. The removal of an unused cable triggered a disabling of the other ports in the bunch, reseating one of these connections in one case and re-inserting the unused cable fixed the block of ports to start working again. ...

Due to errors on the /vol/astro2 filesystem, we had to reboot the fileserver comas1 and take it offline to perform repairs. During this process, both /vol/astro2 and /vol/astro5 have been unavailable.