Jitsi was unavailable after the reboot, because one of the web-interface configurations was still referring to an older certiticate.
The certificate for jitsi.science.ru.nl expired last week and was replaced, but a sub-service was still using a copy of the older certificate. Users were able to open the frontend, but not able to start a meeting. The problem was resolved by updating the copy of the sub-service.
Since midnight there is a problem using Adobe Acrobat Pro DC, you are asked to activate the license. ILS is working on a solution. You can open PDFs using Adobe Reader or some other PDF reader. See: https://meldingen.ru.nl/detail.php?id=1333&lang=en&tg=0&f=0&ii for the original notice.
Just before 23:09 quite a lot of ceph storage nodes became unreachable. This seems to be due to one of the redundant links between two datacenter locations failing for about 4 seconds. This triggered a whole slew of ceph osd processes being killed off and not starting again. A generic configuration change made for all our servers generated an extra interface, which confused some of the osd processes (depending on interface ordering) when starting up....
ILS wifi management notified us that they will upgrade the software of wireless access points, because security flaws have been addressed in the latest release. The maintenance ends on 2022-10-21 01:30. Source: meldingen.ru.nl
Some time ago, some Science users fell for a phishing mail, therefore their account could be misused by spammers to send mail through our mailservers. Therefore our mailservers ended up in blacklists yesterday evening. Some mails could therefore not be delivered to other servers, that mail was returned to the sender. This morning we removed the IP addresses of our servers from several blacklists, we think that the problems are resolved now....
ILS network management announced: Tuesday September 27 23:30 maintenance is carried out on the Eduroam wireless network in FNWI buildings a.o. In most cases connectivity will be interrupted for at least 30 and at most 60 minutes.
ILS network management announced: Friday September 23 18.30 – 22.30 network maintenance is carried out on the DMZ and SRV firewalls. Connectivity interruptions of 20 to 40 seconds are expected. In case of problems, interruptions of 5 to 10 minutes might occur.
A few campus-wide dips in the power supply made network and telephony unusable for about 15 minutes, except in server rooms where UPSs guarantee the power supply. Because 7 modules of desktop switches in Huygens/HFML/Felix had become defective, several hundred network outlets will only have worked again after the relevant switch module had been replaced.
Still unclear what the cause is, several links are strongly degraded. Update: A defective switch has been replaced.