CPK messages are initially sent to the CPK mailing list, you can (un)subscribe via this link. You can also follow the service interruption messages via RSS using the link in the title under the RSS icon. If the CPK takes more time to resolve, any updates are published on this website.
For RU wide service interruption see meldingen.ru.nl.
We will reboot clusternodes and possibly upgrade them from older ubuntu versions to ubuntu 24.04 in the last week of August 2026. If you have clusternodes and would like to plan for an upgrade from ubuntu 22.04 to ubuntu 24.04, please let us know
Our upstream certificate provider HARICA mis-issued several certificates (details) and is in the process of revoking said certificates. When manually renewing the certificate for helpdesk.science.ru.nl, we encountered a configuration oversight and accidentally misconfigured the webserver. This resulted in a redirection loop. We restored the working configuration from backup and reloaded the configuration for the webserver. Update Several thousand more cerficates have again been revoked. This is also led to HARICA being overloaded, because every one is trying to get their certificates re-issued. We are trying to update the certificates where we can.
Thunderbird (and probably other mail clients) using authenticated SMTP to send e-mail complained about an invalid certificate. This was due to the replacement certificate being signed with another intermediate certificate from our certificate authority (Harica), which temporarilly broke the upchain. It was fixed around quarter to one o’clock. This is related to the certificate issues previously described in CPK 1396.
A zero day vulnerability in the Linux kernel forced us to reboot a few of the most open to the world servers, other servers will probably reboot as possible with a fixed kernel. We also installed a workaround to mitigate the issue on all servers.
Since about one week ago, the (central) firewall started to apply internal packet inspection to the regular zone transfers of the internal view of ru.nl. The transfer was then blocked by the firewall before completion, causing our copy of the data, which is required for accessing bass.ru.nl and also internal mail delivery to @ru.nl addresses, to expire after 8 days. This period is relatively short, we have 3 weeks for our own zones, and the effect of expiry was underestimated. ...
An automatic GitLab update caused a database migration failure, making the service unavailable. We resolved the issue by restarting the database and reconfiguring the GitLab migration.
A planned reboot of one of the science firewalls at 10:30 should normally have been unnoticable, however, after the firewall came back online, the keepalived service failed to start, which prevented the firewall’s virtual IP from being advertised. As a result, traffic to several internal services appeared to hang. The issue was resolved by manually starting the keepalived daemon at 10:51.
After an upgrade on Feb 3, the DIY service kept an old process running, so self‑service account creation failed. 12 users hit the “account creation failed” error between 12:01 AM and 9:33 AM. The process was restarted, the new version took over and login creation works again. Sorry for the hassle!
We are working on replacing our incoming mailservers and something went wrong. Due to an unexpected block in the mail flow, around 2800 mails which were queued from around 18:00 on Feb 2nd, were lost. The host queuing the mail, was replaced on Feb 3rd, around 8:30 and decommissioned at around 9:00. Unfortunately, the decommissioning also destroyed the disk of the server with the mail queue of around 2800 e-mails. At 18:00 on Feb 2nd, two servers were accepting incoming mail, but only one of the servers was queuing, the other one was running as expected. ...
Elabftw had an expired certificate, because after the move to a new server, this process was left unmonitored. Both the certificate and the monitoring are now fixed.
A combination of small failures in configuration and monitoring resulted in Daily backups not being run on peck between September 1st and last Friday. Backups are now running again, but other than the Monthly backups, there are no intermediate Daily backups for the volumes on server peck. Updates in monitoring/testing are in progress to prevent this kind of oversight in the future.
After moving the elabftw site to a new server, it was impossible to send e-mail from the server, like when a user wants to reset their password. The changes that were necessary are in between the host and the docker container running elabftw, while figuring out what was needed also caused some general issues reaching the elabftw webserver.