Cpk on C&CZ News

Scheduled firewall reboot broke keepalive service

Simon Oosthoek — Tue, 03 Mar 2026 11:02:00 +0100

A planned reboot of one of the science firewalls at 10:30 should normally have been unnoticable, however, after the firewall came back online, the keepalived service failed to start, which prevented the firewall’s virtual IP from being advertised. As a result, traffic to several internal services appeared to hang.

The issue was resolved by manually starting the keepalived daemon at 10:51.

Self‑creating login broken on DIY

Bram Daams — Thu, 05 Feb 2026 00:00:00 +0100

After an upgrade on Feb 3, the DIY service kept an old process running, so self‑service account creation failed. 12 users hit the “account creation failed” error between 12:01 AM and 9:33 AM. The process was restarted, the new version took over and login creation works again. Sorry for the hassle!

Mail Lost between 18:00 2 Feb 2026 and 9:00 3 Feb 2026

Arnoud Thörig — Wed, 04 Feb 2026 00:00:00 +0100

We are working on replacing our incoming mailservers and something went wrong.

Due to an unexpected block in the mail flow, around 2800 mails which were queued from around 18:00 on Feb 2nd, were lost. The host queuing the mail, was replaced on Feb 3rd, around 8:30 and decommissioned at around 9:00. Unfortunately, the decommissioning also destroyed the disk of the server with the mail queue of around 2800 e-mails.

At 18:00 on Feb 2nd, two servers were accepting incoming mail, but only one of the servers was queuing, the other one was running as expected.

Certificate expired on Elabftw

Simon Oosthoek — Wed, 14 Jan 2026 00:00:00 +0100

Elabftw had an expired certificate, because after the move to a new server, this process was left unmonitored. Both the certificate and the monitoring are now fixed.

Daily backups on peck failed

Simon Oosthoek — Mon, 01 Dec 2025 00:00:00 +0100

A combination of small failures in configuration and monitoring resulted in Daily backups not being run on peck between September 1st and last Friday. Backups are now running again, but other than the Monthly backups, there are no intermediate Daily backups for the volumes on server peck. Updates in monitoring/testing are in progress to prevent this kind of oversight in the future.

Elabftw password reset problem

Simon Oosthoek — Tue, 18 Nov 2025 00:00:00 +0100

After moving the elabftw site to a new server, it was impossible to send e-mail from the server, like when a user wants to reset their password. The changes that were necessary are in between the host and the docker container running elabftw, while figuring out what was needed also caused some general issues reaching the elabftw webserver.

Network changes websites

Simon Oosthoek — Tue, 18 Nov 2025 00:00:00 +0100

To make the network less complicated and more easily manageable, we need to make vlans for the subnets 131.174.8.0/24 and 131.174.9.0/24, this will involve some servers getting a new ip address. This affects firewall rules as well. We must coordinate this with ILS networking and the changes will cause disruptions between a few minutes up to a quarter of an hour.

List of sites:

alice.science.ru.nl
alicecontrol.science.ru.nl
coursetimetable.science.ru.nl
dmarcreport.science.ru.nl
edutools.science.ru.nl
eduview.science.ru.nl

Radius certificate expired (Eduroam login)

Simon Oosthoek — Thu, 13 Nov 2025 00:00:00 +0100

The certificate of the radius server has been replaced this morning, users who use their science account for authenticating to the Eduroam network will not have been able to connect until the certificate was fixed.

You may need to ’trust-on-first-use’ or other setting in your device to properly reconnect and trust the certificates, if that still fails.

Clusternode maintenance day - February 6th 2026

Simon Oosthoek — Tue, 11 Nov 2025 00:00:00 +0100

Every half year we do clusternode maintenance, with at least a package ugprade and a reboot, but sometimes other maintenance can happen, such as changes in filesystems or network configurations.

The upcoming date for this maintenance is February 6th, 2026 (Friday)

The maintenance is done.

ldap unavailable

Bram Daams — Mon, 27 Oct 2025 09:21:50 +0100

A brief service interruption occurred this morning when one of our LDAP servers refused to start. This affected services that depend on LDAP for user authentication. Services such as Eduroam (using science login for authentication) and GitLab experienced authentication failures during this time. We are investigating the source of the failure to prevent recurrence.

Homeserver upgrade memory

Simon Oosthoek — Mon, 27 Oct 2025 08:00:03 +0200

Recently we have experienced some issues with nfs filesystems one of the main suspects was the ZFS filesystem having a lack of cache memory on the machine. In order to improve the situation we will install more memory in the servers. This will cause a downtime of about 10-15 minutes per server. We plan to do this before 10:00 or after 16:00 on Monday Oct 27, 2025.

Update 27 Oct 2025

The memory upgrade was concluded around 09:15 on Monday 27 Oct.

Elabftw upgrade and move

Simon Oosthoek — Thu, 23 Oct 2025 10:35:00 +0200

The ElabFTW server will be upgraded and also moved to another host on Monday October 27th, this will be a little bit more involved than a normal version upgrade, and you can expect some downtime during the move and upgrade.

Update: upgrading was done on the old server, moving to the new server ran into some unexpected issues, we will plan a new date for that.

Update: move was done on Sunday November 2nd, some users report problems logging in, but that appears to be temporary due to elab running on a different ip address and machine.

Physical server move

Simon Oosthoek — Tue, 21 Oct 2025 16:38:03 +0200

On Thursday 23rd we will move the host of the rstudio website and another server to another location. This will cause some downtime, starting from about 9 O’clock and it should be done within 1 hour.

gitlab registry unavailable

Bram Daams — Mon, 29 Sep 2025 09:29:03 +0200

The GitLab registry used by gitlab.science.ru.nl and gitlab.pep.cs.ru.nl is currently unavailable due to a disk failure in the storage backend. The faulty drive is being replaced, after which the registry should be available again.

Miii server issues

Simon Oosthoek — Tue, 23 Sep 2025 12:40:00 +0200

Due to some cleanup changes, the miii server ended up without the proper firewall rules and account configuration scripts that it was supposed to get. We were able to fix the server eventually, but during the day, users were unable to work until it was fixed.

Network reconfiguration vmhosts

Simon Oosthoek — Tue, 23 Sep 2025 12:40:00 +0200

With the changes in the networks, we are able to optimise some vlan configurations to utilise the high speed link, which was impossible before. The network reconfiguration may need a reboot of the server, but otherwise not too much downtime is expected. The reboot takes about 5-10 minutes if necessary.

The list of vm’s affected (only those without redundant functionality are listed):

vmhost05:

mestrenova143
pagespep
pep1
nolaivalkhof
mestrenova150
fsecure
pep2
pages
licensevm1
goudsmit
zaaivm
stormvogel
nolaigoffert
desdapi

vmhost06:

NFS problems under investigation

Simon Oosthoek — Wed, 17 Sep 2025 10:50:00 +0200

When we moved the gateways of our networks from the old location to the new firewalls, we have received some complaints about NFS filesystems having slowness, longer delays or unavailability.

In general NFS should never be a requirement for a clusternode job if you can avoid it, because this I/O is always much slower than local I/O from /scratch.

We are investigating how we can optimise the network to resolve this issue, but we are hard pressed to know the exact cause of the problem. We are assuming the root cause might be that, now more traffic is passing through our firewalls, this degrades the performance in some way.

ssh session hanging due to firewall config

Wed, 17 Sep 2025 09:19:18 +0200

A change in the firewall config had the uncalculated effect that ssh-sessions got blocked/dropped. The config has been reversed.

Server "oliver" needs to be moved

Simon Oosthoek — Wed, 17 Sep 2025 07:30:00 +0200

We need to move a server to another location, during the move some of the services are unavailable, including licenses for:

Cadence Labview Originlab
Cst Maple Server
ExelisIdl Mathematica Synopsys
Genesys Matlab VirtualMachine
GurobiTokenServer Mentorgraphics Xilinx
Hdlworks Moe
IntelCompiler New

Everything should be completed by 10:00.

printing via lp science blocked

Simon Oosthoek — Tue, 16 Sep 2025 11:50:29 +0200

Due to the gateway change the printing traffic followed a different route which ran into default blockages in the central firewall, after the problem was found a fix was implemented to allow this traffic to the printserver via the new route.

websites and other services unavailable from everywhere they were supposed to be

Simon Oosthoek — Mon, 15 Sep 2025 13:14:29 +0200

After switching the gateway for one of our busiest vlans to our own firewalls, a lot of websites were only available from outside that vlan, it took a while to understand the issue and find a fix for it. We now have a temporary fix in place that should make the websites work again.

filemaker pro unavailable

Simon Oosthoek — Mon, 01 Sep 2025 10:57:57 +0200

Due to an unfortunate manual command, the package for filemaker was removed from the server. Service was restored after the weekend, when the backup was restored.

DHZ en daarmee MFA onbereikbaar

Simon Oosthoek — Mon, 01 Sep 2025 10:16:27 +0200

Due to an unfortunate manual command, the package for DHZ was removed from the servers, causing all attempts to login with MFA to fail on for example roundcube and lilo ssh login. Service was restored about half an hour later when the DHZ package was installed again.

Mail forwarding broken to @ru.nl

Simon Oosthoek — Tue, 26 Aug 2025 00:00:00 +0200

Due to a change in how the Microsoft Exchange server validates e-mail, mails sent to an @science.ru.nl address (or similar domains for which we accept e-mail) from an external domain and with a forward to your @ru.nl mail are very likely to be dropped, because the receiving server at Microsoft considers this mail forged, in the sense that the sending client did not address the same domain as the receiving domain. This also applies to our Mail Aliases with @ru.nl addresses in them, when they are addressed from an external mail domain. The forwarded e-mail will be silently dropped, so senders will not notice a problem.

Mattermost broken after automatic update

Bram Daams — Fri, 22 Aug 2025 00:00:00 +0200

After an automatic update, Mattermost no longer starts. The issue has been identified; the solution requires relocating the installation. This may take some time.

Mattermost is now running on a different server as gitlab, which was a lot of work to figure out, but all should be well now. For some people it may have been longer before everything worked, due to dns caching of the hostname mattermost.science.ru.nl to the wrong address.

delay in receiving email

Erik Visser — Thu, 21 Aug 2025 00:00:00 +0200

FNWI users have been experiencing delays in email reception since early this morning.

We have been working on this issue but have not solved it yet.

No emails have been lost, but we suspect certificate issues between the Science mail servers and the central RU mail filter.

Update: the exact cause remains unknown.

During investigations together with RU-ILS, we noticed a long-standing Certificate Chain issue.

However, even after resolving this, the problem remained.

Firewall change for server networks

Simon Oosthoek — Thu, 21 Aug 2025 00:00:00 +0200

In the process of migrating our server networks to new and faster stuff, we are moving the gateways to our (C&CZ) managed firewalls. These firewalls already handle the traffic to our 25Gbit networks, while the older 1Gbit/s connected nodes had the gateway in a temporary router, which came to replace the old DR-HUYG router a few years ago.

While we don’t expect too much trouble, a hiccup may occur for up to a minute or so, while the network changes are being applied.

Sending science mail work in progress

Simon Oosthoek — Mon, 18 Aug 2025 00:00:00 +0200

We are working to upgrade the mail servers, due to caching in various parts, you may get messages about incorrect certificates or other problems, we expect this to resolve itself once we finish the updates.

Update, we apparently still had problems with the certificates, but somehow missed it after last week’s fix. There’s a temporary fix in place, so we can work further on it Tomorrow.

Sending science mail broken certificate

Simon Oosthoek — Fri, 15 Aug 2025 00:00:00 +0200

A change in the configuration management code has broken the certificate on the mailservers, we are trying to fix this properly, but until fixed, mail clients will complain about the server certificate.

Fixed by restarting sendmail

Clusternode maintenance day - August 5th

Simon Oosthoek — Tue, 01 Jul 2025 00:00:00 +0200

We picked August 5th to have a planned downtime/reboot of all clusternodes in the Science Cluster.

This will involve at least package updates and a reboot, if other maintenance can be included we will try to fit this in as well.

Servers and services unavailable due to physical moving on August 7th

Simon Oosthoek — Tue, 01 Jul 2025 00:00:00 +0200

We will be moving servers from the serverroom ak008 (Huygens A-2.008) to other locations. A lot of services and virtual machines will be offline for a few hours while we disconnect, move and reconnect the hardware. Assuming everything goes well, the services should be up after turning the machines back on again.

This CPK will be updated with more servers and information about affected services in the weeks before the move on August 7th 2025.

Network failure in part of network

Simon Oosthoek — Wed, 18 Jun 2025 00:00:00 +0200

There are servers unreachable due to an unknown problem with the 25Gbit network switch in room ak008.

We don’t know the root cause yet.

Update: rebooting the affected switch has resolved the problem.

DOWN    amanda22.science.ru.nl
DOWN    cephgrafana.science.ru.nl
DOWN    cephgw3.science.ru.nl
DOWN    cephgw4.science.ru.nl
DOWN    cephmon2.science.ru.nl
DOWN    cephosd07.science.ru.nl
DOWN    cephosd08.science.ru.nl
DOWN    cephosd09.science.ru.nl
DOWN    cephosd10.science.ru.nl
DOWN    cephosd11.science.ru.nl
DOWN    cephosd12.science.ru.nl
DOWN    cephosd13.science.ru.nl
DOWN    cephosd23.science.ru.nl
DOWN    cephosd26.science.ru.nl
DOWN    cephosd27.science.ru.nl
DOWN    chemotionvm.science.ru.nl
DOWN    containervm02.science.ru.nl
DOWN    dockervm01.science.ru.nl
DOWN    dockervm02.science.ru.nl
DOWN    dockervm03.science.ru.nl
DOWN    ds3-huyg.net.science.ru.nl
DOWN    elabrolabvm.science.ru.nl
DOWN    filemakervm.science.ru.nl
DOWN    home3.science.ru.nl
DOWN    jupytervm.science.ru.nl
DOWN    mariavm02.science.ru.nl
DOWN    mossvm.vm.science.ru.nl
DOWN    rootcell.science.ru.nl
DOWN    vmhost03.science.ru.nl
DOWN    vmhost04.science.ru.nl
DOWN    vmhost07.science.ru.nl
DOWN    zammad3.vm.science.ru.nl

Planned network disruption on some server networks

Simon Oosthoek — Mon, 16 Jun 2025 00:00:00 +0200

Services on networks that are currently behind our old picos switches will be transferred to be behind our firewalls. This may take a few minutes of downtime due to the changes needed for moving the gateway functionality.

If all goes well, it will be one period of a few minutes, if we need to roll back and fix things, there may be a repeat downtime.

All went well, there was a few minutes of interruption on the cncz homepage due to ARP caching issues.

Authentication downtime

Erik Visser — Fri, 06 Jun 2025 00:00:00 +0200

Following the recent update of our LDAP servers certificates, multiple users have reported authentication failures when attempting to log in via RADIUS.

The issue appears to affect only users with RADIUS-based authentication, while LDAP-based authentication continues to function successfully.

Affected users typically include HFML technicians, guest console users, and Science logins with Wi-Fi access.

Servers unreachable

Simon Oosthoek — Mon, 19 May 2025 00:00:00 +0200

In the transition from the older 25Gbit switches to newer switches, two blocks of 4 connections were temporarily unavailable due to a peculiarity of the switches where ports are operated in blocks of 4. The removal of an unused cable triggered a disabling of the other ports in the bunch, reseating one of these connections in one case and re-inserting the unused cable fixed the block of ports to start working again.

/vol/astro2 and /vol/astro5 unavailable

Erik Visser — Mon, 12 May 2025 00:00:00 +0200

Due to errors on the /vol/astro2 filesystem, we had to reboot the fileserver comas1 and take it offline to perform repairs. During this process, both /vol/astro2 and /vol/astro5 have been unavailable.

/vol/astro6 and /vol/astro7 unavailable

Eric Lieffers — Mon, 12 May 2025 00:00:00 +0200

Due to errors on the filesystem /vol/astro7, we had to reboot the fileserver comas2, and take in offline to run repairs on the filesystems. During those repairs /vol/astro7 and /vol/astro6 are not available.

Last time this happend it took 24 hours to complete

Gitlab/Mattermost/Pages offline after package updates

Simon Oosthoek — Fri, 18 Apr 2025 00:00:00 +0200

After the package for gitlab was updated, the service did not come up again, due to a problem with post-install scripts. After rebooting the package installation could finish and service was restored.

DHCP on 25Gbit not working

Simon Oosthoek — Mon, 07 Apr 2025 00:00:00 +0200

Due to a change in the network setup the dhcp servers for the 25 gbit networks (which are being changed to new switches and a new management method) the dns servers were unreachable by the dhcp servers. This caused the dhcp servers to be unable to resolve the names in the configuration to their ip addresses. It was fixed by a change in the central firewall.

ubuntu 20 docker broken

Simon Oosthoek — Tue, 28 Jan 2025 00:00:00 +0100

A package upgrade broke the internal networking on our ubuntu 20.04 docker container VMs. Internal name resolution does not work anymore.

lilo7 access with managed SSH keys

Arnoud Thörig — Wed, 04 Dec 2024 00:00:00 +0100

We have updated the authentication process for login server lilo7 (lilo7.science.ru.nl) as part of a test.

As of now, your ~/.ssh/authorized_keys file on lilo7 is ignored. This change is designed to prevent bad actors from easily gaining persistent access.

The good news is that we can now grant access to lilo7 from outside the previously very restricted IP ranges.

To gain access from a new IP range, you will need to provide us with your public SSH key and the IP range you will be connecting from.

Science services behind eduVPN

Tue, 26 Nov 2024 00:00:00 +0100

De following Science services will be placed behind eduVPN. This means if you want to have continued access, you must use this VPN.

De volgende Science services worden achter eduVPN gezet.

DHZ/DIY
SMTP (mail sending)
LILOs
Webmail

If you have questions are experience problems with setting up eduVPN, please contact our helpdesk.

Science VPN offline with immediately effect

Tue, 26 Nov 2024 00:00:00 +0100

Due to circumstances we are forced to shut the Science VPN with immediately effect.

See this news article where we previously said this would happen on December the 1st.

If you have questions are experience problems with setting up EduVPN, please contact our helpdesk.

rainloop discontinued

Simon Oosthoek — Fri, 22 Nov 2024 00:00:00 +0100

The webmail interface called “Rainloop” has been permanently shut down, due to the upstream project not being maintained anymore. We don’t have the resources to fix security issues or take over the open source project ourselves.

Temporary RADIUS Service Disruption

Bram Daams — Fri, 04 Oct 2024 00:00:00 +0200

The RADIUS service was temporarily unavailable after an automatic update of the FreeRADIUS package. RADIUS is used for authentication of Eduroam, OpenVPN, and Vpnsec when using Science accounts. The issue was automatically resolved when our RADIUS configuration was pushed to the RADIUS servers.

milton fileserver down

Simon Oosthoek — Mon, 30 Sep 2024 00:00:00 +0200

Due to a CPU crash, the milton fileserver was down overnight. After a manual reboot the server was ok again.

fileserver bushel crashed

Simon Oosthoek — Sat, 21 Sep 2024 00:00:00 +0200

The fileserver we call bushel crashed for as yet unknown reasons on Friday evening. From then until the manual reboot, all its shares were down.

All affected shares:

/vol/acfiles
/vol/aerochem
/vol/aggrgtindroplets
/vol/akolkmandata
/vol/albert
/vol/amsbackup
/vol/amscommon
/vol/archivespruijt
/vol/avance600
/vol/beevee
/vol/bioniccell
/vol/bio-orgchem
/vol/bko
/vol/boltje
/vol/ccz
/vol/cczhelpdesk
/vol/cczsecr
/vol/celbio1
/vol/celbio2
/vol/celbio3
/vol/chemprac
/vol/chemstaff
/vol/coastoffon
/vol/cognac
/vol/communicatie
/vol/cursusdossiers
/vol/desda
/vol/devneurob
/vol/diafragma
/vol/dis
/vol/ecogen
/vol/ehef1
/vol/ehef2
/vol/ehef3
/vol/ehef4
/vol/epsylon
/vol/femtospin
/vol/frequencycomb
/vol/fsr
/vol/gissig
/vol/gmi
/vol/goudsmit1
/vol/goudsmit4
/vol/goudsmitoldusers
/vol/grafting
/vol/hafnerlab01
/vol/hafnerlab02
/vol/hafnerlab03
/vol/heatgenes
/vol/heatyield
/vol/hfml-backup
/vol/hfml-backup2
/vol/hfml-backup3
/vol/hfml-backup4
/vol/hfml-backup5
/vol/highres
/vol/honours
/vol/huckgroup
/vol/icis
/vol/ifl-d1
/vol/imappdir
/vol/imappdocs
/vol/imappsecr
/vol/immbureau
/vol/immstaff
/vol/impuls
/vol/introcie
/vol/invbio
/vol/invbio2
/vol/isptinspec
/vol/jupyter
/vol/kaartenbak
/vol/kangoeroe
/vol/khaj
/vol/korevaarstudents
/vol/kuijperslab01
/vol/linova500
/vol/linova800
/vol/liquids
/vol/mbaudit1
/vol/mbaudit3
/vol/mbaudit4
/vol/mbonderwijs
/vol/mbsl
/vol/mlf90
/vol/mlfalgemeen
/vol/mlfsec
/vol/modprspm19
/vol/molbiol
/vol/morph
/vol/mpcoldmollab
/vol/mwstudiereis
/vol/neuroinf
/vol/neuroinf2
/vol/nmr500data
/vol/nmrliquid400
/vol/nmrliquid500
/vol/nmrsolid000
/vol/nmrsolid300
/vol/nmrsolid400
/vol/nmrsolid600
/vol/nmrsolid850
/vol/nmrsolid950
/vol/nmrspectra
/vol/oii
/vol/olcii-archief
/vol/olympus
/vol/pcsm-share
/vol/pcsoftmattersecr
/vol/ppimm
/vol/pracnat
/vol/ruversatest
/vol/sbs
/vol/secrmolbiol
/vol/sigma
/vol/sigmacies
/vol/snn3
/vol/snn4
/vol/soheto
/vol/solids
/vol/spacdata3
/vol/spectra
/vol/spibdossier
/vol/spm
/vol/spm30cobp
/vol/spm30febp
/vol/spm30fecobp
/vol/spm30mkdata1
/vol/spm30mkprojects1
/vol/spm30procobp23
/vol/spmdata3
/vol/spmdata4
/vol/spmdata5
/vol/spmdata6
/vol/spmdata7
/vol/spmexfoliation
/vol/spmmagstm
/vol/spmnanolab
/vol/spmstaff
/vol/ssi
/vol/staff-hfml
/vol/stroom
/vol/studiereis2013
/vol/sucudo-transfer
/vol/tartarus
/vol/tf2devices
/vol/thalia
/vol/ucm
/vol/unl
/vol/uscm-ssi
/vol/uscm-ssi-archive
/vol/vsc
/vol/vsc1
/vol/vsc10
/vol/vsc11
/vol/vsc12
/vol/vsc13
/vol/vsc14
/vol/vsc15
/vol/vsc16
/vol/vsc2
/vol/vsc3
/vol/vsc4
/vol/vsc5
/vol/vsc6
/vol/vsc7
/vol/vsc8
/vol/vsc9
/vol/vscadmin
/vol/wiskalg
/vol/wiskunde

Ubuntu PC password files not updated

Erik Visser — Fri, 06 Sep 2024 00:00:00 +0200

With the introduction of Ubuntu 24.04 by C&CZ, it turned out that the uids/gids of some users had to be changed because they were too small. This was immediately rolled out on servers, since those are always powered on, but needed a workaround on Ubuntu client PCs.

Today it became clear that this workaround ignored new accounts, so first year students and new employees couldn’t use Ubuntu clients. This has been fixed now and all C&CZ managed Ubuntu client PCs in computer labs and the Library of Science have been updated.

gitlab pages unavailable due to failure of the rados gw

Simon Oosthoek — Thu, 05 Sep 2024 00:00:00 +0200

Gitlab pages was unavailable due to a problem with the underlying S3 storage. It took a while to fix, due to confusing messages in the logfiles of the service. title: gitlab pages onbruikbaar door storing in rados gw

gitlab pages broken by failing backing store

Simon Oosthoek — Fri, 16 Aug 2024 00:00:00 +0200

The backing store for gitlab-pages, accessed via the S3 protocol, went into failure mode for still unknown reasons. After restarting several servers of the storage cluster, the service became available again.

update 20-08-2024

It happened again Today at 12:15, fixed quicky (within 20 min)

e-mail delivery issues

Simon Oosthoek — Tue, 13 Aug 2024 00:00:00 +0200

Recently we have received and confirmed issues with delivering e-mail to certain domains, so far:

adelaide.edu.au
applemusic.com
arizona.edu
benwhale.com
biologists.com
communications.ssrn.com
editorialmanager.com
email.arizona.edu
flinders.edu.au
herts.ac.uk
hireright.com
hpeprint.com
inhousevtm.com
insideapple.apple.com
ioppublishing.org
mail.coinbase.com
math.arizona.edu
medicalschemes.com
mq.edu.au
newcastle.edu.au
nonzeroratio.com
skao.int
sussex.ac.uk
sydney.edu.au
telegram.com
uct.ac.za
unimelb.edu.au
uon.edu.au
uow.edu.au
uws.edu.au
vistaprint.com
webofscience.com
westernsydney.edu.au
woodpartners.com
woolcock.org.au

The mechanism by which this fails is getting clearer, but we are still figuring out how to deal with this. What most of these domains have in common is that they are hosted by the same service provider (mimecast.com). It appears to be something with the negotiation failing before the mail can be sent, which was no problem before as mails took another route after delivery failure.

Science VPN problems

Bram Daams — Mon, 15 Jul 2024 00:00:00 +0200

After a network configuration update, we are currently experiencing issues with the Science VPN connections, resulting in no traffic passing through. We are investigating the cause of this issue. As an alternative, we recommend using EduVPN from SURF.

Email sent to too many people

Bram Daams — Mon, 17 Jun 2024 00:00:00 +0200

We apologize for the inconvenience, but due to a software error, an (end date notification email)[https://cncz.science.ru.nl/en/howto/emailcodes/#login-3] was mistakenly sent to too many recipients. Please disregard and delete the email. If the email was intended for you, we will resend it next week. We are working on a fix to prevend this kind of error in the future.

ldap services down

Simon Oosthoek — Sat, 15 Jun 2024 00:00:00 +0200

Science LDAP services are all unavailable, our radius service depends on this, so this also doesn’t work.

The TLS certificate has expired, after renewing the certificate and restarting the service it works again.

Network shares of fileserver bushel could not be mounted login servers and cluster nodes

Bram Daams — Mon, 10 Jun 2024 00:00:00 +0200

Due to a configuration error on the fileserver bushel, the NFS process did not run after a reboot. After correcting this error, the NFS service has been restarted.

DHCP broken in a part of the datacenter

Simon Oosthoek — Mon, 03 Jun 2024 00:00:00 +0200

After a software upgrade and reboot of the AR-114-9 modules, dhcp relay from the Picos network is broken. The cause is still not 100% clear, we will update this incident when we know more.

We tried setting up dhcp servers within the picos network, but this didn’t solve the issue. We now suspect the picos switch itself, which we will try to restart in the morning of June 6th (between 7:30 and 8:30). This will affect all connections behind the picos switches. We hope the downtime will be less than 10 minutes.

license server down

Mon, 29 Apr 2024 00:00:00 +0200

Due to a configuration error the licenseserver for several of our licenses did not start automatically. Previously the server did autostart, however the cause of the configuration change is unknown. The configuration of the license server has been fixed.

license server down

Simon Oosthoek — Wed, 24 Apr 2024 00:00:00 +0200

Due to some unexpected reaction to a config change, the licenseserver for several of our licenses was misconfigured for about 40minutes. The change was reverted, but it took long enough for users to run into license problems.

restore old ceph shares to new locations

Simon Oosthoek — Thu, 18 Apr 2024 00:00:00 +0200

Even though we closed CPK #1359, the user’s shares are not all restored. We have recovered the data from cephfs to temporary storage that is not accessible to users, it will take a bit more time to find new permanent locations for this data.

The storage will not be cluster based anymore, but single server ZFS with snapshots.

Some network shares temporary unavailable

Bram Daams — Thu, 18 Apr 2024 00:00:00 +0200

Some of our network shares were temporarily unavailable earlier due to a configuration error. The issue has been resolved and all shares should be accessible now. We want to assure you that the configuration error that occurred did not impact the integrity or security of the data stored on the file server. We apologize for any inconvenience this may have caused. In the meantime, we’re working on making information about network shares readily available on DIY for your reference.

Brief disruption to Matlab Licensing this morning

Bram Daams — Mon, 15 Apr 2024 00:00:00 +0200

Some users may have experienced difficulty launching MATLAB this morning due to a temporary issue with the license server. We identified the problem and resolved it by manually restarting the server. MATLAB licensing should now be functioning normally. We apologize for any inconvenience this may have caused.

Sieve difference(s) current mailbox server

Erik Visser — Fri, 05 Apr 2024 00:00:00 +0200

After migrating to a new mailbox server on Sunday, March 31, we noticed different behaviour in the processing of mailbox filters (by the Sieve software).

Firstly, the ‘include’ of global spam+virus filtering rules had a side-effect which has been fixed using a newer version of the software.

Secondly, the default behaviour of the ‘fileinto’ statement underwent a change: previously, it featured an implicit ‘stop’ statement, but now it requires an explicit ‘stop’ statement after the ‘fileinto’ to achieve the same effect.

homeserver home1 nfs problem

Simon Oosthoek — Mon, 01 Apr 2024 00:00:00 +0200

home1 had a problem with the nfs server after the reboot. Another reboot fixed it.

mailserver will be replaced Easter weekend

Simon Oosthoek — Fri, 22 Mar 2024 00:00:00 +0100

In Easter Weekend (March 30th 2024) the new mailserver will replace the old one. The final switch will take a short while, during which mail clients (eg Thunderbird, mail on your phone, etc.) will not be able to connect to the science mailserver. (This is a week later than announced, due to some issues that need to be resolved before final migration)

Central mail services (Microsoft @ru.nl) are independent of our mailserver and thus will not be affected at all.

Ceph filesystem failure

Fri, 19 Jan 2024 00:00:00 +0100

Ceph filesystem failure.

Services required for the CephFS filesytem cannot start. Failure of these services wil cause failure to access files on Ceph.

We are in contact with our support party 42on to remedy the problem. Duration is currently unknown.

Update 2024-01-22

We’re working with 42on on the issue and have a meeting scheduled for 17:00 today.

Update 2024-01-23

An initial dentry_recover was successful and according to 42on the Ceph journals are OK. We are hopeful that we can fix the issue with Ceph in the coming days.

GitLab TLS Issue

Mon, 11 Dec 2023 00:00:00 +0100

On December 10, 2023 the main GitLab TLS certificate expired. A new certificate was generated, but it lacked a proper certificate chain. Browsers disregard that chain, but other GitLab components do care, which meant that the registry and the runners were unable to connect to GitLab.

On Monday the 11th we renewed all certificates using Let’s Encrypt. Let’s Encrypt will also auto-renew these.

jitsi videoconferencing unavailable

Simon Oosthoek — Wed, 06 Dec 2023 00:00:00 +0100

Jitsi (or actually the internal prosody service) had an old certificate, causing it to refuse startup after the normal reboot. The expired certificate was replaced and after a restart of the prosody service it works again.

Network problems after maintenance on the RU core router

Simon Oosthoek — Mon, 04 Dec 2023 00:00:00 +0100

After maintenance on the central routers, our servers on the 25Gbit network couldn’t send traffic to the internet. The cause or nature of the problem appears to be in the central RU routing. By resetting the interface to our network, RU Connectivity can fix this, when it occurs again. RU Connectivity contacted their maintenance provider to investigate this problem.

Final solution: After a few failed attempts, it was possible to define a static route in the central routers to our networks.

Jupyterhub restarted

Simon Oosthoek — Fri, 17 Nov 2023 00:00:00 +0100

Some users were unable to login, the server had become unstable, which made a reboot necessary. Unfortunately, the jupyterhub service still needs a manual startup, which didn’t happen until 13:24 hours. The service is now working again. We are working to fix the reboot problem.

DHCP server down due to config error

Thu, 09 Nov 2023 00:00:00 +0100

A typo that could propagate to shutdown the DHCP server had the effect that some network devices did not get an IP address when they were switched on and that others, whose address lease expired in this period, lost their IP address and thus their access to the network..

We’ll improve the process to prevent a typo from bringing down the DHCP server in the future.

Jupyterhub22 refused to start this morning

Bram Daams — Wed, 08 Nov 2023 00:00:00 +0100

After a scheduled reboot of the machine running jupyterhub, jupyter failed to start properly. The service started after invoking a manual start command. The measures taken to resolve this recurring problem have not proven to be sufficient.

DNS broken for z.science.ru.nl

Mon, 23 Oct 2023 00:00:00 +0200

Due to a misconfiguration of the DNS in the z.science.ru.nl zone, all shares were not available during the outage.

Extra tests are added to prevent a future occurrence.

All Science services down 15 minutes Thursday Oct 19 07:00-07:15 due to router reboot

Wed, 18 Oct 2023 00:00:00 +0200

The router for most Science services urgently needs a reboot. This has been scheduled for early morning. In the unlikely case that this reboot fails, ILS Connectivity is on campus to fix it.

Mailman not accepting messages

Erik Visser — Mon, 02 Oct 2023 00:00:00 +0200

In preparation of the migration to Microsoft Exchange Online Protection (MS EOP), we added another mail exchange server (mx5) to be addressed directly by MS EOP. However, having had mx4 in production for some time, we forgot to test mx5 in conjunction with our mailman server (zaaivm). This could result in a bounced mail ’not accepting messages’ for FNWI users (employees and students) using @ru.nl addresses.

Meanwhile, mx5 has been made known to zaaivm thus resolving this issue.

Jupyterhub22 refused to start this morning

Bram Daams — Wed, 27 Sep 2023 00:00:00 +0200

After a scheduled reboot of the machine running jupyterhub, jupyter failed to start properly. The service started after invoking a manual start command.

Roundcube problem

Tue, 26 Sep 2023 00:00:00 +0200

After an upgrade of Roundcube it was temporarily not possible to send e-mails.

After a small change in the configuration file the problem was resolved.

Poster printer kamerbreed broken

Fri, 22 Sep 2023 00:00:00 +0200

The poster printer kamerbreed was broken. It couldn’t be fixed, so we ordered a new HP Designjet Z6 44 inch postscript. The ordering and delivery took way too long, the new poster printer was in operation on November 13. The new printer is twice as fast, so the speed of service will be even better.

When our poster printer breaks down, users that need posters can go to Radboud Post&Print.

Jupyter and Chemotion offline

Bram Daams — Wed, 20 Sep 2023 00:00:00 +0200

For an unknown reason, the Jupiter and Chemotion servers didn’t start. We are currently looking at how we can start the machine again.

The virtual machines have been rebuild, based on their existing disks.

Ceph performance degraded due to broken storage node

Simon Oosthoek — Mon, 18 Sep 2023 00:00:00 +0200

Ceph filesystem storage is experiencing reduced performance, because one of the storage nodes is currently offline. A combination of factors causes this to affect performance, fullness, uneven distribution of data over the storage nodes. We expect this to be resolved when the node is back in the cluster.

update 7 Oct 2023

The cluster is mostly complete again for a while already, but there’s a lot of remaining issues to be resolved by the cluster. Performance may or may not be affected…

Increased chance of SPAM/phishing mails in Science mailbox

Erik Visser — Thu, 14 Sep 2023 00:00:00 +0200

The RU contract with the anti-SPAM/anti-phishing service Proofpoint expired September 21, 2023, which means that ‘Proofpoint End User Digest’ mails are not sent anymore after that date.

C&CZ is migrating Science mailboxes to the anti-SPAM/anti-phishing service of Microsoft Exchange Online Protection. There is an increased chance of receiving SPAM/phishing mails in the Science mailbox during this migration period.

P.S. the RU central mailboxes (with addresses ending in @ru.nl) have already been migrated to Microsoft Exchange Online Protection.

Science and Radboud Newsletter in Spam

Thu, 07 Sep 2023 00:00:00 +0200

The Science newsletter is sent from an @ru.nl mail-account. Recently, the @ru.nl mailservice was changed by adding Exchange Online Protection (EOP). EOP is the successor for Radboud University to the Proofpoint e-mail security filter. EOP added a mail header line to the newsletter

List-Unsubscribe: https://c8ce19ec62454d21b73b7d5a25559d8f.svc.dynamics.com/t/lu/7...

which made the mail appear to be spam to the Science spamfilter SpamAssassin.

1.1 URI_HEX URI: URI hostname has long hexadecimal sequence

When we noticed this, we fixed it by welcome listing (allowlisting/passlisting) the sender address communications-science@ru.nl on September 7, so this mail will never be falsely classified as spam. The Radboud Weekly newsletter had the same problem, so communicatie@ru.nl was also put on the welcome list on September 14.

Some home directories not available

Bram Daams — Mon, 04 Sep 2023 00:00:00 +0200

After the Monday morning reboot, the NFS server on home1 refused to start properly. We are investigating why a manual restart of nfs was needed.

Missed RU mail due to stopped external forwarding

Tue, 22 Aug 2023 00:00:00 +0200

RU mail management let us know that yesterday the forwarding to external (non-RU) mail addresses has been stopped as announced earlier. Unfortunately, mail for several dozens of Science users was/is not forwarded to the Science mailservers. These mails can still be found in MS365 (RU mail), either in the Inbox or in the Deleted Items. RU mail management promised that the forwarding will be corrected tomorrow.

cpu replacement vmhost06

Mon, 14 Aug 2023 00:00:00 +0200

Announcement of maintenance, Wednesday afternoon we are going to replace the cpu of one of our main vmhost servers, meaning vms

gitlab9 (pep) slurm22 pep3 jitsivm poliep indicoimapp2vm pep4 mariavm01 smtp2

will be down for up to 1 hour. several services depend on the mariavm01 (websites, slurm), so they will be affected too.

motherboard replacement vmhost06

Simon Oosthoek — Tue, 01 Aug 2023 00:00:00 +0200

Apologies for the short notice, we are now going to replace the motherboard of one of our main vmhost servers, meaning vms

gitlab9 (pep) slurm22 pep3 jitsivm poliep indicoimapp2vm pep4 mariavm01 smtp2

will be down for up to 1 hour. several services depend on the mariavm01 (websites, slurm), so they are affected too.

Daily backups offline

Simon Oosthoek — Thu, 27 Jul 2023 00:00:00 +0200

Our daily backup system relies on cephfs storage, which is currently offline, see CPK#1337.

This means that as of July 22nd we are unable to perform or restore daily backups. When the cephfs problems are resolved the daily backups should also be OK and restorable again.

NB, this has no effect on the Monthly backups, which continue to work normally.

Cephfs offline

Sat, 22 Jul 2023 00:00:00 +0200

After the power down of the Huygens building we are experiencing a problem with bringing Ceph file system back online.

We currently do not know when the Ceph cluster is operational again.

Update 2023-08-01 10:30

Ceph is working again. This CPK is now closed. CPK#1338 is also closed.

Update 2023-07-31 12:30

After some more support from 42on, we managed to restart the cephfs, we cannot be sure all files are there, but almost all files are. Please let us know if you are missing crucial data, in theory we can find and restore the data, though we have to process 500 million entries to find it, which will take a long time.

VPN service downtime

Tue, 04 Jul 2023 00:00:00 +0200

The VPNsec service will be moved to a new server. This move will cause downtime and existing VPN connections will be destroyed. Downtime is expected not to exceed several minutes.

Mailman disruption

Mon, 03 Jul 2023 00:00:00 +0200

Last friday, a change in the mailman configuration has been rolled out which had the inadvertent effect that mails were not delivered to external addresses anymore. However, these mailman posts were sent successfully to internal Science mail addresses.

The change has been rolled back for the moment but is a necessity meaning that we’re looking for another solution.

router change for most Science services (dr-huyg)

Simon Oosthoek — Tue, 27 Jun 2023 00:00:00 +0200

The connecting router (dr-huyg) for all servers in the subnets 131.174.30.0/24, 131.174.31.0/24 and 131.174.16.128/26 will be replaced.

It is expected that this will cause an interruption of ca. 10 minutes in the connectivity, but unforeseen circumstances may increase this delay.

The reason to do this now is because of the planned power interruption on July 22. The old router hardware has a high probability of failing to survive this.

Science IT services down July 21 and 22 - Huygens building power outage

Tue, 20 Jun 2023 00:00:00 +0200

Friday July 21 from 17:00, we will start shutting down compute clusternodes, in order to prepare for the power outage of the Huygens building Saturday July 22. Other servers will be shut down later. The most important servers (mail, home, file, Ceph, gitlab, loginservers) will be shutdown starting Saturday morning 7:00. We will try to keep basic services (DNS/DHCP, SMTP(mail) and license servers) up during this power outage. RU services are not serviced from the Huygens building, so will not be affected.

Certificate of authentication server expired

Bram Daams — Thu, 15 Jun 2023 00:00:00 +0200

Due to the expiration of an LDAP certificate, it is temporarily not possible to log in to various services. A new certificate is being installed urgently.

Affected services include Eduroam in combination with Science logins, VPN, GitLab and Mattermost.

Downtime Felixdisk and bioboost

Eric Lieffers — Wed, 14 Jun 2023 00:00:00 +0200

Due to a failure in a power distribition unit (pdu) the servers felixdisk and bioboost went down.

Both servers have been connected to another pdu and are now working again.

networking problems due to routing change

Simon Oosthoek — Mon, 12 Jun 2023 00:00:00 +0200

The planned routing change, which should not have caused issues for more than a few seconds, didn’t work as planned and caused problems for up to 15 minutes.

Update 2023-06-12 - 22:00

The situation has become worse, some problems: DNS resolving, some fileservers and jupyterhub are having problems due to the network change. We will attempt to resolve the issue asap.

Update 2023-06-13 - 11:30

After correcting errors (fixed IP addresses) all services are up again. The network configuration is not redundant yet, but user problems should be over.

DDOS on Science mailservers

Simon Oosthoek — Wed, 31 May 2023 00:00:00 +0200

Our smtp mailservers were under attack. In order to prevemt other problems, our configuration limits the number of connections that can be kept open at the same time. We cannot easily distinguish between connections by the attacker(s) and by regular users. When this limit is reached, no new connections can be made.

Therefore sending e-mail using our mailservers can take a long time or will not work at all. There’s a good chance that your IP address will be blocked (max. 1 hour), e.g. if you use loginname@science.ru.nl after June 8, 16:25.

Climate control failure in Huygens Datacenter

Simon Oosthoek — Tue, 30 May 2023 00:00:00 +0200

The datacenter cooling failed around 07:00 this morning. To prevent damage all non-essential systems are being turned off (clusternodes first).

Most fileservers have also been turned off. Due to the urgency, some systems that have been turned off may not be in the location with the problem (Huygens HG04.070).

Around 07:50 the cooling system came online again, about 30 minutes later the temperature dropped to under 25 degrees Celsius.

After ca. 8:35, we started turning systems back on, starting with fileservers and later clusternodes as well. This is now nearly finished. Some hardware cannot easily be managed remotely so that will take a bit longer until someone can turn on the power manually.

Saturday July 22 - Power interruption Huygens building

Simon Oosthoek — Tue, 16 May 2023 00:00:00 +0200

On Saturday July 22, all power, including emergency power, in the Huygens building will be switched off for planned maintenance from 08:00 until 12:30. If all goes well, power will be available from 12:30. We expect to need at least a few hours to get all Science services up again. During this time, all but a few basic services will be unavailable. Most servers will be turned off the Friday before. We will attempt to maintain basic DNS/DHCP, SMTP(mail) and license services, but for instance the science mailboxes, home directories, websites, VPN, fileservers, Gitlab, loginservers and compute clusters will not be available during the off period.

Radboud internet 30 minutes down for maintenance Friday April 21 18:30-20:30

Thu, 20 Apr 2023 00:00:00 +0200

RU ILS Connectivity announced that Friday April 21 between 18:30 and 20:30 the RU network would be separated from the internet for a period of ca 30 minutes, in order to carry out maintenance.

Cancelled: GitLab down April 21 10:00-12:00 (maintenance)

Thu, 13 Apr 2023 00:00:00 +0200

Friday`April 21 at 10 o’clock C&CZ will commence a GitLab server migration. During the migration, working with Gitlab will not be possible.

Because Mattermost is running on the same installation of Gitlab, it will be unavailable during the process as well.

We expect things to be up and running again around noon.

FNWI network maintenance Wednesday April 12 19:00-23:00 Huygens in/near wing 8

Tue, 11 Apr 2023 00:00:00 +0200

RU ILS Connectivity announced that network maintenance will be carried out next Wednesday evening that will cause short disturbances of network connectivity for wired and wireless systems in and around wing 8 Huygens on all floors. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 113- for affected systems) or visiting FNWI ethergids.

FNWI network maintenance Monday April 3 19:00-23:00 Huygens in/near wing 3 except ground floor

Thu, 30 Mar 2023 00:00:00 +0200

RU ILS Connectivity announced that network maintenance will be carried out next Monday evening that will cause short disturbances of network connectivity for wired and wireless systems in and around wing 3 Huygens on all floors except the ground floor. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 107- for affected systems) or visiting FNWI ethergids.

FNWI network maintenance Wednesday April 5 19:00-23:00 Huygens in/near wing 7

Thu, 30 Mar 2023 00:00:00 +0200

RU ILS Connectivity announced that network maintenance will be carried out next Wednesday evening that will cause short disturbances of network connectivity for wired and wireless systems in and around wing 7 Huygens on all floors except the ground floor. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 110- for affected systems) or visiting FNWI ethergids.

Poster printer kamerbreed broken

Fri, 24 Mar 2023 00:00:00 +0100

March 20, the poster printer kamerbreed started producing bad colours. The new print heads that were received a fw days later didn’t solve the problem. Therefore we called for a repair, that will probably be scheduled in the week of April 17.

In the meantime, users that need posters could go to Radboud Post&Print.

FNWI network maintenance Monday March 6 19:00-23:00 Huygens ground floor in/near wing 3 and tunnels/basement A1

Fri, 03 Mar 2023 00:00:00 +0100

RU ILS Connectivity announced that network maintenance will be carried out next Monday evening that will cause short disturbances of network connectivity for wired and wireless systems in and around wing 3 Huygens and in tunnels/basement of the former A1 building. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 109- and 73- for affected systems) or visiting FNWI ethergids.

FNWI network maintenance Wednesday March 8 19:00-23:00 Goudsmit/NMR and Nanolab

Fri, 03 Mar 2023 00:00:00 +0100

RU ILS Connectivity announced that network maintenance will be carried out next Wednesday evening that will cause short disturbances of network connectivity for wired and wireless systems in Goudsmit/NMR and Nanolab. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 72- or 116- for affected systems) or visiting FNWI ethergids.

No ip address on the wired network in Huygens

Simon Oosthoek — Fri, 24 Feb 2023 00:00:00 +0100

The network configuration change of Thursday evening caused wired network connections to fail getting an IP address. The wireless network didn’t have that problem.

Technically, the dhcp options we’re using were not yet allowed on the new distribution switch (DR-H).

Update: the problem should be resolved, but in case of problems, please try rebooting the machine to resolve the issue.

Reboot cn106

Wim Janssen — Thu, 23 Feb 2023 00:00:00 +0100

Because of problems and reacheability machine rebooted.

Cause of problem remains unknown.

FNWI network maintenance Thursday Feb 23 19:00-21:00

Tue, 21 Feb 2023 00:00:00 +0100

Thursday evening between 19:00 and 21:00, all wired and wireless user networks in Huygens, Goudsmit/NMR, Nanolab and FELIX-HFML will be switched to a new router. A downtime of ca. 10 minutes is expected.

Wing 5 Huygens network maintenance Wednesday Feb 15 19:00-20:30

Fri, 10 Feb 2023 00:00:00 +0100

RU ILS Connectivity announced that network maintenance will be carried out next Wednesday evening that will cause short disturbances of network connectivity for wired and wireless systems in and around Wing 5 Huygens. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 111- for affected systems) or visiting FNWI ethergids.

Homeserver nfs service failed to come up

Simon Oosthoek — Mon, 06 Feb 2023 00:00:00 +0100

After the scheduled reboot of the home2 server, the nfs service failed to start. A manual reboot solved the problem.

Wing 1 and 2 Huygens network maintenance Monday Jan 30 and Wednesday Feb 1 19:00-23:00

Thu, 26 Jan 2023 00:00:00 +0100

RU ILS Connectivity announced that network maintenance will be carried out next Monday (Wing 1) and Wednesday (Wing 2) evening that will cause short disturbances of network connectivity for wired and wireless systems in Huygens. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 105- for Wing 1 affected systems and 106- for Wing 2) or visiting FNWI ethergids.

Wing 4 Huygens network maintenance Monday January 23 19:00-23:00

Sun, 22 Jan 2023 00:00:00 +0100

ILS Connectivity announced that network maintenance will be carried out next Monday evening that will cause short disturbances of network connectivity for wired and wireless systems in and near wing 4 of the Huygens building. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 108- for affected systems) or visiting FNWI ethergids.

Maintenance RU internet connection Friday Feb 10 18:00-23:00

Fri, 13 Jan 2023 00:00:00 +0100

ILS Connectivity announced that network maintenance will be carried out on the internet connection of Radboud University on February 10 between 18:00 and 23:00. Network connections may be disturbed several times, each lasting less than 15 minutes.

Wing 6 Huygens network maintenance Monday January 16 19:00-23:00

Thu, 12 Jan 2023 00:00:00 +0100

ILS Connectivity announced that network maintenance will be carried out next Monday evening that will cause short disturbances of network connectivity for wired and wireless systems in and near wing 6 of the Huygens building. For wired network outlets, one can check whether they will be affected by looking at the outlet number (starting with 112- for affected systems) or visiting FNWI ethergids.

Unplanned reboot of vmhost

Bram Daams — Fri, 16 Dec 2022 00:00:00 +0100

After a disk was replaced of a vmhost, a reboot was needed to actually see the disk. Unfortunately, this did not help. A second try, with another disk worked and the machine is rebuilding the RAID.

Accounts on Linux login servers restricted to recent users

Fri, 09 Dec 2022 00:00:00 +0100

Recently, some Science accounts were hacked, after users fell for a phishing mail. These accounts were subsequently abused by internet criminals to send mails from our Linux loginservers. Therefore, we have removed the ability to login on these login servers for all Science accounts that didn’t use these servers anyway. Please contact C&CZ if you want to be able to login to the Linux login servers.

Sending E-mail from lilo blocked

Simon Oosthoek — Wed, 07 Dec 2022 00:00:00 +0100

To deal with the ongoing abuse of science accounts for spam, we have disabled the possibility to send e-mail from the linux login servers. This means that the lilos cannot be used anymore to run mutt/alpine or as socksproxy for mail clients.

For users with cronjobs, note that cronjobs cannot send output via e-mail anymore. Please change your cronjobs to provide feedback in a different way or discuss an alternative solution with postmaster.

Outgoing mail not possible after successful phishing attack

Bram Daams — Tue, 06 Dec 2022 00:00:00 +0100

After several persons fell for a phishing email, several tens of thousands of emails were sent via Science accounts. This causes our mail servers to be blocked by several mail servers on the internet. We are currently cleaning SPAM mails that are still in the outgoing queues. After that, the mail servers will be made available again. We are considering measures to prevent this form of nuisance.

Jitsi certificate mismatch

Simon Oosthoek — Tue, 29 Nov 2022 00:00:00 +0100

Jitsi was unavailable after the reboot, because one of the web-interface configurations was still referring to an older certiticate.

Jitsi certificate expired

Simon Oosthoek — Tue, 22 Nov 2022 00:00:00 +0100

The certificate for jitsi.science.ru.nl expired last week and was replaced, but a sub-service was still using a copy of the older certificate. Users were able to open the frontend, but not able to start a meeting. The problem was resolved by updating the copy of the sub-service.

Adobe Acrobat Pro DC license problem

Fri, 28 Oct 2022 00:00:00 +0200

Since midnight there is a problem using Adobe Acrobat Pro DC, you are asked to activate the license. ILS is working on a solution. You can open PDFs using Adobe Reader or some other PDF reader. See: https://meldingen.ru.nl/detail.php?id=1333&lang=en&tg=0&f=0&ii for the original notice.

Ceph storage interruption

Simon Oosthoek — Mon, 24 Oct 2022 00:00:00 +0200

Just before 23:09 quite a lot of ceph storage nodes became unreachable. This seems to be due to one of the redundant links between two datacenter locations failing for about 4 seconds. This triggered a whole slew of ceph osd processes being killed off and not starting again. A generic configuration change made for all our servers generated an extra interface, which confused some of the osd processes (depending on interface ordering) when starting up. We are reasonably confident we can avoid this from happening in the future.

Eduroam wifi down during upgrade

Thu, 20 Oct 2022 23:30:00 +0200

ILS wifi management notified us that they will upgrade the software of wireless access points, because security flaws have been addressed in the latest release.

The maintenance ends on 2022-10-21 01:30.

Source: meldingen.ru.nl

Mail problems because of spam run after phishing

Thu, 29 Sep 2022 19:27:00 +0200

Some time ago, some Science users fell for a phishing mail, therefore their account could be misused by spammers to send mail through our mailservers. Therefore our mailservers ended up in blacklists yesterday evening. Some mails could therefore not be delivered to other servers, that mail was returned to the sender. This morning we removed the IP addresses of our servers from several blacklists, we think that the problems are resolved now.

Eduroam: Tuesday September 27 23:30 1 hour down for maintenance

Tue, 27 Sep 2022 23:30:00 +0200

ILS network management announced: Tuesday September 27 23:30 maintenance is carried out on the Eduroam wireless network in FNWI buildings a.o. In most cases connectivity will be interrupted for at least 30 and at most 60 minutes.

Friday September 23 18:30-22:30 RU firewall maintenance

Fri, 23 Sep 2022 18:30:00 +0200

ILS network management announced: Friday September 23 18.30 – 22.30 network maintenance is carried out on the DMZ and SRV firewalls. Connectivity interruptions of 20 to 40 seconds are expected. In case of problems, interruptions of 5 to 10 minutes might occur.

Network switches failed after a power failure on campus

Thu, 22 Sep 2022 12:20:00 +0200

A few campus-wide dips in the power supply made network and telephony unusable for about 15 minutes, except in server rooms where UPSs guarantee the power supply. Because 7 modules of desktop switches in Huygens/HFML/Felix had become defective, several hundred network outlets will only have worked again after the relevant switch module had been replaced.

Network failure datacenter

Sun, 04 Sep 2022 16:00:00 +0200

Still unclear what the cause is, several links are strongly degraded. Update: A defective switch has been replaced.

Saturday May 14 adjacent buildings (Mercator, Proeftuin, Logistiek) 5 minutes without network

Sat, 14 May 2022 09:00:00 +0200

RU/ILS network management will switch to new hardware. This will lead to a network interruption of at most 5 minutes.

Coma, coma01 and coma46 network problem

Tue, 03 May 2022 13:47:00 +0200

This afternoon three coma nodes lost their network because of an incorrect network configuration. They must have shown intermittent network problems earlier. It took us some time to find out what caused this network problem, but when found, it was easy to fix.

Astro.ru.nl DNS(SEC) service down

Thu, 28 Apr 2022 12:02:00 +0200

During the regular change of DNSSEC keys that secure DNS traffic, an incorrect key was introduced in the external DNS of ru.nl for astro.ru.nl. This made astro.ru.nl disappear from the internet. This error was partly corrected 2022-05-02 at ca. 14:00 hours, but the automatic process used an not accepted encryption. It took ILS until 2022-05-22 to correct that by hand after we eventually noticed the error. Because the DNS answer that astro.ru.nl doesn’t exist may be cached for 24 hours, the problem was not completely over until 2022-05-13 13:00. The main problems for users were that mail from an @astro.ru.nl address bounced.

Network switch of Astro Coma cluster down

Tue, 22 Feb 2022 13:10:00 +0100

The network switch of the Coma cluster seems to be broken, all attached nodes are separated from the rest of the network. We’ll replace the switch a.s.a.p. and (let) analyze the problem after that.

SUSE Linux 15.3 Eduroam doesn't work with U- or s-number, but does with Science account

Mon, 14 Feb 2022 00:00:00 +0100

February 14, ILS switched off antique versions of TLS (1.0 and 1.1) for the Eduroam authentication on ILS LDAP servers. From then on, SUSE Linux 15.3 clients can’t authenticate with U- or s-number. They only have TLS1.2 and the ILS servers offer TLS1.3 first, after that an error occurs. By using the Science-account to authenticate, these users succeed in connecting to Eduroam.

Interrupted link to new datacenter switches

Wed, 15 Dec 2021 12:45:00 +0100

Due to human error, the connection between the new datacenter switches and the central router was interrupted.

vmhost07 poweroff

Thu, 02 Dec 2021 13:10:00 +0100

Vmhost07 was accidentally shut down. Cause: human error.

labservanttest neurotech2 printvm msql01 indicoimapp ldap2 eftw jupytervm

Ceph storage expansion caused performance issues

Tue, 16 Nov 2021 00:00:00 +0100

As a result of the expansion of the Ceph storage cluster, the cluster had performance and availability issues. The problems were resolved this morning.

Server room network switch powerless

Tue, 12 Oct 2021 11:50:00 +0200

Two modules of an important switch in the main C&CZ server room lost power during the preparation of planned maintenance. This disconnected ca. 75% of the servers in the room from the network. Moving the modules to new PDU’s kimited the downtime to ca. 15 minutes.

License server problem

Mon, 11 Oct 2021 04:40:00 +0200

An error in the management software prevented all license processes from starting correctly at the reboot of the license server. After fixing this error, all licenses were available again.

Fileserver 'flock' overloaded

Fri, 17 Sep 2021 14:30:00 +0200

Course software that had been tested caused an overload of the fileserver when it was used by 100 students. The performance of the fileserver was impaired for all users of network shares of this server.

VPN onbereikbaar

Sat, 24 Apr 2021 00:00:00 +0200

A broken PDU has offlined a switch, which has caused the VPN server to be unreachable (and several other things, which don’t affect users).

Central E-mail/Calendar disruption (exchange)

Wed, 14 Apr 2021 09:30:00 +0200

Due to an emergency maintenance, the central microsoft exchange server is unavailable for 4 hours. This may also affect systems that are dependent on exchange. E-mail and calendar functionality is expected to be restored when the maintenance is done around 13:30 Today.

Ceph problem

Wed, 24 Mar 2021 19:00:00 +0100

During a routine upgrade of ceph, a bug in the latest version manifested itself and made the ceph manager unreachable. After aborting the upgrade and with help from the ceph-users mailinglist, everything became available again using a workaround.

Windows 7 computers disabled in B-FAC domain

Wed, 24 Mar 2021 00:00:00 +0100

Because of security issues the last remaining Windows 7 machines wil be disabled, effective 24-03-2021, as member of the Active Directory Domain B-FAC. Please upgrade these computers to a more up-to-date OS.

Lilo7 restart

Wed, 17 Mar 2021 21:00:00 +0100

To change the network of lilo7, we need to reboot this loginserver. If you want a stable connection to a loginserver during this downtime, please use lilo6 or the soon to be taken down lilo5. For more info see the page on C&CZ loginservers.

Host of several virtual servers broken: Roundcube, websites and others

Fri, 05 Mar 2021 07:45:00 +0100

Yesterday the SSD bootdisk of this VM host reported the first problems. This morning this had the effect of stopping all VMs running on this host. By moving the VMs to a different VM host, the problem has been solved. We will investigate how to best prevent this problem in the future or lessen its impact.

Major RU network maintenance Saturday Feb. 27 08:00-20:00

Sat, 27 Feb 2021 08:00:00 +0100

The ISC announced that Saturday February 27 08:00-20:00 major RU network maintenance work will be carried out. This will mean that all RU services will be unavailable several times for at most an hour. This concerns all RU services including those of FNWI/C&CZ: e-mail, VPN, wifi, BASS, OSIRIS, Brightspace, Syllabus+, Corsa, etc.

Lilo6 down

Thu, 25 Feb 2021 17:30:00 +0100

As of Thursday afternoon, the lilo6 is down due to hardware issues. Because lilo6 was the default linux login server (lilo referred to lilo6), this affected many users of lilo. The impact is limited, because we have alternative lilo’s, namely lilo5 and lilo7. As of March 1st lilo now refers to lilo7, ssh will warn about DNS SPOOFING, which is due to the difference host keys for lilo7 ECDSA SHA256:si3g2elo5m6TShx3PjX0+vF50pZ8NK/iXz/ESB+ZeP0

DNS problems from outside with ru.nl

Sun, 21 Feb 2021 07:10:00 +0100

The central DNS servers for ru.nl for external requests had problems, because they received too many requests, which resulted in science.ru.nl and others not being found. DNS names within ru.nl then will not resolve to an IP address. We enlarged some TTLs (Time-To+lives) to try to lessen the problem. These small TTLs were meant to be able to move a service to a new server in case of problems, but now they just make the problem bigger. After starting VPN you won’t notice this problem, because the internal DNS servers that you use then are not affected. Changes to the RU DNS servers hopefully lessened or removed the problems as of 2021-02-23 14:30.

DNS broken for subdomains of ru.nl

Thu, 11 Feb 2021 00:00:00 +0100

DNS-servers for ru.nl did not serve information about subdomains such as science.ru.nl. Thus no DNS-name will resolve to an IP address at FNWI. A workaround is to use as DNS servers: 131.174.224.4 en 8.8.8.8. If you try to connect to a service for the first time after ca 11:15, you’ll get an error like: “No such domain” or “Cannot resolve”. Restarting RU DNS servers at 12:45 may have fixed the problem. Without a real explanation, the problem went away after a few hours.

Gitlab upgrade

Sun, 07 Feb 2021 04:00:00 +0100

Services will not be available for a while because of a GitLab and Mattermost upgrade.

Science VPNsec disruption

Wed, 03 Feb 2021 13:00:00 +0100

The expiration date of the certificate of our VPNsec service was apparently not yet checked regularly. This made it possible for the certificate to expire. We put a new certificate into place within an hour. Of course we will check this certificate regularly from now on. For Apple/Mac we needed to construct a new mobileconfig, this took some time, because in the meantime RU had moved to a different Certificate Authority. For Apple macOS this was ready at the end of Feb. 4, with a new installation procedure. For Apple iOS (iPhone/iPad) the old profile has to be deleted and a new mobileconfig has to be installed.

DIY temporarily not usable

Mon, 25 Jan 2021 07:15:00 +0100

medewerkers, studenten

Due to a management operation (planned around this time), the DIY website was unusable. Since the time was very early, it’s expected nobody was inconvenienced by this temporary unavailability.

Science smtp service temporarily not usable

Fri, 22 Jan 2021 10:00:00 +0100

medewerkers, studenten

A configuration change unwantedly made the smtp service unusable. When we noticed this, it was repaired immediately.

Very long mail aliases temporarily not usable

Thu, 21 Jan 2021 15:52:00 +0100

medewerkers, studenten

A configuration change had as unwanted effect the disappearance of all very long mail aliases. When this was reported next morning, it was repaired immediately.

Switch crash; gitlab+mattermost, licenses and DHZ

Thu, 07 Jan 2021 00:00:00 +0100

medewerkers, studenten

Due to a simple management command the switch (as-ak008-04) crasht and had to be reset manually. The switch sits between the network and servers for gitlab+mattermost, licenses and the database for DHZ(diy).

Gitlab upgrade

Fri, 27 Nov 2020 04:00:00 +0100

medewerkers, studenten

Services will not be available for a while because of a GitLab and Mattermost upgrade.

Eduroam problem on campus

Wed, 01 Jul 2020 00:00:00 +0200

The ISC announced: For security reasons, the certificate of the wifi server will be replaced in the evening of Friday, July 10. This has consequences for connecting your mobile device to Eduroam when you’re on campus:

• If you get the message that you have to accept the new certificate to use eduroam, choose ‘yes’. You can then use eduroam again;

• If you don’t get this message and can’t connect to Eduroam, choose the wireless network ’eduroam-config’. Accept the terms and conditions. Follow the instructions to reinstall Eduroam.

Webserver 'havik' offline

Thu, 18 Jun 2020 15:45:00 +0200

Several parts have been replaced. We assume the problem, that occurred twice, is now resolved. For dual-boot pcs, the boot menu was served by an alternative method during the repair.

Science radius disruption

Wed, 17 Jun 2020 11:11:00 +0200

Affected : Users of Science VPN and Eduroam based on science account

The certificate of the LDAP servers has been replaced this morning, this has also changed the certificate chain. The radius server uses LDAP as authentication backend and in the radius configuration the certificate chain had to be changed too. This was initially overlooked. Radius is the authentication mechanism used by all VPN servers and Eduroam

Webserver 'havik' offline

Wed, 17 Jun 2020 03:38:00 +0200

The server went down in the same way as the previous time (3rd of June 2020). The cause is most likely a system board problem. This part will be replaced Tomorrow by a support engineer.

Webserver 'havik' offline

Wed, 03 Jun 2020 06:30:00 +0200

by a website) and various websites.

The server couldn’t be reached after the scheduled weekly reboot, not even on its management interface. Because also C&CZ employees work from home and the interruption didn’t get enough urgency fast enough, the interruption lasted too long, apologies for that. The support partner has been contacted and the server has been updated, but the origin of the problem is still unclear. We will also look at making these services more redundant or more easily movable to a different server.

CN00 Slurm master ubuntu 16.04 down

Mon, 18 May 2020 09:50:00 +0200

Due to a failed BIOS upgrade, the hardware of the database server appears to be bricked. We transfered the disks to another machine (cn00) and all database services are now up again, at the cost of not having cn00 running. When the hardware is working well again, we will swap it all back and restore the original situation.

Sperwer Database server storing

Mon, 18 May 2020 06:30:00 +0200

Update May 19th, 12:15 : hardware fixed, situation back to the original state.

Jitsi.science.ru.nl not working properly

Sun, 19 Apr 2020 15:00:00 +0200

Due to performance tuning having gone wrong the jitsi.science.ru.nl conference rooms cannot be joined by more than one person at the moment. Solved by reinstalling server.

Mailserver certificate problem

Mon, 13 Apr 2020 14:00:00 +0200

The new certificate of the Science mailserver hadn’t yet been placed in the right place. The expiration of the old certificate caused a problem for Science mail users, that was resolved by replacing the old certificate.

RU mail erroneously in Spam folder

Wed, 25 Mar 2020 17:52:00 +0100

March 25, a rule “2020 Radboud Universiteit” was added to the Science spamfilter. Recently, this matched RU-central mailings. Therefore RU-wide mailings from e.g. the RU Board and Radboud Recharge have erroneously been delivered in the Spam folder of Science emplyees. The Science spamfilter tries to fight spam and phishing, this is partly manual work in which errors can’t be excluded. C&CZ apologizes for the inconvenience this has caused.

Unplanned GitLab security update

Wed, 11 Dec 2019 06:00:00 +0100

A critical security update for GitLab has been released and is being installed this moment. During the update, GitLab is offline.

Problems with a virtual host

Tue, 05 Nov 2019 05:30:00 +0100

The virtual machine host ‘oscar’ stopped running at 5:30am. The listed virtual machines are not running at the moment. We are working on it. Update: Resolved. Again, a broken LVM snapshot caused the problem.

Power Distribution Unit failure

Wed, 23 Oct 2019 16:00:00 +0200

A failed PDU caused the servers with a single Powersupply to die and restart after the PDU was replaced. Running jobs were not finished on these nodes.

Webserver 'havik' offline

Sun, 06 Oct 2019 10:35:00 +0200

A loose power cable caused an interruption of one of our web servers. The machine is equipped with redundant power supplies. It’s not clear why a single loose cable caused a service interruption.

NFS service interruption

Mon, 23 Sep 2019 06:30:00 +0200

Due to a configuration error, the NFS service on file server ‘peck’ refused to start. After fixing the error, the NFS service started up correctly.

Boot problems of virtual host

Tue, 03 Sep 2019 04:30:00 +0200

At the weekly reboot of vmhost ‘oscar’, it did not boot properly. The listed virtual machines are not running at the moment. We are working on it. Update: most probably, the problems where caused by full snapshots. After removal of the snapshots, the system booted properly.

Mailbox problems for a few users

Fri, 23 Aug 2019 15:10:00 +0200

Due to an administrative error the mailboxen of a few hundreds of users became inaccessible. Mail couldn’t be read from or delivered to these mailboxes until this had been repaired. In this period the mailclient of 19 users tried to read the mailbox and thus possibly reported an error.

Mail problems with newly introduced incoming mail server

Mon, 01 Jul 2019 10:33:00 +0200

A new receiving mailserver was introduced July 1. This server has the most recent version of Ubuntu and other software and settings that are more conforming to the directives (in Dutch). After the introduction, problems appeared with receiving mail from Microsoft Office 365 Exchange Online (mail.protection.outlook.com). An even bigger problem was that some accepted mails were not forwarded, but bounced to the sender. Because that problem couldn’t be fixed immediately, the newly introduced server has been shut down on July 2. At the end of August, C&CZ will test the probable solution. If the test succeeds, the new incoming server will be put into production.

Network interruption central RU network

Wed, 22 May 2019 18:55:00 +0200

Due to a faulty RU core network device, several service couldn’t be reached. This device is in a redundant setup. The ISC network department will start an investigation what exactly happened and how to prevent nuisance in the future.

DNS nameservice interruption for Science domains

Tue, 14 May 2019 16:13:00 +0200

Yesterday afternoon a DNS error with science.ru.nl (due to the introduction of the secure DNSSEC) made science.ru.nl disappear from the internet for many users, when viewed from outside campus. In the evening this error has been corrected. Because of the caching of DNS responses, the last problems should disappear tonight. In the meantime one can use workarounds like rebooting the home router and/or pc, using a different network (mobile provider) or using rainloop.science.ru.nl instead of roundcube.science.ru.nl for webmail access.

Dozens of Science websites down, also Roundcube

Tue, 14 May 2019 15:28:00 +0200

Because a web application used too much memory, all websites of this webserver had problems. After restarting the webserver, all websites could be reached again. We will make sure that web applications get memory limits, which will prevent this problem in the future.

Network interruption Huygens wing 4

Sat, 13 Apr 2019 05:15:00 +0200

Due to a faulty network switch, approx 40 network connections in Huygens wing 4 were temporarily unavailable. The switch has been replaced.

Matlab license server down

Thu, 28 Mar 2019 07:30:00 +0100

This morning, the Matlab license server stopped, because yesterday an error had been made in the upgrade to version R2019a. After the error had been corrected, the license server was restarted, which resolved the problem.

Poster printer "kamerbreed" broken

Tue, 26 Mar 2019 00:00:00 +0100

The poster printer has to be repaired, but we can’t easily fix it. We hope to have the printer operational again in the near future. In the meantime you might use the RU copyshop.

No connections to Eduroam with science accounts

Sun, 10 Mar 2019 18:00:00 +0100

Due to a configuration fault in the Science radius server, many connections to Eduroam for users with a science account failed. It took two days of joint investigation by ISC and C&CZ to resolve this problem.

No Eduroam end for deprecated systems (Windows XP, Android 4.1, ...)

Mon, 10 Dec 2018 05:00:00 +0100

On December 10, the support for the security protocol TLS v1 and v1.1 was discontinued within the Radboud Eduroam wireless network. From that moment on, all wireless clients such as laptops, smartphones, tablets and the like must use TLS version 1.2 or higher.

Unfortunately, the outcome of this change was not as expected. After the change, there were more clients with connection problems than the number of clients using TLSv1. After a period of testing and research, the ISC decided around 12.00 to reverse the change.

SSH connections of older clients/servers

Tue, 11 Sep 2018 07:20:00 +0200

A configuration change was automatically put into production Tuesday morning which had been prepared the previous day on Ubuntu 14_04 LTS, Ubuntu 16_04 LTS and Ubuntu 18_04 LTS in order to increase the level of security of SSH connections within FNWI. The change had the side effect of crashing the SSH daemon on Ubuntu 12_04 LTS and preventing older SSH clients from connecting at all. Users are encouraged to update their SSH clients so that we can gradually increase the level of security again.

Disk problem milton fileserver

Mon, 27 Aug 2018 06:30:00 +0200

One of the data-disks has a currupted file system. During the time slot menioned above, the server was unavailable. At the moment the server is running without the defective disk (milkun16). At the moment, we are trying to fix the disk.

Host hardware of virtual machines broken

Thu, 23 Aug 2018 01:41:00 +0200

Last night the host for a number of vm’s (oscar) broke down. By using a replacement server (joule) this problem has been temporarily resolved. Tomorrow the machine will be repaired ans we will determine when the machines will be switched back.

Network problem IMAP-mailserver because of error during update

Mon, 20 Aug 2018 15:32:00 +0200

While testing a possible update to the IMAP mailserver, an error made the mailserver unavailable for ca. fifteen minutes.

Bug in new IMAP mailserver resolved

Thu, 09 Aug 2018 13:05:00 +0200

The new version of the IMAP mailserver software contained a bug in the handling of SSL connections, that led to connections being dropped and depending on the mailclient led to a mailserver that could hardly be used. This morning a patch for this bug has been installed.

Upgrade IMAP mailserver post.science.ru.nl: Thursday August 9 2018 13:00-13:05

Thu, 09 Aug 2018 13:00:00 +0200

The IMAP mailserver post.science.ru.nl is being replaced by a new server. Therefore connections to the IMAP mailserver post.science.ru.nl will not be possible during ca. 5 minutes.

Server maintenance shares: Friday 3rd of August 2018 12:00-13:00

Fri, 03 Aug 2018 12:00:00 +0200

Three of our main fileservers need firmware updates, this requires the servers to be rebooted several times. The shares will be unavailable for at most one hour.

Affected network disks:

45t acfiles acfiles2 acl-ru-nl$ acs4 aerochem afstudeertest afstudeerwerk akolkmandata albert amsbackup amscommon animal arb astropresentie avd bbb-priv beevee betadidactiek b-fac-backup bioalg bioexamencie bioexamencierw bioniccell bio-orgchem birds bko boltje botany-general carisma carta ccs4 ccz cczhelpdesk cczsecr cdalgemeen cdbiow cdisis cdoii cdomw cdwinst celbio1 celbio2 celbio2 celbio3 ces2017 chemprac chemstaff cncz coastoffon cudotest cursus cursusdossiers dbiwwr-ro dbiwwr-rw dcnadmin desda diafragma docent docenten ds ecogen ecologie ehef1 ehef2 ehef3 ehef4 embdata epsylon erhvedata evsf2data10 evsf2data15 evsf2data16 evsf2data17 evsf2data18 evsf2data19 evsf2data7 evsf2data8 evsf2data9 evsf2schijf1 exoarchief femtospin fsr geminstr geminstr2 gi1 gi2 gi3 giphouse gipsy gissig global global global gmi goudsmit1 goudsmit2 goudsmit3 goudsmit4 goudsmitoldusers grafting hfml-backup hfml-backup2 hfml-backup3 hfml-backup4 hfml-backup-inst hfml-data hfml-engineering hfml-secr highres honours huckgroup hybrids icis icisafstudeer icisbureau iciscop icislaquso icissec icisst ifl ifl-d1 IJSO imappdir imappsecr immbureau immstaff impuls introcie invbio iris isis isptinspec itt iwwr1 janvanhest kaartenbak kangoeroe khaj kolkteaching labservant lambiek leonardo licenses liquidhelium liquids mailmanincludes marcurkasco master mathematica mathsolcie mbaudit1 mbaudit2 mbaudit3 mbaudit4 mbaudit5 mbbioel mbbioel2 mbcortex mbonderwijs mbread mbscon mbsl mbwrite mercator mestrelab mi1 mi2 mi3 micord micro-bulk1 micro-bulk2 micro-labmembers mlf90 mlf90grp mlf90linac mlfalgemeen mlfsec molbiol molchem molchem2 molchem3 molneuro molphtec mol-secr morph mpcoldmol2 mpcoldmollab mpcoldsecr mppstaf multimedia mwstudiereis neuroinf neuroinf2 neuropi nmr500data nmrspectra olcbiocie olcbiow olccudo olcii-archief olympus onderwijskwalificatiedossiers onlyme ons orgdier orgdiermac owc owc1 parp pcb plantkunde-hgl poliepC poliepD pracnat practica project1mpp puc randomwalkmodel rotax ruversatest secrmolbiol secrthch share sigma sigmacies smol snn snn2 snn3 snn4 sofie software soheto solids sos spectra spectra-rw spibdossier spm spmdata1 spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmmagstm spmnanolab spmstaff ssi ssiguest staff-hfml stroom studiereis2013 synorgchem tece teceleiding template-ru-nl$ template-ru-nl$ template-ru-nl$ tf2devices thalia thaliacies thchem tracegas tracegastemp tzacad ucm unl vcbio_prive vergader vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc16 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin vscxray wallpaper wikis wiskalg wiskunde wiskundetoernooi wkru zeegras

RU network maintenance: Monday evening/night July 2 to 3 from 22:00 hours

Mon, 02 Jul 2018 22:00:00 +0200

Monday evening July 2, there will be maintenance on central RU network equipment. From 22:00 hours, central RU applications and network will not be available most of the time. For telephones no disturbance is expected. For the C&CZ services there is a possibility of disruptions: no Internet connection for some time. Please think which processes depend on connectivity to RU central services and prepare for this possible outage if necessary. Future maintenance windows can be seen on the ISC website.

Interruption of parts of the telephone service

Sat, 09 Jun 2018 08:00:00 +0200

Saturday june 9th from 8:00h until approximately 12:00h analog telephone communications via AudioCodes will not be possible. At this time the existing equipment used in the Huygens- and surrounding buildings will be moved and reconnected. Only traditional analog equipment such as DECT phones and such are connected to the AudioCodes. The green (emergency) phones are not connected to these AudioCodes but to the emergency telephone exchange.

The emergency telephone exchange wil go down for ten minutes during the afternoon due to an intteruption of the power supply. The bulk (if not all) of the emergency phones and elevators are connected to this emergency telephone exchange and will therefore not be operational at that time.

No analog telephone communications on Saturday June 9

Sat, 09 Jun 2018 08:00:00 +0200

Saturday June 9th from 8:00h until approximately 12:00h analog telephone communications via AudioCodes will not be possible at FNWI. At that time the existing equipment used in the Huygens and surrounding buildings will be moved and reconnected. Only traditional analog equipment such as DECT phones are connected to the AudioCodes. The green emergency phones are not connected to these AudioCodes but to the emergency telephone exchange.

The emergency telephone exchange wil go down for ten minutes during the afternoon of that day due to an interruption of the power supply. The bulk (if not all) of the emergency phones and elevators are connected to this emergency telephone exchange and will therefore not be operational during those ten minutes.

No Internet (DHCP problem for leased addresses)

Sat, 09 Jun 2018 07:00:00 +0200

Last week, the DHCP infrastructure at FNWI was changed, in order to move from old to new servers. An error was made when we had to move back because DHCP did not run well on a virtual interface. When the first complaints came in this morning, we quickly found and repaired the error. We try to avoid further service disruptions while improving the DHCP service.

FNWI DNS nameserver problem with ru.nl

Tue, 22 May 2018 14:24:00 +0200

Last week, the DNS infrastructure at FNWI was changed, in order to move from old to new servers and to prepare for the use of DNSSEC. A workaround for the internal/external view of ru.nl had an unwanted side effect. This made the FNWI DNS servers lose the info about ru.nl. We had to resort to a different workaround to fix this. We try to avoid further service disruptions while improving the DNS services.

Multiple backup server issues

Sat, 17 Mar 2018 00:00:00 +0100

Backups were incomplete since March 17th. The tape robot couldn’t be used because a tapedrive failed, probably due to dry weather conditions. Therefore holding disks filled up. Only the backup of Tuesday upon Wednesday is possibly complete, but we lack backups of March 21st and 22nd, which is caused by a filesystem inconsistency of the holding disks, repairs already take two days. We’re trying hard to make backups this weekend, but please be cautious with your files, since restores of recently changed files might fail.

Homedirectories onbereikbaar op windows

Tue, 06 Mar 2018 12:00:00 +0100

During the process of restoring the homedirectory of the affected user of the ransomware, the samba server was restarted, however it stopped, but not start. This was noticed only 45 minutes later. We have ajusted our monitoring to help us notice this more quickly.

Gijzelsoftwware op IMAPP mappen

Tue, 06 Mar 2018 08:40:00 +0100

Due to an accidental opening of an e-mail attachment (called Resume.doc), which contained a macro-based ransomware, all the files and directories on the mentioned shares were either removed or encrypted and a message was left on the infected computer to contact a certain URL to get the files back. The files were restored from backup and the PC wiped clean and re-installed. NB. The f-secure software did not recognise this malware yet at the time.

Problem with outgoing mailserver (smtp)

Thu, 08 Feb 2018 02:39:00 +0100

Yesterday morning, we got an alarm because the smtp service did not work properly. The cause was found to be that from various PCs from, among other countries, China, connections were built up with one of the smtp servers, without actually sending an e-mail, as a result of which all these connections were left hanging. If the number of standing connections became too high, the server became unusable for others. This service has to be be accessible from the Internet, because of authenticated SMTP. After a few PCs had been blocked, at 15:39 Fail2ban was turned on, that is also used to protect other services accessible from the Internet, such as mailservers, Linux loginservers, WWW-servers and GitLab servers.

Fileserver maintenance because of reported disc problem

Mon, 22 Jan 2018 08:30:00 +0100

This morning the fileserver reported a disc problem. To diagnose this, we have to upgrade the firmware, which makes downtime necessary. We planned this for next Monday morning because of the Next Business Day support contract. Sorry for any inconvenience.

Affected disks:

acfiles aerochem amsbackup amscommon animal astropresentie avance500 avance600 beevee b-fac-backup bioniccell bio-orgchem boltje carisma celbio1 celbio2 chemprac cncz coastoffon csgi-archief desda ds ehef1 ehef2 ehef3 ehef4 femtospin geminstr2 giphouse gissig global gmi goudsmit1 hfml-backup hfml-backup2 hfml-backup3 hfml-backup-inst highres imappdir imappsecr impuls introcie isis janvanhest kaartenbak kangoeroe khaj linova500 linova800 marcurkasco mathsolcie mbaudit1 mbscon mbsl molbiol mwstudiereis neuroinf neuroinf2 nfstest olympus puc randomwalkmodel ruversatest secres2 secrmolbiol sigma sigmacies spectra spectra-rw spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmmagstm spmnanolab spmstaff ssi ssiguest staff-hfml stroom thalia thaliacies ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde wkru

GitLab temporarily unavailable during upgrade

Tue, 16 Jan 2018 23:00:00 +0100

Because of a planned upgrade, gitlab.science.ru.nl and gitlab.pep.cs.ru.nl will not be available. See also: 1

Expired certificate for IMAP mailserver

Mon, 11 Dec 2017 00:59:00 +0100

The new certificate for the IMAP mailserver hadn’t been put into place correctly. Sorry for any inconvenience.

Incorrect Adobe Acrobat Pro DC package on install disc

Sat, 02 Dec 2017 16:20:00 +0100

We placed an incorrectly packaged version on the install drive. After users reported this, the ISC has created a new package and we have replaced it.

Network maintenance at Science Faculty: upgrade router

Mon, 27 Nov 2017 23:30:00 +0100

New firmware will be loaded into the Heyendaal-East distribution router. During this period the wired and wireless network will be unavailable for approx. 30 minutes.

Homedirectory unavailable

Wed, 22 Nov 2017 07:30:00 +0100

After an update of the samba software, it appears that the smbd service didn’t start up again. The problem was resolved by either rebooting (home1) or manually starting smbd (home2).

SPIB, reviewapp, thesis submission, timetrex intermittent failures

Tue, 14 Nov 2017 20:50:00 +0100

Currently the hardware has problems, causing the machine to randomly turn off and on. We are in the process of resolving this with our supplier.

The problem doesn’t appear to be hardware related, but a problem with the configuration of the OS, which we changed and it seems to be stable now.

Homeservers rebooted daily

Sun, 01 Oct 2017 06:28:00 +0200

Due to a misinterpretation of the correct crontab syntax, the homeservers were rebooted daily since the beginning of this month, instead of, as we intended, once a Month on the first Monday.

Homedirectories unavailable on monday mornings

Sun, 01 Oct 2017 00:00:00 +0200

Multiple Ubuntu 16.04 users are unable to access their homedirectories on machines that have been rebooted on monday morning. Keeping system clocks more in sync seems to have helped somewhat, but no definite solution has been found yet. Therefore we decided to reboot the homeservers only the *first* Monday of each month. In the meantime we still hope for a real solution.

DIY down due to hardware problem

Wed, 27 Sep 2017 15:00:00 +0200

The server of DHZ had a problem with the hardware, the hardware was fixed on 2017-09-29, the website DHZ could be restored more quickly on a different server, limiting the outage to a few hours.

Homedirectories unavailable

Fri, 22 Sep 2017 07:20:00 +0200

This morning, the homeservers were automatically upgraded with normal updates, after which the windows file servicewas not restarted. The problem was resolved by restarting the machines.

OpenVPN not started at reboot

Mon, 14 Aug 2017 06:31:00 +0200

For yet unknown reasons the OpenVPN service did not start at the Monday morning reboot. We started the service manually.

DIY down due to hardware problem

Thu, 10 Aug 2017 08:00:00 +0200

The server of DHZ is having problems, while we are solving this with Dell support, all changes can be handled via us (tel: 53535) or postmaster@science.ru.nl

Emergency power failure Huygens building lasted hours

Sat, 22 Jul 2017 11:38:00 +0200

Saturday July 22 at 11:37, there was a power outage in the Huygens building, also for all equipment connected to emergency power. The power outage lasted long enough to leave even devices and rooms with a UPS battery backup without power. This included all network equipment (routers, switches, wireless access points) and all servers in the C&CZ large server room.

When power returned from 14:08, several devices were defective:

- Half of the fully redundant FNWI router was defective. External companies in the UBC are not doubly connected to their provider, so they had no connectivity until Monday, when their connectivity was switched. Since Monday afternoon, the FNWI router has been repaired, so is redundant again.

GitLab temporarily unavailable during upgrade

Fri, 30 Jun 2017 09:00:00 +0200

Because of a planned upgrade, gitlab.science.ru.nl and gitlab.pep.cs.ru.nl will not be available.

LibreOffice problem using Ubuntu 14.04

Mon, 26 Jun 2017 06:30:00 +0200

LibreOffice and other software suites using Java (jvm) internally are affected by the Ubuntu 14.04 kernel update patching the Stack Clash. It seems that Linux kernel developers, Ubuntu developers and JVM developers are working hard world-wide in finding a proper solution. Work-around ‘JAVA_TOOL_OPTIONS=-Xss1280k libreoffice’. Ubunti released a fix.

Fileserver needed reboot after fatal typo

Tue, 13 Jun 2017 10:45:00 +0200

Apologies.

Affected disks:

acfiles beevee bioniccell bio-orgchem celbio1 celbio2 chemprac csgi-archief csg-projecten csg-staf desda ds ehef1 ehef2 ehef3 femtospin geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 hfml-backup-inst highres impuls introcie isis janvanhest kaartenbak kangoeroe mathsolcie microbiology microbiology2 milkun2 milkun2RenD milkun3 milkun5 milkun6 milkun7 milkun7rw mwstudiereis neuroinf neuroinf2 olympus puc randomwalkmodel secres2 sigma sigmacies spectra spectra-rw spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmstaff ssi staff-hfml stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde wkru

Server TCMR storing na disk-falen

Mon, 29 May 2017 06:30:00 +0200

Sunday evening, one of the archive disks failed. The data on it was lost, as it wasn’t in a redundant setup. After the weekly reboot, the server hung on this disk being broken. It took us a while to get the server up again.

Eduroam wifi behind firewall

Mon, 22 May 2017 22:00:00 +0200

Due to maintenance the wifi network on campus (Eduroam) will not be available for twenty minutes on Monday night between 10 pm and 8 pm. During those twenty minutes your mobile device appears to be connected with the wifi network but you will notice there is no internet connection. After this interruption the connection will be established automatically. During the maintenance period additional short interruptions may occur.

Monday evening the wifinetwork will be placed behind the new firewalls of the Radboud University. When this is done, users of Eduroam on our campus are no longer directly reachable from the Internet. We are doing this to protect wifi users against viruses like the Wannacry virus that was used for the worldwide ransomware attack last week.

Network failure server rack

Thu, 11 May 2017 09:05:00 +0200

Switch es7-huyg-04-070 of a server rack became powerless after maintenance in the computer room of C&CZ. Restoring power and powering up the switch resolved all problems.

RU network maintenance (mainly fixed phones)

Mon, 08 May 2017 23:30:00 +0200

Between 23:30 and 00:30 the fixed phones will be out of service for about half an hour. Between 00.30 and 02.00 hours, a few minutes of internet interruption will take place. At those moments, RU services can not be accessed from outside the campus.

Maintenance network Serverroom FNWI in Huygens building

Mon, 01 May 2017 07:30:00 +0200

The central network switch in the serverroom at the Faculty of Science will be restarted in connection with some software maintenance. Many servers and services are not available for approx 20 minutes.

Eduroam wifi maintenance

Mon, 24 Apr 2017 23:00:00 +0200

The RU wifi network (eduroam) will be unavailable on Monday evening April 24 between 23:00 and midnight for half an hour due to maintenance work.

Some virtual servers temporarily not available

Wed, 12 Apr 2017 07:22:00 +0200

An unattended update of the virtual hosting software (libvirt) triggered a netwerk stack restart. This made the virtual machines unavailable on the network. After a reboot of the virtual hosts (vos and wolf), the virtual machines were available again. Action was not taken immediately, because the alarm mails didn’t reach us, because the mailservers could not be reached. To improve this situation, steps are taken to route the alarms also via alternative channels (sms, whatsapp, signal, etc). Also the network configuration of the virtual machines has been changed, this will hopefully make it less vulnerable to these updates.

Poster printer "kamerbreed" broken

Mon, 03 Apr 2017 00:00:00 +0200

The printer has to be repaired, but the engineer can’t easily fix it. We hope to have the printer operational again in the near future. A new one with service contract has been ordered, in order to make the poster service more reliable. In the meantime you might use the RU copyshop.

Crash coma00 server

Sun, 12 Mar 2017 22:17:00 +0100

Coma00 serves astro0, astro1, astro3 and astro4 volumes, the server crashed on a kernel-BUG, Monday morning the server was restarted and service resumed normally.

Science and RU Internet network maintenance

Mon, 06 Mar 2017 22:00:00 +0100

Monday evening March 6, there will be maintenance on RU network equipment. On March 7 between ca. 01:00-01:30 no network traffic will be possible at the Faculty of Science on the wired and wireless networks. From 23:00 until March 7 02:00 there will be a period where no traffic between RU and Internet is possible. Please think which processes depend on network connectivity and prepare for this outage if necessary.

Science Radius down

Tue, 28 Feb 2017 06:30:00 +0100

The Radius service didn’t start after the regular reboot, restarting the Radius service solved the problem.

VPNsec upgrade

Wed, 22 Feb 2017 15:30:00 +0100

Update of Strongswan server software with extra capabilities

Problem with mail delivery from RU Exchange

Mon, 06 Feb 2017 09:39:00 +0100

Monday morning several employees of FNWI were affected by a fault within the RU Exchange design whereby they could not send their mail to recipients outside of the RU Exchange (i.e. @science.ru.nl, @cs.ru.nl or elsewhere). ISC had added two additional servers to the RU Exchange service the week before, which were not yet allowed to drop their outgoing mail on designated servers for that purpose. Not all FNWI employees using RU Exchange were affected, however, since four out of six servers were still functioning properly and employees using e.g. Thunderbird use servers outside of the RU Exchange for sending their mails.

Maintenance network Serverroom FNWI in Huygens building

Mon, 30 Jan 2017 07:00:00 +0100

The central network switch in the serverroom at the Faculty of Science will be restarted in connection with an software upgrade. Many servers and services are not available for approx 20 minutes.

Network failure in Huygens wing 8

Tue, 24 Jan 2017 10:50:00 +0100

In Huygens wing 8 a network switchmodule crashed. Several users lost their connection to the data and telephone network for some time. The switchmodule has been replaced.

VPNsec (and RU VPN) problem with new RU firewall

Tue, 13 Dec 2016 00:00:00 +0100

The new RU Fortigate firewall, that is in place since December 13, manages fragmented IP packets inconsistently. Depending on the VPN client and its settings, this can cause problems for VPN users. The ISC found a solution for the cenral RU VPN in changing the configuration of the RU VPN as of 2016-12-19 15:50 hours. While waiting for the final solution, a temporary workaround has been used in the Fortigate firewall on April 19, which makes the problem disappear.

Network failure in Huygens wing 5

Wed, 07 Dec 2016 21:15:00 +0100

In Huygens wing 5 a network switchmodule crashed. Several users lost their connection to the data and telephone network for some time. The switchmodule has been replaced.

Maintenance indoor Vodafone cellphone network Huygens

Mon, 05 Dec 2016 00:00:00 +0100

Due to maintenance of the indoor cellphone equipment, this equipment will be out of service for one hour max on Monday December 5th.

Poster printer "kamerbreed" broken

Fri, 02 Dec 2016 00:00:00 +0100

The printer has to be repaired. Spare parts have been ordered. We hope to have the printer operational again this month.

Some machines down due to failed power distribution unit

Mon, 14 Nov 2016 20:40:00 +0100

Due to a failed power distribution unit (PDU) a network switch powered down. As a result of that the mentioned machines lost their network connectivity. After reconnecting the switch to a different PDU, the machines were reachable again.

Servers down due to failed power distribution unit

Thu, 15 Sep 2016 00:27:00 +0200

A few servers powered down due to a failed power distribution unit (PDU), because they were not redundantly connected. After switching to a different PDU, the servers were running again. The redundant connections of important servers will be checked.

RU network maintenance problem

Tue, 13 Sep 2016 01:00:00 +0200

After the maintenance activities as mentioned below, a large number of workplaces of Molecular Biology at RIMLS did not receive an ip-address. After a reset by ISC of a table in the central RU-firewall, the problem was solved.

RU network maintenance

Mon, 12 Sep 2016 23:00:00 +0200

In the night of 12 to 13 September, maintenance on the Radboud University network will be performed from 11pm to 03am. Several service interruptions may occur, like internet access via SURFnet, mail via Exchange, Bass, Osiris and Blackboard. We advise to check which systems might be affected by this and if possible shift network activity to a different time slot. You also might think about systems with alerts via Internet.

Network drives not available after reboot

Mon, 12 Sep 2016 06:30:00 +0200

After the Monday morning reboot, the Samba network drive service was not running on all Ubuntu 12.04 servers. Reinitializing all Samba configuration files solved the problem. Problems fixed at 08:30 for home servers, 10:25 departmental printers, 12:45 departmental server.

Digital exam environment not available due to maintenance

Mon, 25 Jul 2016 16:30:00 +0200

Due to maintenance, the digital exam environment will not be available for several hours.

Power distribution unit failed

Tue, 05 Jul 2016 06:00:00 +0200

Three cluster nodes powered down due to a failed PDU. After replacing the PDU, the nodes came up normally.

Atlas.ti reports: license (almost) expired

Fri, 17 Jun 2016 00:00:00 +0200

When starting Atlas.ti, you get the message that the license expires in a few days or already has expired. There is, however, a three-weeks grace period during which the users will see a warning message but can still use the program normally. The vendor has promised to send the license June 24, and to send it earlier next years. The new license key can be requested from C&CZ.

License server reboot problem

Mon, 30 May 2016 06:30:00 +0200

Due to a problem with a new license the license server didn’t go down for the weekly reboot. When this came to attention, the server was rebooted and the problem was solved.

Altera Ansys Accelrys Genesys Hdlworks Idapro Xilinx IntelCompiler Ingr Maple Mathematica Matlab MentorGraphics Mestrelab Topspin Orcad Originlab StateaseRlm

License server (lepelaar) reboot problem

Wed, 25 May 2016 06:30:00 +0200

Matlab (until 08:45), Origin (09:00), Mestrenova (9:20), Mathematica (9:30) and Orcad (11:20) service not started. Cause yet unknown.

Both Science VPNs down for short maintenance

Wed, 18 May 2016 07:30:00 +0200

The Science VPNs will be down for short maintenance, in order to switch the IP ranges. After this maintenance, there will be enough IP numbers available for the growing use of the new VPN. The old VPN has not been switched off yet, because Ubuntu 16.04 still has problems with the new VPN.

Gitlab temporarily unavailable

Mon, 02 May 2016 21:30:00 +0200

Because of a major security update that will be released tonight, GitLab will not be available. We will update GitLab on Tuesday morning. After that, gitlab.science.ru.nl will be available again.

Gitlab backup accidentially restored on production server

Fri, 29 Apr 2016 14:30:00 +0200

A backup file, dated “Fri Apr 29 05:07:04 CEST 2016” has accidentially been restored on our production GitLab server, where it should have happened on a test server. This caused all repositories, wiki’s, issues etc. to be reset to the state of today 5.07 AM. Commits pushed between 5 am and 2:30 PM should be pushed again. Sorry for the inconvenience.

Server coma00 Monday morning reboot failed

Mon, 25 Apr 2016 06:30:00 +0200

The server comas1 did not start, probably due to some timing problems between the RAID array and OS. Solution: reboot - hardware reset.

RU mailserver blacklisted, for Science too

Tue, 19 Apr 2016 17:37:00 +0200

Yesterday the RU outgoing mailserver smtpout3.ru.nl got listed on the SORBS blacklist of confirmed senders of spam and phishing mails. The reason probably is that recently many students fell for phishing mails, which made their username and password known to Internet criminals. These criminals use that to send spam and phishing mails. Because also the Science mailservers use that blacklist, we did not accept mail from the RU mailserver anymore. The senders received an error message: “Your message wasn’t delivered due to a permission or security issue.” We solved this for our Science users by putting the RU mailserver on our own whitelist.

Power failure

Tue, 29 Mar 2016 10:43:00 +0200

Several services were inaccessible due to power loss of a rack switch caused by a failing PDU (power distribution unit). The PDU has been temporarily replaced.

Sevices include:

home directories, printing, authentication services, licenses, vpn, websites, databases, mailman and dns resolvers.

Hardware replacement webserver

Thu, 24 Mar 2016 12:19:00 +0100

Memory modules and or the motherboard of webserver ‘oehoe’ will be replaced. The mentioned web sites will not be available for a while.

Affected web services:

aerochem,akn,aqua,bgard,boekingen,bomen,cac,cafe,celbi,docenten,exo,filosofie,hfml iconic,ifl,irmacrash,itp2011,iwwr-boookings,kanboard,kndv,labservant,math,mbfys,mbp,mealab, nieuwescheikunde,nlt,pilab,printeger,probiomed,puff,pz,qpl,quantumcs,ra,refbase,rrl, scheikundeolympiade,siam,sigma-lustrum,smartech4health,sws,theorphys,tk,toernooi,tredd, trooster,vsc,w4kangoeroe,wortel,zwaartekrachtsgolven

Hardware failure webserver

Mon, 21 Mar 2016 14:00:00 +0100

A defective memory module caused spontaneous reboots. Since two modules where taken out, these reboots have not occurred anymore. Additional down time will be needed to resolve the problem.

Affected services:

Gitlab upgrade

Wed, 16 Mar 2016 21:40:00 +0100

Because of an upgrade of gitlab to version 8.5.7, the GitLab service was offline for a while.

emoe stuck at reboot

Mon, 07 Mar 2016 06:30:00 +0100

At reboot time some file system (/opt) was not recognized, resulting in a kernel panic. Solutio: Reboot

Server isolator lacking private class networks

Mon, 07 Mar 2016 06:30:00 +0100

For yet unknown reasons the private class network interfaces where incorrectly numbered and activated Solution: Reboot

Gitlab upgrade

Fri, 26 Feb 2016 09:00:00 +0100

Because of an upgrade of gitlab to version 8.5.1, the GitLab service will be offline for a while.

Virtual servers problem

Thu, 25 Feb 2016 07:25:00 +0100

Because of automated updates, Several virtual host had become unreachable. Server vos: gitlab3, bfacdc04, smtp2, redmine2 Server wolf: smtp1, bfacdc03, est2

Rebooted vos and wolf.

miii server problem

Tue, 23 Feb 2016 00:00:00 +0100

For unknown reasons, the miii mail server was unavailable. Rebooted.

IMAP mail server problem

Mon, 22 Feb 2016 06:30:00 +0100

For unknown reasons, the IMAP mail server was unavailable. The service cyrus-imapd was not correctly started. Starting this service service cyrus-imapd start solved the problem.

IMAP mail server problem

Mon, 15 Feb 2016 11:20:00 +0100

For unknown reasons, the IMAP mail server was unavailable. Mail services were halted after some time because of logging activity. As a temporary workaround logging is now performed using UDP instead of TCP.

Fileserver problems

Tue, 09 Feb 2016 09:00:00 +0100

The Ubuntu 14.04 fileservers “flock” and “muckle” had problems a few times. After we reverted to the previous kernel version, the service seems to be stable again.

Matlab license server crashed

Mon, 08 Feb 2016 08:58:00 +0100

Last night the Matlab license server crashed, because the number of open file handles reached the maximum configured, because a user started a lot of Matlab processes. Because we do not yet monitor this properly and it was not reported to us, we could only repair it this morning. We increased the number of file handles by a factor of four. We assume that this will be sufficient to wait till the reboot of next Monday, when it will again be enlarged, this time by a factor of sixteen. We will also on short notice start to monitor the Matlab license server properly.

Server lilo4 Monday morning reboot slowed

Mon, 01 Feb 2016 06:30:00 +0100

The server lilo4 stalled at shutdown during the monday morning reboot. Solution: it continued its reboot after server wielewaal became available.

Webserver down: tftp & egw

Mon, 01 Feb 2016 06:30:00 +0100

Quite a number of websites of ICIS were not available amongst which was eGroupWare. Furthermore, the boot process of workstations was impaired (because of the missing tftp). A reboot of the server seems to be sufficient to solve the issue.

Affected websites:

blackgem brainhealth croatia14 croatia15 cs eduspec eduspecorg egw ethergids fbkr kickstart mhealth oni snn

No free IP numbers in Vlan136

Thu, 21 Jan 2016 11:44:00 +0100

The DHCP server didn’t have free leases on Vlan136, so new devices on Vlan136 didn’t get an IP address, so didn’t have a working network connection. After we freed some unused numbers for PCs with a fixed address, we were able to drastically enlarge the number of leases on Vlan136.

Network failure in part of study area Huygens

Wed, 20 Jan 2016 12:57:00 +0100

In Huygens wing 2 a network switchmodule crashed twice. About 30 pc’s in the Study Area lost their connection to the network for some time. The switchmodule will be replaced on 2016-01-21 at 07:30 am.

Login servers down to activate safer kernel

Wed, 20 Jan 2016 09:30:00 +0100

Webserver down to add hard disks

Fri, 15 Jan 2016 17:00:00 +0100

Affected sites:

aerochem akn aqua bb2syllabus bgard boekingen boekwur bomengids cac cafe celbi dossiers est2b exo filosofie fnwi hfml iconic ifl irmacrash itp2011 iwwrbook kanboard kndv labservant leonardo math mbfys mbp nieuwescheikunde nlt notebooks olympus pilab piwik printeger projecten puff pz qpl quantumcs refbase resourceanalysis rrl scheiolymp setup siam sigma-lustrum sito smartech4health studiereis sws tango taskboard tk toernooi tredd trooster ubuntumirror vsc w4kangoeroe wortel

Virtual servers network interfaces broken after updates

Wed, 13 Jan 2016 07:16:00 +0100

The virtual hosts fret, wolf and vos received updates to libvirt packages, causing the network interfaces of the virtual machines running on them to fail. Solved after reboot (fret) or restart of the libvirt daemons and the virtual machines.

Affected virtual machines:

bfacdc03 bfacdc04 drukker est2 gitlab3 ms2 nwidc01 redmine2 smtp1 smtp2 ts2

Server lilo3 and lilo4 Monday morning reboot failed

Mon, 11 Jan 2016 06:30:00 +0100

The server lilo3/4 stalled at shutdown during the monday morning reboot. Solution: power cycle. Cause currently unknown.

Server comas1 Monday morning reboot failed

Tue, 05 Jan 2016 06:30:00 +0100

The server comas1 did not start due to some possible timing problems between the raid array and OS. Solution: skipping the mount of /vd0 and /vd1 during boot (key S), mounts fixed manually later on followed by exportfs -uaf; exportfs -raf.

Machine failed for unknown reason at boot. Solved by rebooting.

RU Internet connection down

Mon, 23 Nov 2015 15:52:00 +0100

Yesterday at ca. 15:52 a SURFnet core router crashed, taking down the RU Internet connection. At 16:09 the connection was restored. This was reported yesterday on a temporary SURFnet service interruptions page and on the ISC/CIM service interruptions page. Details can be found on the SURFnet Network Tickets mailing list.

Print server printto problems

Thu, 19 Nov 2015 00:00:00 +0100

After a reboot of the print server (and waiting ca. 10 minutes) printers could be attached again.

Problems availability wireless network eduroam

Mon, 16 Nov 2015 08:00:00 +0100

As a result of a short network service interruption last night, there were some problems with obtaining ip addresses on the wireless network eduroam.

Fileserver goudsmit problems

Wed, 04 Nov 2015 23:20:00 +0100

System crashed possibly caused by an erroneous harddisk causing all CPUs to lockup.

RU Internet connection down for maintenance

Tue, 20 Oct 2015 00:00:00 +0200

The ISC has announced that on Tuesday morning, October 20, 2015 between 00:00 and 03:00 SURFnet and the ISC will perform maintenance on the connection of the RU with SURFnet/Internet. From 00:00 to 01:30 there is no connection. Between 01:30 and 03:00 short interruptions may occur. We advise to check which systems can be affected by this and if possible shift network activity to a different time slot. You also might think about systems with alerts via Internet.

Konica Minolta MFPs switched

Thu, 08 Oct 2015 08:00:00 +0200

In the morning of October 8, some KM MFPs will be switched, in order to have the faster/bigger machines at the locations where the print volume is te largest. The replacement of each machine will not take much time.

Affected printers:

pr-hg-00-011 pr-hg-00-089 pr-hg-00-627 pr-hg-00-825 pr-hg-02-038 pr-hg-02-556 pr-hg-02-832

Server lilo4 Monday morning reboot failed

Mon, 05 Oct 2015 06:30:00 +0200

The server lilo4 stalled at shutdown during the monday morning reboot. Solution: power cycle. Cause currently unknown.

Radboudumc buildings power switched off for test

Wed, 30 Sep 2015 16:00:00 +0200

Wednesday September 30, around 4:00 p.m., Radboudumc will carry out a test by switching off the power to almost all their buildings. For the RU buildings this test has no direct effect. Liander states that switching off and on again will have no noticeable effect on the rest of the power net in Nijmegen including Radboud University. However, switching high power always has the possibility of introducing a power problem elsewhere. If research, education or any activity will be seriously hindered by a power problem, you might consider ending this activity in time or taking measures to guarantee continuity with emergency power and UPS.

Brief network interruptions at FNWI

Sat, 12 Sep 2015 09:00:00 +0200

Saturday September 12, between 9:00 a.m. and 1:00 p.m., maintenance work will be conducted on the network router at the Faculty of Science and a number of server switches in C&CZ data centers. This means that between these points in time, a few brief network interruptions with a duration of approximately 1 minute will occur. Many FNWI servers will be unreachable at those moments. The Mitel IP phones will also go out of service momentarily several times. All other phones, however, will continue to operate normally. The maintenance work is necessary to enable resumption of the IP telephony rollout at the Faculty of Science. For further information, please contact netmaster@science.ru.nl.

Mailman mailing list problem

Mon, 07 Sep 2015 06:30:00 +0200

This morning we came to know that mails to mailing lists were not accepted by the mailing list server. The reason was a change that was made last Friday because of the new smtp server mid July. After correcting the change, mails were accepted again.

Network shares unavailable after reboot

Mon, 07 Sep 2015 06:30:00 +0200

During the move this weekend of network shares to a new server, an administrative error was made, that prevented the server from rebooting ok. After correcting this, the server booted ok.

Affected network disks:

acs4 afric afstud afstudeerwerk afstudTest albert bin bioalg bioexamencie bko BKO-TEMPLATE ccz cczhelpdesk cczsecr celbio2 celbio3 CuDoTest CURSUS cursusdossiers diafragma ecogen ecologie embdata evsf2data10 evsf2data15 evsf2data16 evsf2data17 evsf2data18 evsf2data19 evsf2data7 evsf2data8 evsf2data9 fsr geminstr grafting honours huckgroup hybrids ifl-d1 imappdir imappsecr immbureau iobw kolkteaching localdist logs mbaudit3 mbbioel2 mbonderwijs mbsynid micro-bulk1 micro-bulk2 micro-labmembers milkun8 milkun9 mlf90 mlfalgemeen mlfsec molchem3 morph mpcoldmollab mppstaf nmr500data nmrspectra olcbiocie olcbiowetenschappen orgdier owc1 parp petseq project1mpp seq snn3 snn4 soheto studiereis2013 sucudo-transfer tf2devices tracegas unl vcbio_prive vergader vsc16

Interruption public telephone network

Mon, 31 Aug 2015 14:00:00 +0200

Since 14:00 this afternoon it’s not possible to reach phones outside the campus. Also, only external phones on the KPN network and cell phones from all providers can reach the campus.

For central RU service interruptions and planned maintenance, see the combined ISC/CIM interruptions page.

Water damage for pr-hg-03-825

Mon, 31 Aug 2015 00:00:00 +0200

A leak above the KM MFP hg PR-03-825 made this fail beyond repair. We hope to resolve this by replacing it by a little-used KM MFP of the same type.

Network shares unavailable during migration

Thu, 06 Aug 2015 08:45:00 +0200

Starting from 8:45h, the network disks will not be available during a migration to a new server.

Affected disks:

45t, acfiles2, acs4, afric, afstud, afstudeerwerk, afstudTest, albert, arb, bbb-priv, bin, bioalg, bioexamencie, birds, bko, botany-general, bsweet, carta, ccs4, ccz, cczhelpdesk, cczsecr, celbio2, celbio3, CuDoTest, cursus, cursusdossiers, dcnadmin, diafragma, ecogen, ecologie, embdata, evsf2data10, evsf2data15, evsf2data16, evsf2data17, evsf2data18, evsf2data19, evsf2data7, evsf2data8, evsf2data9, exoarchief, flaredisc, fsr, geminstr, gi1, gi3, gipsy, grafting, hfml-data, hfml-engineering, hfml-secr, honours, huckgroup, hybrids, icissec, ifl, ifl-d1, imappdir, imappdir, imappsecr, immbureau, iobw, iris, itt, iwwr1, kolkteaching, lambiek, localdist, logs, mailmanincludes, mbaudit3, mbbioel, mbbioel2, mbcns, mbcortex, mbonderwijs, mb-read-write, mbsynid, mercator, mestrelab, mi1, mi3, micro-bulk1, micro-bulk2, micro-labmembers, milkun4, milkun4rw, milkun8, milkun9, mlf90, mlfalgemeen, mlfsec, molchem, molchem2, molchem3, molphtec, mol-secr, morph, mpcoldmollab, mppstaf, multimedia, neuropi, nmr500data, nmrspectra, nskpromo, olcbiocie, olcbiowetenschappen, ons, orgdier, owc, owc1, parp, pcb, petseq, plantkunde-hgl, practica, project1mpp, rotax, seq, smol, snn, snn2, snn3, snn4, soheto, sos, spmdata1, SSNMR1, studiereis2013, sucudo-transfer, tcbackup, tcscratch, tece, teceleiding, tf2devices, thchem, tracegas, tracegastemp, tracelab, tracelabdb, unl, vcbio_prive, vergader, vsc16, vscxray, wikis, wiskundetoernooi en zeegras

Server vos (gitlab3/redmine2/smtp2) preventive maintenance

Fri, 31 Jul 2015 09:00:00 +0200

The server vos will receive extra disks and need to be reconfigured. Virtual clients gitlab3, redmine2 and smtp2 will be offline during maintenance.

Network shares temporary not available during migration

Fri, 26 Jun 2015 08:45:00 +0200

Starting from 8:45h, the network disks will not be available during a migration to a new server.

Affected disks:

acfiles, aerochem, amsbackup, amscommon, beevee, bfac-backup, bio-orgchem, bioniccell, carisma, chemprac, csg-staf, csgi-archief, desda, ds, ehef1, ehef2, ehef3, geminstr2, giphouse, gissig, gmi, hfml-backup2, highres, impuls, introcie, isis, janvanhest, kaartenbak, kangoeroe, mbaudit1, microbiology, microbiology2, milkun2, milkun3, milkun5, milkun6, milkun7, milkun7rw, molbiol, mwstudiereis, neuroinf, neuroinf2, nfstest, nwibackup, olympus, puc, ruversatest, secres2, secrmolbiol, sigma, sigmacies, spectra, spm, spmdata3, spmdata4, spmdata5, spmdata6, spmdata7, spmmagstm, spmnanolab, spmstaff, ssi, staff-hfml, stroom, thalia, ucm, vsc, vsc1, vsc10, vsc11, vsc12, vsc13, vsc14, vsc15, vsc2, vsc3, vsc4, vsc5, vsc6, vsc7, vsc8, vsc9, vscadmin, wiskalg, wiskunde en wkru.

Server homedcc had old homes after reboot

Mon, 15 Jun 2015 06:30:00 +0200

After the server reboot on Monday morning, the changes to the home directories of the last two weeks had disappeared. It is still not clear how this could have happened. The backups of Saturday June 13 22:55 have been restored.

SMTP server problem because of too short keys

Sat, 13 Jun 2015 00:00:00 +0200

Starting a month ago, mail programs could get an update due to the vulnerability for a LogJam attack. After this update these programs refused connections with a server with a too short key. The old C&CZ authenticated SMTP server had such a short key. Users who reported this problem (with the latest versions of Pine, Thunderbird and recently also Apple Mail), were advised to temporarily use an alternative server. As of today using this alternative server is no longer necessary, because the server smtp.science.ru.nl has been replaced by a new server.

Services unavailable due to power outage

Thu, 11 Jun 2015 20:00:00 +0200

A short power dip interrupted desktop computers and some network switches without UPS backup power on the RU campus. It took approx. 20 minutes before the affected switches became fully operational again. A network switch at the Transitorium reported a failed module until it was reset around 8:30 am on Friday morning. For ca. 46 outlets/users the network didn’t function until 8:40. The UVB reported as cause a cable cut at the Kapittelweg. Liander reports a power disturbance an hour later at that location.

Fileserver tcmr problems

Fri, 05 Jun 2015 16:05:00 +0200

Resolved by a server reboot.

Ubuntu 10.04 cluster / cn99 server problem

Wed, 03 Jun 2015 12:10:00 +0200

The server (cn99) which serviced SUN Grid Engine on Ubuntu 10.04 is no more. It could not find its disks while booting. This will not be solved because the machine is already more than five years old. Clusternodes cn67, cn74, cn75 and cn81 are influenced. They can still be used by logging into them directly.

GitLab server problems

Sat, 30 May 2015 07:34:00 +0200

A configuration error caused the automatic installation of an older version of GitLab (7.7.1) over the the most recent version (7.11.4). Because the GitLab version did not correspond to the state of the underlying database, what caused multiple problems. Among which the gitlab shell for ssh access and inaccessibility of some parts of the web interface. The problems were solved after upgrading to version 7.11.4.

License server reboot problem

Mon, 11 May 2015 06:30:00 +0200

Today the license server booted without configuring the network. The cause of the problem was an error in a configuration file. This has been corrected.

Affected licenses:

Altera Ansys Accelrys Genesys Hdlworks Idapro Xilinx IntelCompiler Ingr Maple Mathematica Matlab MentorGraphics Mestrelab Topspin Orcad Originlab

IMAP mail server upgrade next monday night

Mon, 16 Feb 2015 23:00:00 +0100

The IMAP server will be replaced by a new server next Monday night. During the night, no Science mail can be read from the server, because then the most recent mails will be synchronized to the new server. Mail can be sent in this period, but new mail will start being delivered on the new server from 07:30. A word of warning, one could login on the old server with the first 8 characters of the password, the new server only accepts the full password.

Network maintenance in Huygens wing 8

Wed, 11 Feb 2015 18:00:00 +0100

The configurations in the network switches in the buildings/floors/locations mentioned above, will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible.

Network switch problem

Wed, 04 Feb 2015 16:42:00 +0100

Due to a configuration fault in a C&CZ server switch it became unreachable. Therefore some servers and services were unreachable as well (nameserver, mail etc). After a quick fix the systems were up and running again.

Power dip Huygens building Saturday, January 17

Sat, 17 Jan 2015 07:54:00 +0100

Saturday morning, during maintenance by UVB and Liander, there were a few short unintended power dips in the Huygens building. A large part of the network switches does not have a UPS and thus restarted. After 09:11 all switches were operational again.

Background: January 10 and 17, maintenance has been carried out on the power transformers in the Huygens building. This work should be carried out once every 5 years. Because a power failure of 6-8 hours (duration of maintenance) is unacceptable for the Faculty of Science, last year bypasses have been constructed, to make the transformers able to replace each other. Beforehand, the chance that a power failure should occur, was thought to be particularly small. The next time such a service is planned, in 2020, everyone will be informed in advance.

Upgrade of license server

Thu, 08 Jan 2015 17:30:00 +0100

The license server still runs Ubuntu 10.04. This means that in a few months it would not receive security updates. Furthermore we want to use this server for DNS/DNSSEC/DHCP-service, which makes Ubuntu 14.04 necessary. Because this is a license server, we cannot switch to a new server, but we will place disks with the new OS in the existing server. Because this only means a small amount of downtime, users will hardly notice.

Fileserver bulk problems

Wed, 07 Jan 2015 14:35:00 +0100

Resolved by a server reboot.

Affected disks:

acfiles beevee bioniccell bio-orgchem celbio1 celbio2 chemprac csgi-archief csg-projecten csg-staf desda ds ehef1 ehef2 ehef3 femtospin geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 hfml-backup-inst highres impuls introcie isis janvanhest kaartenbak kangoeroe mathsolcie microbiology microbiology2 milkun2 milkun2RenD milkun3 milkun5 milkun6 milkun7 milkun7rw molbiol mwstudiereis neuroinf neuroinf2 olympus puc randomwalkmodel secres2 sigma sigmacies spectra spectra-rw spm spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 spmstaff ssi staff-hfml stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde wkru

Again IMAP mail server problem

Wed, 24 Dec 2014 17:46:00 +0100

There was again a problem with another harddisk of the IMAP server. This made the server unusable. A reboot solved the problem.

IMAP mail server problem

Wed, 24 Dec 2014 09:20:00 +0100

The IMAP server got overloaded again which could not be due to user usage. The server was rebooted since all I/O processing was stopped and we could not kill any processes. Snapshots have now been completely disabled. It took quite some time before the server had all disks and filesystems in working order.

IMAP mail server problem

Mon, 15 Dec 2014 06:30:00 +0100

There was a problem with a failed harddisk of the IMAP server. This slowed the machine down so that it was practically unusable. The problem was solved by replacing the disk.

Network problem RU

Wed, 26 Nov 2014 09:45:00 +0100

As reported by the ISC, the RU was victim of a DDOS attack. As a consequence of this is the connection to the Internet was interrupted. Measures have been taken to prevent a similar attack having this result in the future.

Network maintenance in Mercator III south and low-rise buildings

Wed, 19 Nov 2014 18:30:00 +0100

The network switches in the buildings/floors/locations mentioned above will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible. This includes the network for access control, climate control etc.

Server nachtwacht preventive maintenance

Mon, 17 Nov 2014 16:30:00 +0100

The server nachtwacht reports having a hard drive that will soon fail. We will replace the disk as soon as possible.

DNS resolver problem

Sat, 15 Nov 2014 11:29:00 +0100

The server that acts within FNWI as first DNS resolver crashed, probably because of a disk becoming defective. This made the network slow or even virtually unusable for a lot of computers in FNWI. Only after a reboot of the server, the problem1 was solved. Monday morning, at the weekly reboot of this server, it would only boot after manually acknowledging the defective disk (problem2), When other servers rebooted at 06:30, a lot of them had problems due to the missing DNS resolver. Further measures to reduce the inconvenience of such a crash have been taken. The move to new DNS servers is in preparation.

Fileserver bulk problems

Wed, 12 Nov 2014 14:45:00 +0100

Resolved by a server reboot.

Affected network shares:

acfiles ARK-Backup Bargerveen Bargerveen-Gebruikers beevee bioniccell bio-orgchem B-Ware-Arnica B-Ware-Dotter B-Ware-Erica B-Ware-Knolrus B-Ware-Lobelia B-Ware-Stratiotes celbio1 celbio2 csgi-archief csg-staf cvi desda ds ehef1 evsf2dataoud evsf2schijf1 femtospin Floron geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 highres impuls introcie isis janvanhest kaartenbak kangoeroe microbiology microbiology2 milkun2 milkun3 milkun5 milkun6 milkun7 milkun7rw molbiol mwstudiereis neuroinf neuroinf2 nfstest nwibackup olympus puc RandomWalkModel Ravon-Algemeen Ravon-Foto Ravon-Projecten secres2 sigma sigmaalmanak sigmacies sigmaexchange sigmasymposium splex spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde

Network maintenance in Mercator III north

Tue, 11 Nov 2014 18:30:00 +0100

Network maintenance in server room Huygens

Tue, 11 Nov 2014 06:00:00 +0100

Because of a software upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be rebooted at 06:00h (AM) exactly. This means that both wired en wireless networks at these locations will be interrupted for 10 to 15 minutes and network traffic will not be possible. This includes the network for access control, climate control etc.

Network problems FNWI

Tue, 11 Nov 2014 06:00:00 +0100

After the network maintenance of 20141111 a single user reported later a problem with the wired network. These problems intensified and grew faculty wide on 20141119 at 13:34 hours. The network switches in the Faculty of Science received so many topology changes of the Spanning Tree Protocol (STP) that network traffic often stuttered. By curbing the STP traffic on 20141120 at 15:32 the stuttering disappeared. We still search for the exact cause, probably it is a housing move.

Network maintenance in part of Huygens

Thu, 06 Nov 2014 06:00:00 +0100

Locations:

Huygens wing 3 all floors
Huygens wing 4 all floors
Huygens wing 8 all floors
Huygens central street between wing 6 and 8 all floors

Network maintenance in peripheral buildings

Tue, 04 Nov 2014 06:00:00 +0100

Because of a software upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be rebooted on Tuesday morning at 06:00h (AM) exactly. This means that both wired en wireless networks at these locations will be interrupted for 10 to 15 minutes and network traffic will not be possible. This includes the network for access control, climate control etc.

Locations: FELIX, HFML, Nanolab, Goudsmit pavillion (NMR), Lab DCN in A-2.002, Kinderopvang Heyendael (both buildings), Logistiek Centrum FNWI, Linnaeus building, Mercator 1, 6th floor

Network maintenance in Huygens wing 7

Wed, 29 Oct 2014 19:00:00 +0100

Because of an upgrade of the network switches in the buildings/floors/locations mentioned above, these network switches will be replaced. This means that both wired en wireless networks at these locations will be interrupted and network traffic will not be possible. This includes the network for access control, climate control etc.

RU network problem

Wed, 29 Oct 2014 09:38:00 +0100

Due to a short DDOS-attack the RU was virtually disconnected from the Internet during ca 7 minutes. All network connected systems may have had problems during that time. Until 15:30 some Eduroam users still had problems acquiring an IP address.

FNWI network problem

Wed, 15 Oct 2014 03:44:00 +0200

The router that regulates all IP traffic between subnets in the Huygens building and surrounding buildings, became too busy. All network traffic, especially UDP had problems. ISC network admins reduced the load by simplifying the ACLs and reducing logging. At about 12:00 the problem was temporarily solved. C&CZ will do a further inspection of all ACLs, with the aim to reduce the load on the router without reducing security for connected systems.

Disturbance of Vodafone VWO data traffic

Wed, 24 Sep 2014 08:30:00 +0200

On the ISC website we read:

Radboud University agreed on a new mobile telephone contract with - again - provider Vodafone. For this purpose, on Wednesday, September 24th during daytime the Radboud subscriptions will be adjusted. Employees with a Vodafone mobile wille have no disturbance in calling or being called that day. But the Vodafone data network might be temporarily unavailable. To restore the data connection it is necessary to turn the mobile phone off and then on again. Of course employees on campus can always use the eduroam Wi-Fi network.

DNS resolver problem

Sat, 20 Sep 2014 10:45:00 +0200

The server that acts within FNWI as first DNS resolver crashed. This made the network virtually unusable for a lot of computers in FNWI. Only after a reboot of the server, the problem was solved. Measures to reduce the inconvenience of such a crash and to make such a crash less likely, have been partly taken and are partly in preparation.

Webserver (klopjacht) disk problem

Tue, 09 Sep 2014 10:46:00 +0200

A defective disk made the server stop serving web requests. A reboot fixed the problem. The defective disk will be replaced soon.

Surfnet uplink offline

Mon, 08 Sep 2014 22:00:00 +0200

Due to an error during the planned maintenance by Surfnet of the RU network interface, the uplink of Radboud University to Surfnet was down.

Fileserver goudsmit problems

Mon, 08 Sep 2014 06:30:00 +0200

One of the filesystems contained errors and prevented the system from starting up.

SMTP server offline for half an hour

Thu, 21 Aug 2014 16:00:00 +0200

Both processor fans have failed and need to be replaced.

Fileserver bulk problems

Fri, 15 Aug 2014 12:04:00 +0200

The fileserver had problems, which were resolved after a server reboot.

Affected network shares:

acfiles ARK-Backup Bargerveen Bargerveen-Gebruikers beevee bioniccell bio-orgchem B-Ware-Arnica B-Ware-Dotter B-Ware-Erica B-Ware-Knolrus B-Ware-Lobelia B-Ware-Stratiotes celbio1 celbio2 csgi-archief csg-staf cvi desda ds ehef1 evsf2dataoud evsf2schijf1 femtospin Floron geminstr2 giphouse gissig gmi hfml-backup hfml-backup2 highres impuls introcie isis janvanhest kaartenbak kangoeroe microbiology microbiology2 milkun2 milkun3 milkun5 milkun6 milkun7 milkun7rw molbiol mwstudiereis neuroinf neuroinf2 nfstest nwibackup olympus puc RandomWalkModel Ravon-Algemeen Ravon-Foto Ravon-Projecten secres2 sigma sigmaalmanak sigmacies sigmaexchange sigmasymposium splex spmdata3 spmdata4 spmdata5 spmdata6 spmdata7 stroom thalia ucm vsc vsc1 vsc10 vsc11 vsc12 vsc13 vsc14 vsc15 vsc2 vsc3 vsc4 vsc5 vsc6 vsc7 vsc8 vsc9 vscadmin wiskalg wiskunde

M1.04 no network

Thu, 14 Aug 2014 16:09:00 +0200

While administering the network switchports for the move of ISIS to Mercator1 floor 0 to 3, ISC network management erroneously also changed the configuration of the network switchports of floor 4. When this was reported this morning, it could be readily remedied.

Failure cn56 disk

Mon, 07 Jul 2014 08:00:00 +0200

One of the mirrored disks failed. We didn’t manage to rebuild the raid set with a new disk. The system is reinstalled with two new disks. After that, data from /scratch could be recovered from one of the old disks.

Network drive migration

Thu, 05 Jun 2014 16:00:00 +0200

The network drives will be migrated to a new server. During the migration, the network drives will be unavailable.

Affected shares:

acfiles, bioniccell, bio-orgchem, celbio1, CSGI-Archief, Csg-Staf, ds, evsf2schijf1, giphouse, gissig, gmi, hfml-backup, hfml-backup2, highres, impuls, introcie, isis, kaartenbak, microbiology, microbiology2, microbiologyftp, milkun2, milkun3, milkun5, milkun6, milkun7, milkun7rw, mwstudiereis, neuroinf, olympus, permissies, puc, secres2, sigma, sigmaalmanak, sigmacies, sigmaexchange, sigmasymposium, Soheto, splex, spmdata3, spmdata4, spmdata5, spmdata6, spmdata7, stroom, thalia, ucm, vsc, vsc1, vsc2, vsc3, vsc4, vsc5, vsc6, vsc7, vsc8, vsc9, vsc10, vsc11, vsc12, vsc13, vsc14, vsc15,wiskalg, wiskunde

Reboot of login servers after network drive migration

Thu, 05 Jun 2014 16:00:00 +0200

The planned move of network drives from an old to a new server caused a lot of “stale NFS file handles”, which made it necessary to reboot some login servers.

IMAP mail server problem

Thu, 05 Jun 2014 14:05:00 +0200

The IMAP server got overloaded again. Restarting the IMAP service was required to bring it back in operation. We think it is related to snapshots and therefore regularly remove invalid snapshots from now on.

IMAP mail server problem

Tue, 27 May 2014 14:05:00 +0200

The IMAP server got overloaded. Restarting the IMAP service was required to bring it back in operation.

Data/file/vol server (heap) problem

Mon, 26 May 2014 06:45:00 +0200

The server had a problem, which had started at the Monday morning reboot. A reboot of the machine solved the problem. Soon all data will be moved to a new server. This will not cause much problems for users, because everybody uses the aliases something-srv.science.ru.nl.

Maintenance wireless infrastructure May 19

Mon, 19 May 2014 20:00:00 +0200

At Monday May 19, a major upgrade of the wireless infrastructure and its maintenance tool (called Prime) will take place by the ISC. Therefore the wireless service (i.e. Eduroam) will be interrupted several times from 8:00 pm. Existing wireless connections will be lost and new connections will not be possible for some time. Due to the complexity of this operation this maintenance will possibly be continued at Monday, May 26, again with interruptions from 8:00 pm.

Maintenance Goudsmit

Mon, 12 May 2014 13:00:00 +0200

One of the systemdisks reported SMART-errors and will be replaced.

lilo.science.ru.nl offline

Mon, 12 May 2014 06:30:00 +0200

Lilo did not shut down correctly during the planned weekly reboot. After a hardware reset, the machine booted just fine

And yet again Print/phpMyAdmin server problem

Thu, 08 May 2014 16:24:00 +0200

Just like yesterday, a reboot of the machine was necessary to restore the functionality. Tomorrow morning, the server will restart with a new kernel. It this doesn’t solve the problem, we will look into a new version of Samba.

Yet again Print/phpMyAdmin server problem

Wed, 07 May 2014 12:49:00 +0200

Just like yesterday, a reboot of the machine was necessary to restore the functionality. We will investigate if we can remediate this problem with newer software versions (Samba, Ubuntu, …)

Eduroam problem

Tue, 06 May 2014 13:40:00 +0200

Some Eduroam users didn’t get an IP number, because one of the IP ranges had no free leases. This ranges contained all @science.ru.nl users. The ISC networking department first decreased the lease time. Now they also make this range free for use by @science.ru.nl users. This will prevent the problem from occuring again soon.

And again Print/phpMyAdmin server problem

Tue, 06 May 2014 12:14:00 +0200

Again a reboot of the machine was necessary to restore the functionality. As a patch we will reboot this server each night.

U-number authentication problem Eduroam

Wed, 30 Apr 2014 13:00:00 +0200

The new RU IdM system erroneously rewrote the RU password hash when data about a person changed. This primariliy affected employees of FNWI, because for this group the Science email address had been incorporated in IdM. This will make it possible to use the Science address as an external mail address when resetting RU passwords. When the problem was recognized, it was fixed swiftly by restoring the RU password hashes from backup.

Peage MFP Huygens building in maintenance April 28/29/30

Mon, 28 Apr 2014 12:00:00 +0200

April 28/29/30, the Peage MFP and the Peage POS near the restaurant in the Huygens building cannot be used. The ISC will then investigate whether Peage can also be made available for employees instead of only for students. April 28, the MFP will be moved to the test location, April 30 it will be moved back.

Poster printer "kamerbreed" broken

Mon, 21 Apr 2014 00:00:00 +0200

The printer has to be repaired. Spare parts have been ordered. We hope to have the printer operational again mid-May.

After replacement still Print/phpMyAdmin server problem

Thu, 17 Apr 2014 13:35:00 +0200

Although the hardware of the server was replaced, the server didn’t react for unknown reasons. A reboot of the machine solved the problem. We will think about ways to tackle this problem.

Print/phpMyAdmin server replacement Thursday April 17 08:30-09:20

Thu, 17 Apr 2014 08:30:00 +0200

Because of the recent problems with this server, we are going to replace the server, in order to prevent further service interruptions.

U:/ home server pile problem

Wed, 16 Apr 2014 17:42:00 +0200

The server had a problem with one partition, which had started during the creation of new snapshots. We waited with the reboot untill after working hours. A reboot of the machine solved the problem. To prevent these problems in the future, we will no longer make local snapshots of homeservers, but of course the daily backups of the homeservers by the backup server will be continued.

And yet again Print/phpMyAdmin server problem

Mon, 14 Apr 2014 13:05:00 +0200

Just like four days ago, the server didn’t react for unknown reasons. A reboot of the machine solved the problem. We plan to replace the server next Thursday 08:30-09:00 hours, in order to prevent further service interruptions.

Again Print/phpMyAdmin server problem

Thu, 10 Apr 2014 12:54:00 +0200

Just like two weeks ago, the server didn’t react for unknown reasons. A reboot of the machine solved the problem. Together with the supplier we will try to find out what to replace to prevent this in the future.

U: / home server bundle problems

Fri, 04 Apr 2014 11:35:00 +0200

From the moment of yesterday’s snapshots (13:00 hours), more and more processes were hung at the server. The first complaints arrived at C&CZ ca. 11:15 hours this morning. Therefore we decided ca. 11:35 to restart the server. The reboot resolved the problem. The number of snapshots will be reduced in order to try to prevent problems due to snapshots in the future.

Printer pr-hg-00-002 did not print from Windows/Mac

Wed, 02 Apr 2014 11:32:00 +0200

Probably due to a too large printjob, the printer queue on the server (printto/printsmb/ooievaar) for the pr-hg-00-002 printer got stuck onWednesday morning. C&CZ was only informed of this problem at the end of Thursday. Friday morning this has been fixed by emptying the printqueue with old jobs.

Print/phpMyAdmin server problem

Thu, 27 Mar 2014 10:40:00 +0100

The server didn’t react for unknown reasons. A reboot of the machine solved the problem.

U:/ home server pile problem

Mon, 24 Mar 2014 17:25:00 +0100

The server had a problem with one partition. A reboot of the machine solved the problem.

Linux login server lilo3 reboot due to NFS-problems

Mon, 10 Mar 2014 12:31:00 +0100

The Linux login server lilo3 didn’t have the homedirectories of some users. It took a reboot to fix this.

File server heap problems

Mon, 10 Mar 2014 06:30:00 +0100

The fileserver failed to start after the Monday morning reboot, because of filesystem errors (LVmd2-7). Manually fixing the errors solved the problem and the machine started successfully.

U: / home server pile and Linux login server lilo/stitch problems

Wed, 05 Mar 2014 13:00:00 +0100

The server had a high load and stopped servicing users, probably because of problems with the creation of new snapshots at 13:00 hours. A reboot solved the problem.

Network interruptions in Huygens wing 5

Mon, 03 Feb 2014 19:00:00 +0100

On Monday evening, February 3rd between 19:00 - 24:00 (7:00 - 12:00 pm) maintanance will be carried out on the network devices in Huygens Wing 5. Therefore the wired and wireless networks will not be available at some moments at all locations in this Wing, on all floors.

Again network interruptions in Huygens wing 2

Tue, 28 Jan 2014 19:00:00 +0100

The announced proceedings on Tuesday, January 21st on the network equipment unfortunately did not take place due to a sudden instability of the network in Wing 1 the same day. In order to prevent further instability issues in Wing 2, it was necessary to solve the problems in Wing 1 first. These problems have been identified and corrected. The network maintenance in Wing 2 will now take place Tuesday, January 28th between 19:00 and 24:00 ( 7:00 pm to 12:00 ) during which the wireless and cabled networks will be off line for some time. This will affect all floors in Wing 2 and rooms along the corridors between wings 2 and 4 (all floors).

Network interruptions in Huygens wing 2

Tue, 21 Jan 2014 19:00:00 +0100

On Tuesday evening January 21 between 7:00 pm and 12:00 pm maintanance will be carried out on the network devices in Huygens wing 2. Therefore the wired and wireless networks will not be available at some moments at all locations in wing 2 and corridors between wings 2 and 4, on all floors.

Network interruptions in Huygens wing 1

Mon, 13 Jan 2014 19:00:00 +0100

On Monday evening January 13 between 7:00 pm and 12:00 pm maintanance will be carried out on the network devices in Huygens wing 1. Therefore the wired and wireless networks will not be available at some moments in many locations in wing 1 and corridors between wings 1 and 3, on all floors.

Network problems

Mon, 06 Jan 2014 11:29:00 +0100

Due to an error by an ISC network administrator, all internal and external RU network traffic was blocked for a short period.

Postponed replacement of home-server "bundle"

Mon, 23 Dec 2013 07:30:00 +0100

The replacement of the old home server “bundle” has been postponed and will now guaranteed take place on Monday morning December 23. Because the data have been synchronized with the new server, there will not be much downtime. The new server should be very dependable: hardware RAID-6, double processors and power supplies and a 5-year support contract from the supplier. The performance has improved, e.g. by using hardware RAID with a 1 GB write cache with battery backup.

Moving network shares to new servers

Mon, 23 Dec 2013 00:00:00 +0100

In the course of these two days, above network shares will be moved to new servers. During the move action, a share will not be available. If you encounter any problems connecting to a network share using Windows, always connect using the “share-name minus srv” naming scheme: \\sharenaam-srv.science.ru.nl\sharenaam. See also Diskruimte#Naming.

Affected network shares:

acfiles2 botany botany-general carta comsol digicd encapson encapson2 exoarchief felix gi2 gi3 hfml-45t hfml-data hfml-engineering ifl iris mailmanincludes mbaudit1 mbaudit2 mbbioel mbcns mbcortex mbdata mbread mbwrite mestrelab mi2 mi3 milkun4 milkun4rw molchem molchem2 molphtec mol-secr multimedia neuropi ns3 nwi-backup onlyme owc pcb planthgl plantkunde-hgl sdisk/software share snn sofie spmdata1 tdisk/cursus tece teceleiding temp tracegastemp wallpaper xpcursus xpsoftware

Interruption of network in Huygens wing 1

Tue, 17 Dec 2013 19:00:00 +0100

Tuesday night December 17 19:00-24:00 hours, maintenance work will be done affecting the data network in Huygens wing 1. The wired and wireless network will suffer service interruptions a few times during that period. The main inconvenience will be on the ground floor.

Service interruption Konica Minolta MFP's outside of Huygens building

Mon, 09 Dec 2013 12:48:00 +0100

C&CZ changed the DHCP-configuration of the KM MFP’s, because KM had told us all KM’s had an identical configuration after the upgrade of the night of December 4. Soon thereafter the KM’s outside of the Huygens building showed problems. After changing the DHCP-configuration back to the previous version and restarting the MFP’s, the problem was resolved. The firmware upgrade of these machines still has to take place.

Konica Minolta MFP's firmware upgrade

Wed, 04 Dec 2013 02:00:00 +0100

Wednesday night, the firmware of the Konica Minolta multifunctionals will be upgraded. This should resolve existing problems, like the not waking up from sleep mode. Please report all remaining problems, for MFP-hardware and paper to KM via phone: 55955 option 4.

Power dip December 3

Tue, 03 Dec 2013 09:36:00 +0100

Tuesday morning around half past nine, there was a short power dip for RU/UMCN, probably due to a switch error at Liander. This power dip, that is not listed on the power interruption website, made all systems restart that were not on UPS power. Because only the network switches in Huygens wing 1 and 7 are on UPS power, a lot of users lost their connection to the network, including wireless and IP-telephony, for about 20 minutes. Apparatus that restarted faster than the network, might have needed an extra restart to restore the connection to the network.

BASS Java 7 problem

Sun, 01 Dec 2013 00:00:00 +0100

The ISC let us know that after the installation of patches installed in BASS during the weekend of December 1, the possibility to use a recent version of Java (version 7) had erroneously disappeared. As of February 11, 2014 this issue has been resolved. The function in BASS to work with all versions of Java SE (JRE) on the client (PC) has been reactivated. Furthermore all JAR-files in BASS have received a security certificate (they have been signed), with which it possible for BASS to work with the latest Java SE versions (Java SE 7.51 en hoger). BASS users who work with Forms, will see pop-up windows asking “Do you want to run this application?”. Of this a description has been made on the RU Intranet.

Gipphoenix network problem

Mon, 18 Nov 2013 06:33:00 +0100

For unknown reasons, the network interface of the server was not online. Running ifdown / ifup eth2 resolved the problem.

Web lectures unavailable for 20 minutes

Fri, 15 Nov 2013 10:35:00 +0100

For unknown reasons one of the servers had a high load and stopped servicing users. A reboot solved the problem.

U: / home server bundle problems

Tue, 29 Oct 2013 16:00:00 +0100

For unknown reasons the server had a high load and stopped servicing users. A reboot solved the problem.

Again IMAP mail server problem

Mon, 28 Oct 2013 13:35:00 +0100

Just like 3 days ago, the IMAP server got overloaded after noon. It took a reboot to bring the service back in operation. We suspect that our making of a backup-snapshot triggers this and now have disabled the snapshot during working hours.

IMAP mail server problem

Fri, 25 Oct 2013 13:23:00 +0200

During an extra backup the IMAP server got overloaded. It took a reboot to bring the service back in operation.

Wireless @ru.nl authentication problem

Wed, 23 Oct 2013 14:30:00 +0200

Yesterday afternoon around 14:30 the ISC conducted a seemingly innocent maintenance on the LDAP-server, but immediately after that auth-requests from Radius were no longer serviced. This made it impossible for wireless users to authenticate with their u/s/e number. Users in the realm @science.ru.nl were not affected by this.

Power dip October 22, ca. 11:00

Tue, 22 Oct 2013 10:55:00 +0200

Tuesday morning around 11 o’clock, there was a short power dip for RU/UMCN. This power dip, that is not listed on the power interruption website, made all systems restart that were not on emergency power. Because only the network switches in Huygens wing 1 and 7 are on emergency power, a lot of users lost their connection to the network, including wireless and IP-telephony, for about 5 minutes. Apparatus that restarted faster than the network, might have needed an extra restart to restore the connection to the network. A department reported that a departmental printer did not survive the power dip.

Scan to email sometimes doesn't work for Konica Minolta MFPs

Sat, 12 Oct 2013 00:00:00 +0200

Update: we changed the configuration of the network switchports of the KM’s, which resolves the problem. This was necessary, because the KM’s do not try hard enough to make a connection with the SMTP-server.

Update: an alternative is scanning to a USB stick. Log in at the MFP with the scan-pin, put a document in the feeder or on the document glass, select Scan and then plug the USB stick into the USB port on the right side of the MFP , on the top side. After a few seconds the MFP recognizes the USB stick and presents the choice “Save a document to external memory”. Choose OK and press Start. See also [RU http://www.ru.nl/publish/pages/687597/uitrol-mf-scan.pdf manual] on the RU-page about the MFP’s.

Again mail problems after supplying password to phishers

Thu, 10 Oct 2013 00:04:00 +0200

Again a Science user supplied his Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam.

PLEASE: do not naively click on a link in an e-mail!

Printto/printsmb/phpmyadmin server problems again

Tue, 01 Oct 2013 15:18:00 +0200

The printserver crashed again, for yet unknown reasons. A reboot of the machine solved the problem.

U: / home server bundle problems

Mon, 30 Sep 2013 13:46:00 +0200

The server crashed. A reboot of the machine solved the problem.

Printto/printsmb/phpmyadmin server problems

Fri, 27 Sep 2013 11:02:00 +0200

The printserver crashed. A reboot of the machine solved the problem.

Poster printer "kamerbreed" broken

Wed, 25 Sep 2013 17:00:00 +0200

The motherboard of the printer had to be replaced. Because there is no maintenance contract for this old printer and spare parts were hard to get, repair has taken a long time.

Printto/printsmb server problems

Tue, 24 Sep 2013 11:41:00 +0200

The printserver crashed. A reboot of the machine solved the problem.

Sound on Linux dual-boot PC's disabled

Mon, 23 Sep 2013 00:00:00 +0200

The availability of sound in Windows appeared to be depending on the state in which Linux had left it. Therefore the sound in Linux on dual boot PC’s has been disabled, in order to have it available on Windows. Sound under Linux will be available again once the HP Compaq Elite 8300 AiO harware has been phased out.

File server heap problems

Tue, 17 Sep 2013 16:25:00 +0200

The fileserver failed to see its disks. A reboot of the machine solved the problem.

File server chunk failed reboot

Mon, 16 Sep 2013 06:30:00 +0200

The fileserver failed the regular Monday morning reboot. Only after a rescue boot and a removal of all snapshots rebooting the machine worked.

Affected disks:

arb botgarden ccs4 excienwi exoarchief FIT gi1 gipsy isis-dhz itt iwwr1 leonardo mercator micord molspec olcwis ons pvs ratio tzacad vb vscxray WiskundeToernooi zeegras

Mail problems after supplying password to phishers

Fri, 06 Sep 2013 10:13:00 +0200

PLEASE: do not naively click on a link in an e-mail!

And again mail problems after supplying password to phishers

Wed, 04 Sep 2013 12:22:00 +0200

PLEASE: do not naively click on a link in an e-mail!

And yet again mail problems after supplying password to phishers

Tue, 03 Sep 2013 13:12:00 +0200

PLEASE: do not naively click on a link in an e-mail!

B/W KM printer pr-hg-00-002 printed in color

Mon, 02 Sep 2013 13:33:00 +0200

The b/w tuned Konica Minolta printer/MFP pr-hg-00-002 suddenly printed in color. C&CZ refunded the student’s budget and stopped the printer until KM had corrected the settings.

DNS nameserver problem

Mon, 02 Sep 2013 04:30:00 +0200

The DNS server on ns1.science.ru.nl didn’t start after the reboot, due to a syntax error in one of the zone files. When this had been corrected, it started without problems.

Fileserver chunk needs reboot

Thu, 29 Aug 2013 07:30:00 +0200

Affected shares:

arb botgarden ccs4 excienwi exoarchief FIT gi1 gipsy isis-dhz itt iwwr1 leonardo mercator micord molspec olcwis ons pvs ratio tzacad vb vscxray WiskundeToernooi zeegras

Yet again mail problems after supplying password to phishers

Thu, 29 Aug 2013 05:30:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Again mail problems after supplying password to phishers

Thu, 22 Aug 2013 22:38:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Mail problems after supplying password to phishers

Thu, 22 Aug 2013 01:45:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Wireless access to BASS problem

Mon, 12 Aug 2013 00:00:00 +0200

From August 12 the wireless network in the Science buildings is being replaced. The new IP numbers appear to not be able to connect directly to BASS. We expect that this will be changed soon.

Windows Roaming Profile problem

Thu, 01 Aug 2013 00:00:00 +0200

An error in distributing a Group Policy Object (GPO) caused roaming profiles to fail for the last week. The start date/time of the problem is unknown, This is an estimate.

Lilo3 reboot for different IP number

Mon, 15 Jul 2013 11:00:00 +0200

Today lilo3 appeared to have an incorrectly chosen IP number. A reboot fixed this.

Mail problems after supplying password to phishers

Mon, 01 Jul 2013 23:50:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Mail problems after supplying password to phishers

Thu, 27 Jun 2013 17:48:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Mail problems after supplying password to phishers

Thu, 27 Jun 2013 17:48:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Mail problems after supplying password to phishers

Mon, 24 Jun 2013 09:00:00 +0200

PLEASE: do not naively click on a link in an e-mail!

Mail problems after supplying password to phishers

Sun, 23 Jun 2013 15:54:00 +0200

PLEASE: do not naively click on a link in an e-mail!

DNS nameserver problem

Mon, 17 Jun 2013 04:30:00 +0200

The DNS server on ns1.science.ru.nl didn’t start after the reboot, due to a syntax error in one of the zone files. When this had been coreected, it started without problems.

Mail problem (IMAP server)

Wed, 12 Jun 2013 13:45:00 +0200

The IMAP server had a problem, which made a reboot necessary.

Maintenance Wireless@RU

Thu, 06 Jun 2013 22:00:00 +0200

On Thursday June 6th from 10:00 pm the wireless networks ru-wlan and eduroam will be unavailable for at least 2 hours. All existing connections will be cut off. The wireless network Science however will not be effected and will be kept available.

BASS RU problem

Mon, 03 Jun 2013 00:00:00 +0200

The central BASS RU environment has been updated last weekend. Part of this upgrade was a change of the web address of the second logon screen, as announced on our BASS page. This morning it became clear that access to the second logon screen via the Port Forwarder didn’t work anymore. Therefore the UCI rolled back the change of the second logon screen. This means that AFTER LOGGING ON TO THE PROXY at https://admin.ru.nl/ BASS is available again via https://bassruap01.uci.ru.nl:8010/OA_HTML/AppsLocalLogin.jsp

OpenID server problem

Fri, 24 May 2013 03:30:00 +0200

During the renewal of the OpenID server a configuration error was made. When users reported problems logging in, we started searching for the origin of problem. When it was found, it could easily be corrected and the server could be restarted with the correct configuration.

Homeserver bundle crashed

Tue, 21 May 2013 17:00:00 +0200

The fileserver crashed. After a reboot everything was back to normal.

Homeserver bundle failed reboot

Mon, 29 Apr 2013 06:30:00 +0200

The fileserver failed to reboot during the regular Monday morning shutdown schedule. It was possible to gain access to the system console only after having removed all power from the chassis. Snapshots were removed using the rescue reboot but rebooting the machine resulted in a faulty filesystem. We were able to boot the system after all filesystems had been checked offline. These actions resulted in a unusual long downtime.

Fileserver stack crashed due to failed hard disk

Mon, 29 Apr 2013 04:08:00 +0200

Problem: Crash due to defective disk Solution: Deactivated disk using hot spare and reboot

Homeserver bundle failed reboot

Mon, 22 Apr 2013 06:30:00 +0200

The fileserver failed the regular Monday morning reboot. Only after a rescue boot and a manual removal of all snapshots rebooting the machine worked.

Disk server stack offline

Mon, 08 Apr 2013 08:55:00 +0200

Problem: Stack: Crash due to failing disk. Solution: Deactivated disk using hot spare

Disk server pile offline

Mon, 08 Apr 2013 06:30:00 +0200

Pile did not shutdown properly during weekly reboot due to a kernel panic. Solution: Executed power-cycle of the system

Mail problems after supplying password to phishers

Tue, 19 Mar 2013 11:45:00 +0100

PLEASE: do not naively click on a link in an e-mail!

Disk server pile offline

Mon, 18 Mar 2013 06:30:00 +0100

Was waiting for interactive input after reporting a warning (^d)

File server miii no private network

Fri, 15 Mar 2013 17:30:00 +0100

While performing maintenance (removing a network card) a network cable was not inserted correctly.

Network problems due to installation of Matlab R2013a

Wed, 13 Mar 2013 13:00:00 +0100

Yesterday Matlab R2013a has been installed. Today at 13:00 hours many servers started to automatically copy this 5.4 GB to their local disc. Some parts of the network were overloaded by all these copying, which made accessing the network slow for many users. The distribution of this software will now be scheduled to happen over a longer period, primarily outside of working hours.

More IP addresses for ru-wlan and Science (wireless)

Mon, 04 Mar 2013 00:00:00 +0100

Monday, March 4th 2013 at 18:00 hours, the number of IP numbers that is available in the FNWI buildings for ru-wlan and Science will be doubled. Because ru-wlan moves to a new range, users of ru-wlan will lose connectivity for at most 15 minutes. There was already a plan to replace ru-wlan and Science within the FNWI buildings by the RU-wide Eduroam and ru-wlan, but the wireless network usage has grown so fast that we can not wait for this plan to be realized. Last week some wireless users at times could not even get an IP address, although the lease time had been brought down to 30 minutes. Therefore this temporary measure became necessary on such short notice.

Short interval in wireless network service

Mon, 18 Feb 2013 00:00:00 +0100

On Monday feb 18 at 6:00 pm there will be some maintenance at the wireless network which will effect the following locations at Toernooiveld:

FNWI cellar A1 FEL Huygens: Library of Science, terrace behind Huygens, cantine, room HG-1.132 KDV1 en KDV2 Linnaeus building Logistic Centre Mercator I Mercator II, ground floor and 7nd floor Mercator III, 2nd floor Transitorium FNWI (ACSW and FELIX) UBC

We expect the service will be completely available again within 30 minutes.

Mail problems after supplying password to phishers

Thu, 14 Feb 2013 00:00:00 +0100

The last few days three Science users have supplied their Science password to phishers. We notice that because Internet criminals use these passwords to get into the Science mail servers (horde webmail, smtp) in order to send spam. This time they even used a fake copy of the horde Science webmail website. The big differences with the real horde Science webmail website are:

the URL is not within science.ru.nl
the connection is not a secure https connection, there is no lock
the username and password do not arrive at C&CZ servers, but in the hands of Internet criminals.

PLEASE: do not naively click on a link in an e-mail!

New Radius server for ru-wlan and eduroam (wireless)

Mon, 28 Jan 2013 00:00:00 +0100

On Monday, January 28th 2013 at 8:00 am, one of the servers that is being used by the wireless network of the RU, will be replaced. This replacement will affect you as a user of the wireless networks ru-wlan and eduroam: There will appear a new certificate when connecting. You can just accept this, after which the connection should work. If this appears not to be the case, then it’s best that you remove your old Eduroam- respectively your old RU-WLAN settings first to activate the new connection .

Homeserver bundle crashed

Wed, 16 Jan 2013 00:00:00 +0100

Because the file server crashed, it had to be rebooted.

LDAP server vernieuwd

Fri, 14 Dec 2012 00:00:00 +0100

Older Fedora desktop PC’s may experience startup problems after an upgrade of one of our LDAP servers. A fix is available and has been applied. If you still encounter this problem, please contact C&CZ.

Mail problems after supplying password to phishers

Fri, 16 Nov 2012 00:00:00 +0100

Horde webmail again appeared to be misused for sending spam. This could happen because a naive user gave the Science password to phishers/spammers. After first stopping horde, early Friday morning we disabled the account of the naive user and restarted horde. Saturday morning it appeared that this short spam-outbreak had caused administrators of hotmail.com to add our mail server to their blacklist. Therefore we switched the IP-number of this mail server Saturday morning.

Homeserver bundle will be rebooted

Wed, 24 Oct 2012 00:00:00 +0200

Because the file server refuses to accept a spare disk, it needs a reboot.

Homeserver bundle unavailable

Mon, 22 Oct 2012 00:00:00 +0200

At the moment, we are solving the problem.

Services unavailable due to power and network outage

Thu, 18 Oct 2012 00:00:00 +0200

During the night of wednesday on thursday a power outage resulted in a network outage in the basement computing facilities. The power was restored to the network equipment using a bypass thus circumventing the UPS at about 09:15. Further checks implied that most servers had not become powerless so that most services became automatically available again. Network drivers on “bundle” had to be restarted in order to get access to home directories for a large number of users. Furthermore, several websites had to be restarted which made it possible for PC’s to boot properly. During the day, an unrelated issue with the RAID storage of “plus” has been fixed as well granting access to the following network shares: sofie, ams*, molchem, mb*, encapson, milkun4, snn, neuropi, digicd. carta, … Since wireless devices were unable to acquire IP addresses, i.e. gain access to the network, a split-brain situation was diagnosed within the DHCP service which was resolved around 13:00.

Announced downtime: home server "pile" down for reboot

Fri, 12 Oct 2012 00:00:00 +0200

Next Friday morning, the home server “pile” will be rebooted. There are problems with the snapshots, which could make a reboot take more time. Therefore we schedule the reboot for early next Friday.

Horde webmail server down because of spam

Tue, 25 Sep 2012 23:05:00 +0200

Yesterday evening, horde webmail appeared to be misused for sending spam. This could happen because a naive user gave the Science password to spammers. First we stopped horde. This morning we disabled the account of the naive user and restarted horde.

Disk server "Plenty" offline

Mon, 24 Sep 2012 06:30:00 +0200

During the weekly reboot (monday mornings), the server got stuck in the BIOS.

Disk server "Stack" offline

Mon, 24 Sep 2012 06:30:00 +0200

Peage top-up unit near Huygens restaurant in maintenance

Wed, 12 Sep 2012 00:00:00 +0200

In order to test new software, the Peage top-up unit near the Huygens restaurant was switched to maintenance mode. This unit is not used often yet, therefore this wil not have caused problems. Students that wanted to top-up their Peage account, could do that only elsewhere on campus. See the http://www.ru.nl/peage Peage website], locations are the halls of the Erasmus, Spinoza and Library buildings.

Announced downtime: home server "pile" down for replacement

Tue, 24 Jul 2012 07:00:00 +0200

Next Tuesday morning, the home server “pile” will be replaced by a new, more powerful server. Because the data have been synchronized with the new server, there will not be much downtime.

Postponed: Announced downtime: home server "pile" down for replacement

Tue, 24 Jul 2012 07:00:00 +0200

The downtime below has been postponed, because we had a few questions on the new server, that could not be answered in time. To be continued…

Next Tuesday morning, the home server “pile” will be replaced by a new, more powerful server. Because the data have been synchronized with the new server, there will not be much downtime. The new server should be very dependable: hardware RAID-6, double processors and power supplies and a 5-year support contract from the supplier. The performance has improved, e.g. by using hardware RAID with a 1 GB write cache with battery backup.

Partly announced downtime for mailman + horde webmail server

Thu, 12 Jul 2012 09:09:00 +0200

This morning, horde webmail appeared to be misused for sending spam. This could happen because naive users gave their Science password to spammers. After we found out who the users were and had them change their password, we decide to also replace a defective cpu fan. Therefore also Mailman mailing lists will be down from 13:00 to 14:00 hours.

SMTP server blacklisted by MS Live Hotmail

Wed, 11 Jul 2012 03:08:00 +0200

This morning, users reported that mail from smtp.science.ru.nl to hotmail users was being bounced by hotmail. We have tried to let the hotmail administrators change this fast, but when this took too long, we changed the IP-number of our smtp-server.

Planned service interruption: file server with problems

Fri, 22 Jun 2012 17:03:00 +0200

A hardware failure of a boot disk of the fileserver stack was reported Friday morning June 22. We decided to repair this after working hours. Thus at approximately 17:00 the defective boot disk was removed from the machine and replaced by a spare one. Enabling the disk, making it bootable, restoring file systems and rebooting the machine (after removing all snapshots) took a lot of time. When this was resolved Friday evening, the NFS/SMB fileservice was not active on the mounted filesystems. It took a reboot Sunday evening to resolve all problems.

Tracelab server poly defective

Thu, 21 Jun 2012 14:12:00 +0200

A hardware failure of the server poly was reported at 2012-06-21 14:12. After a restart of the machine, it stopped working again. No more recoveries were attempted and an identical spare machine was outfitted with the disks from the defective server. Disks had to be synchronized before making the machine available again.

Servers without electric power

Thu, 07 Jun 2012 13:45:00 +0200

A power failure in a rack in a server room brought some C&CZ servers down. After less than two hours all problems were dealt with. Affected systems ware mainly: postvak (Science mail server), bundle (user homedisk), heap/stack (network discs), resser/kookpunt/brievenbus/rustug (mail transport smtp servers)

Planned Service: website-databases and maybe Linux clients

Fri, 20 Apr 2012 17:00:00 +0200

A defective hard disc has been replaced in a server, but the server needs to be rebooted to ensure that this is reboot proof. The MySQL database of roughly 70 websites will therefore be down for a short time. Since this server also provides the Kerberos authentication for Linux clients, Linux clients might encounter service interruptions during a short period.

Eduroam incoming doesn't work for iPhone/iPad/iPod

Tue, 20 Mar 2012 00:00:00 +0100

The UCI network management reports that at this moment the incoming version of Eduroam doesn’t work for iPhone/iPad/iPod. A solution is being worked upon. Eduroam incoming means that one uses the wireless network of a remote institute, with authentication (login/password) being checked by RU or Science.

Windows server "plenty" with xpsoftware unavailable

Thu, 07 Jul 2011 13:00:00 +0200

Thursday July 7, around 13.00 hours the server “plenty” could not be reached. Because this server serves the “xpsoftware” share for the Managed Windows PC’s, all these PC’s had a problem. After the server was restarted and the disks had been checked, it was available again at 14:26.

Downtime Science servers: Sunday July 3, 09:00 - 12:00 hours

Sun, 03 Jul 2011 09:00:00 +0200

In order to improve the cooling of a server room, we plan to move three racks of Science servers a few meters on Sunday morning, July 3. We will have to switch off a lot of servers temporarily. Therefore several services will be unavailable some time starting July 3, 09:00 hours. We expect the downtime will last until 10:00 hours for servers with a lot of different users. The cn compute cluster will probably be fully operational again at 12:00 hours.

Network outage

Wed, 22 Jun 2011 10:55:00 +0200

This morning, in the network hub for Huygens South a UPS (battery power supply) went down, which made a set of network switches loose power. Because of this, users in Huygens wing 1 and spin-off companies lost their connection to the network. After bypassing the UPS, everything was up and running again at 11:30. We are still searching for the exact origin of this outage.

Fixed phone problem

Mon, 07 Mar 2011 00:00:00 +0100

You can’t reach certain fixed phones at the university right now, mobile phones and Skype do work ok though.

Planned Service: Limited computer services

Sat, 12 Feb 2011 07:00:00 +0100

A backup cooling system will be installed in our main computer room. Therefore the air conditioning system must be switched off, which means that most of the computer facilities in this room must be shut down. This includes the cluster nodes cn00 through cn53 and many of the web- and file- (network share) servers. It is advised to expect a very limited service level. We will try to keep all home directories and the mail system available. For detailed information about the impact please contact C&CZ.

Printer lp5

Mon, 24 Jan 2011 00:00:00 +0100

Printer lp5 has been moved to HG00.089. You can’t use this printer at the moment, there’s a problem with the power supply unit.

Plugging in extra disks to wnhome

Mon, 01 Dec 1997 00:00:00 +0100

2 fast-wide 9GB scsi disks are attached to the wnhome wide-scsi controller plugged in:

1 for NetPCs (replacing /vol/netpc from wn0)
1 as staging disk for Amanda (new backup software)

Info