Due to required changes to our network infrastructure and some hardware maintenance, we’re scheduling a maintenance downtime for most D-PHYS servers starting on Thursday, 14th of April 2016, at 5pm. The downtime will last several hours and single services may be down for longer than others or will be down multiple times in a row.
We’ll update this posting as soon as things are back to normal.
Most D-PHYS services will be affected by that downtime, especially file servers and e-mail services, but also some virtual machines and most websites hosted by ISG D-PHYS are affected. (http://www.phys.ethz.ch/ and other AEM-hosted websites are not affected.)
E-mails coming in during the downtime will be held on the sender’s side and will arrive at D-PHYS with a delay. Sending e-mails won’t be possible during the downtime either.
After the migration we will benefit from a faster and more reliable network connection to our servers.
Update at 19:30: Most services are back to normal. Expect further downtimes for home directories and mail later this evening.
Update at 23:00: All services are available again.
Update Fri 09:00: After Thursday’s network migration a defective patch cable caused network problems on Friday morning.
This post is meant to give you a short overview of what has been accomplished in D-PHYS IT by ISG this year. We’ve been hard at work to further improve and extend our services for you, our customers. Some highlights of 2015:
- new team members: both Christian Ringger and Christian Schneider joined ISG this year and have already made significant contributions to our setup.
- new D-PHYS website: the department website moved from the self-hosted Zope system into the ETH-wide AEM/CQ5 content management system. While the hard work of migrating all the content was done by Andreas Trabesinger, we had to sort out a lot of technical details to ensure a smooth transition and to keep D-PHYS’s various web services operational.
- Printing: the majority of the department’s printers have been migrated to the new pia printing system. We are now waiting for Informatikdienste to support student printing in order to complete the project.
- Storage: in 2015 the disk space occupied by data and backup grew from 685 TiB to 929 TiB, further increasing the yearly growth rate. We are also preparing to keep an off-site disaster recovery copy of D-PHYS data on tape.
- Outages: apart from a hardware failure of our mail server on December 2nd and a short interruption on July 2nd our system have been very stable this year.
- System upgrades: 2015 brought OS upgrades for almost every system: Debian Jessie on many servers, OS X 10.11 for the Macs, Ubuntu 14.04 on the Linux workstations and the first pilot installations of our new Windows 10 setup.
- Core services: a lot of infrastructure work has happened in the background to ensure smooth operation and seamless growth of our services in the future. Examples are: more IPv6 work, 802.1x / NAC in our network, a new network zone in the server rooms, an upgrade of our iPXE boot screen and enhanced monitoring.
- IT security: we participate in and support the ETH-wide IT security initiative.
Happy Holidays and see you in 2016!
Thank you to all of you who participated in our IT services survey in late October. More than one third of you completed the questionnaire and the feedback has been extremely positive. The Department Head and ISG have evaluated the results and would like to say thanks to everybody involved.
We have received many helpful comments that deserve an answer. Since the survey was anonymous, we have no way of getting back to each responder individually, so we compiled the questions, answers and comments on a nifty website that we invite you to visit:
You can browse the results, read the comments and click on the little speech bubbles to read our answers. There are a couple of topics that are sufficiently common and generic that we would like to address them here:
- WiFi: yes we know it’s not great, but we cannot immediately fix it as it is a service of Informatikdienste. We are working together with our colleagues of ID to resolve the problems and improve WiFi service on the campus.
- many of the wishes and problems raised in your comments actually have already been solved by us. Also,
- there seems to be a positive correlation between how often people contact us and their satisfaction with our services.
Please talk to us! We can only help you if you tell us about your problem, and we hate unsolved problems. So we’re going to try hard to fix them.
Thanks for you attention and we’re looking forward to serving you in the future.
We had to shut down the D-PHYS mail server on short notice for replacing faulty hardware. Mail service should be back in the evening.
Mails sent to D-PHYS during the downtime will be on hold on the sending side.
Update, 19:30: The dust is settling. We’re soon back to normal.
After several years it was time to update the SSH host keys of our managed Linux machines. Therefore, if you reconnect with SSH, you might get a warning similar to this one:
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in ~/.ssh/known_hosts to get rid of this message.
Offending key in ~/.ssh/known_hosts:1
RSA host key for login.phys.ethz.ch has changed and you have requested strict checking.
Host key verification failed.
This is because your computer has memorized the previous host key and is bailing because the current one is different. This mechanism is designed to prevent users from man-in-the-middle attacks. In our case it can be treated as a mere notification that the SSH key has changed.
In order to get rid of this warning, you simply need to delete the old key from your ~/.ssh/known_hosts file. This can be done either by deleting the entry manually or with the following command
ssh-keygen -R login.phys.ethz.ch
for the machine you try to ssh into.
On the next SSH connection you will be prompted to accept the new key. Power users may also download the full list of SSH keys of all our managed Linux computers.
In order to perform system-level maintenance work, we schedule a maintenance downtime of the D-PHYS mail server today (Thursday, 2nd of July 2015) starting at 17:00. We expect the downtime to take about one hour.
During the downtime all mail services (sending mail, receiving mail, accessing mailboxes, webmail, etc.) will be unavailable. Mails sent to D-PHYS users during the downtime will be held back on the sending side and will be delivered after the downtime.
We will post an update as soon as mail services are back.
Update 19:30: Took a little bit longer than expected, but everything is back to normal now.
Microsoft will provide a final bunch of patches for Windows Server 2003 on July 14th. 2015. After then, no more security and stability fixes are going to be released. This means that still running Windows Server 2003 machines conflict with the ETH Bot (Acceptable Use Policy for Telematics) which requires that every computer connected to the ETH network must be fully updated and secured.
The central IT security group of ETHZ continuously inspects the network streams for signatures of XP and Windows Server 2003 computers. If you have a running Windows Server 2003 machine connected to the public network, please migrate the operating system to a newer version i.e Windows Server 2012.
If you have any questions or need help please do not hesitate to contact the ISG D-PHYS Helpdesk
ISG sits on a pile of older hardware that for various reasons cannot be used in our setup any more but that various people have expressed interest in and that still might be useful for certain scenarios (e.g. lab use or tinkering at home). We will therefore host two grab-your-used-piece-of-hardware sessions:
- Window 1: hardware outside of the ETH live cycle, mainly old computers (PowerPC-Macs and PCs) and TFT monitors, free of charge for both ETH-internal and private use: Wed Apr 22 – Fri Apr 24 in HPT H floor
- Window 2: not-quite-as-old hardware, mostly TFT monitors and printers, free of charge for ETH-internal use, prices for private use according to the rules: Wed Apr 29 and Thu Fri 30 in HPT H floor
As usual, some rules apply:
- this goes to all D-PHYS members
- no registration necessary. Just come by and take whatever is left.
- all items come as they are. We do not have any details or specifications
- there’s no warranty or service whatsoever. All devices have successfully been turned on, but that’s it
- if your item doesn’t turn on, you can bring it back within 5 days and get a full refund (if it wasn’t free in the first place)
- no OS, no software, no manual, no keyboard, often no cables. You get one piece of hardware. All HDs are blank
- all proceeds go to the D-PHYS funds, not ISG
- if you have no use for a computer without OS or software, don’t come shopping
- bring cash
- note that the printers are not meant to undermine the migration to the new printing system! We will not connect those printers to our old print server
It is my pleasure to welcome Christian Schneider into our group. He joins us to replace Elmar Heeb in the Linux team.
That’s just enough Christians for now.
UPDATE 23:00 – maintenance finished, queued mails have been delivered.
As a probable aftermath of last week’s power outage we are experiencing some issues with the file system on our home directory server which can only be repaired offline. We therefore schedule a maintenance window
Today, Monday Feb 2, 2015, starting at 22:00
The duration of the downtime cannot be estimated but should not exceed two hours. During this time you will not be able to access your home folder or receive new D-PHYS email. All incoming mail will be queued for later processing.
Thank you for your understanding.