Title: SDSC/UCSD Border Router Maintenance – 15:00, 14 Aug 2014 Update 17:30, 19 Aug 2014: The primary router is operational after removing some IPv6 routes. SDSC is working with the hardware vendor to upgrade and update the system to prevent similar outages in the future. Update 07:11, 15 Aug 2014: Routes have appeared stable for …
Category Archive: Emergency Maintenance
Emergency Maintenance announcements
SDSC Emergency Maintenance – 19 June 2014, 7-11pm – Linux Patching/Reboot
Title: SDSC Emergency Maintenance – 19 June 2014, 7-11pm – Linux Patching/RebootDescription: SDSC will be applying a critical security update and performing a reboot of the following systems between 7pm and 11pm. System list: ale bastiat beeker brain-mysql brain-web1 bss-prod bss-staging bss-test bss-web bumper buster chron-test cortana cvma3 dough drosophila endor fork ham hildr jcsg-s4 …
SDSC EM – Project and Cloud Storage outage
Update 5 June 2014 This outage was due to one of the redundant switches hanging, causing approximately 15 minutes of outage. This outage was unplanned and unexpected as work has been done to provide redundant, reliable failover of the equipment. SDSC is working with the hardware vendor to determine the underlying cause and will make …
SDSC Emergency Maintenance – 8 April 2014 – OpenSSL patching/rekeying
Title: SDSC Emergency Maintenance – 8 April 2014 – OpenSSL patching/rekeyingDescription: To address the most recent OpenSSL vulnerability, SDSC will be patching and restarting vulnerable websites and services. Some sites may be rekeyed in the process. Please contact your regular support staff if you have any concerns or questions about this maintenance.Date: 2014-04-08
:SDSC Outage – 14 Mar 2014, 11:50 – Blue UPS
Title: SDSC Outage – 14 Mar 2014, 11:50 – Blue UPS Update – 14:36, 14 Mar 2014:Cloud storage service has been restored. There is one node still being worked on, so performance may be slow until that node is brought back online. Update – 13:10, 14 Mar 2014:Project storage nodes have been brought back online. …
Firewall / High Security Services Outage – 28 Jan 2014
Update 15:00, 28 Jan 2013 Some systems and services have been returned to service. Steps are still being taken to restore all remaining offline services Title: Firewall / High Security Services Outage – 28 Jan 2014Description: The primary high security firewall is experiencing an outage for specific services/hosts. SDSC engineers are investigatingStart Time: 13:35Date: 2014-01-28
Outage : merrimac.sdsc.edu (login-linux.sdsc.edu, ll.sdsc.edu) 9 Jan 2014, 12:35
Title: Outage : merrimac.sdsc.edu (login-linux.sdsc.edu, ll.sdsc.edu) 9 Jan 2014, 12:35Description: The system froze and required a hard restart. All sessions and running jobs have been terminated.Start Time: 12:30Date: 2014-01-09End Time: 12:50
Emergency Maintenance – Linux updates – 16 Dec 2013, 2pm
Title: Emergency Maintenance – Linux updates – 16 Dec 2013, 2pmDescription: At 2pm SDSC began installing critical/important updates on php, firefox, nss/nspr and flash-plugin. Please contact regular support channels if any applications or services appears to be adversely impacted.Start Time: 14:00Date: 2013-12-16End Time: 14:30
SDSC Network Interruptions – 5 Nov 2013, 5:30am
Update: 10:00am UCSD ACT and SDSC ENS have resolved the routing issue. Please contact your support channel if you continue to see any interruption Title: SDSC Network Interruptions – 5 Nov 2013, 5:30amDescription: At around 5:30am on 5 Nov 2013, SDSC’s routing fabric had hardware errors. Connections from the commodity internet to SDSC resources may …
SDSC Service Outage – Project Storage – 19 Sept 2013
Title: SDSC Service Outage – Project Storage – 19 Sept 2013Description: At 07:40 on 19 Sept 2013, one project storage node, ps-c02-1, hung causing service interruptions for the below filesystems. SDSC Storage is working to resolve the issue. Updates will be posted. ps-006:/ps-data/cmmap ps-021:/ps-data/dsc2 ps-019:/ps-data/ens ps-009:/ps-data/ngbt ps-038:/ps-data/nsg ps-020:/ps-data/oracle ps-005:/ps-data/rcicdm ps-c02-1:/ps-data/vmware Date: 2013-09-19