City takes steps to prevent future website outages

The City of Champaign realizes the importance to its citizens of our website, and the need to keep it available 24x7x365. Unfortunately the City’s website was down recently for a period of roughly 12 hours. The IT Department had previously identified the potential for problems and had initiated steps to prevent an outage, unfortunately these steps were not fully implemented in time to prevent the site going down.

Traffic to the City’s website has significantly increased since implementing our new design. This increased traffic was in fact the cause of the site failure this Wednesday night, (1/26). Essentially the traffic to our website increased to the point that it exceeded processor usage as defined in the contracted terms of service in our web hosting agreement. When the processor usage was exceeded our website was shut down. The shut down was not due to any equipment failure but entirely to increased traffic. Steps were already underway to prevent an incident of this kind and more are being taken. These steps include the following:

  • On January 18th the City contracted with our web host to move our site to a Virtual Private Server (VPS) which will be dedicated entirely to the City of Champaign. This will eliminate the possibility of our site shutting down due to increased traffic.
  • Migration of the City’s site from its current host to the VPS is underway and is expected to be completed during first week of February.
  • Adjustments have been made to more efficiently handle 404 errors. This will efficiently manage hits from non-cooperative search engines that continue to hit our site in search of objects and images that are no longer part of our site. A significant amount of traffic has been hitting our site from these search engines.
  • Our new VPS host will allow fulltime traffic monitoring which will allow the City to proactively move to increase server capacity should traffic level trends show an increase.
  • Our new VPS host will automatically notify IT staff via email and text in the event of a server failure.
  • The city is now working with an external site monitoring service. This will instantly report to IT staff even if the entire host service should be down. This site will also monitor our site’s response time providing regular performance reports including comparative performance reports.

While we cannot guarantee our site will never go down, we feel strongly that these measures will drastically decrease the odds of this occurring. We apologize for any inconvenience the recent outage may have caused and we will continue to take steps to improve our online offerings.