Skip to main content

https://technology.blog.gov.uk/2020/05/21/managing-gdss-internal-infrastructure-during-lockdown/

Managing GDS’s internal infrastructure during lockdown

Empty GDS office

Since its early days, Government Digital Service (GDS) supported flexible working by being one of the first departments to set up a wireless-only network and support working remotely. This is why GDS built and designed its internal IT infrastructure to be highly resilient, enabling fast and secure access to the internet wherever its employees were, and without using an Always-On Virtual Private Network (VPN). 

These principles were set up back in 2011, when GDS was in Hercules House and a much smaller organisation of less than 50 people. Now, it's GDS policy to encourage all staff to work remotely one day a week, where business needs allow. 

When coronavirus (COVID-19) meant GDS moved to a complete remote-working pattern, we were able to build on these principles and adapt to the lockdown situation.

Services in the cloud

GDS has always designed and built its systems on the internet. 

Our email, calendar and document-sharing is done via Google G-Suite; instant messaging via Slack; and the GOV.UK, GOV.UK Verify, GOV.UK Notify, GOV.UK Pay, GOV.UK PaaS, and GovWifi teams host their infrastructure in public cloud environments. 

In addition, management of GDS MacBooks (Macs) is done via JAMF mobile device management (MDM) hosted in the cloud. Logging data for our internal infrastructure is sent to the cloud using Splunk, which allows a single-pane view of our logs and alerts.

VPN capacity concerns

However, some services such as our HR system and Cabinet Office Intranet require our users to use a VPN when away from the office. Our developers also use the VPN to access their internet-hosted infrastructure, which in addition to other measures is protected by access control lists (ACLs), permitting only authenticated users from the GDS VPN. This is a security control as GDS runs critical digital services for teams across government.

Normally these services are accessible via our office wifi networks. As GDS moved to full remote working, there was a drastic fall in the number of devices connected to the GDS wifi. On 16 March, the last day of significant wifi activity, there were 665 devices connected. This fell to 81 devices by 19 March. 

With all users working remotely we had capacity concerns for the GDS VPN, which is hosted with a public cloud vendor and supports up to 500 users. 

We used our monitoring system to raise alerts when the VPN load was high and began work on setting up an additional VPN endpoint whilst keeping an eye on capacity loads. 

When we moved to remote working on 16 March there was an immediate increase in the number of users on GDS VPN. Since the official lockdown period began on 23 March, this has remained at a consistent high level. However, it has stayed within our limits of 500 users, so the work on the additional VPN endpoint was carried out in parallel with other tasks. 

No Always-On VPN means no bottleneck

Video-conferencing for collaborative working is vital when working remotely. GDS uses Google Hangouts/Meet, which uses greater bandwidth due to video/voice data being transferred over the internet. 

We do not deploy an Always-On VPN, which means most user traffic does not traverse a common VPN endpoint. This would have the risk of becoming a bottleneck and potentially causing a worse experience for users. GDS did not face bandwidth issues because using the VPN is optional. This enabled teams to deliver at pace with various requirements coming in during lockdown. For example, GOV.UK Notify recently sent out its billionth notification

Software patching is still important

‘PatchBot’ is a tool we developed to ensure core applications and Apple software updates are pushed out to users in a timely manner. Users are reminded to run it each week and we regularly monitor patching figures. If users do not run PatchBot regularly they are denied VPN access.

During the switch to remote working, we noticed a decrease in the number of people regularly using PatchBot. We addressed this by sending our staff reminders, which effectively countered this trend.

Get involved

We’re keen to hear from other public sector organisations on how they have adapted to full remote working. Let us know about your workplace in the comments. 

Sharing and comments

Share this page

4 comments

  1. Comment by Richard Thomas posted on

    Well done Mo. Glad to see things still going well ?

    • Replies to Richard Thomas>

      Comment by Mohamed Hamid posted on

      Great to hear from you, Richard 🙂

      Hope to see you soon.

  2. Comment by David Durant posted on

    Loads going on at HackIT (Hackney Council). For example - pulling many services out of VDI (virtual windows desktop) - such as : https://blogs.hackney.gov.uk/hackit/category/hackit.

    Our security model is putting everything on the internet and securing each thing rather than locking things behind a firewall.

    I'm sure there'd be folks here who'd be happy to chat about it all.

    • Replies to David Durant>

      Comment by Jon Willis posted on

      Oh that old document system and firewall sounds horrendous! Nice hack exposing documents. It's clear that GDS's approach has been a breath of fresh air in comparison to old gov/corporate IT, and it's great to hear councils are getting a taste for it too.