The Government Digital Service (GDS) recently hosted a remote GraphQL workshop for around 70 people from across government.
Teams from HM Passport Office (HMPO), National Archives and the Royal Navy walked us through their experiences using this relatively new API query language.
We looked at use cases and showcased some brilliant projects to explore more about when we should use GraphQL over REST API.
What is GraphQL
GraphQL is an open-source API query language and runtime for fulfilling queries with existing data.
Unlike REST APIs, where the server determines the data returned in each endpoint (which can result in clients over or under-fetching data), GraphQL lets API clients request and receive the exact data they ask for. API developers define the data using the GraphQL strongly typed schema and expose it through a single server endpoint called ‘/graphql’. Clients can then request data on a granular level with queries and modify data with mutations.
From this, GraphQL can reduce the number of network calls and bandwidth demands on the server.
What we learned during the day
HMPO: taking a user-centered approach to building 2 APIs using GraphQL
Catherine Walden, senior agile and product manager, and Neeraj Dad, consultant lead architect, discussed technical considerations and user needs that led to HMPO using GraphQL.
Since GraphQL became open source in 2018, HMPO and Home Office have endorsed its use for 2 API products to replace legacy APIs:
- HMPO System of Record Data platform - an internal HMPO system of recording data (passports, application decisions, watchlists, and lost, stolen and recovered passports)
- Data Validation Applications (DVA) Digital API - a web application and API that supports over 80 government departments in running identity checks for services and business processes
Catherine discussed how both products had increasing demands and evolving customer needs. There were some technical considerations to make sure:
- sensitive data was protected
- rigorous security processes were in place
- user needs were met
Neeraj talked about important lessons they learned from their legacy APIs and common API challenges, including:
- understanding that APIs should evolve as user needs change
- designing APIs upfront and remaining flexible to change depending on how users request data
- dealing with different user needs after versioning
- reducing over-fetching and under-fetching data as it can have cost, security and performance implications
These lessons influenced HMPO to choose GraphQL as they identified that it could help overcome the challenges of past APIs, meet user needs and improve data sharing. HMPO used Apollo as its GraphQL implementation tool.
The DVA Digital API team used GraphQL to build an API that was:
- hierarchical - customers can view the hierarchies of data structures and can request only the data they need
- strong typing - DVA Digital can define their type system so customers know what data is available to them and can only query and receive data that is valid in the system
- product-centric - DVA Digital can easily react to customer requirements by giving them control of their requirements and learn from data usage
The DVA Digital team concluded that GraphQL can be a steep learning curve but it is important that developers consider GraphQL as a new way of building APIs and always focus on user needs when choosing how to build their APIs.
National Archives: choosing GraphQL for Transfer Digital Records API
Suzanne Hamilton, software engineer, spoke about the National Archives Transfer Digital Records (TDR) project.
TDR stores information about consignments and files in a relational database. The team wanted to build an intermediate component between the frontend and backend processing and the database. They explored different GraphQL approaches and considered REST before deciding to use Sangria, a GraphQL server. They chose GraphQL because they:
- have several types of clients with different data requirements
- anticipate multiple data sources and want to connect relational data about files, transfers and users
- want to reduce over-fetching data from the database
- want to reduce the number of network calls to data
- want to benefit from GraphQL’s strongly typed schema
The team wanted to try something new and a small internal API was a safe place to experiment.
Some lessons they learned along the way were:
- there’s a lack of mature client libraries in some languages
- sometimes having a single URL, such as ‘https://example.com/graphql’, makes it hard to add security measures like IP restrictions for admin queries
- having to handle both HTTP error status codes and GraphQL errors makes error-handling in the API clients more complex
Navy NELSON Programme: visualising data better through GraphQL
Chris Spence, data architect, discussed the Royal Navy’s NELSON programme and the NELSON Data Platform (NDP) mission to provide accessible and coherent access to different types of data.
Previously, data and systems on ships could get locked up in individual silos, making it difficult to access and share. The Royal Navy built the NDP as a single point of integration so that an application could access data numerous times from one place without having to fetch data from multiple backend systems.
When choosing how to build their APIs, the team evaluated both REST APIs and GraphQL. They decided to use GraphQL because:
- GraphQL lets the team generate and share automated documentation of code with their users
- the Navy deals with sensitive data, and the GraphQL strongly typed schema gives them stringent controls over the shape and nature of responses to queries
- there are limited communication links to ships at sea, and GraphQL is more bandwidth-efficient and less chatty than REST APIs
- GraphQL can neatly connect different data sources and stitch schemas together
Chris discussed how GraphQL has helped build APIs that can cater for their wide user base who have varying data needs. For example, one user might need data on engine fuel rates whereas another user may only need data on engine tanks. With GraphQL, each user can query what they are interested in and receive only that.
The Royal Navy also used GraphQL Voyager, a tool that helps visualise your schema. For example, the Royal Navy used this to maintain a live radar feed so users could query data like ship location.This made it easier to share and discuss data with developers and senior stakeholders in a more interactive way.
The Royal Navy took a user-centered approach to developing a GraphQL API to better understand the use of fuel. They developed some user experience standards and collaborated with data teams to build the application.
Some important lessons the team learned along the way were:
- choosing the right application framework for the requirements is important
- as the schema gets more complicated, you end up with more nested code and developers might struggle to manage this
- breaking up the schema if it gets too complex, or using API federation instead
What's on the horizon?
GraphQL can be a useful way to build APIs and use cases in the public and private sector are developing fast.
We are adding more guidance on GraphQL in our next iteration of the API Standards, focusing on the situations where GraphQL may be a suitable alternative to REST APIs.
Where possible, we recommend building RESTful APIs in our API standards, unless there is a clear user need to do otherwise. Developers should have a good understanding of how user needs interact with their APIs and make design decisions that support them in meeting their goals.
We’re hoping to see more teams experiment in the open like The National Archives.
To learn more and to share your work with us, you can get in touch with us on the API Standards group or leave a comment below. If you’re in the public sector, you can also join the #graphql and #apis cross-government Slack channels.
1 comment
Comment by Uri posted on
Thank you so much for sharing your experience!
I'm writing as a member of The Guild (http://the-guild.dev) which maintains and creates a lot of the tools you used and mentioned.
We would love to help with anything you want, for free of course.
We love supporting initiatives like that for governments and open data.
We can help with thinking about the right ways to grow with GraphQL, as there are many different approaches and it can get confusing sometimes to understand the vast amount of tools in the GraphQL ecosystem.