Discovery report: Trainee Data Management System

National School of Healthcare Science - Health Education England

Version: 1.0

Recipients: Stuart Sutherland, Chanelle Peters, Alex Levine, NSHCS

Authors: Dr John Gribbin, Stephen Hale, Adam Spencer and Dr Helen Taylor- Lagom Strategy. Dr Neal Todd and Paul Vetch - Torchbox

Issued: 11 February 2022

Contents

1 Discovery overview

1.1 Executive summary

1.2 Discovery phase goals

1.3 The scope of the Discovery

1.4 Project team

1.5 Discovery phase activities

1.6 Key associated documents

1.7 Background to this Discovery

1.8 Overview / method

2 Findings

2.1 Purpose and context

2.2 Data and technology

2.3 Service management

2.4 The current user experience

2.5 Users’ suggested improvements

3 Validating user needs

3.1 Analysis method and outputs

4 Concept development

4.1 Overview

4.2 Assigning responsibilities for responding to self-service requests

4.3 Accessing and updating trainee records

4.4 Searching for trainee data

4.5 Consuming trainee data

4.6 Permissions to view trainee data

5 Landscape analysis

5.1 Manchester Academy for Healthcare Science Education

5.2 NHS Education for Scotland

6 Inclusivity and accessibility assessment

6.1 Digital skills

6.2 Connectivity

6.3 Accessibility

7 Recommendations

7.1 Context for these recommendations

7.2 Overarching recommendations

7.3 Data and technology

7.4 Service management and governance recommendations

7.5 Suggested roadmap

Annex 1: Discovery research

Annex 2: Discovery outputs

Annex 3: Surveys

1 Discovery overview

1.1 Executive summary

The Discovery was conducted by digital agencies Lagom Strategy and Torchbox between November 2021 and February 2022.

The Discovery focussed on informing the procurement of a new trainee information system for the National School of Healthcare Science (NSHCS).

The research activities were conducted in line with the definition and guidance of a digital service Discovery phase mandated by the Government Digital Service (GDS) Service Manual.

All publicly funded digital services are subject to such a Discovery phase before proceeding to the next phase of technical development and design, and passing subsequent service assessments.

The Discovery has concluded with a prioritised user needs backlog and a set of recommendations for NSHCS to consider.

The context for the recommendations

The current system is not fit for purpose

There is an increased risk of doing nothing

There is an opportunity to improve the quality of the School’s data

It would be difficult to integrate Healthcare Science into HEE’s Trainee Information System

A new system has to fit in with external and internal governance standards

Overarching recommendations

Proceed with the procurement of a new trainee management system

Agree upon the boundaries of the new service

Agree upon the scope of a minimum viable product (MVP)

Do some scoping work to understand what the boundaries of this work are/could be across the School

Offer this work as a platform for others in HEE to build on

Embrace the procurement of a new system as an opportunity to be considerate and ambitious about what data the School collects

Explore new ways to engage trainees in the collection of their data

Data and technology recommendations

Prototype and evaluate the viability of a new service based on a Low-code or CRM platform

Prepare yourselves for the data migration challenge

Relate the service design to HEE's technical standards and principles

Determine the value of sending data to National Data Service

Carry out a GDPR compliance review

Find an implementation partner

Service management and governance recommendations

Redefine and then formalise the management of the service

Establish a hierarchy for data across the School

Embed access permissions within the new system

Roll out training to support users with the new system

Create a roadmap to maintain momentum following this Discovery

1.2 Discovery phase goals

At the start of the Discovery we used the kick off workshop to prioritise a set of questions to explore in the Discovery:

Capture and prioritise the needs of the identified user roles

Understand the technical and data landscape in the NSHCS

Understand wider HEE technical landscape. Identify how work aligns with standards and where it differs

Investigate and propose options to replace the existing system

Develop an options appraisal and set of recommendations about how to best meet identified user needs – considering costs, technology, interoperability and time

1.3 The scope of the Discovery

The scope of the Discovery was limited to the current NSHCS trainee management system, and our findings and recommendations address this system directly.

Our research did inevitably reveal insights about wider processes in the School and HEE more broadly. Some of our findings and recommendations address these wider insights.

Some of these findings also lead to further work that should be considered by the School.

NB: Throughout this report we refer to a system and service. The term system refers to a database users will interact with to process and seek data. The term service refers to a wider trainee management process that the system sits in the middle of.

1.4 Project team

Discovery team

Dr John Gribbin - Service designer and support user researcher

Stephen Hale - Support service designer

Adam Spencer - Senior user researcher

Dr Neal Todd - Senior data and technical architect

Dr Helen Taylor - Delivery manager

Paul Vetch - Subject expert

National School of Healthcare Science

Stuart Sutherland - Head of Digital and Information Systems

Alex Levine - Education & Digital Systems Manager

Chanelle Peters - Educational Systems Support

1.5 Discovery phase activities

The Discovery was conducted between November 2021 and February 2022.

These activities allowed us to generate the insights and recommendations in this report and the other Discovery outputs:

Kick off workshop with key project stakeholders

Reviewed 9 online and documentary sources

Conducted 14 stakeholder interviews

Conducted 16 one-to-one user interviews

Analysis of a user experience survey with 10 responses

Analysis of a user needs survey with 16 responses

Ran a user needs workshop with 6 user representatives

Developed 5 user proto-personas

Created and prioritised 52 user needs with the NSHCS team

Presented a project show-and-tell

Performed a landscape analysis including desk research and 2 interviews

Ran a co-design workshop with 4 user representatives

Developed a set of concept prototypes and tested these with 5 user representatives

1.5.1 User research participation levels

Participation from users across all research activities was to a satisfactory level required for a robust Discovery.

It is important to consider the Discovery as the start and not the end of ongoing user research.

1.6 Key associated documents

Proto-persona profiles

User needs backlog

Prototypes

Note: all project documentation has been gathered and shared. A full list of outputs is available in Annex 2.

1.7 Background to this Discovery

The NSHCS currently uses an Access database to maintain data on Healthcare Science (HCS) trainees, so that the School can appropriately manage and report on the trainees that they are responsible for.

Trainee data from the system is critical to decision making across the School, as well as the wider HCS profession.

The School commissioned this research to inform the procurement of a new trainee management system that better meets the needs of its users - both processors and seekers of trainee data.

1.8 Overview / method

1.8.1 Stakeholder overview

We conducted semi-structured interviews with stakeholders to ensure that their needs and views were understood and accounted for in the research process.

We completed 14 interviews with stakeholders, including those responsible for decision making across the School and the wider organisation of HEE.

Stakeholders articulated the importance of a new system to the School, as well as the complex environment that a new system would have to operate within.

Our analysis of stakeholder perspectives is presented alongside our analysis from the user research in the findings section of this report (Section 2).

The stakeholder interviews took place in December 2021 and January 2022.

See Annex 1 for a full list of participants.

1.8.2 User research

We consulted staff across each of the School’s departments to develop a thorough understanding of their contexts, behaviours and needs with trainee data and the incumbent data management system.

The current School departments:

Accreditation and Admissions

Assessment and Research Evaluation

Digital and Information Systems

Operational Management

Programme Support

Qualitative research

16 one-to-one interviews

User workshop with 6 user participants

Co-design workshop with 4 user participants

5 concept feedback sessions

Quantitative research

16 respondents to the online user needs survey

10 respondents to an online experience survey

1.8.3 User roles

Our research addressed the following user roles, as defined in a user research planning call at the inception of the Discovery:

Data processors

Those who input, update and manage all trainee data collected by the School

Data processors work across all of the School’s departments and can access all available data, but some may have specific responsibilities or interests, e.g. Assessment and Research Evaluation colleagues tasked with updating trainee progression and outcomes data

Data seekers

Those who access (or request others to access) trainee data for specific purposes, e.g. to find the contact details for an individual trainee, to know the training officer of an individual trainee, or to produce bespoke reports on trainee cohorts

Data seekers work across all of the School departments, but some of the typical roles that demonstrate this ‘data seeking’ behaviour are Training Programme Directors and Stakeholder Engagement Managers

School trainees

Those who are currently studying on one of the School’s training programmes, e.g. the Scientist Training Programme, and the Higher Specialist Scientist Training Programme

Data processors manage information about these individuals, but trainees do not have access to this data themselves - user research was conducted with this specific role to further explore the prospects of a self-service function for trainees to update their own information where applicable, e.g. a change of contact details

1.8.4 Prioritising the user needs backlog

A user need is something that a person has to get done.

This need is present regardless of any existing products or services in place that they might have to interact with.

User needs are written using the syntax below:

As a... [the type of person/role that the user is]

I need to… [the task that the user wants to do]

So that... [the end goal that the user wants to reach]

The NSHCS team were invited to a session to prioritise the user needs on 5 January 2022.

The team used research evidence to inform their initial round of prioritisation of user needs using the MoSCoW (Must, Should, Could, or Won’t) method.

Image: Screenshot from the backlog of prioritised user needs. The backlog is available as an output.

1.8.5 Concept development

See Section 4.

1.8.6 Service journey mapping

A service journey map documents the touchpoints involved in a current service.

The service journey map for this project was informed by a service walkthrough with the Digital and Information Systems team on 25 November 2021.

It was then reviewed in a second walkthrough with Alex Levine on 15 December 2021.

The service map documents four key administrative journeys involving trainee data:

Adding a trainee to the system

Updating trainee information

Generating reports

Viewing trainee data in OneFile

It aims to visualise who is involved at each of these stages, what technology is being utilised and what potential pain points are.

Image: A screenshot of the completed service journey map

1.8.7 Service blueprint

A service blueprint describes a hypothetical, optimal journey throughout a service.

It is not restricted by the boundaries of the current service and represents an ideal that a new service can aim towards.

For this Discovery, we created a blueprint based on the four key areas outlined in the service journey mapping process:

Creating a new cohort of trainees in the system

Updating trainee information in the system

Managing trainees (broadly and those in difficulty)

Enhanced reporting of trainee data

Within these four areas, the blueprint highlighted features that are ideal and in scope for a trainee management system.

The blueprint also highlights some features that may fall outside the scope of a trainee management system, however would be valuable to meet the needs articulated through this Discovery.

Most of these features fall into the category of case management solutions, rather than trainee management solutions.

Image: A partial screenshot of the completed service blueprint

The service blueprint was then used alongside the user needs backlog to influence the technology options review (Section 7.3).

2 Findings

2.1 Purpose and context

2.1.1 Importance of data collection

It is clear and obvious why the School needs to collect trainee data. In part, it is enforced in order to comply with regulatory requirements.

“It is our statutory duty in service of public policy … We need to collect data to support trainees.” - Stakeholder

Good data collection also enables the School to communicate with and report on trainees.

“It’s critical for us to have up to date data on trainees because it’s how we contact them and how we report on them.” - Stakeholder

This is fundamental in enabling people across the School to carry out their day-to-day duties.

“It’s absolutely business critical. It’s the bread and butter of what we do and we wouldn’t function without it.” - Data processor

Whilst collection of trainee data is of critical importance at the moment, it is also clear that this importance will grow as the number of healthcare science trainees grows.

“The number of trainees is increasing and the proportion of trainees in difficulty has also increased.” - Stakeholder

2.1.2 Quality of the data

The importance of trainee data is clear, however there are questions about the quality of existing data. For some within the School, there are concerns about oversight of the entire trainee population.

“We don’t have an overall view of the training population.” - Stakeholder

This was emphasised through the document review, which took into account the findings of the School’s Strategic Plan (Annex 1). The plan itself highlights several problems in the area of workforce planning and analysis. Most notably, it highlights that current data about healthcare science is unreliable and not easily accessible.

Anecdotally, we have also heard statements about the quality of the School’s data from external sources.

“When we get trainee details from the School, the number of errors is huge. Out of 300 trainee names, approximately 50% of them had an error in either their first name, last name or email address.”
- Landscape interview

Whilst this is not based on a robust audit of the School’s data, it does present some evidence that the quality of trainee data is a wider issue.

Questions about the quality of current trainee data suggest that there are issues within the current data management process, which will likely increase as current processes face additional strain from the increasing trainee population.

The importance of the data and this increasing strain on current processes has led to the point where the School needs to look at a new trainee management system to support their work.

“The weight of the work needs a more efficient tool to support it.” - Stakeholder

2.1.3 Context of the current system

We have heard that the current system benefits some people across the School more than others. For those wanting basic information about trainees, the system does a reasonable job of providing this information.

“If you just need to know who a trainee is, where they are based and what year they are in to make a decision, you might think it’s brilliant.” - Data processor

For others, this level of information is not sufficient to make adequate decisions; primarily because of the lack of qualitative information about trainees.

“Although we have a basic level version of truth within this system, we haven’t got the qualitative stuff on what we’re doing with them.” - Data processor

We have also heard that the frequency of which people use the system varies depending on role. For some, this is every day, whilst for others it is much less frequent.

“I use the system absolutely every day. I’m not sure I’d say every hour but some days I definitely do.”
- Data processor

“I could probably go two weeks without going into [the system], it’s based around the queries I get. Half of the time, the information I need is outside of the system anyways.” - Data processor

Use of the system is also cyclical for some people, with usage increasing in line with the assessment cycle.

“Access varies depending on the stage of the assessment cycle, sometimes multiple times a day at other times less often.” - Survey respondent

2.1.4 Duplicate data collection across the school

Whilst our research has focused on the current trainee management system, we have uncovered some evidence of overlap with the collection of trainee data across the School.

Data collected by other teams was outside of the scope of our research, therefore we are unsure about the exact scale of this issue. However, there is some evidence that this might be a wider issue.

In particular, people within the Commissioning Team have talked about the collection process of trainee contact details, names and entry details directly from HEIs. This process is different to that carried out by the Digital Team and suggests duplication.

“The portal has around 31 questions that a HEI has to fill in. It includes the contact details of trainees, if it’s someone who has been nominated to a programme, if it’s a direct entry through recruitment.” - Stakeholder

The commissioning data is then stored outside of the current trainee management system and used as part of the process to ensure that institutions receive correct payments.

“Trusts input trainee email addresses as the way in which they contact the trainees. If the institutions don’t fill this in properly then it delays their payments.” - Stakeholder

Stakeholders have suggested that an ambitious piece of work should look to combine all of this information within a single system.

“Each month there is a schedule where the Commissioning Team sends money to each trust. Against that we have the names of trainees, and how much they need. If we are really going for it, there’s no reason at all why that information couldn’t be exported straight to the Team. That’s set up on a completely separate system at the moment.” - Stakeholder

2.1.5 Data collection opportunities

Stakeholders have expressed potential missed opportunities with the current data that is collected as part of the trainee management process.

Equality, Diversity and Inclusion (ED&I) data in particular, represents an opportunity to collect more data and use this information to inform policy.

“Differential attainment is a big issue. To understand why BAME staff score lower or fail higher, don’t get through the system as well as white counterparts. That is an issue and in order to understand that we need to collect their data.” - Stakeholder

To do this well, however, the School needs strong ideas about what data they are collecting and why they are collecting it.

“There is a question of what problems the team is trying to solve by collecting this data. Are they collecting it for the sake of collecting data or are they analysing specific outcomes?” - Stakeholder

This may mean going further than focusing on protected characteristics of the equality act and identifying wider factors that may influence trainee progression.

Examples include caring responsibilities or travel to an institution, which may impact on a trainees ability to successfully carry out their training.

“We go further to include caring responsibilities. That might be important if students have kids to drop off at particular times for example so trainees can’t get to particular places or sessions.” -	Stakeholder

“Things like travel to an institution can be difficult sometimes. They might not be able to afford to travel because they get paid later than they start a programme. My understanding was that some people were dropping out as a result of that.” - Stakeholder

Thinking deeply about data that is collected can then inform interventions by the School to improve trainee outcomes. It also allows the measurement of these interventions to determine successes and failures.

“If you see a high number of people who potentially fall into these problems, you can identify support mechanisms that could be put into place prior to people arriving. This could help the school to best target their limited resources. They might have schemes that kick in when there are a number of students meeting particular criteria.” - Stakeholder

2.1.6 The School and HEE’s Trainee Information System (TIS)

As an organisation, HEE are not clear about the ambitions for TIS.

“It’s unclear what the prime ambition of TIS is. Is it the trainee system for everyone or just medical and dental?”
- Stakeholder

We have heard this confusion reflected from stakeholders who feel like they need clarity in order to make informed decisions about TIS and other trainee management systems across HEE as a whole.

“If TIS is just going to be for medical and dental then other teams can go and do what they need to do elsewhere to fulfil that need.” - Stakeholder

The TIS team have expressed a desire to incorporate HCS trainees, as well as trainees from other professions across HEE.

“I think HCS [healthcare science] is in scope for TIS. It would be nice for that to happen ultimately. It would be nice if we had a single system that was used for all trainees.” - Stakeholder

Despite this desire to incorporate healthcare science trainees, it is clear that TIS don’t have the resources to incorporate the School in their current roadmap of work.

“We tried mapping HCS data to but it was too much, we didn't have the capacity, and still don't, to deal with it. I was hopeful we would be able to but there was a lot more in there and more controversial needs that I anticipated.” - Stakeholder

As a result this work was commissioned to look into alternative trainee management solutions that could be delivered at pace to deal with current and increasing pressures.

“Two years ago we started working with HCS [healthcare science] to retrofit the data model of TIS. The conclusion was that it was feasible and practical but we didn’t have the capacity to do the work.” - Stakeholder

“Last year we were trying to deal with remote working and an increased volume of trainees in difficulty. Those two things were rightly leading to pressure from HCS [healthcare science] colleagues but TIS still wasn't ready for us.” - Stakeholder

Although a trainee management system outside of TIS is being sought, healthcare science staff would be open to TIS as a solution for the School if it was appropriately resourced and developed for the needs of healthcare science.

“As long as it meets our needs I don’t mind what system we end up using … I don’t think anyone would mind if that was TIS as long as it was properly resourced and developed with HCS in mind.” - Stakeholder

“The decision making in the past was about how much of the current system could you live with? We’d need something that fits around and is informed by our prioritised user needs.” - Stakeholder

Despite an openness to TIS as a potential solution, there is little confidence that the TIS Team could be resourced to make this happen in an appropriate time frame.

“I would be concerned about the [TIS] Team delivering in any sort of timescale that is acceptable. I’m not sure they have a big enough development team.” - Stakeholder

“I don’t think that [the] TIS [Team] will be able to develop something useful in their current form … They won’t deliver in a timescale comparable to purchasing something off the marketplace.” - Stakeholder

2.1.7 Other professions within HEE

Whilst the School is facing issues with the way in which their trainees are managed, it has been suggested that they are not alone in tackling this problem across HEE. There are other professions who manage trainees in a similar way.

“I’m looking at Pharmacy because they manage a group of trainees. Dental manages their trainees in the same way, as do Public Health.” - Stakeholder

Stakeholders have therefore recognised the potential in the work being carried out by the School, even suggesting that the system utilised by healthcare science might become the standard for other professions across HEE.

“It would be nice if the new system was portable to other professions. I think there are other professions in HEE who have been looking at TIS but [they] might not be able to accommodate them.” - Stakeholder

“TIS at the moment doesn’t cover all of our trainees. I would be more comfortable if TIS was looking after junior doctors and we’ve got another system that’s looking after everybody else.” - Stakeholder

2.1.8 HEE’s internal landscape

In talking to Stakeholders, we have uncovered a range of criteria that any potential recommendations should take into consideration. Much of this criteria stems from the strategic direction of HEE.

“The criteria for making those decisions are value for money, minimising digital risk and fitting in with HEE’s strategic journey.” - Stakeholder

Currently HEE are striving to follow assurances from external bodies when making decisions around technology and digital ways of working.

“Part of the assurance is from the Cabinet Office, there’s something called a technical code of conduct. We have to adhere to that.” - Stakeholder

“The other assurance is about service delivery assessment, which is monitored by NHSX. Those standards are about delivering services to meet user needs.” - Stakeholder

“Digital risk is making sure that we don’t buy anything that doesn’t fulfil HEE’s legal obligations as a public sector organisation. Mainly that means complying with GDPR and accessibility standards.” - Stakeholder

Following the impending merger with NHS England, HEE may find themselves facing even further scrutiny to adhere to NHSX standards. Primarily because they are then joining the body responsible for upholding those standards across the industry.

In addition to these external standards, HEE also has a set of internal architectural principles that seek to underpin best practices across the organisation.

“It’s things like: a greener agenda, sustainability, open standards, web rather than client-server and application architecture.” - Stakeholder

“The background for our decisions is buy first, build second because building costs money. If our internal building of something isn’t meeting needs then our next step would be to buy something.” - Stakeholder

These internal principles reflect that it may be difficult to justify a business case for a bespoke solution as part of this work.

“If we are going down the bespoke development route, I’ll be honest, there has to be a very strong business case and show that there’s nothing in the market that could possibly meet that requirement.” - Stakeholder

2.2 Data and technology

2.2.1 Data model

In researching the data model of the existing system, we have seen that it is not complicated. The system has a relatively flat structure.

“The database is relatively uncomplicated in structure, centred around core tables representing trainees and their training data.” - Stakeholder

“SQL views are used to create composite representations of data from multiple tables. These views are queried by the Access frontend.” - Stakeholder

This opens up options for a transition to an alternative platform because there is less complexity and specificity to address immediately.

For example, data cleaning and migration is simplified and there is little need to closely reproduce current interfaces and workflows because they aren't strongly coupled to the model.

There is the potential to make it adaptable for use by other professions, e.g. Pharmacy, that have similar data models. This opens the possibility for a system that can be used by multiple professions that don't follow the TIS data model.

The data model is adaptable to being evolved for higher value work raised as being of value in user interviews, e.g. the Progression Review Process.

The nature of the data doesn't carry high risk to information security, reducing constraints on platform options that might otherwise be the case.

“There are no legal regulations around the data beyond those of the UK GDPR. Educational records can be retained indefinitely and all data is retained in the database year-on-year.” - Stakeholder

“The data is considered to be low risk in terms of security threats from external bad-actors.” - Stakeholder

2.2.2 Scale of the service

The number of trainees managed by the current system is modest, numbering in the mid-hundreds per year, relative to medicine (in the tens of thousands).

“Approximately 550 trainees are added per year across the four programmes with a current total of 4,100.”
- Stakeholder

This may reduce potential logistical issues with transferring trainee data to a new system, even if historical trainee data continues to be preserved.

The modest relative number has been a barrier to prioritising a move to TIS because addressing the changes required for the significant data model differences is seen as disproportionate to the number of trainees it would benefit.

“HCS needed the model to be certain ways. If it ends up in development work I have to justify that to business managers that are only interested in medical, they're our main stakeholder.” - Stakeholder

The system is only used by School staff with a low concurrency.

“There are approximately 50 users of the system with around three typically using the system at any one time.” - Stakeholder

“The system is available to all (and only) staff at the School.” - Stakeholder

The relatively low number of users may simplify any training needs when transitioning to a new system. A new, faster, easier service would be expected to increase the concurrency of use.

Any costs of a new system that are based on the number of users (e.g. a per-seat model for off-the-shelf products) should factor in the potential for trainee self-service because this would increase the user base by a factor of at least 10 (depending on the number of years trainees can expect to be in training and active on the system).

2.2.3 Data ingestion

There are a number of data sources for creating new cohorts of trainees and synchronisation.

A Master Commissioning File creates skeleton trainee records

Registration forms flesh out the skeleton trainee records

OneFile updates are manually collected at least once a week

These are not automated. They require expert manual intervention to collect and pre-process the data before importing it.

A replacement system needs to account for the potential requirement of a similar process. Exploration is needed to see if automation is possible.

2.2.4 Data egress

There are a number of systems that rely on the database to supply trainee data in order to set up accounts.

Eportfolio: tracking training portfolios

ZOHO CRM: managing stakeholder relations following completion of training

Clarity Informatics: trainee 360 appraisals

ENetAssess: final exit assessment tool

They are not integrated - data is exported to spreadsheets for subsequent use.

A replacement service would not need to integrate either, but there would be benefits to having reporting capabilities that simplify the generation of the exported data.

2.2.5 Ecosystem

The trainee system does not exist in isolation. There are a number of other systems in use for managing trainees and their training programmes that it needs to interoperate with and maintain data consistency.

We have seen from user feedback, and acknowledged in the service design, that limitations in data gathered, speed and reporting flexibility result in data consistency issues between data processors.

“The [candidate reports] 'process' consists of multiple stages driven by multiple data sources developed and maintained by different teams.” - Data processor

“I don’t have enough information regarding progression on the database. Lots of data is in our email inbox rather than the trainee database or their files.” - Data processor

“We look at our own spreadsheet more. It’s only if something comes up that we check the trainee database.” - Stakeholder

“We also have a separate database on Access which is for financial activity. That’s lying dormant.” - Stakeholder

“It would be good if whatever HCS end up with would integrate with Oriel. HCS has an E-portfolio system too. That should have standards around interoperability.”
- Stakeholder

A replacement service either needs to interoperate with the same services with robust data consistency or, taking on their functionality, replace them.

It further needs to factor in how it may need to interoperate with new HEE services that are in the pipeline, e.g. a future CRM.

“At the very least the product used by the School needs to consider how it would react to a future CRM solution.”
- Stakeholder

2.2.6 Accessing data

The current technology limits easy access to the data.

“The use of Access in and of itself is a barrier, e.g. interface constraints and the need to run the Access application on a user’s computer.” - Stakeholder

“Use of a single, shared global user for the system. Multiple authentication layers. Long database access time.” - Stakeholder

From user feedback we have heard that this encourages the use and sharing of out-of-date copies of the data exported to spreadsheets at an earlier date, or using other source of data maintained independently (thus going out of sync and invalidating a single source of truth).

2.2.7 Reporting

The current system lacks powerful analytical capabilities and has limited data export for users.

Without being able to flexibly inspect and report on the data from within the Access database tool, nor export all the data they need to do it externally, users are not able to make full use of the data directly themselves.

“Partial trainee data can be exported from the system via the Access frontend as Excel files.” - Stakeholder

Users can request Alex to produce ad-hoc reports for them but this is a bottleneck and means that the user has to hope what they request is going to be right, rather than be able to experiment with the data.

“Alex can export via direct access to the database for ad-hoc reports.” - Stakeholder

There are important reports that can't be generated directly from the Access database tool and need to be created externally and with privileged access to the source database. A replacement service would ideally allow such reports to be created natively.

“Completion Tracker report is generated at the end of the training year.” - Stakeholder

A primary need in a new training service is to enable all users (including, potentially, self-service trainees) to fully and flexibly make use of the data.

“If there was one big post-it note for a goal of a new service it is that users are able to flexibly interrogate the data and create their own reports.” - Stakeholder

It is worth noting that powerful reporting functionality is not necessarily required within the same system that collects and manages trainee data: TIS exports its data and delegates all reporting to the HEE National Data Service.

“We send all the data to NDS and it is all reportable via Tableau. We don't have any in-built reporting in TIS.”
- Stakeholder

2.2.8 Service design

We have heard various reasons to support the introduction of a new system across the School, notably for reasons relating to functionality, maintainability and standards conformance.

This is recognised within the School and the current Access database is no longer in active development.

“The system is now effectively in maintenance mode.”
- Stakeholder

The current Access database is over reliant on one person to support the operational needs of all the users. A new service should implement design and features that removes or minimises this operational reliance, freeing resources for strategic work that is on the backlog.

“Alex is the full development team.” - Stakeholder

“Alex provides all user support.” - Stakeholder

“Users being unable to generate reports using their own custom parameters results in administrative burden creating ad-hoc reports.” - Stakeholder

A new service can provide specialisms in service design that are currently not possible just from expertise in the trainee management domain. Accessibility is a key specialism that will improve user experience.

“The interface is not explicitly designed to meet any accessibility standards.” - Stakeholder

Security, including role-based access, is a specialism that will be increasingly important if there is self-service for trainees.

“The system has not had security assessments or accreditations.” - Stakeholder

There is a drive within HEE to improve the conformance and design principles of the services HEE uses. The current standards would be difficult to achieve with the current system even if it were under active development. A new service would have to adhere to these standards.

“HEE architectural principles. The whole organisation has signed up to it.” - Stakeholder

“The Enterprise Design Authority has the application compliance checklist that has all the standards.”
- Stakeholder

“We have to adhere to the Cabinet Office technical code of conduct.” - Stakeholder

2.2.9 Bespoke solutions

We have heard the issues faced with developing and maintaining the bespoke trainee system. This is reflected more widely in concerns about tools and services used throughout HEE.

“The Executive board's main thrust at the moment is about digital risk.” - Stakeholder

Reducing the dependence on bespoke solutions, instead using Commercial off-the-shelf (COTS) products, is seen as a way to reduce risk in meeting standards, reduce the training requirements to use them and reduce the costs and delivery time of functionality.

“We've got quite a few highly customised systems causing no end of headaches … So as close to out-of-the-box and vanilla that we can get, brilliant, absolutely brilliant.” - Stakeholder

“My preferred systems option would always be off-the-shelf. And with the least amount of development and bespoke work will be easier.” - Stakeholder

COTS solutions put focus on the ownership of service delivered - the quality of data and its use - over the development and maintenance of the service.

COTS solutions may compromise the ability to fully meet all user needs compared to a fully implemented bespoke solution, however this is compensated for by an improved user experience for most users and for most of the things they need to do.

2.2.10 National Data Service

The HEE National Data Service (NDS) provides facilities to send data from HEE systems and services from which any set of potential users can report on the data.

“We take data from wherever we need it and put it in there and people can report on it. It connects to the Tableau reporting instance, it can also connect to a version of Power BI in another service.” - Stakeholder

“Whatever goes on downstream from us, you're not usually worried about it. If someone wants to do reporting by Tableau that's fine, if they want to do it by cloud API, that's fine. If they want to export it into Excel and fiddle about with it then that's fine. If they want to use Alteryx or Python or whatever else it is, then that's fine as well.” - Stakeholder

This allows reporting to be done outside the systems and services that collect and manage the data. This reduces or eliminates the need for those systems and services to provide reporting capabilities. TIS follows this model.

“We send all the data to NDS and it is all reportable via Tableau. We don't have any in-built reporting in TIS.” - Stakeholder

Pre-defined reports and dashboards can be created by domain experts within the systems and services, as well as allowing ad-hoc analysis of the data.

The data sent to the NDS can be fully defined by the system sending it and does not need to conform to any specific standards or compatibility with other datasets in the NDS.

“We don't define the data structure, it would be a fool's errand to attempt to structure that stuff, at least at this stage. It's just a reporting back end.” - Stakeholder

There may be benefits to a wider range of HEE stakeholders and data consumers if, eventually, healthcare science trainee information was available in the NDS.

“I imagine healthcare science will be similar to TIS. It will just do whatever collections it needs to do and they will give us a cut every now and again. So that we can keep people up to date on what they need to know about the system.” - Stakeholder

2.2.11 Trainee Information System (TIS)

We heard of the challenges in trying to align the healthcare science data model with the medical data model used by TIS, placing a barrier in using TIS for the management of healthcare science trainee data.

“We tried mapping HCS [healthcare science] data, but it was too much, we didn't have the capacity, and still don't, to deal with it. I was hopeful we would be able to, but there was a lot more in there and more controversial needs that I anticipated.” - Stakeholder

Obtaining adequate resources to pursue it in more depth to find a possible solution faced competing interests from stakeholders.

“HCS already had an established Access database, whereas pharmacy and ACP had a spreadsheet, so it's a lot easier to work with them to fit into our model than it was for HCS, which is completely understandable. HCS needed the model to be certain ways. If it ends up in development work I have to justify that to business managers that are only interested in medical. They're our main stakeholder.” - Stakeholder

Giving trainees the ability to self-manage their data is seen as an important part in maintaining the quality of the data and reducing administrative burden.

“The trainee self-service part of the roadmap is our focus and has been for a long time.” - Stakeholder

The TIS experience of incorporating trainee self-service into the system is likely to provide valuable lessons to any new healthcare science trainee data service.

2.3 Service management

In researching the user experience of the current system, we have heard several findings that relate to the wider management of the service and data across the School.

2.3.1 Responsibilities for data

There is a wider confusion about who is responsible for different aspects of trainee data across the School.

This confusion means that people are reluctant to edit trainee data, whether it is in scope for their role or not.

“I don’t really touch the database. I can [update data], but because you never know what’s going on behind the scenes. There might be a reason why the Programme Team hasn't changed it. Instead of interfering with that I just don’t really touch it.” - Data processor

This confusion over responsibilities also leads to gaps and inaccuracies in trainee data. In particular, when data is spread across multiple systems, e.g. OneFile.

“Occasionally OneFile will be more up to date than the database because sometimes people won’t update OneFile and the database. For example a trainee might get in touch with the Programme Office and change their supervisor but forget to change it on OneFile, so we won’t know about it.” - Stakeholder

“I’ve got a trainee that’s showing up on our OneFile as working in Southampton. He logs into his OneFile and it says he’s at Birmingham - which I know he is, but our OneFile still says Southampton. So somewhere along the line, there’s been a gap. It’s that maintaining of the system [which is the issue].” - Data processor

Whilst people are aware that these gaps exist, another problem exists in that people don’t take ownership to fix the incorrect or missing data when they identify it.

“I can’t understand why when there’s gaps [in data], people don’t fill in the gaps. They know they’re there - go and fill them in!” - Data processor

“If something doesn’t look right, we need to have that initiative to check.” - Data processor

2.3.2 Reliance on others

Stakeholders noted that people across the School have a reliance on others to do things inside the system rather than doing it themselves.

“My perception is that there is an over reliance on people to do things that the system could do.” - Stakeholder

Our research with users confirmed that this is happening across the School. In particular, Alex faces the brunt of these requests to do things for other people.

“I have to always go back to Alex and if he’s not around it’s a problem. We are always relying on Alex to do things for us, so downloading or uploading data.” - Data processor

There are several reasons for this over reliance on others. One of which is people lacking confidence in using the system.

“I started this role during Covid so I haven’t had any face-to-face, in office experience of accessing this data … In an office you can ask the person sitting next to you [for help].” - Data seeker

Whilst this lack of confidence appears genuine, a more concerning issue is that people are avoiding the system because they are unwilling to use it.

“It was easier - and I feel guilty about this - to ask someone ‘can you pull me that number?’ than it was to teach myself how to learn a new system that was slow.” - Data seeker

“It’s a long process. You have to have a login for the VPN and hope you are navigating the system properly. People would rather ask someone who’s always on it.” - Data processor

“Any time I need to access it I ask someone else. I just try and avoid it like the plague.” - Data processor

2.3.3 System updates

Users find updating the system itself to be a frustrating process.

“It’s a bit of a patchy plaster-fix process.” - Data processor

“Alex has to push through additional updates for whatever reason and then there’s a new version of the system. You have to uninstall then install something new.” - Data processor

This source of frustration for users also opens up the possibility of digital risk, with people using older versions of the system. This is something that stakeholders suspect is happening and users have admitted to throughout our research.

“The amount of times we’ve had to download a new version of it, I can imagine a lot of people are on older versions of the database.” - Data processor

“I’ve seen lots of emails about updates [to the system], which I haven’t kept up with and so can no longer access the database.” - Survey respondent

2.3.4 Sustainability of the service

It is clear that there is an over reliance on others to work inside the system across the School and that there is a poor experience in terms of keeping the database version up to date.

These factors raise questions about the sustainability of the system.

This is particularly the case given that so much of this reliance falls upon the product manager of the current system.

There are several issues associated with this. Most notably, people recognise that if Alex were to take leave for any reason it would create significant problems for the School.

“If our expert goes on holiday for two weeks and the system goes down, I’ve got no database for two weeks and that’s just not a sustainable position for us.” - Data processor

“We’re always asking [Alex] to do things, if he wasn’t around I think it would be a huge concern.”
- Data processor

People also recognise that relying on one person to do particular tasks leads to a bottleneck getting things done, no matter how efficient that individual’s processes are.

Distributing these responsibilities across the team would reduce risk and make processes more efficient.

“[The product manager] is our go-to person, but he’s everybody’s go-to person so he’s overstretched.” - Data processor

“We’re in a really vulnerable position where there’s just one person making these changes.” - Data processor

2.3.5 Autonomy of users

Whilst users have avoided the system due to several frustrations, they have also expressed a desire to do more things for themselves in the system.

In part, this is to make their own working patterns more efficient.

“Getting the database to work for myself would save us all a lot of time.” - Data processor

Stakeholders have also expressed a desire to do more in the system themselves. There is an awareness that asking people to provide data is currently a time consuming task for those interacting with the system.

Consequently they are unlikely to ask for information that isn’t business critical but might have a small chance of aiding decision making.

“Sometimes there are bits of information that would be useful for me to know but I don’t ask for it as the team is already overburdened and it’s adding on to their workload for something that might not be business critical, but is interesting to look at.” - Stakeholder

2.3.6 Higher value work

Maintaining the system in its current inefficient state is a time consuming process for everyone involved. The impact of this on day-to-day work has been made clear through our findings to this point.

A further byproduct of this inefficiency is that it leads to missed opportunities to carry out higher value work across the School.

“Fixing the system means that [we] can do more in terms of efficiencies and process, since we wouldn’t be dealing with the granular day-to-day.” - Stakeholder

An example of higher value work is to address the current review of progression processes across the School.

“We could redesign the mid-term progression process.” - Stakeholder

“Our mid term review of progress currently generates data about every trainee. The backlog at the moment is so large that we are working on cases months after an issue is discovered.” - Stakeholder

What this review would look like is outside of the scope of this discovery, however we did capture a need around harder measures to support decision making in this area.

“Annual Review of Progression are self-declared answers and we give a grade based on how we think they’re progressing. I think it’s quite useful for the future to have some harder measures.” - Data seeker

2.4 The current user experience

2.4.1 A clunky interface

The incumbent data management system has been consistently described by participants as ‘clunky’ and ‘cumbersome’ to use.

“It works and I get what I need from it, but it takes longer and is more complex than it needs to be.” - Data seeker

Research participants provided a range of examples where they perceive the current service to be clunky.

For example, when attempting to access and make use of trainee data contained within the system.

“Where it has the training officer’s details, we can’t physically copy-and-paste from that. It’s literally a case of having both screens up and manually typing it out.” - Data processor

“Only one person at a time can access a trainee's record.” - Data processor

And also when navigating the current interface.

“If I’m in a trainee record and I need to get back to home, there’s no way of getting home. I find myself having to log out and log back in. I’m looking at all the buttons and I cannot see something that takes me back to home.” - Data processor

This overall sentiment is also recognised by stakeholders.

“One of the problems with the current system is that the Programme Office find it difficult to extract the information that is needed by themselves or others.” - Stakeholder

2.4.2 A slow and time-consuming experience

As well as clunky, some participants discussed how interacting with the current system can often be a slow and time-consuming process.

“To update one bit of information on 10 trainees can take me 2 hours on certain days, whereas it can take me 10 minutes on others.” - Data processor

This slowness seems to have been exacerbated because of the requirement for School colleagues to work remotely via a VPN when working from home during the Covid pandemic.

“A lot of the time, you can’t get onto the VPN and it takes us ages to log onto the database.” - Data processor

“When I first started working from home, I could still access it, but it was just a hell of a lot slower.”
- Data processor

Some believe the combination of slowness and clunkiness makes users avoid the system.

“Sometimes that just puts me off. It can take a few minutes to log into the system. To connect with the VPN we need another password, which isn’t the same every day.” - Data processor

Some participants understandably expectat the system to work like the other services they use personally and professionally.

“When it’s slow, it’s frustrating. Anything that can be done to speed it up. I think you take it for granted in the broadband world, but you get used to it.” - Data processor

Similar to the system’s perceived clunkiness (Section 2.4.1), stakeholders have also acknowledged pitfalls with how the current system is accessed remotely.

“It’s underused because of the user experience in how it is accessed.” - Stakeholder

2.4.3 Concerns around accessibility

While discussing their current user experience, one participant questioned whether the incumbent system could adequately support someone with access needs.

“I don’t know how accessible it is. To me, it doesn’t stand out that it is.” - Data processor

As part of this discovery’s user research, School staff members were invited to complete an online survey that asked specific questions about their current user experience. One of these questions asked respondents about whether they had any specific accessibility issues or needs with the current system.

One respondent discussed how their eyesight and optical surgery can impact their experiences when consuming trainee data and using spreadsheets in general.

“[It can be] harder to scan information in horizontal lines, engaging for long periods of time in activities that require switching between data views can become tiring [...] I prefer to use slightly larger font sizes, when available.” - Survey response

The discomfort described in their feedback would suggest that the current solution does not adequately meet their accessibility needs.

One stakeholder cited the digital risk to HEE for failing to comply with accessibility standards.

“[A] digital risk is making sure that we don’t buy anything that doesn’t fulfil HEE’s legal obligations and duties as a public sector organisation. Mainly that means complying with GDPR and Accessibility standards.” - Stakeholder

2.4.4 Deliberate avoidance of the service

As discussed in Section 2.3.2, it was observed that some users rely on others because they are reluctant to use the current system.

For some, this avoidance is a personal choice influenced by the perceived slowness and clunky user experience.

“I’ve sort of steered away from it because of how problematic it is and how slow and clunky it is. I go in when I’ve exhausted other options and I have no option but to go in. If I can find another way, I’ll go for that first.” - Data processor

“So once we have the latest version, we download that to our desktop and just keep using the same one until we get a new version because it takes absolutely forever. You could wait all day just to get into it.” - Data processor

However for others, their avoidance is being forced because of a lack of desired functionality.

“We’ve got some information, but because it doesn’t fulfil everything, we’ve got departments working on their own.” - Data processor

“I know how difficult it is for people to do their jobs using the system at the moment. Use of information in multiple places has grown a lot as people have become more demanding and want to know more information.” - Stakeholder

For example, some users are using Excel spreadsheets as these enable greater flexibility when manipulating data or enabling them to collect more information than is permitted by the current system.

“From what I can work out the Programme Office is using spreadsheets as a first resort rather than the database.” - Stakeholder

“We’ve created our own sheets to give us the information we want because we can’t have it off the database.” - Data processor

One participant even suggested that their offline Excel spreadsheets are more valuable to them than the current system.

“[If the system was gone tomorrow], there wouldn’t be much of an impact because we have our own spreadsheets.” - Data processor

2.4.5 Duplicated processes and effort

As discussed in Section 2.4.4, some participants have developed workarounds to better meet their needs because the current system lacks certain capabilities.

For example, making notes and setting reminders about individual trainees and their affected data.

“You need to remember to make separate notes for things like maternity leave dates.” - Data processor

“Someone taking a break for six months, it won’t automatically change the dates in other places. So to do that I have to set a calendar invite in the database to remind myself to go back later and change the dates manually.” - Data processor

Or needing to create copies of spreadsheets containing trainee information and then redacting certain data fields so that they are able to share only necessary data with their School colleagues.

“We’ve had to create duplicate spreadsheets that don’t have all the information so that the other teams can come in and have a look.” - Data processor

As well as issues related to a lack of interoperability between the multiple systems used currently to manage all data collected by the School.

“Nothing talks to anything which just makes everybody’s job harder because you’ve got to do things three or four times.” - Data processor

Affected users are very much aware of this duplication and how unhelpful and detrimental this can be to existing workflows and processes.

“It’s a bit of a nightmare at the moment. We’re working like we’re in the dark ages in terms of storing all this information on worksheets.” - Data processor

“It’s halfway there for what we need it to be. It’s so much duplication.” - Data processor

“All this duplication, it’s not good for anybody.”
- Data processor

And stakeholders have also recognised these issues which have contributed to this issue of duplication.

“They have needs that won’t be developed at the moment as it’s either outside of current skill sets or time capacity. At the moment these needs are being met by Excel sheets.” - Stakeholder

2.4.6 Human error

While talking through their behaviours with the existing system, some users discussed how human error may occur and can compromise the quality of trainee data.

“Human error is natural. We will make mistakes. We’ve all put the wrong data in or forgotten to update something and we’re constantly battling having out of date or incorrect data about trainees.” - Data processor

For example, incorrectly updating details about a trainee’s new workplace supervisor or training officer.

“When a trainee gets a new [training] officer, we have to update that on the system. When we add a new one, the old one stays there until you untick it. If someone doesn’t untick it then when we pull off a report, that trainee will appear as a duplicate.” - Data processor

Or being unaware they have moved between trainee records and updating a different one with incorrect data.

“Sometimes I don’t think it gives you enough warning that you’ve moved between trainee files.” - Data processor

One user also referred back to their concerns around the current interface’s perceived slowness too.

“There’s some human error. There’s some aspects of the system being so slow and clunky that you think ‘oh, I’ll do that later.’ I often think ‘oh I don’t want to do that now and watch it whirl.’ I’m sure the team does that and I’m sure that’s how things get forgotten.” - Data processor

And some users suggested that those less experienced with the incumbent system may be at a greater risk of creating a mistake.

“Somebody who’s new might not know they have to report a training officer change, so information gets missed or delayed or doesn’t get updated.”
- Data processor

“If they’re not exercising the skills to be on it or understand how to click this or should they be doing that. We all see it as the master database, so to accidentally mess up something - it can be detrimental.” - Data processor

2.4.7 Missing and outdated information

Several participants described their frustrations when accessing trainee data and finding either outdated or even missing information from the current system.

“If I go into the database and it’s got null everywhere, I don’t know where that trainee is and I don’t know if they’re attributed to the particular trust I’m looking at.” - Data processor

As discussed in Section 2.4.6, human error can often be the cause for these data anomalies.

“Quite often that revised completion date [for a post-maternity leave trainee] gets forgotten about so when I’m downloading a report in March for people I’m expecting, I’ll either have people I shouldn’t have or people I’m expecting that I haven’t got because of updates not being made.” - Data processor

“Occasionally OneFile will be more up to date than the database because sometimes people won’t update OneFile and the database.” - Stakeholder

But for some, the root of these issues seem to lie more with their avoidance of the current system (Section 2.4.4).

“Someone else will send out some comms on a Wednesday that will go to an old training officer because they haven’t used the latest version.” - Data processor

“I won’t have the most up to date email address because it might have been downloaded from a few weeks ago.” - Data processor

Either way, outdated and missing data can cause disruption and delay.

Some users have alternative sources of data, but sometimes the only solution is for them to wait for the necessary updates or changes to be made.

“If the region isn’t there, that’s the biggest issue I have with missing data. In that case, I’ll do it via the hospitals or just do it again in a month’s time when it’s likely the data will then be there.” - Data seeker

2.4.8 Triggers for trainees to get in contact with the School

Trainees will get in contact with the School for a multitude of different reasons with varying levels of complexity.

“We have a general inbox which any enquiry will come into. There are weird and wonderful things coming through every day.” - Data processor

An interaction could be a relatively minor and straightforward transaction, e.g. updating contact details.

“When we send an email to the trainees about booking people onto the assessments, they’ll say ‘oh this is my new email address, please can you update it’, so we still get those types of requests.” - Data processor

Other examples included a change of name, an updated address or to update important details related to training, e.g. a change in training officer or workplace supervisor.

“Any change requirement. That could be from them changing their name, changing their training officer to transferring to a different employer.” - Data processor

Another reason could be a change of circumstance to a trainee, e.g. maternity leave or another interruption to their training that requires an extension.

“Trainees might get in touch about periods of time when they are not in work or if they have requested an extension to their training.” - Data processor

“They might have extensions to training. They might have interruptions to training. They have to change their completion timelines. We have to put in information about when they go on maternity leave. It’s all that sort of stuff about an individual where things happen to them during their training programme.” - Data processor

For these types of requests, system users (predominantly the Programme Support Office team) currently have a more complicated procedure to follow when updating trainee information (see Section 2.4.5).

2.4.9 Difficulties in communication between the School and its trainees

Although not the focus of this Discovery’s research, it is worth noting some of the difficulties mentioned by users in relation to their communication with trainees.

Some issues are specific to trainee data. For example, trainees not informing the School about changes to their personal information, e.g. updated contact details.

“When people say they haven’t received an email, in most cases, they have been communicated with. When they haven’t it’s because they haven’t informed us of up to date contact details.” - Data processor

There are also more extreme cases where trainees fail to notify the School about any important changes to their circumstances, e.g. maternity leave; risking the data maintained about them becoming very inaccurate.

“Some [trainees] are very good, some are not so good. Some people have gone on maternity leave, come back and never told us.” - Data processor

A sample of trainees were also engaged to better understand their context around trainee data and their overall interaction with the School.

Interestingly, some trainees suggested that the procedure for updating personal data is not always obvious.

“I think if it’s at the footer of a bulletin, then it won’t be getting read. I don’t think I’ve seen it enough for me to know that’s what I need to do.” - Trainee

One user also discussed how trainees are not aware that the School makes use of multiple systems, and may naturally assume that updating information in one place is all that matters.

“Some trainees think that all the systems are linked. So if it’s been changed on the database, I know they think automatically it’s changed on OneFile. It’s not.” - Data processor

Aside from issues with trainee data, some users suggested that trainees may be developing a fatigue and apathy to the School’s communications.

“Trainees are bombarded with emails so probably don’t read half of them. It can cause big problems because we are sending out quite major information.” - Data processor

The feedback from some current trainees seems to confirm this assumption.

“I do get them [emails]. I don’t read them. I have them go to a separate folder where I can just ignore them.” - Trainee

“They send out a regular newsletter but I think the amount of trainees that probably engage are few and far between. It’s kind of like the local paper. It’s quite generic. I think people are looking for things relevant to them.” - Trainee

2.5 Users’ suggested improvements

2.5.1 A web-based service

As discussed in Section 2.4.2, some users find accessing the current service to be time-consuming and often difficult, particularly because of the need to use a VPN.

“We have to access it through the VPN and it is working better than it was, but if you haven’t got VPN then it just says ‘computer says no’ so you can’t get onto it.” - Data processor

Or some of the issues that can occur when required to download and use a system on a work device.

“I put the front-end on my desktop and then it got swamped [by everything else].” - Data seeker

Understandably, some have suggested the need for a future service that moves away from this technology.

“We want to be able to go to the database without logging into a VPN.” - Stakeholder

More specifically, a preference to access trainee data through a web-based platform.

“I’d prefer a web-based system so you can access it anytime and anywhere.” - Data processor

“It has got to be web-based. It can’t be what we’ve got currently with Access.” - Data processor

It was also suggested that this would also complement the School’s ability for staff to continue to work remotely.

“Whilst we’re in the office and we’re hardwired into the system it works okay, but if we want to work remotely anywhere, then it’s not fit for purpose.”
- Data processor

2.5.2 A mobile experience

While discussing current uses of tools and equipment when accessing trainee data (e.g. laptops connected to dual display monitors), some users valued the potential of a mobile or tablet experience.

“I suppose there’s always the opportunity [for mobile].” - Data seeker

This was notably desirable from those likely to be in situations where they are away from their office devices and in need of specific data more urgently.

“Say we’re at the audiology conference and I can go tap-tap-tap: we have this many audiology trainees.”
- Data seeker

“The power of having that on a tablet and saying ‘oh what was your question about the number of so and sos on what programme?’ If I could access it on my tablet and give them a number there and then, that would be amazing.” - Data seeker

Whereas users who are more likely to be updating volumes of data from a static work environment were less enthused about a mobile experience in future.

“I do have a work mobile, but I wouldn’t necessarily want to use it on that, whether it was this database or another.” - Data processor

With some even being reluctant to the idea entirely.

“The data is quite confidential on there, you don’t want people to take screenshots and share it [on a mobile].” - Data processor

2.5.3 Training and upskilling

Most participants recognised that because not all users (including themselves) engage enough with the current system, this has most likely contributed to a lack of confidence with the system.

“I think everyone has access to the database, but if you don’t use it often enough you just wouldn’t know where to start.” - Data processor

“​​It’s not the database that’s the problem, it’s me not interacting with it.” - Data seeker

And so most users welcome the opportunity to learn and upskill with the systems used for managing trainee data.

“I think it’s really important that people are trained on how to make changes.” - Data processor

Or just enough onboarding and guidance so that they can be self-sufficient and not waste their time or rely on others to complete tasks for them related to trainee data.

“At the moment I’m not an expert user of the system, so it’s not a good use of my time.” - Stakeholder

“I think my team would welcome learning about a system to a basic level. Certainly if they could troubleshoot some issues.” - Data processor

2.5.4 Alerts and notifications

Some users suggested that a future iteration of the system would be more valuable to the School if it were more than just a repository of data.
Specifically, if the system could provide prompts, triggers and alerts to users so that they can better maintain data.

“We need to make sure that our data is up to date but we also need a system that helps support us to make sure that it is always up to date.” - Data processor

For example, being reminded about trainees that are expected to be returning soon from their maternity leave.

“If the system could alert us to that so it prompts us to take action. Whether that is changing their status from maternity to active or whatever it is we do because that then has a rolling impact on how we manage their data.”
- Data processor

Rather than having to work outside of the current system and the potential risks that could be incurred from this.

“We have to set reminders in Outlook. We have to put things in my diary for me to revisit. And we’ve had a few instances recently where because that’s been missed, that’s then had an impact on the trainees and I’m really keen to avoid that.” - Data processor

Some users had greater expectations for a future service (see Section 2.5.9). For example, alerting training providers so that the School can ensure their compliance when it comes to their accreditation status.

“Say [a department] didn’t return to us their accredited application forms - that’s a non-compliance and the system could automatically write out to them. You’re able to set deadlines and then run off a report for who hadn’t responded.” - Data processor

2.5.5 Bespoke reporting

For most users, the current service’s functionality for reporting was described as being quite basic and limited.

“It would be great to have reports pulled off it. You can do, but it’s limited. You have to do one big spreadsheet and then filter it afterwards.” - Data processor

And therefore requiring additional manual effort.

“How it works at the moment is that it’s a big data dump, so we have a big CSV file and edit that and provide the data that they’ve asked for.” - Data processor

To an extent that some have avoided asking colleagues for specific trainee data because of this burden.

“Sometimes there are bits of information that would be useful for me to know but I don’t ask for it as the team are already overburdened.” - Stakeholder
Understandably, several users have suggested that a future service must have more comprehensive and customisable reporting capabilities so that they can easily access and interrogate granular permutations of data.

“There’s a lot of data in there and it would be good to manipulate it to what we want rather than the bog standard reports. […] It would be good to have something that gives you the opportunity to put in the parameters from the start.” - Data processor

“Is there a report builder which would give us some flexibility. Can we just say ‘I want to look at this, this, this and this’ and it just pulls it?” - Data processor

“A way that we could ask questions and have the database model answers for us to reduce manual effort.” - Survey response

2.5.6 Building a bigger picture with data

Some users already access trainee data so that they can inform decision-making (e.g. resource planning) or have more knowledge about a specific trainee’s circumstances.

“It’s nice to know how many people are on the Programme and in what discipline to plan for future assessments. How many examiners do we need?” - Data seeker

“A lot of trainees will email me asking for advice but they don’t really say where they are, what region they’re in, what hospital they’re in. So there’s just a little bit of understanding who they are and a little bit more about their background before I give them any advice.” - Data seeker

However, some users discussed how the School could do more to collect data and use it in a more intelligent way.

For example, analysing historic data to observe interesting patterns around trainee pathways.

“I think there’s always been a desire to do quite a lot of reporting to try and understand the background of people who come onto these programmes. Does it attract people with PhDs or undergrads?” - Data processor

“It could be triggered by us saying ‘well we don’t do particularly well for this particular professional group, is there anything we can do to increase the interest in the programme around there?’.” - Data seeker
Or reviewing trends specific to training officers and workplace supervisors, enabling the School to potentially intercept and respond when appropriate.

“Over time you could see that this particular training officer always has problems with their trainees, we’d then be able to build a better picture of what’s happening.” - Data processor

Or monitoring programme and assessment data to build a bigger picture and stronger narrative about trainee behaviours and to support those in potential difficulty.

“It would be good to add flags to show for example if an assessment is a second attempt but being treated as a first attempt. We need to follow the story of a trainee in the database.” - Stakeholder

“Maybe pull a report about all the ones who are about to breach their particular timelines so we’ve got a list of people that we can start doing some work with.” - Data processor

2.5.7 Better supporting trainees in difficulty

Some suggested that the School could better support trainees who are in difficulty.

“The least met need is on the individual support side when a trainee is in difficulty.” - Stakeholder

For example, some users discussed an absence of data recorded about any trainee issues.

“Trainee issues. The database doesn’t give me that information.” - Data processor

“If there’s a trainee who is in difficulty for example and they try to determine what the problem is. It might be a quality issue or a personality issue, but there’s no way to record any of that. There’s no history for the trainee.” - Data processor

And similarly to Section 2.5.6, some need a comprehensive narrative about every trainee (e.g. monitoring assessment data) and also a historic log of their interactions with the School.

“When did they start? Therefore how far along should they be? If there’s information there around whether they’ve had interruptions in the past. That sort of thing.” - Data seeker

“Just having a plotted history of where they’ve intersected with the School. Whether Accreditation have been involved, whether Programme Support have been involved and just getting that history of what’s led them to where they are now.” - Data seeker

By having this data available to users, some have indicated that this will enable them to better support trainees in difficulty because they can be more informed and prepared about each trainee’s unique circumstances and therefore respond in the most appropriate way.

“Say I had an email about a trainee in difficulty, I could then just have a quick look to see who they are and then be better informed when I then offer them support.” - Data processor

2.5.8 Self-service functionality for trainees

Most users value the potential of a self-service function for trainees to update their own information.

“Self-service where the trainees can manage their data and inform us about changes in circumstance using the same product that we use would be perfect.” - Stakeholder

“It would be really useful if trainees could access and update their own information. Often they will contact a university and assume that the university will update us from there but that’s not the case. It would be great if the trainees could check themselves.” - Data processor

Some users also provided compelling examples of when and how a potential self-service function could be beneficial to both trainees and School staff, e.g. recording trainee maternity leave dates.

“The start of their maternity leave and when they’re anticipating to return and if they want to return on part-time training. They put all that information in and the calculator would provide them with their revised completion date.” - Data processor

A small sample of trainees were also consulted and each one was supportive of the ability to manage their data.

“A self-service that’s very clear that you engage with regularly or that you know of and it’s quite obvious, it would then make sense that you go and update it.” - Trainee

“Just in general, it cuts out the middle-man. It’s like online banking, you don’t have to go into your branch anymore.” - Trainee

However, some users were reluctant about trainees updating all information and suggested that some data items (e.g. updating their training officer) would require additional measures for quality assurance purposes.

“A self-service, but a self-service with controls so a trainee can’t just change any old thing without us knowing about it. For example, they could change their training officer but a change of training officer potentially impacts on the accreditation status of the department. And we need to know that they’re qualified to do that role.” - Data processor

“I’d be a bit cautious about a trainee being able to update who their training officer is themselves.” - Data processor

The ability to approve or reject changes was desired by some users if a self-service function became a reality.

“If there was maybe something where they could do it themselves and we just have to approve it from our end.” - Data processor

Generally speaking, there was consensus that a self-service function would be a positive development and some participants contemplated whether this could become one component of an all-inclusive interface for all communication between trainees and the School.

“If you were to change any details and you were told that you only had one system to interact with - that would be quite simple from a user point of view and I imagine that would be similar for the School’s perspective too.” - Trainee

“To do all of our comms with the trainees from one system would just make things so much neater and more compact. Whether there was a self-service function for trainees themselves to update their data.” - Data processor

2.5.9 A single source of truth

Some users emphasised the need for, and current lack of a single source of truth about trainees.

“For me, it’s a full record of all of our trainees through the lifetime of the healthcare science programme and we haven’t got that.” - Data processor

“I only want us to record the information once. I don’t want people to have to copy and paste information from different spreadsheets.” - Data processor

A comprehensive and up to date record of trainee data that is accessible to every department within the School.

“Just keeping a track and keeping everything about each trainee together in one place so it’s easy to access.” - Data seeker

“We all need to be able to access the same software and information in real time. If one department makes a change, all other departments are notified of that.” - Data processor

“We should destroy our spreadsheets and have confidence in the system. It should be easy to use and access, that would support us to make changes in the School.” - Stakeholder

Some users gave specific examples of how this could noticeably improve their current tasks and processes.

“We might have a situation where a trainee might be removed from programme. We’d have to trawl through all the emails, all the minutes and any notes to build a case. It would be very helpful if [...] it was all in one place. Rather than look in 5 or 6 different places and speak to 3 or 4 different people.” - Data processor

For some suggestions, this would mean introducing more data than what is already collected by the current system, e.g. documenting reviews of progression.

“Processes like the review of progression which is a really big task for us. At the moment it’s done very manually and I feel that should be attached to the record of an individual trainee as it’s a critical part of their record.” - Stakeholder

Or offering functionality to record a log of all communication between a trainee and the School.

“It would be helpful if the system showed what communications have happened with trainees and on what date to help us with that.” - Data processor

But for some users, a single source of truth really means a holistic, integrated source of all information important to the School and its departments; with trainee data featuring as just one aspect of this ideal system.

“We need one system that manages workplace departments, manages HEIs, manages the training officers and workplace supervisors and also all the information about the trainees.” - Data processor

With any scenario around centralising data, it will require a review of the School’s data management processes.

Or at the very least, improve interoperability between the multiple, disparate systems currently used by the School.

“A system that talks to our other systems. We have a standalone CRM system. If there was any way that those two could connect?” - Data processor

2.5.10 Specific access and permissions

Within the current system, there are no specific hierarchies or unique data permissions for School staff.

“The system does not have user types/roles that control access to parts of the data and functionality. All users have the same abilities.” - Data processor

However, some users have suggested the need to introduce specific access controls.

“Not everybody should have access to the same data. [...] I think we could set up a system whereby there’s certain permission levels set for individuals and teams.” - Data processor

Particularly in relation to concerns around ensuring data privacy and confidentiality, especially if the School were to pursue users’ aspiration for a single source of truth on trainee data (see Section 2.5.9).

“Information around training support that is kept outside of the system. There are confidentiality issues associated with that but that information should also be inside of the system without being accessible to all.” - Data processor

Finally, it was also suggested that by offering specific access rights and permissions to manage data, this may also bolster the confidence of users to interact more with the service and not rely on others (see Section 2.3.2).

“That might make people use it more because they know they’re not touching anything. We’ve got to restore people’s confidence and I think that might be a way of doing it.” - Data processor

3 Validating user needs

In total, 52 user needs were identified from user interviews, workshops and surveys.

The School’s staff were invited to complete an online survey to validate 35 of these user needs.

The survey had 16 respondents in total, of which:

5 work in Assessment and Research Evaluation

3 work in Digital & Information Systems

3 work in Accreditation and Admissions

2 work in Operational Management

2 work as Training Programme Directors

1 works in Programme Support

*The Discovery observed that some School staff members work in multiple departments but associate themselves with one more than others (e.g. part of the Programme Support team), but focus predominantly with Assessment and Research Evaluation processes. This has been taken into consideration as part of the analysis.

3.1 Analysis method and outputs

The level of user need was scored by participants on a scale from 0 - I have no need... to 4 - I definitely need to…

Each need was then categorised into Very High, High, Medium and Low priority user needs depending on the average score of the responses.

Please refer to the full data contained within the user needs validation survey analysis spreadsheet for more detail about the scoring of the user needs.

The levels of user need are shown in this chart:

Stacked bar chart: Validated levels of user need

The 35 user needs were also themed and the level of validated need across these categories are shown in this chart:

Stacked bar chart: Validated levels of user need by research theme

3.1.1 The level of need is greatest for accessing and interrogating data

Of the eight needs themed as Essential data, five were validated as Very High or High and only one as Low.

These results are not unexpected when the key purpose of the current system is to function as a repository of trainee information for School staff.

Of the seven needs themed as Data interrogation, four were validated as Very High or High and only one was ranked Low by survey respondents.

The two needs validated as Very High, were:

UN45: I need to access high-level summaries about the trainee data collected so that I can get an instant overview of the information available (e.g. quick stats)

UN46: I need to easily segment available trainee data by one or more attributes (e.g. region, specialism, cohort, programme) so that I can find trainees relevant to my enquiry

These reflect the Discovery’s qualitative findings reported in Section 2.5.5 and 2.5.6 around bespoke reporting and for the School to build a bigger, more holistic picture of their trainee data.

3.1.2 The level of need is greater for needs around data quality

All three needs related to Data quality were validated as either High or Medium. These were:

UN37: I need to know when each data item was last updated so that I can be confident that the information recorded is accurate

UN39: I need to choose from pre-selected options when inputting specific data items (e.g. a standardised list of trainee specialisms) so that information is consistently recorded and the risk of discrepancies in data is removed

UN40: I need to have any incomplete data items for an individual trainee clearly highlighted so that myself or a School colleague can rectify this issue

While these results are not unexpected, they re-emphasise the Discovery’s findings around concerns with human error (Section 2.4.6) and users seeking improvements to mitigate the impacts of missing and outdated information (Section 2.4.7).

3.1.3 The level of need around trainee support differs between School teams

Of the four needs themed as Trainee support, two were validated as Low, one as Medium, and only one as High.

The High need was:

UN36: I need to collate all evidence about a trainee in one location so that the School can make important decisions with robust data that is not fragmented and/or duplicated across multiple places - single source, reduce duplication

Which correlates with the Discovery’s finding around ambitions for a single source of truth about trainee data (Section 2.5.9).

The validation results may suggest that these user needs are of a lower priority than others in general. However, it is important to emphasise that only certain School teams require access to information about trainees in difficulty.

And when analysing survey responses at a segmented level, Operational Management, Programme Support and Training Programme Directors ranked all four of the Trainee support user needs as Very High.

Whereas for Accreditation & Admissions, Assessment & Research Evaluation and Digital & Information Systems, the same user needs are all validated as Low.

3.1.4 The level of need is lower for updating trainee data

Of the nine needs themed as Updating data, eight were validated as Low and only one as High.

The High need was:

UN31: I need to be clearly signposted about which data item I am accessing (e.g. a specific trainee) so that I am confident that I am viewing the correct information

This High need reflects the Discovery’s findings around mitigations to reduce human error (Section 2.4.6).

“Sometimes I don’t think it gives you enough warning that you’ve moved between trainee files.”
- Data processor

The large majority of Updating data user needs being ranked as Low is likely because not all respondents were data processors and some may only seek information and not be required to update it.

Additionally, some School departments are responsible for updating specific data items, but not others, and so these individual responsibilities may have caused the overall validation results to be reduced.

4 Concept development

4.1 Overview

We developed a set of low-fidelity concept prototypes informed by the Discovery’s user research and user needs.

The concepts explored:

Searching trainee records and
viewing a individual trainee’s profile

Bespoke reporting on trainee data

Aggregated data about trainees in difficulty

A trainee self-service function and
service administrator dashboard

Five data processors were invited to participate in a concept feedback session.

Each participant was asked to assess the viability of these concepts and suggest where further improvement could be made for designs to better fulfil their user needs.

An exported Miro board of collated feedback from these sessions is included as an output of this Discovery.

Designs were iterated based on initial feedback and an updated version of these concept prototypes has been included as an output of this Discovery.

Below is a summary of important feedback worth reflecting on and to potentially explore further as part of an Alpha phase.

4.2 Assigning responsibilities for responding to self-service requests

The initial proof-of-concept for a trainee self-service function tested well with each participant.

“That all looks good. It looks like what I'd expect to see for something like this.” - Data processor

And participants were very helpful to provide further examples (and suggested vocabulary) of data changes which trainees may request in addition to the three examples presented in the prototype.

“Extenuating circumstances to training. So any type of extensions, any pauses to training.”
- Data processor

“What if they move trust? They move workplace. Sometimes a trainee will say they've moved. They have to seek approval.” - Data processor

Most importantly, participants emphasised the need to indicate which School department or individual team member is responsible for responding to each change request submitted by trainees.

And most suggested ways for possibly fulfilling this need:

“Perhaps maybe two drop-downs? So the first one is the team and then the second one would be a member within that team?” - Data processor

"I think it would be good so that everyone sees the list, but maybe if something could be added to show the status as to where it is, so if Accreditation are reviewing or working on it? Then maybe at the end, it generates a tick when they've completed their bit.” - Data processor

However, some participants also discussed that some trainee self-service requests will involve multiple teams within the School and a future interface must therefore also reflect this complexity.

“We do have some requests that crossover into different teams. For example, the training officer change will have three teams involved in. The accreditation team will do their quality checks to make sure that an appropriate person is nominated. The digital team action it through OneFile and then we need to go and update the database to reflect the change." - Data processor

4.3 Accessing and updating trainee records

Based on the Discovery’s user research findings, a prototype was developed to explore how to present trainee data in a less ‘clunky’ and cumbersome way.

A suggested option was to remove the use of tabs and multiple sections within a trainee’s record and instead display their data in one single view.

This concept tested well with every participant:

“This does look really good. So you can see all of the details.” - Data processor

“I prefer this to having different tabs.”
- Data processor

A single view would likely require users having to scroll through a trainee’s profile, and so some participants also valued the introduction of local navigation to reduce the burden of this additional user interaction.

“I think it's quite good that you can scroll or just click on that to see progress and status.”
- Data processor

The prototype also included content indicating that a user could view when a trainee’s record was last updated. While the overall concept of this was desirable, most participants suggested the need for more specific details about what data items were updated and by whom.

“If it could tell you what the last change was. Even if that was another box. Cos really other than telling me it happened on the 22nd, it doesn't tell me anything.” - Data processor

“That would be useful cos then we could know who's been in and done what.” - Data processor

“Perhaps you can see last updated by? Or maybe we word it differently to something like Case Manager and we can see who is managing them?”
- Data processor

4.4 Searching for trainee data

One concept prototype presented a more comprehensive search function than what users described as being available to them when using the incumbent service.

This concept tested well with most participants:

“All of those are really sensible. And I'd probably use all of those at some point or another.”
- Data processor

“I can't think of anything extra that I'd need there.”
- Data processor

And where necessary, participants suggested where their user experience could be further improved. For example, additional search filter options.

“You've got Trust, but then you'd need a sub-filter for hospital sites. Some departments may be accredited, but some departments might still be going through the process.” - Data processor

“I do think regions would be useful, especially as our TPDs are responsible for certain regions in the country. If there are issues in the South West, we know exactly who to raise this with.” - Data processor

Prototype feedback also reinforced the user needs identified during the Discovery about how School users tend to access trainee data. Either by searching for an individual trainee or by segmenting data to identify trainees with shared attributes.

“Generally, we're searching by name. What we also have to do is search by trust and by specialty. If we're accrediting a department, we need to know how many trainees they've got and what status they're at.” - Data processor

“The more options we've got to narrow that search down saves time. That would be really valuable.”
- Data processor

This particular prototype also presented a suggested view for how a user’s search results could be displayed. Understandably, the different School departments had their own unique preferences.

“I think that I'd want to see the trust as well because we tend to use that heavily. We nearly always steer towards the trust first.” - Data processor

“Email address is very useful. I don't think I need to look at the institution, but that's just me.” - Data processor

And one participant suggested whether search results could be customised each time to meet the preferences of the individual searcher.

“If you had customisable filters across the top, that would be useful. Whoever needs whatever access, then they can click on what they need and obviously my needs will be different to someone else.” - Data processor

The prototype also included the option to save specific searches for later reuse. Some participants were able to suggest situations where this might be useful.

“Every month, we have the trainee management panel and we need to go into the database and check some details. We would need to go in multiple times, so that saving search would be fantastic.” - Data processor

But for others, this addition would not be of great value.

“The search would be different every time.”
- Data processor

Finally, the prototype suggested the option for users to search for and update data items in bulk. This also tested well and participants were able to suggest particular situations where this could improve efficiency.

“That sounds fantastic. You go to change the one training officer and the system recognises that there's other trainees at the same trust and it flags it up and you can tick the ones you need to.”
- Data processor

“Annual monitoring. That would be a bulk upload of about 400 or 500 to update.” - Data processor

“Things like the assessment results. At the moment, 300 trainees will need their pass or fail status putting in and that would be a manual job if we were expected to do it.” - Data processor

4.5 Consuming trainee data

Participants were shown a concept prototype which explored how users could potentially view and consume aggregate levels of trainee data.

For some users, this would be a useful addition.

“We don't have anything like this so that makes it more valuable.” - Data processor

But for others, this would have lesser value because of their current roles and needs.

“This isn't maybe what I need it for.” - Data processor

“We wouldn't be looking at that. We'd be looking specifically at sites where there are difficulties or we need to know more information so that we can accredit.” - Data processor

This is understandable as those recruited to participate in the five prototype feedback sessions were predominantly processors of trainee data, not consumers.

Information around trainees in difficulty was a recurring suggestion from tested users about data they would be more inclined to consume and interrogate further.

“I think a particular question I'd ask would be about trainees in difficulty. Are they from a particular region or trust?” - Data processor

One of the prototypes did explore this user need.

This concept included a list of trainees in difficulty which was of value, but most users emphasised the need for greater context about why a trainee is classed as being in difficulty from the onset (e.g. specific categories), which could determine their next course of action or support.

“It would be good to know the category of difficulty.” - Data processor

“The thing is - are they on the programme in difficulty or are they signed off on sick in difficulty? We wouldn't contact them if they were under a doctor's care because that would be inappropriate for us.” - Data processor

Some participants also valued the suggested ability to view a historic list of trainees previously classed as ‘in difficulty’, enabling them to better meet some user needs around reviewing trends or patterns in trainee data.

“Yes. That's part of our overview. How many trainees have passed, have we had any that have failed or are there any that have left because of their environment? That may tell us that it's a difficult department to train in and it may not be suitable for accreditation.” - Data processor

This low fidelity prototype also included some visual presentations of data. Feedback around this varied based on individual preferences for consuming information.

“As long as it's clear, I don't mind if it's pictorial or numbers.” - Data processor

“I have spent a lot of my time producing graphs to present information like this so that would be useful to have ready made.” - Data processor

4.6 Permissions to view trainee data

Some users during this Discovery have described the need for tiered levels of access to trainee data (Section 2.5.10), and so the prototype feedback sessions probed participants further about these possible controls.

It would seem permissions are desired most for users accessing information specific to a trainee’s personal circumstances, e.g. whether they are considered to be in difficulty during their training.

“Those who don't need to know it, don't need to see it for confidentiality reasons. There could be any reason for a difficulty.” - Data processor

Some users need to be aware that a trainee is in difficulty, but do not need to know the intricate details as to why.

“I don't think everybody should see this. Whilst I might know the name of somebody in difficulty, I don't know what's gone on. There needs to be some sort of security [privacy] here.” - Data processor

Whereas some users suggested they need to know the specifics of each trainee’s circumstances as it will likely affect their responsibilities, e.g. the Accreditation Team quality assuring training providers.

“We do need to see a lot of that. We need to see any absences that's affected anything to do with the quality of training.” - Data processor

Based on this initial feedback, the Programme Support and Accreditation teams have been suggested as those who possess the most access for trainee data.

“My immediate thought is that only certain people get to see if they're in difficulty or if there are issues or if there are illness issues. And that is the Programme Support and ourselves.” - Data processor

“I probably don't need access support. I'd need to see absences just for completion information or for managing their assessments.” - Data processor

While reviewing the trainee record concept prototype, some participants suggested ways of indicating levels of access permission to users could be to display certain data fields as being locked or password protected.

“There needs to be some sort of security here. Whether it's password protected.” - Data processor

“So that would need to be locked.” - Data processor

5 Landscape analysis

We took a detailed look at three comparable services, at various stages of their development to see what lessons could be learned.

5.1 Manchester Academy for Healthcare Science Education

Manchester Academy for Healthcare Science Education (MAHSE) are responsible for coordinating the development of healthcare science programmes across partner institutions.

This case study is based on a conversation with Kate Smith, MASHE Manager, who is responsible for maintaining the organisation’s trainee data in their own trainee management system.

Lessons and insights

Purpose and context

Get data from the National School in a spreadsheet. Use that spreadsheet to add more data, as students are enrolled. That data is then entered into the University database

Data includes a student ID and student email address. It’s information needed to get in touch with and track trainees throughout a programme

Have some access to Power BI for student data from University of Manchester students

Data model

HCS database is in Access. Everything is uploaded from spreadsheets into Access: ID, name, personal email address, student email address, trust, home nation status

Home nation is necessary as HEE will pay for English and Welsh students. Scottish and Irish students have to be charged separately

Training officers are bolted on to that, so people can be grouped like that

Students are given iPads so the system is used to record that and the associated serial number

Capturing training officer information

In year one, someone manages a trainee’s rotations, but that person tends to not end up being the final training officer

Therefore trainees are emailed at the end of year one to find out who their training officer is because that tends to be when they know who they are. Response to this email is quite high - approximately 85% after chasing a couple of times

There is also a follow up in year two to ensure that everything is correct

With Covid, this information has been more important because iPads have to be posted to a trainee supervisor’s hospital address

Managing trainees in difficulty

Managed better for HSST trainees. Students have multiple comment boxes in their record where administrators can go in and add notes if someone is having issues. A student record then separately highlights all of the issues that a student has had over a programme. It also shows their current status at the bottom of the record

STP trainees only have one box for comments as there are limitations as to how much free text can be entered in the system

With trainees returning from a break, this is mostly managed by the University. It’s possible to run a status report in Access once a month to see who is due back and then plan accordingly

Although data is included in the system, it’s usually the bare minimum needed, e.g. if a trainee sends a detailed email documenting their circumstances, it might be summarised to a couple of lines in the system. This information then isn’t kept anywhere else

Image: Notes form inside the database used to capture details of trainees in difficulty or taking a break from their programme.

Accessing the system

Working from home initially led to issues because the system couldn’t connect to new IP addresses

Now there is access through a VPN, which needs to be logged into regardless to get onto a shared drive. “It works fine, I don’t think it’s any slower than when we were working in the University.”

5.2 NHS Education for Scotland

NHS Education for Scotland are responsible for managing healthcare science trainees across Scotland.

This case study is based on a conversation with Dr Robert Farley, Healthcare Science Associate Director, and Joanna Cuthbert, Head of Product and Delivery at NES Digital.

Lessons and insights

Context

NHS Education for Scotland have an in-house team who have built a suite of apps including trainee management

The system is called TURAS and specifically contains a module called Training Programme Management (TPM). This system caters for all cohorts including Medical, Dental, Psychology and Healthcare Science

The system captures data including the names of trainees, details on supervisor/second supervisors, discipline of trainees and trainee email addresses. Information on trainers is captured in a separate area of the system

The volume of HCS trainees across Scotland is smaller than that in the NSHCS - suspected to be ten times fewer trainees

Image: An overview of the TURAS system dashboard page, including the Training Management Programme module.

Similarities with TIS

The TPM system was originally designed for medicine, “healthcare science was bolted on”. There are workarounds to make this work as the terminology is different for HCS compared to medicine

“Our system does everything that we need it to, but it’s clunky because it has far more fields than we really need. We depend on knowing the shortcuts and idiosyncrasies of the system that we are working with.”

”I should get some paper, cut some windows out and press it up to the screen so I can only see the fields relevant to us.”

There are issues about making fields created for medical trainees to work for HCS trainees. System asks for GMC numbers, which isn’t relevant to HCS (who need HCPC details)

Medical training posts are limited every year so they reuse training numbers. This doesn’t work with HCS either, those trainees get a unique number so can’t use that field

There are issues with inputting trainees if a trainee is already in another aspect of the TURAS system, e.g. TURAS learn

Managing trainees in difficulty

Trainees in difficulty are mostly managed outside of the system at the moment

A new module is being developed for TURAS to improve reporting, however it is focusing on whether training centres have issues rather than individual trainees themselves

Major issues are fairly rare outside of extenuating circumstances for trainees. This is mainly because they are dealing with a much smaller number of trainees overall. The rarity means that when it happens it’s dealt with hands on outside of the system

Adding and maintaining trainee information

This is a manual process beginning with Microsoft forms where training supervisors are asked to fill in trainee details. Once data is available they are manually added to the system

Trainees rarely get in touch to update changes in their personal details. This is usually an issue when sending out quality checks at the end of the year

Scotland email addresses changed recently to NHS.Scot. That has led to issues with people dropping off regulatory registers

Adding and maintaining trainee information

The system is locked down to audits made available by the digital team. This results in spreadsheets with unneeded data, which requires significant filtering to get the appropriate details needed for a report

There is a strong desire to create bespoke audits to pull off reports containing appropriate information

Data to do with assessments is held by individual programmes, as they have their own way of doing assessments. Therefore this data isn’t fed into the trainee management system and there isn’t a way of tracking progression

The reporting/audit function doesn’t meet current needs, the team has to go through a digital team to get appropriate reports

6 Inclusivity and accessibility assessment

6.1 Digital skills

User research was conducted with all key roles that make use of the current service (see Section 1.8.3).

Each respondent to the user needs validation survey was asked to self-assess their digital skills using the 9-point GDS Digital Inclusion Scale. Participants were also provided the options of Prefer not to say and Don’t know.

Their responses are presented in this chart.

Image: User research participants self-assessed digital skills.

Important: all surveys were online surveys shared to School staff email addresses. It is appreciated that this will likely bring in a bias towards users with a higher level of digital literacy.

This has been considered when interpreting results, however there is no reason to suggest that the individuals recruited to participate in user research were not representative of all users in terms of digital skills.

The latest Digital Inclusion Scale figures on the UK population are taken from the Government Digital Inclusion Strategy, updated in December 2014 and presented in the visual below:

Image: the Government Digital Service’s Digital Inclusion Scale.

Basic digital skills to Expert

79% of the UK population are a 7 (basic digital skills), 8 (confident) or 9 (expert). This segmentation should be capable of using any Service Standard level digital service offered to them if they have internet access.

Approximately 95% of survey respondents identified within this group, which is higher in comparison to the UK average population.

Never have, never will to Willing and unable

14% of the UK population are a 1 (never have, never will), 2 (was online, but no longer) or 3 (willing and unable). This segment of the UK average population will have an assisted digital need.

There was no evidence in the Discovery of participants identifying with this group and it is unlikely that School staff would fall into this category as there will be some expectation for them to use digital processes and technology in their professional capacity.

Reluctantly online to Task specific

7% of the UK population are a 4 (reluctantly online), 5 (learning the ropes) or 6 (task specific).

One of the survey respondents (approx. 5% of the total sample) identified as being Task specific, which demonstrates parity with the average UK population and also suggests that some intended users may have a level of assisted digital need.

Next steps

Further research should be carried out during Alpha with users on the lower end of the Digital Inclusion Scale to better understand their individual needs when accessing trainee data and where assisted digital support would be most appropriate.

With approximately 40 staff in total, embedding a dedicated assisted digital support team within the School does not seem feasible or cost-effective.

However, the Discovery does recommend the rollout of more training support for School staff (see Section 7.4.5) which should help to build users’ digital skills and confidence so that they can be more self-sufficient.

This could hopefully result in more staff becoming recognised as a source of digital support for others, mitigating the current risk of only one individual being identified as the go-to person for support.

6.2 Connectivity

The Discovery’s findings suggest that all School staff currently access and will continue to access trainee data through a desktop computer or laptop while located in an office environment or when working remotely.

However, some participants did express an interest in consuming trainee data as a mobile user experience.

As the Discovery prototypes were low-fidelity and created as a proof-of-concept, they were not tested as a mobile experience with research participants.

Next steps

We recommend that the Alpha explores mobile testing of reporting functionality as part of its user research. This should be conducted with a range of smartphones, tablet devices and operating systems (e.g. iOS, Android) to identify any potential issues that may be triggered when accessing trainee data.

As some users discussed frustrations with VPN access and the ambition for a web-based system (Section 2.5.1), it would be helpful for research to further explore the impact of specific technology choices and related connectivity issues, e.g. optimal broadband conditions.

6.3 Accessibility

It is likely that there will be some users of this service who have access needs (according to The Big Hack, 19% of working age adults in the UK have a disability).

Research in this area by The Big Hack shows that disabled people are more likely to encounter barriers to accessing digital services and lack the skills they need to get online.

And as reported in Section 2.4.3, one survey respondent from the School disclosed about having access needs and how they currently support themselves.

Next steps

Section 7.3.1 recommends exploring a commercial, non-bespoke platform for a new service in Alpha. Such platforms will provide out-of-the-box support for accessibility needs, however the implementation of the service on such a platform will have a bearing on the degree to which the service is accessible.

For example, how well design patterns (such as GDS ones) can be applied given the components available in the platform.

We recommend that the Alpha evaluates the broad accessibility support in platforms used during prototyping.

Additionally, perform prototype usability testing with users with access needs and/or using assistive technologies.

7 Recommendations

Our recommendations cover four themes:

Context for these recommendations

Overarching recommendations

Data and technology recommendations

Service management and governance recommendations

They are supported by a suggested roadmap (Section 7.5) and a recommended set of Alpha experiments (Section 7.5.1).

7.1 Context for these recommendations

7.1.1 The current system is not fit for purpose

Our Discovery research has emphasised the importance of data collection to the School.

Data collection is an essential part of compliance with regulatory requirements, as well as enabling the School to monitor and communicate with trainees.

The importance of this data is clear, however the current system is placing an unnecessary burden on users who need to access trainee information.

Whilst the current system works to an extent, significant work is involved in the management of the system to maintain current functionality.

Our research has further highlighted inefficiencies for both data processors and data seekers, which represents a significant opportunity to improve the service.

7.1.2 There is an increasing risk of doing nothing

Our research has also highlighted the changing environment of healthcare science training.

In particular, the number of trainees is increasing year-on-year. Alongside this, the proportion of trainees who are facing some level of difficulty is also increasing.

The increased number of trainees (including those in difficulty) further increases the burden of an inefficient service on users.

There is a risk that this burden on School staff becomes overwhelming if the School continues to use the current system in this environment. This poses an immediate risk to the quality of data collected and the School’s ability to manage and report on its trainees.

7.1.3 There is an opportunity to improve the quality of the School’s data

Whilst the importance of trainee data is clear, we have heard anecdotal evidence questioning the quality of the data collected by the School.

Section 2.1.2 outlines that the School doesn't currently have an overall view of the training population, which can lead to difficulties in informing decision making about workforce planning and policy.

Through our landscape interviews, we have also heard anecdotal evidence about errors in data provided to other institutions.

Whilst this Discovery seeks to inform the procurement of a new trainee management system, these findings also offer the School the opportunity to reflect on the type and quality of data that is collected.

7.1.4 It would be difficult to integrate Healthcare Science into TIS

Throughout this Discovery, we have heard contrasting perspectives about the purpose and ambitions of TIS.

We have heard that TIS would like to incorporate HCS into its database, however it appears that it is not adequately resourced to make this happen 1) in an appropriate time frame, and 2) designed with the user needs of HCS in mind.

If TIS did attempt to onboard HCS, it appears that a copy of the medical system would be the starting point for this work.

We have heard the dangers of this approach through a landscape interview with NHS Education Scotland, who find themselves in a similar situation whereby they are using a system that was designed for medical trainees.

The lessons from this case study suggest that this approach would not be satisfying for the School and is unlikely to meet the needs identified throughout this Discovery.

Moreover, people involved with the previous TIS and HCS work suggest that work to integrate couldn’t be resourced.

Subsequently, we agree with the decision to procure a separate trainee management system for HCS.

From an overarching HEE perspective, it would nevertheless be worthwhile to calculate the resource and cost needed to repurpose TIS with HCS and other disciplines in mind. This would allow the organisation to make future business case decisions about trainee information systems with maximum confidence.

7.1.5 Criteria for decision making

Throughout this Discovery, we have heard about internal and external criteria that should inform the School’s decision about a replacement trainee management system.

From an external perspective, it is expected that this work will comply with:

CDDO’s Technology Code of Practice

NHSX Service Standards

GDPR regulations

Accessibility standards

HEE also has internal standards that this piece of work will be expected to comply with, most notably:

Buy first, not build first

Web-based rather than client server

Value for money

Minimising digital risk

We were mindful of this criteria when developing our recommendations.

This ensures that the recommendations are consistent with HEE’s ongoing strategy and will support the School to make an appropriate business case for this work.

7.1.6 User experience improvements

Throughout this discovery process, we have also heard about user experience improvements that are expected from the work.

Many of these improvements can be delivered through the creation of a new service, designed with users in mind.

To achieve this, it is important to continually reflect on the user needs backlog and service blueprint that are provided as outputs from this work.

Both of these artefacts have their origins in our user research and represent the needs of users.

The needs in the backlog have been validated by users and further prioritised by the project team, which will help Alpha work to address priorities for a new system and supporting service.

We have also been mindful of these needs when developing these recommendations.

7.2 Overarching recommendations

7.2.1 Proceed with the procurement of a new trainee management system

Our Discovery research has reinforced the need to procure a new system to manage trainee information across the School.

Whilst the current system is sufficient to collect and manage trainee data in the current environment, it is not sustainable in the medium to long term.

The increasing level of trainees and proportion of trainees in difficulty is putting a strain on the current service that will soon outpace the capabilities of existing processes.

Furthermore, it is clear that the School is ambitious in their desire to collect better data and improve trainee management. Aspirations such as trainee self-service are not feasible on the current system, whilst other requested features cannot be delivered through iterative improvements.

The system is described as being in ‘maintenance mode’, and it has reached its limit in terms of what can be done to improve it at this point.

We recommend procuring a new system that is chosen in line with the prioritised user needs and service blueprint produced through this Discovery work.

7.2.2 Agree upon the boundaries of the new service

There are different models of data management that can be adopted to carry out this work.

Before progressing with any procurement, it is important that the School consider what the boundaries and constraints of the new system will be.

In particular, there are three areas of service and data management that should be considered.

7.2.2.1 Consider the difference between the boundaries of the service and the boundaries of the system

Currently, the system sits in the middle of a service that is surrounded by other platforms. Most significantly of which is OneFile.

OneFile plays an important role in defining the boundaries of the service going forward and decisions need to be made to reflect this.

There needs to be boundaries between the new system and OneFile, so that a hierarchy of truth can be established. This will enable both the system and OneFile to remain in sync with each other, whilst retaining OneFile as an integral part of the service.

Users go to OneFile at the moment to complete specific tasks, such as a trainee updating training officer information. The School should consider where tasks such as this fit in the ideal future service, especially if trainee self-service is an aspiration for the new system.

A new system may have to be designed with OneFile integration in mind from the outset. Whereas a radical approach may be to remove OneFile entirely from the service, moving all its trainee data functionality into a new system.

Section 7.4.2 further expands on this recommendation to suggest how OneFile might be used alongside a new system to make a new service work well.

7.2.2.2 Consider how the new system will deal with reporting functionality

Other areas across HEE, including TIS, use the National Data Service to facilitate reporting on data (Section 2.2.10).

The School needs to consider whether this is an approach that they would like to take with the new system.

There is no requirement to send data to the NDS. An integration would not need to form part of a minimum viable product. Conversely, it could provide options for reducing the requirements that a trainee system needs to fulfil user needs around reporting.

Reporting functionality would not necessarily need to be delegated to external tools taking their data from NDS. A new system could provide the simple reporting capabilities and dashboards and delegate the complex ones.

Even if reporting is done within the new data the School should consider whether there is value in making at least part of the data available in the NDS for others in HEE to make use of, without them needing access to the system.

If the chosen approach is to send data to the NDS, any new system should be designed with this in scope from the outset. If not, this becomes less important.

Technical design would be relatively straightforward, primarily that the system should provide an API for the NDS to consume.

The more interesting element is the design of the data being provided, as this would not necessarily be a straight copy of the internal data model.

7.2.2.3 Consider how the new system will deal with the data of historical trainees

The School should also consider the boundaries of a new system in relation to historical trainee data.

This is about setting out a strategy for where the responsibilities of the system end and the responsibilities of the CRM platform begin.

If the goal is to seamlessly transition from the system to CRM once a trainee becomes an alumni, then this should be designed from the outset. If the ambition is for the system to maintain historical trainee data in the long term then integration with a CRM tool may become less important.

Maintaining historical trainee data in the system may also have GDPR implications. Maintaining this data may require a systematic review at the end of every year to remove data that it is not appropriate to keep in line with legal requirements.

A key factor in determining the ongoing retention of trainee data is how it is used in reporting, particularly longitudinal reporting. Of relevance to this is a roadmap programme in TIS to archive trainees seven years after they leave, putting the data into the NDS and deleting it from TIS (reporting is done via the NDS).

7.2.3 Agree the scope of a minimum viable product (MVP)

Whilst we have suggested the procurement of a new trainee management system, it is unlikely that any system will meet all of the user requirements immediately.

Therefore, it is important that the Team consider what features are necessary to form an MVP.

We also recommend that the Team consider which features are not within the scope of an MVP, to form a longer term road map for the new service.

Based on the backlog of prioritised user needs and service blueprint, we recommend working towards and testing concepts for an MVP that:

Collects minimum viable data on trainees managed by the School (meets the most important processor and seeker needs, whilst being viable to collect)

Has appropriate identity management for School staff

Enables exporting of data into appropriate formats for both data processors and seekers

Can provide an overview of trainee cohorts

Is capable of recording case management information to support trainees in difficulty

Once the scope of an MVP has been agreed, it can be used to inform a decision about which platform to proceed with (Section 7.3.1).

It will also inform a series of Alpha experiments on a chosen platform to further test suitability (Section 7.5).

7.2.4 Do some scoping work to understand what the boundaries of this work are/could be across the School

Throughout this discovery, we have heard some anecdotal evidence of duplication of effort in data collection across the School (Section 2.1.4).

In particular, we have heard about duplication within the Commissioning Team who collect trainee names and contact information from higher education institutions (as well as other information).

Similarly, we have heard about an unsatisfactory experience across the commissioning process, suggesting the potential to better meet user needs.

Whilst this is out of scope for this piece of work, stakeholders have suggested that an ambitious trainee management service might widen its scope to consider data acquisition and sharing between the Digital Team and the Commissioning Team.

To understand the depth of work involved in this challenge, we recommend carrying out a further, short discovery focused on the data and supporting processes in place to enable the commissioning of trainees.

Best case scenario, this work has the potential to lead to better integration of data across the School with the new system as a focal point for this data.

Worst case scenario, this would lead to a set of recommendations to support the Commissioning Team to carry out their work in a more user centred way.

A short discovery would involve a research plan designed to identify the user needs of the Commissioning Team, which could then be validated and compared with the needs identified in this piece of work.

Significantly, this would outline similarities and differences that will inform the development of a new system by allowing the Team to make an informed decision about incorporating commissioning data or leaving it out.

7.2.5 Offer this work as a platform for others in HEE to build on

Whilst TIS is the platform of choice for managing the data of medical and dental trainees, it appears that there is scope for a complementary system that caters for other trainees across HEE.

We recognise that the School is at the forefront of this work by carrying out this piece of Discovery research before other professions.

This represents an opportunity to improve trainee management on a wider scale than just within the School itself.

Therefore we recommend that HEE carry out further discovery work to uncover the needs of other professions and use this to understand whether the approach taken by HCS can become a viable alternative for other professions across HEE.

Taking this approach benefits HEE as an organisation and also helps to future proof the work carried out by the School.

If other professions end up taking their own approach to trainee management in parallel, there is a risk of future rationalisation decisions being taken at the strategic level of HEE.

The organisation may find it difficult to justify multiple trainee management systems, particularly in an environment where a HEE custom build already exists in TIS.

Furthermore, increased desire to adhere with NHSX service standards (Section 2.1.8) may drive this rationalisation. The Service Standard talks about solving a whole problem for users, collaborating across organisational boundaries where necessary - in this case that would be a single trainee management system that could meet the needs of the organisation, rather than multiple disparate systems and approaches.

In the short term, we have heard that Pharmacy and Advanced Clinical Practice trainees have a data model that is closest in resemblance to the School’s. Therefore these professions represent the best place to look for potential collaboration.

We suggest a piece of discovery work to identify the user needs of those professions. Those needs can then be compared with that of the School to determine whether a HCS system could become a system for professions that are unlikely to fit into the model of TIS.

Whilst this is extra work at HEE’s expense, it can be carried out at pace with relatively little cost. The potential opportunity from this extra discovery significantly outweighs the cost.

This work could be carried out whilst the School is preparing themselves for the extra work ahead, with no impact on the School’s timeline to complete their own work.

7.2.6 Embrace the procurement of a new system as an opportunity to be considerate and ambitious about what data the School collects

In carrying out this work, the School has an opportunity to reflect on trainee data currently collected and consider any areas for improvement.

Throughout this Discovery, we have heard that the proportion of trainees in difficulty is increasing and this is placing a strain on current processes (Section 2.1.1).

The School has an opportunity to think about what data could be collected to better understand issues such as the increasing number of trainees in difficulty.

Factors such as trainees having caring responsibilities or trainees living a significant distance from their training location have been identified as potential sources of difficulty.

The School should consider what data they could collect to begin to quantify these factors and others. Ideally, this will provide further insight into why the proportion of trainees in difficulty is rising.

If the School is capable of better using data to understand this area, it enables them to design, implement and evaluate interventions targeted at specific groups of trainees. This might be those actively in difficulty or those who fall into a group at risk of being in difficulty in the future.

Ultimately, this would allow the School to target limited resources at groups of trainees most in need to both reduce the numbers of trainees currently in difficulty and ideally prevent trainees reaching that point in the future.

Image: prototype dashboard with real time tracking of trainees in difficulty.

Once the School has identified potential data to track, it can be built into the data model of a new system.

Our prototypes have explored what this could look like, by suggesting intelligence that can be gathered if the School broadens the data model in use.

7.2.7 Explore new ways to engage trainees in the collection of their data

Our research has highlighted that the needs of trainees aren’t well served by the current processes surrounding data collection and management (Section 2.5.8).

This work represents an opportunity to better meet trainee needs as well as the needs of users within the School.

The most opportune way of meeting trainee needs is by utilising a new system as the focal point for trainee interaction with the School. This can be achieved through trainee self service.

The self service process should begin by directly onboarding new trainees and allowing them to enter their data directly. This vastly streamlines the current approach of trainees entering data via a JISC survey and then subsequent semi-manual imports of data to create a core trainee record.

Once trainees have onboarded in this way, the trainee facing component of the system can then act as a dashboard for trainees to view their data and request changes.

Personal information such as changes to contact details could then be completed by a trainee without the need for intervention by School staff, increasing efficiency.

The School would also retain the ability to validate requests to change professional information, such as a training officer or employment information.

Image: trainee dashboard prototype highlighting self service elements of changes to professional information.

A self service option could also aspire to house communication records between the School and a trainee. This would allow the School to refer to specific communications with trainees when needed, rather than the current approach of having to search the School email inbox, which is a significant burden on staff.

Image: Prototype of a trainee hub showing historical communications with the School.

7.3 Data and technology

7.3.1 Prototype and evaluate the viability of a new service based on a Low-code or CRM platform

Discovery reviews validate the assumptions that the current Access database is unsuitable for meeting the evolving needs of trainee management and that TIS is not a viable route to replace it.

7.3.1.1 Technical options review

In this Discovery we looked at options for platforms that a new system could be based on.

We considered classes of platform rather than specific commercial offerings or technology choices. The class of platform that is most suitable is the first level to identify.

This Discovery gives recommendations for which classes appear to offer the best opportunities for a new system.

We suggest that these are used as the focus of Alpha to test their viability in meeting user needs - expressed both in the user needs backlog and inherent in the Standards (see Section 7.1.5).

Platform options

Between Buy and Build there’s a wide spectrum of possibilities.

At a high level there's striking an appropriate balance of “custom to your needs” vs “out of the box”.

More nuanced is: the potential for flexibility vs extent to which you’ll benefit from existing functionality; your ability to exercise complete control over the end-to-end process in the way that you do now vs practicality and total cost of ownership.

We considered four classes of platform:

Student Information System

A Student Information System (SIS) as a Commercial Off-the-Shelf (COTS) solution has resonance with the School's requirements: It has an existing data model which understands tertiary ed in the UK and also understands the business processes involved in managing them

Examples: Tribal products like SITS; Ellucan Banner; PeopleSoft

Customer Relationship Management platform

Customer Relationship Management (CRM) platforms are potentially attractive since they provide a lot of built in functionality and capability which caters for the simpler requirements you have such as the need for a customisable data model together with more advanced features such as workflow and inherent concept of case management.

CRMs provide a mature user interface for working with the data including reporting and dashboarding and good support for interoperability.

Examples: Salesforce; Zoho

Low-Code platform

Low-Code is a development environment and delivery platform that uses interfaces and configuration to build applications

At the simple end this can be choosing a range of individual tools that fulfil specific parts of what you need and then using an automation tool to make components talk together and to manage workflow

At the other end of the spectrum sit more enterprise-capable solutions which provide an end-to-end toolkit for designing and building completely custom web applications using pre-built components and integrations that make the process quicker and easier, with all support and hosting built into the package. We looked at this end of the spectrum in consideration of the range of personas that would use the trainee system

Examples: OutSystems; Mendix

Custom web application

Building a web application in a supportable technology framework is a traditional solution to meeting user needs exactly, with advantages in supporting all relevant standards, using existing components and services such as the GOV.UK design system, GOV.UK PaaS

Such flexibility comes with compromises for the level of investment required to reach it and risks associated with a build strategy (we have heard evidence of a preference for Buy over Build from HEE's Enterprise Architecture team)

We also considered whether any existing services used by the School could be suitable for a trainee system by adaptation or extension. We concluded that this was unlikely to be the case.

OneFile would not be suitable as a complete solution (vs being an integrated system, as currently). It is not capable of having its data model extended and appears incapable of functioning with an external identity provider (a desirable function).

The Zoho CRM used by the School provides some of the advantages of a more fully featured CRM solution but cannot serve as an identity provider and is less customisable (though, it might warrant more detailed consideration in an Alpha phase).

Evaluation criteria

We evaluated the four classes of platform against a number of criteria identified during the Discovery as being important to a new trainee management system.

Against these criteria we applied a RAG (Red-Amber-Green) rating:

Red: The platform is not well suited

Amber: The platform could be suitable but has caveats - there are compromises to make

Green: The platform is well suited

The criteria we looked at were:

SSO ready: Will the solution work with an external IdP service so that single sign-on could be achieved?

Access control: Does the solution have an innate concept of access control / management enabling roles and permissions to be defined?

Data modelling: Does the solution have a flexible data model that can be customised to meet the specific requirements of the School around trainee data?

Management: To what extent will the solution allow an admin user interface and workflow to be customised to the School’s specific requirements?

Reporting and dashboards: Does the solution allow for the creation of custom reports and dashboards? Will the solution allow the School to be self sufficient in configuring their own reports and dashboards?

Trainee self-service: Can the solution support direct access by trainees to view and change their contact details?

Technology Code of Practice: To what extent does the solution align with the CDDO Technology Code of Practice?

Integration and APIs: To what extent does the solution anticipate integration with third-party systems (ePortfolio, LMS for example)? Does the solution provide a flexible read/write API?

Purchase and setup costs: What is the comparative cost associated with the purchase and initial build or configuration of the solution?

Ongoing costs: What are the comparative ongoing costs associated with licences, support and maintenance for the solution?

Scope for reuse: Could the solution be practically reused or adapted for reuse by other disciplines?

The assessment is summarised in an artefact in the Discovery outputs: Platform options evaluation grid. This shows the RAG ratings and key points contributing to the rating.

Image: platform options evaluation grid.

SIS platforms have some drawbacks: they assume they are being used by the organisation providing the learning.

They are enterprise scale, so expensive, and have a lot of unneeded functionality.

They are opinionated giving less flexibility to customise.

CRM platforms have more baked-in core functionality and mature interfaces for reporting.

Trainee self-service may require greater customisation.

The user-based costs can become punitive.

Low-code platforms have a flexible data model, allow for flexible administrative and self-service interfaces and adaptable workflows built from out-of-the-box components.

They have good interoperability with other systems by design and functionality-based costs.

Custom builds can assure compliance with standards and security and are flexible in meeting user needs.

The School would own the product, with the ability to reuse, however it would also own the maintenance burden.

7.3.1.2 Technical platform recommendations

No one platform option will provide the best solution across all criteria. An appropriate balance needs to be struck between flexibility, self-sufficiency, and robustness.

Overall, the evaluation grid favours the Low-code and CRM platforms as having the potential to meet the needs of a trainee system identified during this Discovery.

Criteria for which these platforms are well suited (Green RAG rating) align with many of the service blueprint elements (counted as blue circles in the assessment grid).

Many of the needs could be addressed more rapidly and at a lower cost and risk than a custom build.

The SIS platform, although having a surface resonance, is not as adaptable to the specific needs of healthcare science trainees. At the same time they have feature sets that would not be used and complicate user experience.

We recommend selecting one instance of a Low-code platform and one instance of a CRM platform in Alpha and testing the viability of implementing the trainee system on these platforms. This can be done through a series of Alpha experiments, some of which we suggest in Section 7.5.1.

These platforms are amenable to rapid prototyping so should allow for a variety of experiments in Alpha and provide the opportunity for interactive user testing.

Although there will be some variability in the functionality and approach between different instances of each platform they will be similar enough that it will not be critical which instances are used. The objective will be to test the viability of the class of the platform - the particular instance itself would necessarily be selected in a Beta phase even if viability was confirmed.

Working with an implementation partner (see Section 7.3.6) will aid in abstracting out any instance-specific knowledge needed in Alpha.

Most Low-code and CRM platforms offer a free trial period and for an organisation such as HEE extended trials are likely to be readily negotiable.

7.3.2 Prepare yourselves for the data migration challenge

With the recommendation to replace the current Access database tool with a new one comes the need to migrate trainee data to it.

There is the possibility to connect the underlying Azure database to a new system in some cases, retaining its use and data model.

However a migration provides the opportunity to make longer term improvements to the data model and the extent and quality of the data.

A consideration that will apply whatever system is chosen is the scope of the trainees that will be managed. A new system could manage:

All trainees, including the data of those that have completed training

Only trainees in current training and those that will come onto the system as new annual cohorts

Only future cohorts, with the current Access database tool used in parallel for current trainees until they have completed training

Bringing all trainee data to a new system requires the most effort for migration, but gives the cleanest slate for long term operation of the service. In this scenario the new system replaces the old one at a nominal point in time.

The School should consider early on the practicalities of such a switchover and plan when such a switchover is best made that accommodates development timelines and business critical periods of use of the system.

The School already has some experience of this when migrating data to the current Azure-backed database.

Migrating data as-is to a new system is one aspect of a migration.

The School will also need to consider the changes in the data model that may arise from the new service design and factor them into a migration plan.

Data may be transformed or made redundant, in addition to the data model being extended (for example, adding the ability to historically track ‘In Difficulty’ status).

Migration also needs to consider the processes required to switch over access to data from external systems (for example OneFile and Sharepoint) in order to maintain data integrity and continuity.

7.3.3 Relate the service design to HEE's technical standards and principles

As well as functional needs, a new system will have to meet the standards set out in a GDS and/or NHSX assessment.

Additionally we recommend that a service is designed with reference to HEE's own technical standards and principles throughout Alpha.

These standards and principles are encapsulated in HEE's Architectural Principles documentation, Application Compliance Checklist and the CDDO Technical Code of Practice.

7.3.4 Determine the value of sending data to National Data Service

Section 7.2.2.2 discussed using the NDS to act as a potential source for reporting. Section 7.3.1 discussed the ability for recommended platform options to flexibly report within a new system.

Even if most reporting requirements could be met within a new system there may still be value in sending data to NDS, for example so that other HEE stakeholders with an interest in trainees from multiple disciplines have access to the data without requiring access to the new system itself.

In determining value the School should consider what representation of the data is most appropriate to send to the NDS. It does not necessarily need to be a duplicate of the new system’s data model.

For example, an anonymised subset of trainees with a subset and/or transformation of their data may be best suited to needs outside of the system.

The recommended platform options will generally have flexible data export options.

If there is value in sending data to NDS then using them to create an API for the data will be a useful Alpha experiment.

Consuming APIs is NDS's preferred approach for obtaining data.

7.3.5 Carry out a GDPR compliance review

In concert with refining the data model during Alpha we recommend taking the opportunity to review the purposes for the collection of all data in the context of the data protection principles within the UK GDPR.

We have seen that the current data model contains potentially sensitive personal data that is retained indefinitely and can be shared without strong constraints on its dissemination.

Such a review should also extend to the requirements for, and obligations on, the retention of the data.

A review will also assist in the design of appropriate access controls discussed in Section 7.4.3.

The recommended platform options to explore in Alpha, Low-code and CRM, will be GDPR compliant as a Data Processor in regards to factors such as data storage and data security (though that should be validated for any specific implementation that is pursued).

However, as the Data Controller, the School is responsible for ensuring that the data model and service design maintains full compliance, as that cannot be achieved by the Data Processor alone.

Undertaking lightweight Data Protection Impact Assessments (DPIAs) for key user journeys explored during Alpha are a good way of assisting with compliance reviews and demonstrating the mitigation of risk to data protection.

7.3.6 Find an implementation partner

Commensurate with the recommendation to explore and validate the use of either a Low-code or CRM platform as the solution for a new system is the recommendation to work with an implementation partner in Alpha.

A partner with experience in implementing solutions of both platforms for one or more specific instances of each will accelerate the breadth and depth of the experiments that can be performed.

Although the platforms are designed for developing solutions without requiring expertise in coding, visual & UX design, accessibility, etc, resources within the School would be better directed at defining the requirements of the solution in Alpha.

Expertise should though be gained after adoption of a specific instance because there are long term advantages to being self-sufficient in evolving the functional capabilities of the system (without the responsibilities of maintaining the underlying platform).

7.4 Service management and governance recommendations

Whilst this work offers the opportunity to procure a new trainee management system, it also offers the School an opportunity to make improvements and efficiencies to the way that the trainee data management service is managed.

The increasing volume of trainees suggests that a scalable, continuously improving service is needed to reflect the changing landscape within HCS training.

Importantly, purchasing a new system alone will not solve all of the School’s problems. It is important that supporting structures are in place so that the result of the work does not lead to the Digital Team facing the same challenges.

7.4.1 Redefine and then formalise the management of the service

It is clear that the inefficiencies of the current service place a high burden on the Digital Team at the moment.

The changing HCS environment means that this burden is unsustainable. It is also the case that the School is missing out on opportunities to do higher value work.

Firstly, we recommend that responsibilities are better distributed across the team to reduce the burden on individuals and reduce the overall risk of people moving roles.

Going forward, there will be a need to manage continuous improvement of the Digital Team, the platform, and staff across the School. The Team should evenly distribute these new responsibilities to achieve this efficiently.

Key to this will be maintaining the backlog of user needs to drive improvements of the new system.

It is also critical that the Team ensure that standards of data entry and upkeep are upheld in the new system.

There is also a need to focus on higher value work that can be carried out as a result of more efficient processes overall.

Examples of this include reviewing the annual progression processes (outlined in Section 2.3.6) and thinking about the overall data collection strategy to better implement interventions across trainee cohorts (Section 7.2.6).

The choice of platform will also influence the degree of project and operations management across the Team.

For example, a Low-code implementation partner will require a degree of management, however a bigger degree of management would be required if the team decided to do a custom build of a new system.

7.4.2 Establish a hierarchy for data across the School

We have heard users express a lack of confidence in data within the current system.

This work offers an opportunity to improve this confidence so that users can be sure that system data is correct.

In part this can be achieved through cultural change, by establishing a hierarchy for data across the School.

To do this the Digital team should decide whether OneFile or the system is the primary source of truth and the first place people go to update information.

Ideally, the system becomes the first place people go to update information, which means that when users spot inaccuracies across other systems, they know to rely on the information inside the system.

This could be further developed into a new system, where automated syncing of data between the system and OneFile should be explored.

Additionally a single account user experience would be particularly useful for trainee self service, as well as benefiting staff.

The capabilities for OneFile to use an external identity provider (IdP), i.e. not its own IdP but one used by a new system, and an API that supports user creation and bidirectional data syncing would support this.

It was not possible to determine these capabilities during this discovery, however we recommend pursuing this line of enquiry.

7.4.3 Embed access permissions within the new system

Our recommendations to this point have highlighted the potential to better use trainee data to support decision making across the School.

To achieve this, the School needs to be considerate about what data it collects in pursuit of its goals. In some cases, this will include collecting personal data in addition to that already collected.

To support this process, the School should integrate data governance measures into the design of a new system.

In particular, this can be achieved through access permissions to view or edit particular data.

In testing our prototypes, we explored who should be able to access personal trainee data (Section 4.6).

We found that for the most part, School staff should be able to see high level information about a trainee, however they might not need to see granular levels of personal information.

For example someone might be able to see that a trainee is in difficulty, but not be able to see a detailed explanation of why.

Image: Access permissions to edit or view appropriate information.

Building these permissions into a new system represents good data governance and is important to make sure that the School comply with GDPR regulations.

Access permissions will also enhance a culture of ownership across the School, whereby users are certain that they are responsible for updating data that they are capable of editing in the system.

7.4.4 Roll out training to support users with the new system

A new system represents an opportunity to develop a training programme to support users as they transition from the old system.

The goal of a new system is to find one that is intuitive enough for people to go in and do things without too much prior knowledge. Therefore, training on how to use the system may be fairly straightforward to develop and deliver.

A bigger challenge may be convincing users to engage with the system when they have previously admitted to deliberately avoiding the system (Section 2.4.4).

We therefore also recommend that this training is an opportunity to reestablish existing responsibilities for data, and reaffirm new responsibilities.

Emphasising cultural change in this way will reduce the burden on individuals and improve the longer term self-sufficiency of the service.

7.5 Suggested roadmap

To maintain momentum following this Discovery, we recommend developing a roadmap for action.

In particular we recommend addressing five key recommendations as soon as possible.

These five things make up a post-discovery phase of work that will pave the way for the things you will need to do later.

Do these five things now:

Satisfy yourselves about unmet needs across the School and organisation (HEE)

Satisfy yourselves with an appropriate system strategy

Decide on the scope of the new Trainee Management System

Decide which system best meets your needs and begin procurement

Begin an Alpha to test hypotheses against the selected system

7.5.1 Suggested Alpha approach

Once you have made the suggested decisions about the strategy and scope of a new system, begin an Alpha stream of work.

The Alpha phase will reflect upon these decisions and test riskier assumptions within a new system, as an experiment or proof of concept of the potential new service.

Depending on the recommendations you adopt and prioritise for Alpha, experiments could include:

Explore how staff can search for trainees in the system

Explore trainee profile pages

Test methods of identity management (both assigning profiles to School staff and trainees)

Explore the creation of trainee cohort dashboards

Continue to test concepts around providing an overview of trainees in difficulty

Explore ways of exporting data directly from searches

Explore case management functionality

Test access permissions

Explore trainee self-service, in particular the submission of trainee data

Explore trainee messaging through the system

Test integration with other systems (Sharepoint, OneFile, Oriel, CRM systems, National Data Service, and the National Web Platform)

These are in addition to the standard set of GDS recommendations to use the experience of building prototypes in the Alpha to:

Find problems with the design of your service and decide how you’ll solve them

Make some estimates about how much your service will cost

Identify the biggest risks for a Beta phase, as early as possible

We also recommend that you use the prototype feedback from this Discovery (Section 4) as a starting point for deciding what Alpha experiments to pursue.

Annex 1: Discovery research

Research sources

The following materials have been reviewed during the Discovery phase:

Healthcare Science overview dated 4 October 2019

HCS /TIS Collaboration dated 11 August 2020

HCS/TIS - Field Validation dated 19 July 2020

Stories for the HCS On-boarding dated 6 February 2020

HCS Admin roles and permissions dated 11 August 2020

Programme Naming dated 7 October 2020

New Request Form - Health Care Science (HCS) v6.

Data Service Register of Data Flows 2021 - v4

Baseline_Audit_of_Data_Flows_Report_v5

Stakeholder interviews

A range of stakeholders were interviewed during the Discovery phase, including:

Alex Levine, Education & Digital Systems Manager, National School of Healthcare Science - Health Education England

Robert Pink, Programme Manager Trainee Information Systems (TIS) - Health Education England

Stuart Sutherland, Health of Digital, National School of Healthcare Science - Health Education England

Elisa Lakhan-Hector, Enterprise Architect - Health Education England

Carol-Anne Jollie, National Data Service - Health Education England

Angela Daly, Director of Operations, National School of Healthcare Science - Health Education England

Assad Roachford, National Commissioning Manager - Health Education England

Mahroof Husssain, regional diversity and inclusion manager, Health Education England

Carol Higgison, Head of Assessment and Research Evaluation - Health Education England

Temilolu Odunaike, Education Officer, Education Department - Health and Care Professions Council

These stakeholder were interviewed from a technical perspective:

Alex Levine, Education & Digital Systems Manager, National School of Healthcare Science - Health Education England

Elisa Lakhan-Hector, Enterprise Architect - Health Education England

John Thompson, National Data Service - Health Education England

Alistair Pringle, Service Manager - Trainee Information Systems (TIS) - Health Education England

Landscape stakeholder interviewees

Dr Robert Farley, Healthcare Science Associate Director - NHS Education Scotland

Joanna Cuthbert, Head of Product and Delivery - NHS Education Scotland Digital

Kate Smith, MAHSE Manager - Manchester Academy for Healthcare Science Education (MAHSE)

User participants

All the users below were involved in telephone interviews, the user needs workshop, co-design workshop or concept feedback sessions.

User interviews

Administrator, Programme Support - Health Education England (data processor)

Lead administrator, Programme Support - Health Education England (data processor)

Lead administrator, Programme Support - Health Education England (data processor)

Head of Programme Support - Health Education England (data processor)

Training Support Case Manager, Programme Support - Health Education England (data processor)

Educational Systems Support, Digital and Information Systems - Health Education England (data processor)

Stakeholder Engagement Manager, Digital and Information Systems - Health Education England (data seeker)

Lead Administrator, Assessment and Research Evaluation - Health Education England (data processor)

Deputy Head of Programme Support, Programme Support - Health Education England (data processor)

Accreditation and Quality Manager, Accreditation and Admissions - Health Education England (data processor)

Project Manager, Accreditation and Admissions - Health Education England (data processor)

STP Training Programme Director - Health Education England (data seeker)

HSST Training Programme Director - Health Education England (data seeker)

Trainee Pharmaceutical Scientist - NHS Trust (trainee)

Trainee Clinical Biochemistry Scientist - NHS Trust (trainee)

Trainee Respiratory and Sleep Scientist - NHS Trust (trainee)

User needs workshop

Head of Programme Support, Programme Support - Health Education England (data processor)

Training Support Case Manager, Programme Support - Health Education England (data processor)

Educational Systems Support, Digital and Information Systems - Health Education England (data processor)

Stakeholder Engagement Manager, Digital and Information Systems - Health Education England (data processor)

Lead Administrator, Assessment and Research Evaluation - Health Education England (data processor)

Lead Administrator, Assessment and Research Evaluation - Health Education England (data processor)

Co-design workshop

Head of Programme Support, Programme Support - Health Education England (data processor)

Lead Administrator, Accreditation and Admissions - Health Education England (data processor)

Lead administrator, Programme Support - Health Education England (data processor)

Accreditation and Quality Manager, Accreditation and Admissions - Health Education England (data processor)

Concept feedback sessions

Training Support Case Manager, Programme Support - Health Education England (data processor)

Lead Administrator, Assessment and Research Evaluation - Health Education England (data processor)

Lead Administrator, Accreditation and Admissions - Health Education England (data processor)

Project Manager, Accreditation and Admissions - Health Education England (data processor)

Deputy Head of Programme Support, Programme Support - Health Education England (data processor)

Annex 2: Discovery outputs

These artefacts have been delivered in accordance with the project proposal:

Notes from stakeholder interviews

User research notes and analysis from user interviews

Survey results and analysis

User proto-persona profiles

User journey maps

User journey maps

Service journey map

User needs backlog

Service blueprint

Platform options evaluation grid

Concept prototypes

Show and tell presentation

Discovery report (this document)

Discovery presentation deck

These other distinct outputs have also been delivered in accordance with the project proposal but captured elsewhere:

Concept development (Section 3)

Landscape analysis case studies (Section 4)

Annex 3: Surveys

User experience survey data

NSHCS staff were signposted to a user experience survey.

11 staff completed the survey and further data about their responses have been included in the summaries below.

What device(s) do you typically use to access the current system and/or its data?

How often do you make use of the current system?

Other responses:

“As and when required, can't be specific.”

“Use the data on a weekly basis but not via the database due to technical issues.”

“I did have access but no longer use it and rely on others if I do need information.”

“Access varies depending on the stage of the assessment cycle, sometimes multiple times a day at other times less often.”

Have you ever required any help or support when using the current system and/or its data?

User needs validation survey data

NSHCS staff were signposted to a user needs validation survey.
16 staff completed the survey and further data about their responses have been included in the summaries below.

What team/office do you work for within the NSHCS?