The Proof is in the Pudding: The Importance of Impact Measurement when Supporting Disabled Students in Higher Education

Overview

This article will examine the various ways support for disabled students in higher education can be measured, and why this is important from the perspective of the student, the university and for the continued development of inclusive practice. Diversity and Ability (DnA), a disabled /neurodivergent led social enterprise, teamed up with the University of Cambridge to pilot an ‘in-house’ style of support. DnA was selected by the University to be the sole provider of assistive technology (AT) training for 58 international students, using a measured approach.

A Measured Approach to Support Provision and Inclusive Practice

A measured approach is important because it tells us more about the quality of the support, and whether it is actually effective and enabling for the students. Historically there has been a black hole in terms of regulating the quality of assistive technology training funded through the Disabled Students Allowance (DSA). Under the DSA, students are awarded up to £5,529 worth of specialist equipment and training is provided to show the student how to use it. However, crudely put, a race to the bottom ‘cheapest quote wins’ scenario has often dictated what kind of training the student receives. We only have metrics concerning the take-up of support, which tells us little about the quality and impact (Non-Medical Help (NMH) Take Up and Recommendations, November 2017, Student Loans Company - SLC). According to most recent data collected by Student Finance England (SFE), 33.65% students took up the Assistive Technology Training they had been awarded. These standalone figures indicate a low take-up rate and perhaps insinuate that the services are underused and therefore not needed, which could be seen to provide the Department for Education (DfE) with evidence to support reductions in funding. Whilst these figures are important to have, without any kind of quality measurement, they are potentially damaging, do little to reflect the reality of the sector, and certainly do nothing to tackle issues surrounding quality.

We can understand so much more if support services are audited for quality. Without measurement, we can’t clearly see the difference between good and bad practice, leaving our students vulnerable to the latter. Using a measured approach, best practice can be documented and shared, ‘raising the game’ in the industry. In order to heighten the student experience and indeed the retention rates of students in higher education, we must use an anticipatory approach that tells us whether the support has been effective and the student has really engaged with it. Currently, across the sector, we are unable to judge, for example, if support was tailored to the students' needs, if it was integrated with study skills, if it positively or negatively impacted the students' grades, supported their mental health, their enjoyment or confidence in studying.

If we are to ensure students receive the high-quality support they deserve and support services serve their purpose in terms of boosting attainment, retention and satisfaction whilst protecting diversity, we need to ask ourselves the following questions:

1. Is the support delivered using a measured or tokenistic approach?
2. How do we measure impact?
3. How can we act on our findings to improve support services and inclusive practices

The Pilot

International students are not eligible to apply for the Disabled Students Allowance, so the University of Cambridge ensured a budget was made available to pay for support they might need, through their International Disabled Students’ Fund. This fund is designed to provide international students with equivalent support to that available to their disabled UK peers. DnA was selected to provide the assistive technology training due to their track record of providing The University of Cambridge, amongst others, with quality assurance demonstrating that support is measured, relevant and enabling.

Through the integrated practice of goal setting, student feedback and Individual Learning Reviews (ILR’s), DnA provides a level of transparency that shows what happens in each support session and the impact this has on the students learning experience. To summarise, DnA is able to demonstrate to the University:

• what software had been trained on in each session
• how study skills were integrated, ensuring a strategy focused approach
• how goal setting and metacognition were embedded
• the impact of the support scored against personalised goals
• technical issues encountered
• the student’s shift in confidence
• how tailored the support was to the student's needs
• whether the student enjoyed their training

These quality measures are important to the University as it enables them to have direct insight and control over the efficacy of the support delivered to their students. This, in turn, allows them to maintain their service level, student satisfaction and reputation. Whilst the last decade has seen a huge increase in disabled students entering higher education, due to changes, the legacy of the next 5 years will be the retention and attainment of disabled students. Through measuring the support that is delivered to their students, the university can proactively augment these figures and implement an anticipatory approach to inclusion, ensuring all has been done to support their students to achieve their potential.

There is also a strong business case in support of a measured approach to inclusive practice. Gathering data, numbers and metrics as opposed to anecdotal stories makes it much easier to build a case that supports investment in such services and secure ongoing funding. Ultimately, higher education institutions are businesses that need to provide accessible services to their customers (the students). The customer is paying for an education that they can actively participate in and should be able to do so regardless of their learning style. ‘Value for money’ is what businesses look for when implementing new systems, and it’s clear that this approach not only provides value in monetary terms but also reputation and client satisfaction. Money that might be spent on compensatory demands, judicial reviews and legal redress, for example, due to poor accessibility, compounded by lack of transparency and accountability over support provided, are diverted by using a proactive anticipatory approach.

In addition, there are also legal and moral drivers for providing high-quality support with measurable outcomes for students. It’s critical to ensure the institution is operating in line with the Equalities Act and that disabled students have the same access to materials, services, opportunities and premises as their non-disabled peers. From a moral perspective, the arguments for inclusion are obvious. Everyone benefits. We are all unique, think, learn and work differently- so a system that takes into account the rich diversity of the student body and values neurodivergence as an asset will be winning in the long run. Students thrive, create new innovations and nurture diverse thinking when given the space and support to do so.

The Results

Feedback was collected by DnA and Cambridge from students undertaking support through this pilot, and the data shows clear benefits of the approach taken. The pilot showed that the Cambridge/ DnA support model enabled students to access support significantly faster than accessing it through the DSA - average time spent from application to award was only 3 days (in comparison to 2-3 months spent on DSA applications). It also showed an increased take-up of support, showing higher engagement. Under this pilot, students received a high quality holistic service that actively looked to build their confidence and independence. It is also clear that general ‘student satisfaction’ increased, as well as grades achieved. This is all clearly illustrated by data and feedback collected by DnA and The University of Cambridge.

Data collected by DnA: Students were asked to give a score out of 10 on the 4 themes below:

This data was collected by asking students to fill out a brief feedback form after their first and last sessions of support with DnA. You can read the feedback form that was used here: https://diversityandability.com/community/learners/feedback/. This feedback is important as currently there is no sector-wide data reflecting the quality of support and the impact it had. Without feedback from students concerning the support they receive, making service improvements is problematic as we are missing the crucial voice of the end-user.

Data collected by the University of Cambridge: students asked to rate their experience on the 4 themes below:

This data set compares student satisfaction of those who received DSA funded support and those who received Cambridge funded support, collected by the Disability Resource Centre at the University of Cambridge. The data shows that student satisfaction was consistently higher when receiving Cambridge funded support.

To Conclude

To summarise, a measured approach to disability support benefits both students and their universities and is not difficult to implement. Careful documentation of support provided is required in many other sectors, even within the DSA, so why not for Assistive technology Training? The longer-term benefits of measured high-quality support that equips individuals with core learning and working strategies go beyond the walls of the University. These are valuable life skills that will aid graduates to find and stay in gainful employment. A longitudinal approach to inclusion in this sense is so much more than supporting a student through academia - it will serve them beyond, in the world of work in a sustained way. Considering digital literacy is a hugely important skill set in today's digital age, it’s all the more important to ensure our students are using it effectively, and we can only make sure of this if we have transparency and enthusiasm to make inclusion fair and equitable for all.

To keep up to date with DnA’s and the DRC’s comings and goings, you can follow them on Twitter- @dna_matters and @Cambridge_DRC