The Impact of ExamReaders in Exams for Students with EAL, SEN or Low Reading Ages

Introduction
The aim of this project was to identify to what extent ExamReader Pens are able to support EAL, SEND and students with low reading ages in their GCSE exams. EAL students with identified SEND are at the greatest risk in regards to low attainment; when compared to EAL student with no recorded SEND, those with School Action, School Action Plus and statements are 16, 24 and 40 NC months behind their counterparts (Strand, et al. 2015). Students with both EAL and SEND are denied access arrangements (for example readers and extra time) during public exams due to their EAL status. However, they are able to make use of ExamReader Pens (for example C-Pen ExamReader) which have been approved by the Joint Council for Qualifications (JCQ) for use in exams without any special access arrangements. 

Design 
The study took the form of a quantitative experimental methodology which compared the attainment of a control and test group; followed by a small qualitative questionnaire which focused on student perceptions of the support provided by the ExamReader. There was an initial small scale pilot study (16 students – year 9) followed by the larger study (46 students – year 11). In the pilot study the groups were drawn from a population consisting of year 9 (KS4) computer science students who meet either the EAL, SEND or low reading age criteria. The identified 16 students were paired based on either their SEND status (K or statement), EAL status (fluent or emerging) or reading age (12 or below). From this paired population the students were randomly divided into two groups of 8; test group and control group.
In the larger study a test group of 23 students was drawn from the population of year 11 science students (all of the students in year 11); every student in this population who met either the EAL, SEND or low reading age criteria was included in the test group. This was done to ensure that students (known to have EAL, SEND or low reading ages) taking their GCSE exams would not be unfairly disadvantaged by the study (i.e. denied access to a digital technology which it was believed would improve their assessment grades). Instead the 23 test students were randomly matched with 23 students with a similar prior attainment and gender (without EAL, SEND or low reading ages) to make up the control group. 

Intervention 
In the pilot study both the test and control groups underwent a series of three short (including a baseline) computer science assessments. The year 9 students in the test group had access to an ExamReader, the students in the control group did not. Neither group had access to an ExamReader during the baseline assessment.
In the larger study both groups again underwent a series of three science assessments (mock, mock 2 and actual exam). The students in the test group had access to an ExamReader, the students in the control group did not. Again neither group had access to an ExamReader during the baseline test (mock 1). A qualitative questionnaire made up of Likert scale and open questions was used to collect the year 11 test group students’ perceptions about the ExamReaders following their actual GCSE exam. 

Outcome Measures
In the pilot study the students’ scores from the two assessments were compared to their baseline scores (Table 1). The assessment data suggested that the difference between the test and control groups had a positive effect size of 0.17 (first assessment – low effect size – approximately 2 months’ progress) and 0.38 (second assessment – moderate effect size – approximately 5 months’ progress). 

Computer Science 

                           Test          Test 2         Baseline         Diff 1      Diff 2 
 Test                    3.0            4.9                2.7               0.4           2.7 
 Control              5.0            6.6                4.6               0.0           2.0 
 Diff                   -2.0          -1.7               -1.9               0.4           0.7 
 SD                    3.46           2.17               3.34            2.51          1.73 
 Effect Size      -0.58          -0.79             -0.57            0.17          0.38 


                        Table 1: Year 9 Assessment Data Analysis 

In the larger study students’ scores from the two assessments (mock 2 and actual exam) and the baseline assessment (mock 1) where compared for the control and test groups students’ scores (Table 2). The assessment data again suggests that the difference between the test and control group had a positive effect size of 1.03 (mock 2 – very high effect size – approximately 12 months’ progress), 0.12 (actual exam – low effect size – approximately 2 months’ progress) and 0 (mock baseline – no effect – approximately 0 months’ progress). 


Science 
                       Baseline       Mock2         Actual   
Test                   1.26            1.74              2.74 
Control              1.26            0.85             2.59 
Diff                    0.00           0.89             0.15 
SDev                  1.08           0.87            1.29 
Effect size          0.00          1.03             0.12 


                    Table 2: Year 11 Assessment Data Analysis

Table 3 shows the questionnaire results where 41% of the 23 (test group) year 11 students used the ExamReaders in all or most of their exams; 36% used the ExamReaders in some of their exams. 71% of the students who made use of the ExamReaders during their exams agreed or strongly agreed that they helped them to understand more questions. 59% of these students agreed or strongly agreed that they helped them to attempt more questions. The remaining students in each instance neither agreed nor disagreed. 

                                                    All             Most            Some          None
 ExamReader use in Exam           2                 7                    8                5 
                                                     9%             32%              36%           23% 

                                                 Strongly Agree   Agree        Neither agree nor disagree        Disagree        Strongly Disagree 
Understand more questions              1                     11                      5                                         0                         0
                                                         6%                  65%                 29%                                      0                         0
Attempt more questions                   3                       7                       7                                         0                         0
                                                       18%                  41%                  41%                                     0                          0



Table 3: Year 11 Questionnaire Data Analysis 

The students were very positive about the ExamReaders in the open question section of the questionnaire describing them as: "A good piece of technology that can help a lot of people”; “They are brilliant for words you don't understand"; "It was good, the school should use them again for the next year coming up"; "Alright I believe that the ExamReader helped a lot in the exam and I am happy I get to use it in my exam"; "It helped me read the words I didn’t understand"; and "It helps to understand words your (sic) not familiar with". 

Results
In the pilot study the students in the test group (using ExamReader) made more progress from the baseline test to the first test (0.4 marks) than the control group (0 marks); and from the baseline test to the second test (2.7 marks) than the control group (2.0 marks). These results and the data outlined in the previous section seem to suggest that the ExamReaders were able to support and have a positive effect on attainment for EAL, SEND, and students with low reading ages. 

In the larger study the students in the test group (using the ExamReaders) made more progress in their second mock exam (0.48 grades on average) than in their first (baseline) mock exam; compared to the control group who made less progress (-0.41 grades on average). There was a difference of nearly 1 grade (0.89 on average), with the test group outperforming the control group. The test group made more progress in their actual GCSE exam (1.48 grades on average) than in their first mock exam; compared to the control group who made slightly less progress (1.33 grades on average). There was a slight difference of 0.15 grades on average. Although, this difference seems slight, it is important to note that none of the students in the control group have EAL, SEND or low reading ages. Therefore, any progress made by the test group towards matching the grades of the control group is very positive. The ExamReaders appear to have ensured the academic progress of vulnerable (EAL, SEND and low reading ages) students.

Percentage progress from Mock to Actual - Control 
 1         2          3           4           5 
 1         7          7           5           1 
 4%     30%     30%     22%     4%

Percentage progress from Mock to Actual - Test 
1             2              3              4           5           6            7           8 
1             3              5              9           2           0            0           1 
4%     13%            22%        39%       9%        0%          0%       4% 

Table 4: Year 11 Percentage Progress 

Table 4 notes that in the control group 30% improved by 3 grades; 22% improved by 4 grades; and 4% improved by 5 grades between their mock and actual exam. In the test group 22% of the students improved by 3 grades; 39% improved by 4 grades; and 9% by 5 grades between their mock and actual exam. In general, the students in the test group
with access to the ExamReaders made higher levels of progress. 


Conclusions 
The results of this study seem to support similar research from Garner Education Services Ltd (2016) which concluded that ExamReaders can support dyslexic students in achieving an improved grade in their GCSE English Reading paper; (5 out of the 6 students, who undertook the paper with the assistance of the reader pens, achieved a real increase in their results) whilst also improving emotional well-being, improved confidence and attitude to learning. The results of the Higgins and Raskind (2005) study indicated significant and moderate gains in performance when students were able to use the ExamReader.
It has been suggested that characteristics associated with SEND may block a student’s access to the content of the test (Thurlow, et al., 2009), with the result being that students are not able to show their knowledge and skills simply because the assessment itself has created a barrier to doing so (Thurlow, et al., 2010). This study indicates that ExamReaders can benefit students with EAL, SEND and low reading ages by providing them with a means to independently undertake exams. The students can take an exam in the same room as their peers by plugging in the headphones. Furthermore, the ExamReader can be used during public exams (GCSE’s) without any special access arrangements and can provide support to all EAL/SEND students. It is important that students are given time to familiarise themselves with the reader pen technology (Thurlow, et al., 2010). 

References 
Strand, S., Malmberg, L. & Hall, J. (2015) English as an Additional Language (EAL) and Educational Achievement in England: An Analysis of the National Pupil Database. 

Garner Education Services Ltd. (2016) The C Pen Exam Reader – A Quantitative and Qualitative Study. 

Higgins, E. & Raskind, M. (2005). The Compensatory Effectiveness of the Quictionary Reading Pen II on the Reading Comprehension of Students with Learning Disabilities. Journal of Special Education Technology, 20(1), p.31-40. 

Thurlow, M., Moen, R., Lekwa, A. & Scullin, S. (2010) Examination of a Reading Pen as a Partial Auditory Accommodation for Reading Assessment. 

Thurlow, M., Laitusis, C., Dillon, D., Cook, L., Moen, R., Abedi, J., & O’Brien, D. (2009) Accessibility principles for reading assessments. Minneapolis, MN: National Accessible Reading Assessment Projects.