STEER leads institutional research and evaluation (contributing evidence to strategic objectives) and externally funded research. Our externally funded research has an impact on national policy and also brings knowledge into Hallam to enhance institutional policy and practice.
Evaluation of the National Mixed Methods Learning Gain Project
This evaluation contributed to an exploration of various methodologies for measuring learning gain in further and higher education. The National Mixed Methods Learning Gain Project (NMMLGP) aimed to use an online assessment programme that included a combination of test and self-reporting survey questions which sought to evaluate students’:
- critical thinking and problem-solving skills
- attitudes towards their study experience
- engagement with their studies.
The project was initiated by HEFCE in 2016, and was due to finish in the academic year 2019-20. It was reviewed in late 2018 by OFS, who took it over in April 2018 following HEFCE’s closure. On the advice offered by the STEER evaluation team, OFS decided not to progress with the third round of data collection and closed the project. Further work was commissioned to seek student perceptions of learning gain.
Working with students researcher was an important part of this project. You can read the reflections from John Adesola and Adam Weston on this blog, which accompanies a summary piece “Lessons in measuring ‘learning gain’: One size fits none” .
Jones-Devitt, S. Pickering, N, Austen, L. Donnelly, A. Adesola, J. & Weston, A. (2019) Evaluation of the National Mixed Methods Learning Gain Project (NMMLGP) and Student Perceptions of Learning Gain, Bristol: Office for Students, available at https://www.officeforstudents.org.uk/publications/evaluation-of-nmmlgp/
Office for Students National Conference on Learning Gain (March 2019) https://www.officeforstudents.org.uk/news-blog-and-events/events/ofs-national-conference-on-learning-gain/
Observing the observers: Using digital storytelling for organisational development concerning ‘Critical Whiteness’
Funded by the Leadership Foundation for Higher Education Small Development Project grant 2018.
This project aims to test the utility of digital storytelling as a) an intervention for mediating difficult conversations about positive cultural and behavioural change, b) a method of data collection, and c) as an innovative way of sharing evidence and expertise. The focus of the project is the discussion of ‘Critical Whiteness’ within organisational development. Previous research by the project leaders concluded that the absence of Critical Whiteness discussions may be creating a barrier to addressing the Black and Minority Ethnic (BME) attainment gap through curriculum change. We believe that approaches to address implicit bias often focus on individual rather than organisational responsibility for change. Thus, Critical Whiteness is an important and often overlooked area of positive cultural and behavioural change within organisations. This development project will collect and analyse data from one Higher Education Provider and develop a Digital Practice Guide to support the development of further work across the sector. The project runs until July 2018.
Austen, L. & Jones-Devitt , S. (2018) Observing the observers: Using digital storytelling for organisational development concerning ‘critical Whiteness’, York: Advance-HE, available at https://www.lfhe.ac.uk/en/research-resources/research-hub/small-development-projects/sdp2018/sheffield-hallam-university-po.cfm
Austen, L. & Jones-Devitt , S. (2018) How ‘Comfortable’ is Your Organisation in Having Difficult Conversations?, AdvanceHE, available at https://www.advance-he.ac.uk/news-and-views/organisations-difficult-conversations
AdvanceHE Towards Embedding EDI in the Curriculum: Collaborative Network Meeting’ (march 2019)
NEON BME Learning Working Group (December 2018) Observing the observers: Using digital storytelling for organisational development concerning ‘critical Whiteness’
Austen, L. & Jones-Devitt , S.(2019) Are you sitting comfortably? Then we shall begin. Exploring the discomfort of discussing critical Whiteness within higher education.
Evaluation of the National Teaching Fellowship Scheme
Funded by the Office for Students (formerly HEFCE), this report evaluates the impact and relevance of the NTFS across the higher education sector. This research was co-led by Dr Liz Austen (STEER) and Professor Colin McCaig (CDARE) with significant work from Alan Donnelly (STEER) and Dr Christine O’Leary (SHU/SBS). Data was collected from a range of stakeholders between August and November 2017. The report found that the NTFS was valued across the sector and made recommendations for increased impact within and beyond institutions.
The final report was published in August 2018 and can be accessed here.
Austen, L. Donnelly, A. McCaig, C. and O’Leary, (2018) Evaluation of the National Teaching Fellowship Scheme: Final Report, Bristol: Office for Students, available at https://www.officeforstudents.org.uk/media/914c7e2b-0879-4531-a2c5-585b7d8b2d3a/ofs2018_ntfseval.pdf
Use and Abuse of the Student Voice: Leaders’ Responsibilities for Making Positive Use of Student Evaluations of Teaching in Higher Education
Funded by the Leadership Foundation for Higher Education Small Development Project grant 2017-2018.
‘The Use and Abuse of the Student Voice’ investigates the challenges for academic leaders created by the need to develop education in the context of evaluation of teaching by students. Emerging findings from the project were shared at a national workshop on 20th October 2017 at Sheffield Hallam University. The workshop aimed to help colleagues to make best use of the student voice to recognise and support teaching excellence. The workshop participants’ feedback will also be used to further define and develop the project’s leadership materials. The full report has been published by Advance HE and can be found here.
Jones-Devitt , S. & LeBihan, J. (2018) Use and Abuse of the Student Voice: Leaders’ Responsibilities for Making Positive Use of Student Evaluations of Teaching in Higher Education, York: AdvanceHE, available at https://www.lfhe.ac.uk/en/research-resources/publications-hub/index.cfm/SDP2017-05
The project set out to achieve: gaining evidence-based insights into the use of co-design and peer-learning as conduits of confidence-building and belonging of BME students; developing a scalable approach to building confidence and fostering belonging of all students; raising awareness of the need to think differently about explanations of BME underachievement. The team found that the biggest barrier concerned staff engagement as – no matter how much evidence was presented – other facets of institutional provision were identified as having priority, resulting in inertia. Considerable emotional labour was expended in trying to elicit change within a resistant culture. Despite this, there have been some very positive developments and enlightening lessons.
Austen, L, Heaton, C, Jones-Devitt, S. Pickering, N. ( 2017) Why is the BME attainment gap such a wicked problem? The Journal of Educational Innovation, Partnership and Change 3 (2) available at https://journals.gre.ac.uk/index.php/studentchangeagents/article/view/587
Jones-Devitt, S., Austen, L., Chitwood, L., Donnelly, A., Fearn, C., Heaton, C., Latham, G., LeBihan, J., Middleton, A., Morgan, M., Parkin, H. J. and Pickering, N. (2017) Creation and Confidence: BME students as academic partners….but where were the staff? The Journal of Educational Innovation, Partnership and Change 3 (2), available at https://journals.gre.ac.uk/index.php/studentchangeagents/article/view/580
Liz Austen #LTHEChat Wednesday 7th March 8-9PM (GMT) “Can we talk about Whiteness?”
Austen, L. and Jones-Devitt, S. (2017) Challenging wicked problems and folk pedagogies to address the BME attainment gap in Higher Education, Equality Challenge Unit, 8th December 2017
Austen, L. and Pickering, N. (2017) Creation and Confidence: BME students as academic partners at Sheffield Hallam University, React Conference Reacting to the ‘Hard to Reach’ Through Student Engagement Initiatives, 2nd May 2017
Austen, L and Heaton, C. (2016) The wicked problem of BME student attainment, Closing the gap: Research and Practice on Black and Minority Ethnic Student Attainment in Higher Education, , University of Kent, Monday 27th June 2016
Dunne, L. Lowe, T. Simns, S. Jones-Devitt, S. Chitwood, L, Morgan, M. Le, T (2016) The REACT Project: Perspectives on enhancing inclusive approaches to Student Engagement, JISC Change Agents Network, 2nd June 2016
Equalities Objectives Launch Event – Stall (February 2018)
Funded by the Quality Assurance Agency for Higher Education.
Jones-Devitt, S. Austen, L. Irwin, B. McDonald, K. Parkin, H. J. (2017) Are all students ‘hard to reach’ in a digital higher education (H.E.) context? The Journal of Educational Innovation, Partnership and Change 3 (2), available at https://journals.gre.ac.uk/index.php/studentchangeagents/article/view/526
Jones-Devitt, S., Austen, L., Parkin, H. J. (2017) Integrative Reviewing: a flexible research process for exploring complex phenomena, Social Research Update, Surrey: University of Surrey, available at http://sru.soc.surrey.ac.uk/
Austen, L. Parkin, H. J., Jones-Devitt, S., McDonald, K. and Irwin, B. (2016) Digital capability and teaching excellence: an integrative review exploring what infrastructure and strategies are necessary to support effective use of technology enabled learning (TEL). QAA Subscriber Research Series. Gloucester: QAA, available at www.qaa.ac.uk/publications/information-and-guidance/publication?PubID=3115
Austen, L. Parkin, H.J., and Jones-Devitt, S. (2017) How can digital capability promote teaching excellence? Exploring guidelines for digitally-capable teaching excellence, Higher Education Academy Annual Conference, 5th July 2017.
Parkin, H.J. and Austen, L. (2016) Digital capability and teaching excellence: an integrative review exploring what infrastructure and strategies are necessary to support effective use of technology enabled learning (TEL), Heads of E Learning Forum, 8th November 2016
Parkin, H.J. and Austen, L. (2016) Digital capability and teaching excellence: an integrative review exploring what infrastructure and strategies are necessary to support effective use of technology enabled learning (TEL), JISC Student Experience Group Meeting, 12th October 2016
Images attributed to funding/sponsoring body