Virtual Internships Step 7 - Programme Evaluation & Intern Appraisal

Levels 2 & 3: Learning and Learning Transfer & Application

Levels two and three in this model of evaluation relate primarily to learning transfer and application and given the prioritisation on learning varied from organisation to organisation, it was not surprising that the focus on these metrics was mixed.  Indeed, learning and transfer or application of learning to the job were also not formally assessed in the majority of organisations. (See the ideas in practice box above as an example of an effective approach.) This is an important level of evaluation for organisations where a key programme objective was to develop interns¡¯ competence and to build their work-readiness. However, the reality was that many organisations measured outcomes at this level in a more ad-hoc way.  For example, feedback provided by immediate managers and project leaders represents a form of evaluation of intern¡¯s performance in the in-company projects. Equally, feedback from panels observing end of programme internship project presentations represents a form of evaluation with regard to their internship project.

Formalising such measures does seem like a useful initiative, as a number of managers spoke about the importance of the rate of development evident in interns as key in hiring decisions at the end of the programme. A number of organisations also tracked engagement throughout the programme. For example, to what extent did interns engage with online resources, attend social and other events. These represented an important pulse check on engagement with the programme.

Level 4: ROI

Evaluation of ROI was generally limited. For most organisations, the objective of the programme was to use the internship programme to create a talent pipeline. Many organisations used conversion rates of interns to graduate hires as a key metric in terms of measuring the ROI of the programme.  For many organisations target conversion rates were as high as 50-70%. A number of organisations also tracked diversity and inclusion metrics in terms of the conversion rates for different demographic groups.

Some tensions between perceived programme objectives and the more suitable evaluation approaches did emerge. In one organisation the evaluation survey distributed to intern managers focused wholistically on the programme. However, the line manager we interviewed perceived that the feedback form did not reflect the priority of the programme for managers on the ground, which was to attract and retain suitable talent with specific competencies. This highlighted the importance of developing a shared understating of the objectives amongst key stakeholders in the organisation.

Although not raised as a KPI in any organisation, we suggest that organisations should track longer-term career outcomes of interns versus other routes of entry to the graduate programme. How do their careers play out relative to other hires for example? These longer-term outcomes would provide a very useful metric for organisations to track.

Intern Performance

Similar to programme evaluation, the approaches taken to appraising the interns were generally focused around gathering feedback from their line manager with regard to their daily work and work-related projects. The added importance of the line manager in virtual programmes was called out again here in terms of their role of engaging more actively and regularly in tracking performance with the intern due to the remote nature of working. This varied from informal check-in¡¯s, which was the more common approach,  to a few organisations that had a more formal process, often driven more by the manager than HR.

¡°[the intern] doesn¡¯t have formal goals updated on to our performance management system. However, as a manager, we created a document to list out his goals and we manage that as very much an organic document and the way I see it is this will be a really useful tool when he comes to pull together his CV so he can articulate really clearly what his goal was, how he met it and key outcomes, etcetera. We¡¯re just trying to set him up as best we can for life after his college course.¡±

One manager who did not have a formal approach for managing the intern¡¯s performance, did call out to the intern the expectations, deliverables and targets that would be expected of a graduate on the organisations graduate programme and used this as a means to motivate them to strive towards those goals during their internship.

The intern specific projects and related presentations also provided opportunity for HR, L&D, and panels of judges to evaluate the projects, the presentations, and the interns. Depending on the nature of the projects, performance of interns could be evaluated and compared here if the project had very precise measurable outcomes e.g. selecting the best stocks.

In contrast to how organisations approach the performance management of full-time employees, interns were generally not set and managed against precise performance or indeed learning and development objectives. One organisation did however set performance expectations for the intern which were the ¡°same as other employees but toned down a little¡±. Interns were evaluated against key performance indicators around collaboration, development, growth in skillset, impact of their work and making connections. Another organisation set and tracked interns against formal performance objectives related to their role. A food organisation tracked performance, learning and knowledge transfer and it was in the remit of the line manager to set these objectives from the outset with the intern and review progress every 3 months. The more formal performance management of interns was considered useful to decisions about recruiting the intern for future positions and indeed fast-tracking this recruitment process. The decision on whether or not to set more formal objectives and goals depends on the objectives of the internship programme. If the programme objectives include intern performance and satisfaction with their jobs, then academic evidence consistently finds that setting specific, challenging goals improves performance substantially[2]. Combining this with feedback improves performance further still.

¡°Yeah, so we would actually ¡­ We would measure them the same way that we would measure across the organisation obviously at a more junior level, but all our measurement would be of our behaviours. So the ¡°for the customer¡±, create the future, experiment and learn fast, and get it done together which is great for us because we're really measuring them on how they've performed in their role against our culture in terms of the collaboration piece. And it's not about just what they've done. It's about how they've done it as well, would be the key performance indicators for us. So we would gather that feedback near the end of the programme because that would then inform the decision to direct to offer for the likes of our grad programme¡±. 

[1] Garavan, T.N., McCarthy, A., Sheehan, M. et al., (2020), Training and Organisational Performance: A Meta-Analysis of temporal, institutional and organisational context moderators. Human Resource Management Journal, 1-26.

Phillips, J.J. & Phillips, P.P. (2011). Measuring ROI in Learning and Development: Case Studies from Global Organizations. Alexandria, VA: ASTD

[2] Centre for Evidence Based Management (2016) Rapid evidence assessment of the research literature on the effect of goal setting on workplace performance, CIPD.

[3] Garavan, T.N., Hogan, C., Cahir-O¡¯Donnell, A., Gubbins, C. (2020), Learning & Development in Organisations: Strategy, Evidence and Practice, Oaktree Press. Chapter 13.

[4] Blume, B.D., Ford, J.K., Baldwin, T.T & Huang, J.L. (2010), Transfer of Training: A META-Analytic Review, Journal of Management, 36 (4), 1065-1105.