More students are going to French universities than ever before. Over the past decade, registration for degree courses in healthcare and law has risen by 30% and 18% respectively. Encouraging success for such a large number of students from diverse educational backgrounds represents a major challenge for the French education system.
Faced with this situation, the French government adopted a number of policies to increase the proportion of students completing their undergraduate studies. Many seek to enter degree courses they are ill prepared for, paving the way to low performance, discouragement, and subsequent drop-out. A majority of those failing the final examinations of their first year (about 50 % of each entering cohort), are likely to do so due to a mismatch between course requirements and their skills.
Free advice
The 2009 Active Orientation (AO) policy informs high-school students about their chances of success in the degree courses they intend to apply to. Most universities elect to give students written feedback on the quality of the match to their chosen degree course, given their grades and motivation letter. As these recommendations are not binding, it is important for universities and policymakers to assess whether prospective students take this feedback into account when deciding where to enroll, particularly in the case of the weakest students who run the highest risk of failure.
In their paper, Nina and Nicolas focus on students who are encouraged to reconsider their enrollment choices because their skills, notably in mathematics and abstract reasoning, may not be sufficient to complete the degree. The data they use stem from the departments of a large French university, one of which elected to give all prospective students the type of feedback described above, while the other departments do not. The researchers take advantage of this natural experiment to compare enrollment rates for the different departments before and after implementation of the feedback policy, thus obtaining an estimate of the policy’s causal effect.
Changing course
The researchers find that receiving negative feedback reduces the proportion of students enrolling for the degree course by 7 percentage points. This drop is a sizeable effect when compared to an average enrollment probability of 35 per cent before implementation of the policy. This indicates that the academically weakest applicants, who may also be the least well informed about course requirements, reconsider their choices when advised against enrolling. In other words, the AO policy does seem to help attenuate mismatches between students and degree courses.
The deterrence effect of receiving negative feedback varies in size among the relevant degree courses and is not always significant at the 10 percent level. The effect is greater for students living in the same region as the university, and for those whose chosen specialization in high-school is less relevant to the subject of the intended degree course.
Different approaches may lead to sizeable differences in effect sizes. A 2016 companion study by Nicolas uses the same data but takes advantage of a threshold grade in mathematics below which university staff are expected to indicate their reservations about the student’s choice. This allows him to compare the decisions of students whose grades are just below the threshold to those with grades just above it. Apart from the advice they receive, these two groups are very similar. In this setting, negative feedback diminishes the probability of enrollment by about 14 percentage points. The difference between the two studies’ results suggests that students close to the threshold are the most likely to modify their decision after negative feedback.
Thinking ahead
Nina and Nicolas aim to conduct a closer examination of the effectiveness of the feedback policy, looking at the destination of students who change their mind. They also want to investigate the possibility of changes in drop-out rates after the first year, in mean grades in first-year exams, or in graduation rates after three years. To evaluate the AO policy, it is crucial to find out if it helps students achieve better results, and if those students who ignore negative feedback do worse in their future studies.
Extract of the TSE Mag#18 Hiver 2018
