Quality assurance tools

When developing and discussing education-related policies, it is important to have a good overview of relevant quantitative parameters. The fact sheet is a tool that provides a better picture of a number of important statistics relating to intake, student population, study progress. These statistics serve to contextualise discussions and planning around these issues, as well as to evaluate previously implemented actions. It is important to always consider these data in the context of the specific study programme.

The quality assurance staff can provide support in contextualising the figures and additional statistics can be used to answer more specific questions about numerical trends and developments within the study programme.

The annual fact sheet consists at least of following data:

Intake

• The absolute number of first-time and non-first-time students
• The ratio of Belgian to foreign students
• Gender distribution (M/F)
• Prior schooling (only Belgian degrees)

Student population

• The absolute number of Belgian, EEA, and non-EEA students
• Student participation: number of students obtaining 0 ECTS vs 1 or more ECTS of their registered credits.
• Median age
• Gender distribution (M/F)

Study progress

• Average study efficiency compared to the faculty and university average
• Drop-outs against faculty and university average
• The destination of the drop-out students: do they start a new programme within the same faculty, within VUB or do they leave VUB entirely.
• Average study time against faculty and university average

A coherent and well thought-out curriculum is, of course, more than the sum of its parts. A very suitable instrument for mapping the interrelationship of the different programme components is the study programme matrix.

With the help of a programme matrix, the course units of the study programme can be linked to (among others) the learning outcomes, work and evaluation forms. Especially the reflection of the course council on the overview offered by the programme matrix is relevant for the quality development of the study programme. The matrix provides an overview of the existing situation and its interrelationships, and thus forms the starting point for further analysis, vision development, coordination, and policy.

The programme matrix can be tailored to specific needs and situations of the programmes. The overview makes it possible to get a clear idea of e.g. (the consistency of) following elements:

• A link between the programme-specific learning outcomes and the domain-specific learning outcomes;
• A competence matrix, linking the programme components to the programme-specific learning outcomes;
• A link between the course units and the teaching methods used in these course units;
• A link between the course units and the way in which these course units are evaluated. It also differentiates between formative and summative evaluation.

Depending on the programme's priorities, other elements can be investigated, such as:

• Internationalisation, which can be used to map out the activities surrounding internationalisation of the curriculum;
• Workload, to identify the workload of students per week;
• Study material.

Because bachelor's and master's programmes have different learning outcomes, in principle, a separate programme matrix is filled in for each bachelor's and master's programme.Also for bridging programmes, preparatory programmes and programmes for working students completing a matrix can generate relevant insights.

The quality assurance staff can advice during this exercise and the interpretation of its results.

As key participants in our education students play an essential role when it comes to monitoring the quality of that education. One of the ways in which they do this is to provide feedback on the study programme through regularly organised focus group discussions.

The aim of the student focus groups is to identify strengths and possible areas of improvement of a specific study programme. The focus discussion provides more qualitative input that can complement the rather quantitative results of the student feedback. It allows for more depth and students have the opportunity to nuance their answers and help determine priorities. In addition, work is always tailored to the study programme, since the topics and questions of the focus discussion are determined in consultation with the course council.

A small group of students, usually between four and ten, take part in the discussion. Some representativeness within the participating students is sought, for example, a minimum representation for each year of study. Unlike student feedback, the focus here is not on individual course units, but on the level of the study programme. It is possible, however, that certain subjects are dealt with specifically, at the request of the study programme or at the initiative of the participating students.

The quality assurance officers can moderate these discussions and can draw up a descriptive and action-oriented report afterwards. The results are discussed by the course council, possibly leading, for example, to strategic goals being supplemented, to the profile of the study programme being sharpened on the basis of the input provided or to expectations being adjusted.

Because students are important stakeholders in education, they are explicitly asked for their opinion on that education. One of the quality assurance instruments used to do this is student feedback. In principle all bachelor's and master's programmes (coordinated by VUB), bridging and preparation programmes are evaluated annually by students through this survey.

The results provide insight into the extent to which students are satisfied with the quality of education. Student feedback can help lecturers, course councils and the institution to reflect on the quality of education, detect and adjust any problems, and further optimise the education.

Student feedback consists of several components:

• a questionnaire per course unit;
• a questionnaire about the study time experience;
• a questionnaire assessing satisfaction with the study programme, and the generic competences acquired during the programme;
• a questionnaire on general satisfaction with Vrije Universiteit Brussel and its central services.

The questionnaire per course unit consists of a standard questionnaire that gauges student satisfaction on five aspects: goals, content, guidance, study materials, and evaluation. In addition, students can indicate their satisfaction with the lecturer or co-tutor of the course unit. For many components students can add qualitative responses through open-ended questions.

Student feedback is organised twice a year, at the end of each semester, for course units that students have taken in the respective semester. All course units are evaluated, including internship, bachelor’s and master’s thesis. The questionnaire about the study programme is offered to bachelor's and master's students who are at the end of their study trajectory. The general satisfaction questionnaire is offered to all students at the end of the first semester.

The results of student feedback are reported after each round via the survey platform. The aggregated results are accessible to the entire VUB community. Comments from the study programme questionnaire and the general satisfaction questionnaire are accessible to the chair of the course council concerned, the dean of the faculty to which the study programme belongs and the Vice Rector for Education and Student Affairs. The results per course unit are accessible to the lecturers attached to the course unit, the chair of the course council responsible for the course unit, the dean of the faculty to which the course unit belongs and the Vice Rector for Education and Student Affairs.

The general results of student feedback are to be discussed by the course council. From a quality culture point of view, this council can also act as a forum for the exchange of good practices, by discussing individual results and actions taken - obviously with approval of the lecturers involved. 

The quality assurance officers can support the course council. Lecturers and teaching teams can also discuss individual results with an education expert from Education and Student Affairs. There is also a specific procedure for acting on reported major issues or recurring issues within the same course unit.

On a regular basis, a central survey is sent out to recent graduates. This survey provides both the institution and the course councils with information that can be used in the further development of the quality of education. The graduates' experience can shed a light on how well the programme aligns with professional practice and how smoothly graduates find their way in their careers. 

Course councils are additionally encouraged to involve alumni in their activities, besides this central survey and representation in the course council, through specific surveys, alumni events, guest lectures, interships, field visits, etc.

The alumni surveys vary over the years, but they all focus on the employment component and the relationship between the study programme and the professional field. Other aspects of the educational process are given a place in student feedback and the student focus groups.

• Study data e.g.: some factual data are collected to match with the internal data and respondents can report on their prior education, on any study delays (and their causes) and on any additional courses they may have taken after completing the surveyed study programme.
• Employment e.g.: how long after graduating did respondents find work, in which sector are they employed or are they looking for work, do they work full-time or part-time, what is they current job title,...
• Education e.g.: opinion questions are used to gauge the extent to which the study programme matches the professional career of the alumnus. This part examines, among other things, the added value of the internship and foreign study experience for the future career and asks about strengths and areas for improvement of the study programme.
• Institution e.g: what is the graduates general experience with Vrije Universiteit Brussel.
• Personal data: respondents can add some personal details and if they wish to keep in touch, which are used to improve the alumni network of Vrije Universiteit Brussel.

General results of the alumni survey are made available to the VUB community and alumni through the Education & Student Affairs website. Course councils receive a detailed report concerning their study programmes, provided the response rates are appropriate.

Study programmes can use a work field consultation to obtain input from representatives of the professional field on various aspects of the study programme. It is a way to check for example how well the study programme prepares students for certain careers or for jobs in the intended domains, and to make sure programmes act on recent developments, are clearly profiled, and train for the required competences. In addition, this consultation can result in a stronger connection between the study programme and the professional field.

A work field consultation can be executing in a variety of ways. It is up to the course council to decide where the focus of the consultation lies and which aspects are explored in more depth based on the information it needs. The survey can be specifically aimed at the professional field of one study programme or a cluster of study programmes; or it can include one very sepcific part of the professional field or even on the broadest possible representation of the entire professional field. Themes discussed are the study programme, the profiling of the programme, the final competencies of the graduates, the programme-specific learning outcomes, internships, cooperation between the study programme and the professional field, and so on.

Different methods can be used to ask about the professional field:

• Participation of professional field representatives in a programme council
• A focus discussion
• A permanent sounding board group or advisory board
• A questionnaire or email survey

The quality assurance officer can support the course council in determining the focus and themes of the survey and can also provide practical support. The internal channels offer documentation and a step-by-step guide.

In the context of educational quality, it is not only important to look internally at the study programme, but also to look outward. How do other, similar study programmes profile themselves? What specialisations are covered? How do the various players in the educational landscape distinguish themselves? The process of comparing a study programme with similar programmes at other institutions at home and abroad is called benchmarking. By looking at our (national and international) neighbours, we can learn a a great deal about our own profile, vision and identity.

A benchmarking will vary depending on the questions the course council wishes or needs to tackle. At the base the benchmarking compares one's curriculum with similar programmes at other institutions, ideally nationally and internationally. A number of relevant programmes to compare with will be selected. For some domains there are also studies or reports with recommendations, prepared by trade associations or umbrella projects (such as Tuning). It is appropriate to include them. The same applies to study programmes that are subject to professional recognition, for which specific guidelines exist.

A benchmarking can also focus on a specific aspect of the programme, such as learning outcomes, master theses, intake, study success, final attainment level, internationalisation, etc.

The quality assurance staff as well as online documentation on the internal VUB channels can provide support at various steps in the process.