Quality assurance tools

When developing and discussing education-related policies, it is important to have a good overview of relevant quantitative parameters. The fact sheet is a tool that provides a better picture of a number of important statistics relating to intake, student population, study progress, and internationalisation. These statistics serve to contextualise discussions and planning around these issues, as well as to evaluate previously implemented actions. It is important to always consider these data in the context of the specific study programme; therefore no 'targets' or 'goals' are explicitly used.

The annual fact sheet consists at least of following data:

Intake

• The absolute number of first-time and non-first-time students
• The ratio of Belgian to foreign students
• Gender distribution (M/F)
• Average age

Student population

• The absolute number of Belgian, EEA, and non-EEA students
• The ratio of day students to work students
• Gender distribution (M/F)
• The previous education of the Belgian students (ASO, BSO, KSO, TSO)

Study progress

• Average study efficiency compared to the faculty and university average
• Drop-outs against faculty and university average
• Overview of what percentage of students obtain a diploma after [x] years
• Average study time against faculty and university average

Internationalisation

• Absolute number of outgoing students
• The type of study contract
• The destination of outgoing students (over the last two years only)
• The average external number of ECTS taken and acquired

The quality assurance staff can provide support in contextualising the figures. Based on the fact sheet provided, and in consultation with the quality assurance officer, a decision can also be made about the extent to which additional statistics can be used to answer more specific questions about numerical trends and developments within the study programme.

A coherent and well thought-out curriculum is, of course, more than the sum of its parts. A very suitable instrument for mapping the interrelationship of the different programme components is the study programme matrix.

With the help of a programme matrix, the course units of the study programme can be linked to (among others) the learning outcomes, work and evaluation forms. Especially the reflection of the course council on the overview offered by the programme matrix is relevant for the quality development of the study programme. The matrix provides an overview of the existing situation and its interrelationships, and thus forms the starting point for further analysis, vision development, coordination, and policy.

The programme matrix can be tailored to specific needs and situations of the programmes. The overview makes it possible to get a clear idea of (the consistency of) following elements:

• A link between the programme-specific learning outcomes and the domain-specific learning outcomes;
• A competence matrix, linking the programme components to the programme-specific learning outcomes;
• A link between the course units and the working methods used in these course units;
• A link between the course units and the way in which these course units are evaluated. It also differentiates between formative and summative evaluation.

The analysis can be supplemented with the following elements:

• Internationalisation, which can be used to map out the activities surrounding internationalisation of the curriculum;
• Workload, to identify the workload of students per week;
• Study material.

Because bachelor's and master's programmes have different learning outcomes, in principle, a separate programme matrix is filled in for each bachelor's and master's programme. For bridging programmes, preparatory programmes and programmes for working students completing a matrix can generate relevant insights.

The quality assurance staff provide a template. They provide support in completing the matrix and interpreting the results. A number of support materials were also developed to facilitate completing the study programme matrix and the accompanying reflection. Ask your quality assurance officer about it.

Not only do students participate in education at the institution, but they play an essential role when it comes to monitoring the quality of that education. One of the ways in which they do this is to provide feedback on the study programme through regularly organised focus group discussions.

The aim of the student focus groups is to identify some of the strengths and possible areas of improvement of a specific study programme. The focus discussion provides more qualitative input that can complement the rather quantitative results of the student feedback. It allows for more depth and students have the opportunity to nuance their answers and help determine priorities. In addition, work is always tailored to the study programme, since the topics and questions of the focus discussion are determined in consultation with the course council.

A small group of students, usually between four and ten, take part in the discussion. Some representativeness within the participating students is sought, for example, a minimum representation for each year of study. Unlike student feedback, the focus here is not on individual course units, but on the level of the study programme. It is possible, however, that certain subjects are dealt with specifically, at the request of the study programme or at the initiative of the participating students.

During focus discussions, students as important stakeholders are given the opportunity to give their opinion on the quality of education, to identify points for attention and suggest improvements. The ultimate goal of focus discussions is to achieve further quality improvement through a discussion of results.

The quality assurance officer composes the group of students for the focus discussion in consultation with the programme council chairperson and, if applicable, student representatives, and defines the themes in consultation with the programme council. The quality assurance officer conducts the discussion with the students and draws up a descriptive but action-oriented report afterwards. After approval of the participating students, the report is sent to the chairperson of the responsible programme council. The latter presents the report to the members of the programme council. The members discuss the input and decide how to follow up on the issues that were discussed. This can lead, for example, to strategic goals being supplemented, to the profile of the study programme being sharpened on the basis of the input provided or to expectations being adjusted.

Because students are important stakeholders in education, they are explicitly asked for their opinion on that education. One of the quality assurance instruments used to do this is student feedback. All bachelor's and master's programmes - with the exception of inter-university programmes for which quality assurance is coordinated by a partner institution - as well as masters of education and bridging and preparation programmes - are evaluated annually by students through this survey.

The survey has a multiple purpose. The results provide insight into the extent to which students are satisfied with the quality of education. Student feedback can help lecturers, course councils and the institution to reflect on the quality of education, detect and adjust any problems, and further optimise the education. The institution also uses quantitative feedback for reporting and quality assurance purposes. Lecturers primarily use qualitative feedback for quality-improvement purposes.

Student feedback consists of several components:

• a questionnaire per course unit;
• a questionnaire about the study time experience;
• a questionnaire assessing satisfaction with the study programme, the training facilities and the generic competences acquired during the programme;
• a questionnaire on general satisfaction with the Vrije Universiteit Brussel and its central services.

The questionnaire per course unit consists of a standard questionnaire that gauges student satisfaction on five aspects: goals, content, guidance, study materials, and evaluation. In addition, satisfaction with each teacher who is a lecturer or co-tutor of the course unit is assessed. For all components of the student feedback, except the study time experience, students can use free fields to further explain their answers.

Student feedback is organised twice a year, at the end of each semester, for course units that students have taken in the respective semester. All course units are evaluated, including internship, bachelor’s and master’s thesis. The questionnaire about the study programme is offered to bachelor's and master's students at the end of the second semester. The general satisfaction questionnaire is offered to all students at the end of the first semester.

The results of student feedback are reported after each test via https://studentenfeedback.vub.ac.be/. The results of the study programme questionnaire, the general satisfaction questionnaire, the study time experience and the aggregated results of the questionnaires on course units are accessible to the entire VUB community. Comments from the study programme questionnaire and the general satisfaction questionnaire are accessible to the chair of the course council concerned, the dean of the faculty to which the study programme belongs and the Vice Rector for Education and Student Affairs. The results per course unit are accessible to the lecturers attached to the course unit, the chair of the course council responsible for the course unit, the dean of the faculty to which the course unit belongs and the Vice Rector for Education and Student Affairs.

Education and Student Affairs expects the results of student feedback to be discussed by the programme council. The quality assurance officer can support the programme council in this.  In any case, it concerns the aggregated results, the results of the questionnaire per course unit that go beyond the level of the individual lecturer or the individual course unit, the study time experience and a questionnaire in relation to the study programme. In addition, the programme council provides a forum for the exchange of good practices to the extent that these emerge from student feedback. Where the quality culture within the programme council allows this and where it makes sense, it is recommended - with the approval of the lecturers involved and with a view to quality improvement - to also discuss individual results from student feedback.

Information documents have been developed for programme councils, lecturers, and education teams that can support evaluation of the results of student feedback. Teachers and education teams can also always discuss results from student feedback with an education expert from Education and Student Affairs.

Every four to five years, Education & Student Affairs organises a central survey of alumni who have graduated in the last 1-5 years. This survey provides both the institution and the course councils involved with information that can be used in the further development of the quality of education. The previous central alumni surveys took place in academic years 2016-2017 and 2020-2021. The 2024 survey is in preparation.

Within the framework of the alumni survey, the following study programmes are included: initial master's programmes, master-after-master's programmes and specific teacher training programmes (as of the 2024 alumni survey, masters in educations are also included). Interuniversity study programmes are included if the Vrije Universiteit Brussel is designated as the coordinating institution in terms of quality assurance. Majors are not included separately, but in the survey it can be indicated which major was concerned, if applicable.

-- The survey is currently being reviewed and updated. The questionnaire of 2024 will slightly deviate from the structure mentioned below. --

The alumni surveys of 2016 and 2020 consisted of 5 parts, focusing on the employment component and the relationship between the study programme and the professional field. Other aspects of the educational process are given a place in student feedback and the student focus groups.

• Study data: Here, factual data are requested regarding the course of study of the alumnus in question. Respondents are asked to register their prior education, to indicate any study delays (and their causes) and to indicate any additional courses they may have taken after completing the surveyed study programme.
• Employment: Here you are asked to enter data on your career between the moment of graduation and the moment of the survey. Questions such as "In which sectors have you worked in the past?", "What is your current job level?", "Do you work full-time or part-time?" are addressed in this section.
• Education: Here, opinion questions are used to gauge the extent to which the study programme matches the professional career of the alumnus. This part examines, among other things, the added value of the internship and foreign study experience for the future career and assesses the strengths and weaknesses of the study programme.
• Institution Here, the relationship of the alumnus with the Vrije Universiteit Brussel as an institution is assessed.
• Personal data: These data are intended to improve the alumni work of the Vrije Universiteit Brussel and to optimise the data bank.

General results of the alumni survey are made available to the VUB community and alumni through the Education & Student Affairs website. Course councils receive a detailed report concerning their study programmes. In addition, an aggregated report is made at the institutional level.

Course councils are expected to discuss the results of the alumni survey at a course council meeting and include them in their quality assurance work. The quality assurance officer can support them in this. Course councils are encouraged to involve alumni in their activities, besides this central survey and representation in the course council, through specific surveys, alumni events, guest lectures, field visits, etc.

Study programmes can use a work field consultation to obtain input from representatives of the professional field on various aspects of the study programme. After all, the study programme prepares students for academic and non-academic careers, and the input from the work field is therefore very valuable. In addition, this consultation can result in a stronger connection between the study programme and the professional field.

The programme council decides where the focus of the questions lies and which aspects are explored in more depth based on the information it needs. The survey can be specifically aimed at the professional field of a study programme or a cluster of study programmes. Themes discussed during a professional field survey are the study programme, the profiling of the programme, the final competencies of the graduates, the programme-specific learning outcomes, internships, cooperation between the study programme and the professional field, and so on.

Different methods can be used to ask about the professional field:

• Participation of professional field representatives in a programme council
• A focus discussion
• A permanent sounding board group or advisory board
• A questionnaire or email survey

The professional field survey can also focus on a very specific part of the professional field or on the broadest possible representation of the entire professional field. To ensure that the input is not one-sided, a professional field survey is organised with at least five representatives who can offer different perspectives. The action-oriented report of this survey is discussed by the members of the programme council. They determine what actions must be taken. These can be included in the strategic goals of the programme and/or bring about adjustments in the shorter term.

The quality assurance officer can support the course council in determining the focus and themes of the survey and can also provide practical support. The internal channels offer documentation and a step-by-step guide.

In the context of educational quality, it is not only important to look internally at the study programme, but also to look externally. How do other, similar study programmes profile themselves? What specialisations are covered? How do the various players in the educational landscape distinguish themselves? By looking over the wall at our (national and international) neighbours, we can learn a lot about the education we offer.

The process of comparing a study programme with similar programmes at other institutions at home and abroad is called benchmarking. The results of a benchmarking exercise can be used in profiling the study programme and can contribute to a well-founded review of the programme.

The course council takes the initiative for benchmarking. It determines the number of study programmes to be included in the comparison and also chooses which programmes (both national and international) are relevant. For some areas there are also studies or reports with recommendations, prepared by trade associations or umbrella projects (such as Tuning). It is appropriate to include them, at least when processing the results of the benchmark exercise. The same, of course, applies to study programmes that are subject to professional recognition, for which specific guidelines exist.

Initially, (specific aspects of) the study programme are compared with similar programmes at other institutions, but course councils are free to extend the benchmarking exercise to other domains, such as learning outcomes, intake, study success, final attainment level, internationalisation, etc. The results of the benchmarking exercise are recorded in a report and discussed in the course council.

The quality assurance staff can provide support at various steps in the process. Course councils and teaching staff can find documentation, a step-by-step guide and templates for data collection and the final report on the internal channels.