Appreciation of an Alternative: Dutch Alumni Look Back on Their Science Master, a Work-Based Learning vs a Research-Oriented Approach

The Science, Business and Policy Program (SBP) of the University of Groningen has been a work-based learning program for science master students since two decades. This program, a direct consequence of the Bologna declaration of education in Europe, has a distinctive position in European science curricula. Here, we use a web-based survey (N = 242), comparing alumni perceptions on the SBP program (N = 111) with a research oriented master (ROM, N = 131). Alumni from both programs predominantly had a content-driven motivation to choose for either the ROM or SBP format. For SBP alumni, this preference was also inspired by the master's focus on society and a career outside academia. We assessed self-perceived learning outcomes using an extended version of the Science Student Skills Inventory. Both groups were positive about the academic level of their program. Scientific content knowledge and practical research skills were perceived as sufficient by both groups, although significantly higher in the ROM group. Both groups were equally positive about obtained skills in academic reasoning and would have preferred more disciplinary content knowledge. The SBP group perceived their skills on teamwork, multidisciplinary working, leadership, ethical thinking and project work to be significantly higher compared to ROM alumni. The SBP group felt significantly more prepared for a career outside academia and the ROM group for a career in academia. Relatively the ROM group missed career preparation. Both specific work-based learning and classical research approaches were seen as valid ways to obtain a high-quality, science master level, although for different career ambitions.

Keywords: work-based learning; science; Business and Policy; alumni perceptions; career; curriculum comparison

The Science, Business and Policy Program (SBP) of the University of Groningen has been a work-based learning program for science master students since two decades. This program, a direct consequence of the Bologna declaration of education in Europe, has a distinctive position in European science curricula (Grooters et al., [23]). The notion that science should focus on research only is no longer sufficient (Agre & Leshner, [2]) and also applies to educational activities of universities. Classically, in many European countries, science education is mainly focused on preparing for a research career. Research training alone however does not facilitate the much-needed connection with the labour market (Bologna Working Group, [6]). There is a call from society that education should be much more synchronised with the work field (Dutch Ministry of Education, Culture and Science, [18]) and that students need different skills to be employable beyond academia (Hart, [26]). This discrepancy between supply and demand on the labour market for higher educated science alumni (Mahbub et al., [33]) stretches a need for universities to bridge the gap between science and society (Matthews et al., [35]).

A promising solution lies in providing work experience within the academic framework. Work-based learning (WBL) is an educational form enabling this. WBL is also criticized upon (cf: Lester & Costley, [32]) but comparable information on WBL in practice is still not sufficiently available, making it unclear whether critiques can be generalized over other program types (Lester et al., [31]). In this paper, we examine perceived differences by the key players of two science programs offered at the Faculty of Science and Engineering (FSE), University of Groningen, the Netherlands: students of a research-oriented master's (ROM) program, and students of a WBL master's program, the Science, Business and Policy (SBP) program. The SBP program is a work-based learning program that is classified as a European best-case practice of WBL (WEXHE). Because this program exists for almost two decades and the FSE offers a substantial number of alumni for both the SBP and ROM-program, a comparative evaluation of programs can be done.

Our main research question is: "To what extent do ROM-alumni and WBL-alumni differ in the perception of their master's programs, specifically with regard to bridging science and society?"

More specifically, we want to know whether the WBL-program is distinctive, and qualitatively comparable to a ROM-program based on alumni perceptions; which skills alumni perceived to have acquired and missed during the master's program; to what extent alumni appreciated, and were satisfied with the program; and finally, did alumni feel prepared for the career they were heading for?

To answer those questions from a (former) student perspective, a web-based survey is conducted among alumni who graduated between 2 and 18 years ago. They answered questions comprising a long-term reflection on the content and experience of their master, using (partially) standardized questionnaire items suitable for comparison between programs.

Situating the development of a European bachelor-master structure (Bologna Working Group, [6]), a differentiation of science master curricula was initiated. For science education, this included also an alternative society-oriented program that was initiated in the Netherlands in 2002 (Hermans & Vermeend, [27]). The Faculty of Science and Engineering (FSE) of the University of Groningen developed a 60 ECTS program: SBP.

FSE students follow a 2 year 120 ECTS master program (ISCED level 7). In the first year, discipline-specific research courses are taken as well as a research thesis (of usually 30–40 ECTS). In the second year, students can choose between:

• A Research Oriented Master in which students do a second research thesis (30–40 ECTS), supplemented with an essay, a colloquium and more research courses. This route is aimed at acquiring scientific knowledge and preparing for an academic career.
• The Science, Business and Policy program in which students first do two courses with a theory part on Business and Policy and group projects for real clients in the business sector or with governmental agencies. The courses are followed by a relatively long individual work placement (six months, 40 ECTS). During this work placement outside academia the students learn on the spot to become a science advisor and apply academic knowledge to societal, (i.e.-business or policy) problems (for a more detailed description and learning outcomes: Grooters et al., [23]).

SBP can be described as a WBL-program. The educational strategy of SBP is to offer work experience within the curriculum to students at companies or (non-) governmental organisations where they can apply their academic knowledge and combine and integrate this with Business or Policy. Hereby students can work on their employability and skills required for a non-academic career.

The concept of work-based learning is explained in many different ways for many different groups. In this article, our focus is on science students in a conventional cohort of young full-time students who follow a regular study route (i.e. excluding adult learners) and we focus on the program of the last year of their master. We define WBL as: "The term 'work-based learning' logically refers to all and any learning that is situated in the workplace or arises directly out of workplace concerns." (Lester & Costley, [32], 562).

The number of work-based learning programs offered at universities is rapidly increasing, which is not surprising given the multiple positive influences of WBL. First, WBL-programs have shown to effectively foster students' personal, professional and academic development. WBL students gain—next to the regular academic theoretical development—more practical expertise and skills and reported increased levels of confidence, competence, the ability to reflect and understand, as well as an ongoing desire to learn and develop (Costley & Stephenson, [15]; Stephenson & Saxton, [41]).

Second, WBL-programs can have distinct benefits for organizations that offer work placements. To start, the work placement project itself brings value when the WBL-student and WBL-organization match well. The work placement project adds knowledge to the organization's intellectual capital (Garnett, [21]) and students' input can lead to organizational changes at several levels of the organization. Work practices may be changed, and new business directions can be identified (Costley & Stephenson, [15]). Furthermore, these organizations might also be the first potential employers after a successful work placement. For employers in general, the WBL-programs seem to deliver suitable job candidates. Employers of small and medium enterprises seem to value WBL-programs and on the job experiences more compared to classical educational programs (Burgoyne et al., [10]).

From an academic educational point of view, an important argument for WBL is providing education that optimizes career perspectives and enables students to keep up with requirements from the work field. Modern education can thereby focus on the ability to train students in dealing with the ongoing changing environment, the enormous pace of technological change and complex context, as stressed important in literature (Barnett, [4]; Brew, [8])

There are also a number of critiques concerning offering WBL at science education on a master's level. First, WBL in general is sometimes considered a non-academic practice. In the Netherlands, gaining experience during a placement is seen as typical for applied education (bachelors, ISCED level 5) and not as master's level (ISCED level 7) education. Critics mention WBL's absence of a proper curriculum, a lack of theoretical exams, written assignments and academic specialization (Lester & Costley, [32]).

Second, by transferring part of the academic education to a place outside academia you also transfer control over the curriculum and learning outcomes, which might not be desirable (Powell & Walsh, [39]). Adding to this, there are assumed uncertainties about learning objectives (Saunders, [40]) and academic skills in WBL, that are unclearly formulated (Hughes et al., [28]) and poorly assessed (Brodie & Irving, [9]).

The third point of critique is perhaps more specific for science education. From literature, we know that science programs usually tend to value scientific content knowledge over skills required to apply the content knowledge (Wieman, [42]; Wood, [44]) and therefore it may not be surprising that large universities seem to be resistant to educational change within science education. (Anderson et al., [3]; Wieman et al., [43]) Lastly, it seems difficult to organize WBL in practice in science education in such a way that it creates a sustainable difference. In literature, multiple large projects funded by the government are described. These projects were in the United States (Dancy & Henderson, [16]), the United Kingdom (Grove, [24]) and Australia (Gannaway & Hinton, [20]), respectively. They all had the aim to reform science curricula, but did not create a lasting persistent change. Thus, although, there is a noticeable amount of evidence that WBL-programs can be an effective way of educating and are in fact a rather valued educational form, some critiques given may very well be real issues. In this current evaluation study, we investigate how the pros and cons of a WBL-program are seen by alumni of the SBP program. Specifically, we want to learn how the WBL-program's academic quality and learning outcomes are perceived by the SBP alumni in comparison to ROM alumni.

To answer our research questions, a survey was conducted among alumni. This survey contained questions about perception of the study program, that are shown in this article and about career perception, which will be published separately. Permission for the distribution of the survey has been given by the ethical committee of the University of Groningen (CETO submission number 63,104,880).

In total 2,473 FSE alumni were approached (for the selection procedure see, appendix A). The participants invited were all alumni of programmes that offer SBP and ROM in the second master's year. In total, 242 alumni who graduated between the academic years 2001/2002 and 2017/2018 from the FSE of the University of Groningen completed the questionnaire and were included for analysis. More specifically, 111 SBP alumni (32.0% of approached SBP alumni) and 131 ROM alumni (6.2% of approached ROM alumni) completed the questionnaire. Respondents were able to skip questions if they did not want to answer, therefore the number of participants varies per item.

Table 1 shows participants' age and gender distribution. On average, participants were between 29 and 32 years old. For the SBP group, there was a significant difference between the age of male and female participants (F(1,109) = 4.87, p =.029). Male SBP participants were on average 2.27 years older than female SBP participants. There were no other significant differences found in gender and age.

Table 1. Age and years since graduation, per gender/program

The alumni graduated on average 6.5 years prior to participation in the survey (Table 1). There is a significant difference between the SBP and ROM alumni for graduation year (F(239,1) = 94.06, p =.016). The ROM group graduated on average 1.26 years more recent—prior to taking the survey—compared to the SBP group. Further exploration shows an interaction effect between alumni type and gender for graduation year (F(1,235) = 4.03, p =.046). Although within each group of alumni no effect were found for gender, a significant difference in years since graduation was found between SBP male participants and ROM male participants (F(1,123) = 10.86, p =.001). Hence, among ROM male participants, there were more recent graduates (on average 1.83 year more recent than SBP alumni). These biases have to be taken into account in the interpretation of further results.

Table 2 shows the disciplinary backgrounds of the SBP and ROM group. Biology alumni form the largest group of respondents. The second largest group had a medical discipline, biomedical alumni in the SBP group and medical pharmaceutical alumni in the ROM group.

Table 2. Percentual disciplinary background of alumni, SBP group n = 110, ROM group n = 131

The questionnaire was based on previously developed standardized questionnaires in addition to non-standardized questions constructed by the researchers of this paper. All questions were checked on questionnaire design recommendations (cf., Dijkstra et al., [17]) and were optimized through pre-testing. The pre-test comprised of cognitive interviewing (Beatty & Willis, [5]; Collins, [14]). The questionnaire was first tested by students during a course on survey design. Then, a revised version of the questionnaire was pre-tested again. Based on these results, the questionnaire was extensively discussed by all authors of this paper and finalized. Questions that turned out to be unclear during the pre-test where adjusted or removed. In Appendix C the pre-test procedure is further described in detail.

Preceding academic quality assurance in general is the need for universities for each bachelor's and master's program to formulate learning outcomes for degree programs they offer (Mercer-Mapstone & Matthews, [37]). Matthews and Hodgson ([34]) developed a tool, the Science Student Skills Inventory in order to measure science students' learning outcomes. It consists of several closed-ended questions on academic skills. We used the SSSI to explore to what extent SBP and ROM alumni perceive having gained several academic skills throughout the program and how these perceived skills compare across the two different master's programs.

To assess academic skills, all six original items of the SSSI were used: (1. Scientific content knowledge in your field(s) of study; 2. Communication skills (oral scientific presentations); 3. Writing skills (scientific writing); 4. Quantitative skills (mathematical & statistical reasoning); 5. Team work skills (working with others to accomplish a shared task); 6. Ethical thinking (ethical responsibilities and approaches)). The following question was asked for every skill: "During the entire master's degree programme I followed at the FSE, I improved (skill) ... ". Participants had to evaluate these statements on a 5-point-scale (ranging from I totally disagree to I totally agree).

To the SSSI four additional skills were added, based on the SBP's learning outcomes (7. Practical research skills (i.e. lab work and modelling); 8. Academic reasoning (i.e. analytical and critical thinking); 9. Project-based working skills (to achieve a certain goal with limited resources); 10. Leadership skills (i.e. guiding & managing a team)). Furthermore, participants were asked to assign their master's program a general grade (ranging from 1 to 10), and to evaluate statements on a 5-point scale (ranging from I totally disagree to I totally agree) about their master's program in relation to its value in the faculty's curriculum and career preparation (Generally speaking, I believe the SBP track ... 1) is a positive addition to the curriculum of master's programmes that can be followed at the FSE; 2) should remain available as an optional master's track within a master's programme of the FSE; 3) contributes to students' career preparation for a career in academia; and 4) contributes to students' career preparation for a career outside academia).

In this survey, three open-ended questions were posed (Table 3). Responses were coded by two independent coders and an intercoder reliability (Cohen, [13]) was calculated for each of the coded questions based on answers of 100 respondents. Table 3 shows the items that were presented to the participants, as well as the reliability of the coding process (Cohen's Kappa). These values suggest an intercoder reliability with almost perfect agreement (McHugh, [36]).

Table 3. Open-ended questions

The data were collected using survey software Qualtrics. Before participation, respondents were asked for informed consent. After giving consent, participants answered demographic questions and the open-ended questions. Then, items from the SSSI, additional learning goals and program satisfaction were presented to respondents. Lastly, participants were asked several questions on their first job after graduation and current job. The latter questions go beyond the scope of this paper and are part of another study. After data collection, any identifying information was removed from the data. Statistical analyses were carried out with statistics software SPSS. Depending on meeting assumptions, (i.e., normality, equality of variances), parametric tests (Univariate ANOVA's) or non-parametric alternative tests (Welch & Brown-Forsythe tests) were carried out.

For the open-ended questions, using an inductive coding strategy, participants' responses were coded based on non-predefined categories. The data was first coded by two student-assistants, not involved in the data collection. Then, overarching coding categories were created that merged similar coding categories (For the code book see table A1 in Appendix B). Lastly, all data was coded based on these merged categories by the second author of this article. The first author double-coded 100 responses (50 SBP, 50 ROM).

The SBP alumni and ROM alumni responded in a different way to the open-ended question "Motivation of choice" (Figure 1). Study content was frequently mentioned by both groups. For the SBP alumni frequent motivations to choose the SBP program also included the focus outside academia and the focus on society. For the ROM alumni, their choice was motivated mostly by the alumni's personal interest. The focus on academia (research) was a motivation for 14% of this group, among the SBP alumni nobody mentioned this. Other motivations were recommendation of others (1% for the SBP group and 14% of the ROM group) and career in general.

Graph: Figure 1. Motivation of choice of the study program, open-ended question, SBP group n = 110, ROM group n = 125.

On average, alumni of FSE evaluated their education rounded an 8 out of 10 but ROM alumni give a slightly, but significantly, higher grade than SBP alumni (Table 4) when it comes to evaluating their master's program F(1,238) = 5.01, p =.026. Table 5 shows means, standard deviations and percentage of agreement (participants who scored 4 or 5 on the Likert-scale) with statements on appreciation, academic level and career preparation. Both groups generally perceived their program is a positive addition to the curriculum. There is a significant difference between the different groups of alumni in their perception of whether their master's program should stay available in the curriculum F(1,221) = 9.04, p =.003, R2 =.04). Robust tests of equality of means (Welch & Brown-Forsythe) confirm that there are significant differences despite inequality in homogeneity of variances (1,193.63) = 9.07, p =.003. The difference is also visible in the percentage of agreement where the SBP group had a higher percentage of agreement (95.5%) than ROM group alumni (84.8%).

Table 4. Grade alumni give to their study program, scale 1–10

Table 5. Appreciation, academic level and career preparation of second master year, measured with a statement, agreeing on Likert-scale 1–5 (1 strongly disagree – 5 strongly agree) ** p < 0.01, * p < 0.05

SBP alumni only were asked questions on their satisfaction with the track and whether they would recommend their track to others. For both statements, SBP alumni generally agreed with the given statements.

In their perceived academic level of the program (Table 5), almost all participants scored 4 or higher. This shows that the alumni generally agreed with the statement that their master's program led to obtaining a sufficient academic level. There is a significant difference between the two groups, SBP alumni score a bit lower in their perception (M = 4.06) than the ROM group (M = 4.42), F(1,237) = 12.11, p =.001, R2 =.05.

Figure 2 the results of the SSSI, assessing the learning outcomes more specifically. Zooming in on the alumni's reflection on their learned skills, both groups indicate that they have increased their scientific knowledge fairly. ROM alumni thought they improved their scientific content knowledge during the whole master's program more so than SBP alumni (F (1,236) = 5.35, p =.022. R2 =.018). In terms of writing skills and quantitative skills, both groups indicated that they have increased properly. No difference was found between groups. Alumni experienced growth on oral communication skills (F(1,236) = 6.53, p =.011. R2 =.023) and teamwork skills (F(1,236) = 46.06, p =.000. R2 =.160), whereby SBP alumni felt that they have experienced more growth than ROM alumni. Robust tests of equality of means (Welch & Brown-Forsythe) confirm that for both these skills there are significant differences despite inequality in homogeneity of variances (1,232.) = 52.31, p =.000). SBP alumni scored significantly higher on ethical thinking than the ROM group (F(1,236.40) = 24.60, p =.000. R2 =.091).

Graph: Figure 2. SSSI skills precepted by alumni right after graduation on Likert-scale (1 strongly disagree- 5 strongly agree). SBP group n = 109–111, ROM group n = 130–131. *** p < 0.001 ** p < 0.01, * p < 0.05.

Figure 3 shows the results for the added skills. For practical research skills (lab work), both groups indicated that they have increased fairly. The ROM alumni thought they improved their practical research skills during the whole master's program more so than SBP alumni, F(1,326) = 4.51, p =.035, R2 =.015. For academic reasoning (i.e. analytical thinking), both groups indicate having increased skills and no difference was found between the groups. In terms of project-based working skills, growth was also mentioned, whereby SBP alumni felt that they have experienced more growth than ROM alumni, F(1,36) = 12.85, p =.000 R2 =.048. Robust tests of equality of means (Welch & Brown-Forsythe) confirm that there are significant differences despite inequality in homogeneity of variances (1,238.42) = 13.07, p =.000. For leadership skills, there is also a significant difference, F(1,236) = 54.41, p =.000. R2 =.195, with the ROM group finding they did not develop during the study. The SBP group scores M = 3.20, which means only slightly more agreement on development. Robust tests of equality of means (Welch & Brown-Forsythe) confirm that there are significant differences despite inequality in homogeneity of variances (1,214.31) = 57.59, p =.000.

Graph: Figure 3. Additional skills precepted by alumni right after graduation on Likert-scale (1 strongly disagree- 5 strongly agree). SBP group n = 110–111, ROM group n = 131. *** p < 0.001 ** p < 0.01, * p < 0.05.

For the ROM group, the most frequently mentioned acquired skills were scientific/research skills (Figure 4), followed by disciplinary content knowledge. For the SBP group the frequently mentioned skills acquired vary more and included teamwork, personal development, communication and disciplinary content knowledge.

Graph: Figure 4. Acquired skills in final master year, open-ended question. SBP group n = 106, ROM group n = 126.

For the most important skills that participants missed that should have been included in their master's program (Figure 5), disciplinary knowledge was most often missed among SBP alumni (33%) and was missed almost equally by ROM alumni (29%). However, ROM alumni most often missed career preparation skills (45%). A smaller percentage of the SBP alumni missed career preparation skills as well (29%). Other missed skills by both groups were on communication. A part of the SBP group also wanted to have acquired more business skills (25%).

Graph: Figure 5. Missed skills after final master year, open-ended question. SBP group n = 91, ROM group n = 110.

When alumni reflected on career preparation (Table 5), both groups scored well for preparation for an academic career. However, there is a significant difference between SBP alumni and ROM alumni in their perception of being prepared for a career in academia, F(1,240) = 95.43, p =.000, R2 =.28. SBP alumni thought that their master's program prepared them less for a career in academia than the ROM group. Conversely, ROM alumni thought that their master's program prepared them less for a career outside academia than SBP alumni F(1,235) = 95.48, p =.000, R2 =.33).

In this article, we aimed to find an answer from an alumni perspective to the question whether work-based learning (WBL) and research oriented master's (ROM) programs differ with regard to bridging science and society. Being a new educational development we wanted to find out if the work-based learning master's program Science, Business and Policy (SBP) offers sufficient basis for a successful societal science career. To answer this question, web-based surveys were conducted in two groups of alumni who had completed different programs at the same faculty and university, the SBP group (WBL) and the ROM group (research oriented master). From SBP alumni's perception the WBL-program was considered distinctive, qualitatively good and on sufficient academic level. Participants appreciated SBP and reported that the program prepared the students well for a different career than the classic research program. It is perceived suitable for those who aspire a career outside academia, where experience in the work field is so important.

First, we had to check whether the SBP group is indeed comparable to the ROM group. Both group samples include well over 100 alumni. The response rate was higher in the SBP alumni group, which can be explained by the reminder email this group received. Possibly, SBP alumni felt more directly addressed or involved in improving the SBP program by participation in the survey. Diversity was the overarching theme in the participants' backgrounds. In SBP and ROM we find alumni with different disciplinary background, of different ages and gender. To review a multidisciplinary program, it is advantageous that alumni from various backgrounds have responded. Fortunately, we found this in both alumni groups, with minimal differences between the groups.

There were however some noticeable differences between the groups in general characteristics. SBP participants graduated on average a bit more than a year earlier than FSE participants. This bias could be explained by the more intensive alumni-university relation of the SBP alumni. This is important information since they have had longer development time, might respond differently due to longer involvement or had more difficulty to retrieve details about their education because of the longer time frame (Bradburn et al., [7]). The bias in years since graduation was particularly seen for males and not for females among SBP alumni. Also, it could be that we are actually measuring a preference difference. Perkmann et al. ([38]) suggests that academic engagement (explained as professional interactions between scientists and external organizations) is associated with personal characteristics, for example, being male and being senior. Perhaps such engagement increased the attractiveness of participating in the survey. Therefore, when interpreting the data, it should be kept in mind that the groups had different characteristics. To the question whether SBP is perceived distinctive as a study program, the answer seems confirmatory. The first argument is the consideration of students to choose for a specific program. Earlier on the diversity theme was already mentioned. Nowadays this goes beyond just choosing a different discipline. Literature shows that academic success is related to the learning style of the student, in which individual characteristics play a unique role (Busato et al., [11]). Learning from a societal perspective and future ambition are also variables that can play a role in educational preference. This seemed to be visible in the difference between reported the motivation of study choice between both groups. Furthermore, in study choice, we see that ROM alumni chose the program more often based on recommendation of others, for the SBP alumni who participated this was only rarely the case. There seems room for improvement with regard to the worth of mouth strategy in promoting the SBP program.

The SBP and ROM groups also differed in their perceived learning outcomes. Compared to the ROM group, SBP alumni reported more development of oral communication skills, ethical thinking, teamwork skills, project-based working skills and leadership skills. These are all skills that are specifically trained during the SBP program (Grooters et al., [23]), but only incidentally in the ROM programs. The SBP group perceived they developed scientific content knowledge and practical research skills less than the ROM group, but in general still sufficient. This finding can be explained by the difference in focus of the study programs but can also be partly due to an initial difference in the type of student (i.e. students might already prefer a program with a more "commercial" style, or might already possess more talent for skills that are focused on in the SBP-program). For instance, it has been found that individuals with interest for basic research are less likely to collaborate with the industry (Abreu & Vadim, [1]), whereas individuals who are initially more interested in the applications of research are more likely to engage with industry (Lawson et al., [29]). Looking at perceived acquired skills, we recognize similar dynamics. In line with the approach of the study programmes, SBP alumni reported more acquired skills in communication, hands on experience, project-based skills, multidisciplinary thinking, teamwork and personal development, where the ROM group mainly reported to have developed scientific or research skills. Furthermore, it is surprising that FSE alumni feel confident about their communication skills, since these are often weak in recent graduates (Geissler et al., [22]). Overall, the outcome of perceived skills match with the SBP alumni's motivation to choose for SBP, a focus on society (i.e. outside academia).

We also investigated whether SBP, as a distinctive program, still was perceived as qualitatively comparable and up to standards on an academic level. Alumni in both groups feel that they have increased their academic skills in their program. For most skills, alumni felt that their program provided sufficient development. Exceptions to this are seen in the ROM group's leadership skills (moderate growth), whereas the SBP group did not score any skills insufficiently. For writing skills and quantitative skills, no differences were found between the two programs. Also, no differences were found for academic reasoning, a skill that is often considered responsible for academic level (Gaast van der et al., [19]). When the academic level is assessed directly both groups score high, though perceived attainment of a sufficient academic level is slightly stronger in the ROM group. Besides appreciation for the academic level, the overall level of appreciation of both a classic research (ROM) program and an alternative WBL (SBP) program was highly positive. Alumni saw the program they have followed as a positive addition to the curriculum of educational programs. In line with this they also found that the program should remain available as an optional master's track, the SBP group even more so than the ROM group. Apparently, offering an alternative, more societal program is highly desirable for this group of students. Focusing on perceived career preparation, we found different trends between programs. SBP alumni felt more prepared for a career outside academia, while ROM alumni felt more prepared for a career within academia. This is also what one could expect considering the programs' targets. It forms an argument to offer different types of education and to prepare for different types of careers. At the same time, career preparation skills were missed mostly by the ROM group, which also indicates the importance for the ROM group. SBP alumni also report missing career preparation skills but to a much smaller extent. In both groups around one third of the alumni wanted more disciplinary content knowledge. Surprisingly about one fourth of the SBP alumni wanted to develop even more business skills then offered within the program. Overall, the SBP group indicated they would recommend SBP to others and were satisfied with the program. SBP Alumni seem satisfied with a work-based learning program as SBP where they can learn about other things than gaining knowledge and preparing for a PhD. This is also reflected in the literature where academic success is defined broadly by students as 'the accomplishment of the learning process; gaining subject knowledge; and developing employability skills '(Cachia et al., [12]). And especially the latter seems to be a process that differs from individual to individual in how this is achieved optimally. The potential advantages described in literature (Costley & Stephenson, [15]; Stephenson & Saxton, [41]) of a WBL-program seem to be represented in the SBP format. Alumni report personal, professional and academic development in combination with practical experience. Hereby one can expect the WBL to contribute to making students employable, to connect academia with the work field and to bring science and society more together. What about the hurdles of WBL? Actual and future problems on organisation, like changing a science curriculum in general, increased online education and organising on a bigger scale are apparently possible to overcome. Based on the self-reports of alumni the critiques on WBL mentioned in literature seemed not to be experienced in the SBP group. The academic level is perceived well and learning outcomes and skills are different but abundant and highly valued. In the case of SBP, the University remains leading in grading and has pre-set learning outcomes and criteria (Grooters et al., [23]). It can hereby control the academic quality. The history of the SBP program proves, by its existence over 20 years, that a science curriculum can be adapted in this way and that WBL can be a major part of a curriculum instead of a voluntary extracurricular task done in the students own time to boost a resume. The organization of WBL asks for a larger engagement of staff, but the development of the SBP program proves that it is feasible, also on a larger scale (Grooters et al., [23]). All together it seems possible to offer science education in an alternative way compared to the classical approach without losing academic quality or decreasing academic level. A practice-oriented approach should have a place in science education and can be implemented in curricula that caters to the distinctive group of science students with interest in society. This would fit perfectly into the values of most academies to be a socially engaged institution (Lee, [30]).

For a next step in the evaluation of WBL master programs like SBP, it would also be interesting to investigate how the SBP alumni performed after their graduation and to see whether the effects reported here continue to play a role in their professional live. This would allow to study the impact of WBL education in science on the career perspective. Furthermore, it will also be interesting to extend research (Lester et al., [31]) on this matter from the point of view of other actors, for example, academic staff and work placement provider/employers. In a follow-up of this study the impact of the implementation of WBL on the work floor and the academy itself will be researched.

We thank M.J. van den Nieuwenhof, J.Zevenberg, M.van Rijssel, B. Piersma and H.J. van der Windt for their helpful comments and thank student-assistants Allard de Vries and Nina van den Velden for coding and data support.

No potential conflict of interest was reported by the author(s).

SBP alumni participants were recruited from the SBP alumni subject pool of the Science, Business and Policy database. The SBP program keeps its own alumni database containing only alumni who granted permission to contact them (almost all). All of the SBP alumni from the database got an invitation to join if they graduated before 2019 (n = 347). The SBP alumni also received a reminder. The ROM alumni, who followed a 2 year research program, were recruited from the general alumni subject pool of the University of Groningen (SBP students excluded). In total, 2,126 invitations were sent to the ROM alumni that granted permission for contact. They received an invitation once, i.e., no reminder. Programs that do not give access to SBP were excluded from the ROM group, i.e., all engineering programs and technical business administration.

For both groups, invitations were send via email. In this email, participants were informed that they were invited for an online study about their perceptions of the master's programme(s) they followed at the FSE of the University of Groningen, that this study was aimed at improving the FSE's knowledge regarding teaching programmes, that it could be used for making evidence-based choices with regard to the FSE's future curriculum and for writing external reports. Participants had to grant informed consent before they could participate in the research. This consent also indicated the anonymisation and integer use of data. Partial completion of the survey was also possible. When finished, participants were thanked for their participation. There was no reward, bonus or compensation system for filling in the survey. Contribution to this research was completely voluntary.

The coding procedure was as follows: the code was scored as present (1) or absent (0) in the answer of the alumnus due to the coding justifications. An example is given in Table A-1 for the question 'Acquired Skills' (what skills did you acquire?). For "Skills missed" and "Motivation of study" choices similar justification tables were used. For all open-ended questions, multiple answering options per respondent were possible, and therefore coding categories are not mutually exclusive, it is possible that one answer is coded into multiple categories.

Table A1. Coding justifications "Acquired skills"

Pre-testers were 30 bachelor students following the second year bachelor's course "Questionnaire and interview design" of the study "Communication and Information sciences" (from now on CIS) at the University of Groningen. Every CIS-student was instructed to carry out and analyse cognitive interviews based on a first draft of the questionnaire. The CIS-students received interviewer training and relevant literature (e.g., Beatty & Willis, [5]; Collins, [14]) to prepare them for carrying out the cognitive interviews.

The CIS-students practiced with two methods of cognitive interviewing. First, the think-aloud method, which implies that participants are asked to evaluate a questionnaire by verbalizing all their thoughts during the entire process of filling out the questionnaire (Haak Van den et al., [25]). Participants are asked in advance to focus on—for instance—comprehension of the question and retrieval of the information. Second, CIS-students practiced the probing method. Probes are questions aimed at retrieving how participants came to their answers. These questions can be asked during filling out the questionnaire, or after completion of the questionnaire (e.g., "What does the term 'academic skills' mean to you?"). Probes are often predetermined and included in an interview protocol, but can also be spontaneously devised based on the verbal or non-verbal behaviour of the participant. CIS-students were obligated to employ the think-aloud method, but were free in their choice to include the probing method.

Participants were 29—at the time of the interview—SBP students, 14—at the time of the interview—master's students of an unrelated master program and 35 master's alumni (from different backgrounds). All participants were between their early twenties and late thirties and around seven out of ten participants were female and three out of ten were male. All CIS-students had to analyse three interviews. In total, 78 cognitive interviews were conducted, on which 90 analyses were carried out (i.e., some students analysed (recordings of) cognitive interviews that were conducted and also analysed by other students, thereby doubling some cases). One cognitive interview was never analysed more than twice.

The choice to carry out cognitive interviews with members of similar populations (e.g., other master's students and master's alumni) and not with actual members of the population of the final study (SBP alumni) was a conscious choice, based on the fact that the SBP population is relatively small, and the whole population is targeted in the actual study (i.e., no sample of the population was drawn).

Participants of the cognitive interviews were informed about the purpose of the study, and instructed and able to practise to think-aloud during filling in the questionnaire. Before the actual study started, all participants had to give explicit consent by filling out a consent form. After about 30 interviews data saturation took place. Still, all input from the pre-test assignment of the CIS-students was used by the (second and third) author of this paper to refine and optimize the questionnaire.

This optimized questionnaire was cognitively tested again by the (second) author of this paper and a student-assistant in a cumulative way: after each cognitive interview the questionnaire was adjusted based on the findings of the prior interviews). In this way, the adjustments that were made could be tested in the next cognitive interview to make sure the adjustments did not result in new cognitive problems or unclarities. The sequence of cognitive interviewing and adjusting the questionnaire was carried out until no cognitive problems or unclarities were found during the cognitive interview. After six of these cumulative cognitive interviews no new cognitive problems arose, and as such, the final questionnaire was constructed together with the other authors of this paper.