Short-term training programs for immigrants in the German welfare system: do effects differ from natives and why?

Abstract

We evaluate the individual employment effects of four types of short-term training for immigrants and natives in the German welfare system and identify differences in the effects determined by unobservable factors. Based on comprehensive and unique administrative data, we apply propensity score matching in a dynamic setting to estimate the treatment effects and suggest the identification of unexplained effect differences between groups. The results show a mixed picture: two types of training increase the employment chances of immigrants, whereas the other two are rather ineffective. Effect differences between immigrants and natives predominantly result from socio-demographic composition.

JEL codes: I38, C14, J61

1 Introduction

Immigrants face a higher risk of unemployment and are more often dependent on welfare benefits than natives in many countries. In Germany, their labor market situation is of particular concern.¹ In 2011, more than every third welfare recipient was an immigrant (36 percent, comprising first and second generation), while the corresponding share in the population was less than 20 percent (Statistisches Bundesamt 2011). Persons claiming welfare benefits are forced to actively look for employment and to enhance their employment chances. In order to improve the employability of welfare recipients, German welfare agencies offer a number of active labor market policy (ALMP) programs. Given the overrepresentation of immigrants in welfare, this group can be perceived to have also a higher need of support by ALMP.² Nevertheless, relatively little is known so far about the impacts of ALMP programs for immigrants. This lack of evidence is not specific for Germany. There are only a few international studies that analyze ALMP programs with a particular focus on immigrants.³

A reason for the scarce evidence may be the lack of ALMP programs specifically designed to the needs of immigrants. In Germany, for example, with language courses being the only exception, immigrants participate in programs that have been designed for all welfare recipients. Within Germany’s ALMP programs, so-called short-term training programs have been widely used for native and immigrant welfare recipients. These programs last for a maximum of 3 months comprising aptitude tests, intensified counseling, courses improving job search skills, and measures providing specific skills and abilities required for employment. Programs’ content can be flexibly arranged according to the individual’s needs. Participation should help to improve the employability by increasing the search efficiency and improving productivity.

Similar programs are used in many other countries, usually referred to as job search assistance programs. Leaving aside country-specific differences in labor market institutions, detailed program characteristics, and enrolment rules, the overall picture of the employment effects of these programs tend to be positive.⁴ Whether these programs reach their purposes for immigrants (like for natives) is not clear ex ante and varying impacts can occur for a number of reasons. Differences in placement may result from a different value of the programs for immigrants compared to natives. If programs are designed with respect to the needs of the average native welfare recipient, then they may not work (well) for immigrants, and policy makers should think about redefinition of programs addressing the specific needs of immigrants. On the other hand, if programs are particularly effective for immigrants even though not initially designed for this group, policy makers may intensify the use.

We evaluate the effects of four short-term training programs on taking up employment for male native and immigrant welfare recipients in Germany. In particular, we try to answer the following research questions: First, how effective are programs for immigrants (compared to natives)? And, second, if differences in effects are observable between natives and immigrants, can we identify potential reasons of these differences? Determining the source of differences in program effectiveness between groups is important. If, on the one hand, differences in program effectiveness are driven by differences in the socio-demographic composition of native and immigrant welfare recipients, this implies a general potential for welfare agencies to improve the targeting of programs to participants. If, on the other hand, differences are due to the immigrant characteristic, then this points to other reasons, e.g. discrimination, and the use of programs for either immigrants or natives should be reconsidered.

The empirical analysis is based on comprehensive register data of the inflows into welfare in 2006. Individuals are followed until July 2008. Regarding the first question, we estimate effects of participation in a short-term training program during the first year on welfare using propensity score matching estimators in a dynamic setting, where treatment effects vary conditionally on the preceding duration in welfare. To answer the second question, we decompose differences in treatment effects in observable and unobservable components. To do so, we identify differences in treatment effects between natives and immigrants that result from the attachment of the individual to the immigrant group (immigrant fixed effect). The estimation is carried out with a matching estimator considering only treated individuals of both groups, where group attachment is the treatment in question. To preview our results, the picture is mixed. On the one hand, courses providing skills and aptitude tests have positive effects on employment. These effects differ only slightly between immigrants and natives once we control for observable differences between both groups. On the other hand, job search training and combined programs are ineffective irrespectively of migration background.

The rest of this paper is structured as follows. Section 2 presents details of the German welfare system and of the programs evaluated. The data used in the empirical analysis are described in section 3. Section 4 discusses the evaluation approach. The empirical results are presented in sections 5 and 6. The final section concludes.

2 Institutional framework and theoretical effects

The German welfare system was substantially reformed at the beginning of 2005 (see e.g. Jacobi and Kluve 2007). In order to be eligible for welfare benefits, persons have to be aged 15 to 64 years and be able to work for at least 15 hours per week since then. According to the legal definitions, unemployed welfare recipients are persons who have not been employed in the past and are not eligible for unemployment benefits and persons who have exhausted their unemployment benefits claims.⁵ Unemployment is not a prerequisite for receipt of welfare benefits, and employed individuals are also eligible if the household income is too low. Claimants have to register with the local welfare agency and are obliged to participate in ALMP programs. Placement in programs results upon decisions of caseworkers regarding the needs of the welfare recipients. Most programs, including short-term training, are exclusively targeted at unemployed welfare recipients.

Table 1 provides some selected numbers on the welfare system concerning the entitled persons and the corresponding spending for the years 2005 to 2008 (the years from the welfare reform in Germany to the horizon of our empirical analysis). As becomes obvious, on average about 5 million people were entitled to welfare benefits. However, referring to the years from 2006 onwards a slight decline from about 5.4 million to 5.0 million persons could be established. The spending amounted to more than 30 billion Euro per year for welfare benefits. Corresponding to the shift in the number of entitled persons, spending declined slightly between 2006 and 2008. In contrast, the numbers for the spending on ALMP emphasize the 2005 reform goal to activate all welfare recipients. Whereas in 2005 only 3.1 billion Euro were spent overall, this number increased by more than 50 percent up to 4.7 billion Euro in 2008.

Table 1 The welfare system and short-term training programs

Within the scope of ALMP programs, short-term training programs are a quite frequently used measure. During the years 2005 to 2008, between 411 and about 628 thousand welfare recipients per year have participated in these programs.⁶ Short-term training programs consist of three different types of measures (modules) that can be offered separately or in combination and allow a flexible implementation in line with the specific needs of the welfare recipient and the options of the local welfare agency.⁷

The first type of courses are aptitude tests (Eignungsfeststellungen) which last for up to four weeks. These tests are used to assess the suitability of participants in terms of skills, capability, and labor market opportunities for specific occupations. During the assessment process occupation specific skills are provided which shall help to improve employment chances in the respective occupations. At the same time, the caseworker gets a better knowledge about the skills and labor market prospects of the welfare recipient. This should increase the effectiveness and efficiency of his or her placement efforts. The measures of the second type of short-term training programs aim at improving the applicant’s presentation and job search abilities (Überprüfung der Verfügbarkeit/Bewerbertraining). The activities support the individual’s efforts to find work or efforts by the welfare agency to place him/her, especially through job-application training, counseling on job search possibilities or measures assessing the person’s willingness and ability to work (work-tests). Measures of the second type are promoted for up to two weeks and will be referred to as job search training in this analysis. The third type contains practical training of the participants (for up to eight weeks) providing necessary skills and techniques required for placement in employment or vocational training (Vermittlung notwendiger Kenntnisse und Fertigkeiten). The courses cover, for example, specific working techniques like business administration or computer courses. We will refer to this form of training as skill provision. Finally, combinations of modules, e.g. a job aptitude test followed by a computer course, could be granted for a maximum of twelve weeks. This is the fourth type of training considered here and will be referred to as combined training programs. The modules can be flexibly arranged. In some cases, combined training programs consist of two modules, while in other cases all three modules are combined.

Financial support during the programs is provided by the Federal Employment Agency (FEA) and covers course costs, examination fees, and travel grants as well as child care. In addition, participants receive welfare payments. All short-term training programs are intended to qualify welfare recipients for employment. There is no intention of a threat effect, i.e. the assigned participants quit welfare in order to avoid participation. Assignment to programs depends on the decision of the caseworker, who interviews and advises the welfare recipient about requirements for and obstacles to employment. If this assessment reveals the need for participation in a short-term training program, the caseworker decides about the type offered, i.e. whether the welfare recipient should be trained in a single or a combined program. In case of the latter, the number and the sequence of modules is defined ex ante. (Therefore, they can be assumed to constitute a distinct type of program).

The institutional set-up of short-term training programs implies two channels through which programs affect the job search of the participants and, therefore, the employment chances. First, the modules that improve or support the job placement – on part of the welfare agency or the self-contained job search of the participants – can be expected to improve the search behavior of the participants by increasing the intensity as well as the efficiency of the search efforts. More efficient job search will lead to an increase in the job offer arrival rate, which increases the probability of leaving welfare. However, it will make job seekers more selective with respect to potential job offers and induces a negative indirect effect on the transition. The overall effect is then the sum of the positive direct and the negative indirect effect. Second, participation in short-term training could improve the job-relevant skills and therefore increase the job opportunities of the participants. Increasing the skills is equivalent to increasing productivity which enables participants to apply for jobs associated with on average higher wages. In terms of job search theory this equals a shift of the wage offer distribution to the right. According to Mortensen (1986), an increase in the mean of the wage offer distribution increases the reservation wage by an amount less than the increase in the mean, and, therefore, this will increase the probability of leaving welfare as well.

These theoretical considerations imply positive effects of participation in a short-term training program on the probability of taking up employment. However, for the theoretically derived positive effects to hold in reality there have to be potential employers willing to engage the participants in the programs. Despite the increased search effort or the higher productivity obtained in practical training, participants may still possess a productivity too low to be remunerated by the market. In that case, there will be no positive effects of participation.

3 Data

The empirical analysis is based on merged information of five different sources of administrative records provided by the FEA for a sample of inflows into welfare in Germany between January, 1st 2006 and December, 31st 2006. The main source is the Integrated Employment Biography data set (Integrierte Erwerbsbiographien, IEB). The information included in IEB covers the years 1990 to 2007 on a daily basis and provides a detailed characterization of the labor market histories and the current situation of the welfare recipients, including information on participation in the four types of short-term training programs. To ensure that inflows in the data are not short-term recurrences of welfare episodes, for example due to false reporting or data errors, only persons are regarded who have not been registered in welfare for three months before the sampling date. Since entitlement to welfare is means-tested with consideration of the wealth and the income of further household members, we merge information on further persons living in the households that are recorded in the Benefit History Master Records (Leistungshistorikgrunddatei, BHMR).

In our analysis, we distinguish between immigrants and natives. Immigrants comprise all foreigners and naturalized persons. Foreigners are persons, who do not possess German citizenship. The naturalized group contains German resettlers (from the former Soviet Union and Eastern Europe) and naturalized foreigners. Although citizenship is recorded in IEB, identification of naturalized foreigners and German resettlers could only be carried out partially from this dataset. To identify resettlers we consider information on the immigration date recorded in the Job Seeker Statistics (Arbeitsuchendenstatistik, ASU) dating back to 1990, which contains resettler status explicitly. To identify naturalized foreigners, we use information from the IEB for the years 1990 to 2007 and in addition the Employment History Records (Beschäftigtenhistorik, EHR) for the years 1975 to 1989. A person with German citizenship at the sampling date who was recorded being a foreigner in any spell since 1975 is treated as a naturalized.⁸

Immigrants are not equally distributed across welfare districts in Germany. For example, in East Germany the number of immigrants is far lower than in the West, and even in West Germany strong regional disparities can be observed with a higher concentration of immigrants in cities and urban regions with industrial production. In contrast, rural areas usually exhibit smaller numbers of immigrants. To consider the local labor market conditions specific to the distribution of immigrants in the estimation of treatment effects, the analysis sample was drawn in a 1:1 ratio of immigrants and native Germans on regional level. In a first step, the sample of immigrants was randomly drawn from the total inflow population into welfare in 2006. Then, in a second step, for each randomly drawn immigrant from a welfare agency district, one native German was drawn from the same district. Therefore, immigrant-native German ratios are balanced across districts and should mitigate regional imbalances in the distribution of immigrants.⁹

For our analysis, we restrict the sample to male unemployed welfare recipients aged 18 to 57 years at the sampling date. Welfare recipients younger than 18 years are excluded so that the estimates are not affected by compulsory schooling. Individuals aged 58 years and above are eligible for so-called relaxed welfare receipt. Within this scheme active job search is not required for benefit entitlement and claimants can rely on welfare until (early) retirement age. The final sample for the analysis contains 50,815 observations of which slightly more than half are natives (25,953) and the rest are immigrants (24,862).

We identify for each sampled person the first assigned program during the welfare spell and evaluate participation in a short-term training program against nonparticipation in any other program at the time of starting the program. Our outcome of interest is the effect of short-term training on the drop-off rate from welfare conditional on employment uptake of the individual, where employment comprises jobs subject to social security contributions in Germany. The binary outcome variable can be observed on a monthly basis until July 2008 and has been merged from the Employment Statistics Register (Beschäftigtenstatistik, ESR).¹⁰ The spectrum of available independent variables in our data comprises sociodemographic characteristics, details of the household composition of the individual, aspects of qualification and experience, a comprehensive labor market history, and regional information. Further information on the independent variables is provided in section 5.1.

4 Evaluation approach

4.1 Estimation of treatment effects

The empirical analysis follows the potential outcome approach of causality, see e.g. Roy (1951) or Rubin (1974). Using conventional notation, let Y¹ and Y⁰ denote the two potential outcomes, where Y¹ is the outcome when the individual participates in the program, and Y⁰ is the outcome, when the individual does not participate. Since the individual cannot be in both states at the same time, one of the potential outcomes is unobservable and direct calculation of the treatment effect is impossible. To provide an estimate of the counterfactual state, we use information of non-participants. We apply a matching estimator with dynamic definition of participation and non-participation conditional on the elapsed welfare duration to estimate the average effect of treatment on the treated (ATT).

Conditioning on the elapsed welfare duration is sensible in the German context for a reason emphasized by Sianesi (2004).¹¹ In a comprehensive ALMP system a person will join a program at some point, provided the individual remains in welfare long enough. Consequently, the reason why an individual is not observed as participating in a program is that the person has already left the welfare system, or the time horizon of the analysis is too short. In line with that, participation and non-participation have to be defined dynamically, i.e. with respect to the point in time in which the comparison is made. Persons who have neither entered a program nor left welfare up to a specific point in time are defined as non-participants of interest or ‘waiters’ (in the sense that they are waiting to be allocated to a program). Thus, non-participation can be interpreted as the default state for each individual, and everybody is a non-participant until entering a program or leaving to take up a job. Individuals who are defined as non-participants at the moment we start our comparison may enter a program at a later point in time.

The evaluation approach in the dynamic setting could be formalized as follows. Let U = {0, …, U _max} define the discrete elapsed welfare duration of the individual since registration at the local welfare agency. Furthermore, let u denote the point of time during the welfare spell in which the program of interest starts and D _u the treatment indicator with the discrete time index: D _u  = 1 if the individual starts a program at time u of the welfare spell, and D _u  = 0 if the individual remains on welfare at u. Program effects are estimated for time t, i.e. the time since the program started. The hypothetical outcomes for time t given a treatment at time u are then defined as $Y_{t,u}^1$ for individuals who received the treatment at u and $Y_{t,u}^0$ for individuals who did not receive the treatment at least up to time u. The parameter of interest for each u is the average effect in t for individuals starting a program in period u of their welfare spell compared to not joining at u:

$$\begin{array}{ll}{\Delta }_{t,u}^{\mathit{\text{ATT}}}& =E(Y_{t,u}^1-Y_{t,u}^0|D_u=1,D_1=\cdots D_{u-1}=0)\\ =E\left(Y_{t,u}^1\right|D_u=1,D_1=\cdots D_{u-1}=0)\\ -E\left(Y_{t,u}^0\right|D_u=1,D_1=\cdots D_{u-1}=0).\end{array}$$

(1)

Whereas the first term is identified in the data by the observed outcome of the participants, the second term has to be estimated. Simply using the observable non-participants’ outcomes may lead to biased estimates due to self-selection. To solve the potential selection bias by matching, we have to assume conditional independence, i.e. conditional on a set of relevant covariates X the non-participation outcome Y⁰ is independent of the participation decision. Instead of conditioning on single X, we use the propensity score p(X) = E(D = 1|X), i.e. the probability of participation in a program (see Rosenbaum and Rubin 1983). The propensity scores are estimated by probit models. For the dynamic case, the dynamic conditional independence assumption (DCIA) is:

$$Y_{t,u}^0\coprod D_u\left|p\right(X_u),D_1=\cdots =D_{u-1}=0,$$

(2)

i.e. the hypothetical outcome at time t after not participating up to time u is independent of program participation at time u, conditional on the propensity score p(X _u ) measured at time u. The DCIA ensures that treated and non-treated individuals are comparable in their non-treatment outcomes at time t conditional on p(X _u ), conditional on claiming welfare benefits up to time u, and conditional on not receiving treatment before u. In addition, the availability of non-participating analogues for the participants has to be ensured (common support), i.e. P r(D = 1|X _u )<1 (Smith and Todd 2005a).

4.2 Estimation of the Immigrant Fixed Effect (IFE)

Considering effect heterogeneity in the treatment effects between immigrants and natives for a particular program can be used to reveal important insights. Assuming that identical programs are provided, differences could be, on the one hand, due to differences in the composition of the groups, i.e. the distributions of the relevant characteristics for program participation and labor market success may be different. Hence, when conditioning on all these variables no further differences should remain. However, on the other hand, if residual differences persist between groups, these differences are solely due to the group attachment of the individual. In this case, programs do not work in the same way for natives and immigrants with identical characteristics. Since we are interested in the effect difference due to the attachment to the immigrant group, we call this residual part the immigrant fixed effect (IFE). Identifying the part of the effect difference that cannot be explained by differences in sociodemographic composition is important for the targeting of programs to welfare recipients. The heterogeneous value of an identical treatment may result from a set of confounding factors associated with the attachment of the individual to the immigrant or the native German group. To estimate the immigrant fixed effect, we apply the following estimation procedure based on a matching estimator. To abbreviate notation, we suppress the indicators of the dynamic setting.

Starting point of the analysis is the raw differential ${\Delta }_{\mathit{\text{Dif}}}^{\mathit{\text{ATT}}}$ in treatment effects between immigrants and natives, that can be obtained by subtraction of the estimated treatment effect for native Germans from the estimated treatment effect for immigrants:

$${\Delta }_{\mathit{\text{Dif}}}^{\mathit{\text{ATT}}}={\Delta }_{M=1}^{\mathit{\text{ATT}}}-{\Delta }_{M=0}^{\mathit{\text{ATT}}},$$

(3)

where M is a binary index indicating immigrants if M = 1 and native Germans otherwise. The corresponding estimates for ${\Delta }_{M=1}^{\mathit{\text{ATT}}}$ for the immigrants and ${\Delta }_{M=0}^{\mathit{\text{ATT}}}$ for the native Germans are obtained as described in the previous subsection. The estimated effects are given by

$${\Delta }_{M=1}^{\mathit{\text{ATT}}}=E\left(Y_{M=1}^1-Y_{M=1}^0\left|p\right(X_{M=1}),D=1,M=1\right),$$

(4)

i.e. the estimated ATT for the immigrants conditional on the estimated propensity score of this group, p(X _M = 1), and

$${\Delta }_{M=0}^{\mathit{\text{ATT}}}=E\left(Y_{M=0}^1-Y_{M=0}^0\left|p\right(X_{M=0}),D=1,M=0\right),$$

(5)

the analogue estimator of the ATT for the native Germans conditional on the propensity score of this group, p(X _M = 0). For the sake of illustration, we have also added the group attachment indicator to the potential outcomes within each group.

For the estimation of the IFE, we can also apply the potential outcome framework. The treatment of interest is now the group attachment of the individual (where M = 1 denotes immigrants and M = 0 natives) and we are interested in the difference between two potential treatment effects: The first is the expected treatment effect of training for immigrants with immigrant characteristics X _M=1, $E(Y_{M=1}^1-Y_{M=1}^0|X_{M=1},D=1,M=1)$. The counterfactual state refers to the hypothetical treatment effect of training for immigrants, if they had the same expected labor market success as natives with identical immigrant characteristics X _M = 1, i.e. $E(Y_{M=0}^1-Y_{M=0}^0|X_{M=1},D=1,M=1)$. The IFE is therefore given by:

$$\mathit{\text{IFE}}=E(Y_{M=1}^1-Y_{M=1}^0|X_{M=1},D=1,M=1)-E(Y_{M=0}^1-Y_{M=0}^0|X_{M=1},D=1,M=1).$$

(6)

Since we consider treatment effects of short-term training as the hypothetical outcomes in the identification of the IFE, we only consider participants as the population of interest. Unfortunately, the second term on the right-hand side of Equation (6) cannot be estimated directly from the data since the labor market outcomes of natives ($Y_{M=0}^1,Y_{M=0}^0$) are not observed for the immigrants (M = 1).

To estimate Equation (6), we have to provide a proxy for the unobserved hypothetical treatment effect of the immigrants. The estimated treatment effect of native participants who are matched on all relevant labor market characteristics to immigrant participants in short-term training programs can serve as an approximation. Again, we use the advantage of a balancing score to condense the information of the distribution of the X _M = 1 into a single index function, b(X _M = 1). The estimation of this balancing score is arranged by estimating a probit of being in the immigrant group, M = 1, on the comprehensive set of available covariates used in the estimation of the treatment effects. By doing so, we can resemble the counterfactual hypothetical treatment effects of the immigrants when they were like native Germans. Conditional on the estimated balancing score, we have to assume mean conditional independence of the expected treatment effect of natives from immigrant status, i.e.

$$E(Y_{M=0}^1-Y_{M=0}^0)\coprod M\left|b\right(X_{M=1}),D=1.$$

(7)

If assumption (7) holds, we can replace the second term on the right-hand side in Equation (6) with the estimated treatment effect of natives possessing the observable characteristics of the immigrants:

$$E(Y_{M=0}^1-Y_{M=0}^0|b\left(X_{M=1}\right),D=1,M=1)=E(Y_{M=0}^1-Y_{M=0}^0\left|b\right(X_{M=1}),D=1,M=0).$$

(8)

5 Estimated employment effects

5.1 Implementation and quality of the estimates

For the estimation of program effects we stratify our data by migration background and quarter of program start. Since short-term training usually is assigned early during a welfare spell, the number of treated observations declines over time. This is in particular true for job search training, which – due to its content – is mainly used in the first quarter of a welfare spell. As the numbers of treated individuals in this and the other forms of training become too small in later quarters for estimation, we concentrate our analysis on quarters with a sufficient number of participants. In total, we estimate treatment effects for 28 different strata.

For the propensity scores, we have estimated separate probit models for each group, each treatment, and for up to the first four quarters of the welfare spell. Each model estimates the probability of starting a program in quarter u, conditional on X, conditional on having reached the welfare duration of u ∈ {1, …, 4} quarters, and conditional on not having received a treatment before u in the welfare spell. Explanatory variables regarded cover sociodemographic characteristics, details of the household composition of the individual, aspects of qualification and experience, a comprehensive labor market history, and regional information. We consider the age of the individual in four classes (18 to 24 years, 25 to 34 years, 35 to 49 years, and above 50 years) and whether the person is married or lives in cohabitation. Details of the household composition cover the number of welfare recipients (like spouses or partners), the number of welfare recipients under age 18, number of persons who are not able to work, and the number of children under age 15 living in the household. In addition, we consider the total size of the household that can include further persons not dependent on welfare. The (formal) qualification is given by the degree of schooling (no schooling, basic schooling, intermediate schooling, professional schooling, and higher secondary schooling providing a university entrance qualification) and the occupational training the person possesses (no occupational training, basic occ. training, intermediate occ. training, university of applied sciences degree, university degree). Regional information is regarded by dummies for the sixteen federal states in Germany and a dummy for East Germany.

The individual labor market history is considered for up to 72 months (6 years) before (potential) program participation. The characterization takes account of episodes of employment, unemployment, searching for a job, participation in ALMP programs, and out of the labor force. To enable incorporation in the empirical models, information is aggregated in a first transformation into six-months intervals by summing up the number of 14-day spells in the respective intervals. Since spells are defined in 14-day episodes, a possible limitation of this transformation is that two spells are weighted equally independently of whether they occurred subsequently or were interrupted. A second transformation, therefore, counts the number of non-interrupted spells within six-months intervals. In a third transformation, the average durations in the five labor market states during the last six months and last 24 months prior to (potential) program participation are regarded. A fourth transformation normalized the relative time spent in the different states over six-months intervals. In addition to the detailed labor market history, a dummy variable indicates that the person has not been employed in any form (full-time or part-time employment subject to social insurance contributions or minor employment) during the years 2001 to 2005.

We distinguish the available information into 22 blocks of variables. The specifications of the final models used in the estimations were obtained by starting with the full set of variables and a stepwise dropping of jointly insignificant variable-blocks (indicated by F-tests) in order to provide a parsimonious specification.The exact specifications of the estimated 28 propensity score models cannot be presented here. Results reveal, that especially age, educational attainment, vocational qualification, household composition, region, and employment history within the last two years before (potential) program participation are relevant factors that must be accounted for when estimating the effects of short-term training programs.

Valid estimation of treatment effects requires, that the covariates included in the propensity score are balanced between treatment and comparison group after matching. To check this, we applied a procedure suggested by Smith and Todd (2005b):

$$\begin{array}{ll}{X}_{k_u}& ={\beta }_0+{\beta }_1\widehat{p}\left(X_u\right)+{\beta }_2\widehat{p}{\left(X_u\right)}^2+{\beta }_3\widehat{p}{\left(X_u\right)}^3+{\beta }_4\widehat{p}{\left(X_u\right)}^4\\ +{\beta }_{5}D+{\beta }_{6}D\widehat{p}\left(X_u\right)+{\beta }_{7}D\widehat{p}{\left(X_u\right)}^2+{\beta }_{8}D\widehat{p}{\left(X_u\right)}^3+{\beta }_{9}D\widehat{p}{\left(X_u\right)}^4.\end{array}$$

(9)

Equation (9) was estimated for each variable $X_{k_u}$ included in the propensity score of program participation in quarter u. Afterwards, the null hypothesis of β ₅ to β ₉ being jointly zero was tested. The test indicates, whether there are differences due to the treatment indicator conditional on a quartic polynomial of the propensity score. If ideal balancing is achieved all those coefficients should be zero. The test is passed in 95% or 1,387 of 1,460 cases at the 1% significance level (see Table 2). Thus, balancing quality is high and sufficient to obtain valid treatment effects.

Table 2 Balancing test for the estimation of treatment effects ^a,b

To ensure common support in each estimation sample, we delete all treated individuals whose propensity score is smaller than the smallest or larger than the largest propensity score in the respective group of non-treated individuals. As can be seen from Table 3, in most cases all treated individuals are on support.

Table 3 Common support condition for the estimation of treatment effects ^a

The outcome, drop-off from welfare conditional on employment uptake, is measured monthly from the first month of the sequent quarter after (potential) participation onwards for up to twelve months.¹² The treatment effects are estimated using kernel density matching applying the algorithm provided by Leuven and Sianesi (2003). Standard errors are calculated by bootstrapping with 250 replications.

5.2 Estimation results

Table 4 shows the estimated ATTs for participants starting one of the four short-term training programs during the first year on welfare. The results indicate, that aptitude tests have a positive impact on the probability of natives and immigrants to take up employment. This positive impact does not depend on the exact timing of the training. However, the absolute size and the significance of the effects differ across quarters and also between the two considered groups. For programs starting in the first quarter of welfare receipt, we observe larger treatment effects for natives than for immigrants. One year after starting the program, the ATT for natives amounts to 9.4 percentage points, whereas the corresponding value for immigrants is 6.5 percentage points. The effects one year after program start are substantial when compared to the overall outflow rate from welfare at this point in time. On average, 25 percent of the natives leave the welfare system within 15 months after inflow. For immigrants we observe an outflow rate of about 20 percent.

Table 4 Estimated ATT ^a

In contrast to the first quarter, for aptitude tests starting in the second and third quarter we observe, that treatment effects are larger for immigrants than for natives. During the whole observation period, the estimated effect for immigrants amounts to more than 10 percentage points, whereas the corresponding estimate for natives ranges between 6.4 and 9.7 percentage points. In the fourth quarter, the picture is again reversed. We now find large treatment effects with a magnitude of more than 15 percentage points for natives, while the effect for immigrants amounts to about 10 percentage points.

Compared to aptitude tests, job search training is rather ineffective. We estimate insignificant employment effects for natives in all considered quarters. For immigrants the estimated effects are also insignificant in the second and third quarter. Only in the first quarter, we find significantly positive effects at the end of the observation period. Nine months after starting the program, the average effect amounts to 6.4 percentage points. Thereafter, however, the effect declines to about 4 percentage points one year after program start.

Contrary to job search training, skill provision seems to be more effective. For natives who start training in the first quarter of the welfare spell, we observe positive employment effects. The training increases the probability of participants to take up a job and to leave the welfare system by more than 10 percentage points in the second half of the observation period. For immigrants we also observe positive employment effects. However, compared to natives effects are smaller in magnitude and only slightly significant. One year after program start the probability to find a job is increased by 6 percentage points. In the remaining quarters, employment effects are mostly insignificant for natives and immigrants. Only for immigrants participating in the third quarter, we observe increasingly positive effects during the observation period.

Even though some training modules show a positive impact on the probability to take up employment, the combination of two or three modules in one program is rather ineffective. For natives we find no significant effect, irrespective of the quarter considered. A similar picture arises for immigrants for the first and second quarter. In the third quarter we find increasingly positive employment effects for this group during the observation period albeit the degree of statistical significance is low. Nine months after program start the estimated effect amounts to 7.4 percentage points, but the effect is significant only at the 10% level.

Overall, the results indicate that aptitude tests work best both for natives and immigrants, irrespective of the timing of treatment. When applied early during the welfare spell, skill provision is helpful for welfare recipients, too. Job search training and combined training programs are rather ineffective. Aptitude tests and skill provision are programs that have a direct focus on employment uptake. They aim at increasing qualifications demanded by the labor market. Aptitude tests, for example, include the screening of participants’ potential to work in occupations looked for by employers and the teaching of skills relevant in these occupations. Therefore, aptitude tests improve the matching quality between participants and open jobs. Skill provision also aims at increasing qualifications demanded by employers. However, the focus of skill provision is more general than in the case of aptitude tests. It involves mainly qualifications that are not occupation-specific but are basic requirements for employment uptake in many occupations (e.g. computer courses). In contrast to aptitude tests and skill provision, job search training does not provide a direct path to employment. It can be seen as a preparatory program for employment uptake. The program helps to get in touch with employers, but does not provide specific skills required by them. This might explain the disappointing employment effects of this program. A combination of job search training with skill provision or aptitude tests might be more effective.

The fact that we do not find positive employment effects for combined training programs can have two potential reasons. First, in our data we are not able to differentiate which programs are combined. There might be effective and ineffective combinations which overlap and drive our estimation results to get insignificant. Second, there seems to be a particular selection into combined programs. Caseworkers assign the program in particular to welfare recipients with multiple disadvantages and adverse labor market histories. The combined training programs may not be sufficient to address and overcome these problems.

Our results add to the evaluation literature of training programs by showing that there can be considerable effect heterogeneity between different training modules. The overall picture in the literature indicates modestly positive employment effects of short-term training (see e.g. Kluve 2010 and Card et al. 2010). The same is true for our results. Yet, looking at different training modules reveals that some modules are more effective than others. The more employment-oriented a program is, the more beneficial it seems to be for participants.

6 Estimated IFEs

6.1 Implementation and quality of the estimates

For the estimation of the IFEs, we follow a matching protocol similar to the one used for the estimation of treatment effects above. Based on the estimated ATTs for natives and immigrants for the four considered short-term training programs starting in the first quarter of the welfare spell, we have calculated the expected treatment effects on the individual level and have kept only the information of the participants.¹³ Based on these samples, we have then estimated the balancing scores for the likelihood of being member of the immigrant group incorporating all available information in the data. For the specification of the balancing score we applied the same selection procedure of the relevant variables using F-tests and the Smith-Todd-balancing test afterwards to support the inclusion of the single variables in the balancing score. Treated natives with balancing scores identical to the scores of the treated immigrants were matched to treated immigrants, where kernel density matching was applied weighting the matched partners according to the distance in balancing scores. Standard errors were calculated by bootstrapping with 250 replications. The resulting effect gives us the average difference in program effects between immigrants and natives which is due to the IFE keeping all observable characteristics constant.

In the specifications of the balancing score models for the estimation of the IFEs, we use on average more covariates than we used for the separate estimation of the treatment effects of short-term training for immigrants and natives. Nevertheless, it turns out again that especially age, educational attainment, vocational qualification, household composition, region, and employment history within the last two years before (potential) program participation should be accounted for in these models. Table 5 shows that covariates are balanced very well and that matching quality is of the same high degree as in the first matching step. It has to be noted though, that the share of persons, who are not on common support and are excluded from the analysis, is larger as in the estimation of treatment effects. It varies between 4.2 percent (aptitude tests) and 13.0 percent (job search training).

Table 5 Balancing test for the estimation of IFEs ^a and common support ^b

6.2 Estimation results

Section 5.2 showed that the treatment effects of the considered training programs differ to some extent between natives and immigrants. Therefore, the question arises what might cause these differences. Are they due to differences in the observable characteristics of the two groups or are they due to unobservable differences subsumed in the immigrant fixed effect? To answer this question, we estimate the IFE and Table 6 displays the corresponding results. The first row of each block in the table depicts the raw differential of differences in the ATTs between immigrants and natives for the respective program. This raw differential is calculated from the results presented in section 5.2. The p-value denotes statistical significance of the difference in ATTs of natives and immigrants. One can see that raw differentials are relatively small and mostly statistically insignificant. This implies that short-term training programs have on average a similar effect on both natives and immigrants.

Table 6 Differences in treatment effects between natives and immigrants ^a

We are, however, more concerned whether the programs have similar effect on comparable immigrants and natives. For this, we need to look at the magnitude and significance of the IFE. The third row of each block in the table is denoted by IFE and shows the estimated part of the raw differential which is due to the IFE. In other words, the IFE indicates by how much the treatment effect of a program is changed due to the migration background holding all other factors fixed. The IFE of 0.014 for aptitude tests three months after program start, for example, means, that participating immigrants have on average a 1.4 percentage points larger treatment effect than native participants with identical socio-demographic characteristics. Therefore, immigrants tend to benefit more from aptitude tests than natives holding all other characteristics constant. However, this immigrant fixed effect is not statistically significant at the conventional levels as can be seen from the p-value. Six months after program start, we observe a similar picture. The IFE is positive but insignificant. During the following months, the IFE increases and reaches a maximum of 6.4 percentage points nine months after program start. In this month, the effect is statistically significant at the 10% level. At the end of the observation period, the IFE decreases to about 3 percentage points and lacks statistical significance again. When looking at job search training, skill provision, and combined training programs, we do not estimate statistically significant IFEs. For all three programs, the IFE tends to be negative at the end of the observation period. In absolute terms, it is largest for skill provision and nearly identical for job search training and combined programs.

The absence of significant IFEs indicates that there are no differences in the effectiveness of training programs between natives and immigrants with otherwise identical characteristics. Vice versa, differences in the employment effects arise because of differences in observable socio-demographic variables like, for example, age or educational achievement. Thus, there seems to be a general potential for caseworkers to improve the targeting of programs at participants. The optimal assignment of programs is, however, beyond the scope of this paper.

7 Conclusion

Based on comprehensive administrative data providing rich and unique information on immigrant and native welfare recipients in Germany, we have evaluated the effects of four different short-term training programs on the exit-rate from welfare by take up of employment and have identified differences in the treatment effects between both groups. To take account of differences due to the timing of the intervention during the welfare spell, we have applied a propensity score matching estimator in the dynamic setting. Afterwards, we have estimated the part of the effect difference between immigrants and natives that results from group attachment solely (immigrant fixed effect). It determines differences in employment chances of participants with and without migration background that cannot be explained by differences in composition.

The estimation results of the treatment effects exhibit substantial effect heterogeneity. Participation in aptitude tests results in positive employment effects for both immigrants and natives. Skill provision is valuable for employment when provided early. Job search training and combined programs, in contrast, are ineffective. Thus, programs enhancing occupation-specific knowledge are more effective than programs focusing on qualifications which are not of direct use for potential employers.

Our estimates of the immigrant fixed effect do not indicate significant differences in treatment effects between immigrants and natives beyond observable characteristics. These results reveal a general potential for welfare agencies to improve the overall use of programs and the targeting of programs at participants based on observable characteristics. On the other hand, they show that there is scope for improvement in the activation of immigrant welfare recipients. Assignment of immigrants to job search training and combined training programs should be reconsidered. The use of aptitude tests and skill provision at an early stage of the welfare spell should be extended. An adjustment of activation into this direction can contribute to reduce the still high number of immigrant welfare recipients in Germany.

Endnotes

¹ See OECD (2008 2012) for a comprehensive description of the situation.

² Possible reasons for this over-representation can be different (un)employment dynamics of immigrants compared to natives, i.e. immigrants may be more likely to stay unemployed or less likely to stay employed. Uhlendorff and Zimmermann (forthcoming) show that the first reason provides the relevant explanation, and immigrants need more time to find a job. Nevertheless, employment stability does not differ between immigrants and natives. This is a further argument for the expected higher need of support of welfare recipients to get employed.

³Clausen et al. (2009) evaluate the effects of ALMP programs on the hazard rate into regular employment for newly arrived immigrants in Denmark. The programs are part of the integration policies specifically designed for facilitating the labor-market integration of newly arrived immigrants (introduced in 1999). In addition, Cohen-Goldner and Eckstein (2010) evaluate a government provided training program for highly-skilled female immigrants from the former Soviet Union in Israel. Heinesen et al. (2013) investigate the employment effects of ALMP programs for non-western immigrants in the Danish welfare system. Butschek and Walter (2013) provide an overview of the effectiveness of ALMP programs targeted at immigrants in European countries. See also Rinne (2013) for a comprehensive overview on the evaluation of immigration policies (including the evaluation of ALMP programs.)

⁴ Besides others, job search assistance programs have been evaluated by Crépon et al. (2005) for France, by Blundell et al. (2004) for England, by Graversen and van Ours (2008) for Denmark. For Germany, evaluations have been provided by Hujer et al. (2006) or Wolff and Jozwiack (2007). Thomsen (2009b) provides a detailed synopsis on the effects of these programs from empirical studies for nine European countries.

⁵ Welfare recipients differ in their composition from short-term unemployed persons. See Thomsen (2009a) for an analysis of differences in employability between unemployed persons and welfare recipients in Germany considering a comprehensive spectrum of socio-demographic information.

⁶ Short-term training programs were introduced for unemployment benefits recipients in Germany in 1997/1998. In 2005, welfare benefit recipients became eligible to participate in the programs, too. Since 2009, short-term training programs are subsumed in the group of “programs of activation and vocational integration” (MaSSnahmen zur Aktivierung und beruflichen Eingliederung) in Book III of the German Social Code.

⁷ Short-term training programs can be provided on-the-job within firms and off-the-job. If provided off-the-job, activities are conducted by specialized service providers (Bildungsträger). Evaluation of the treatment effects of on-the-job courses may be complicated due to potential windfall gains of the supporting employers that have to be considered. For this reason, we concentrate the analysis on off-the-job courses only.

⁸ Unfortunately, the administrative records of the FEA contain neither information about the place of birth nor about the parents of the individual. Moreover, since minors (persons under 15 years of age) do not appear in any of these data sources, we cannot identify immigrants who were naturalized at an early age and cannot distinguish first and second generation immigrants.

⁹ The native Germans reflect a regional-identically distributed sample of immigrant welfare recipients, but do not reflect a random sample of native German welfare recipients.

¹⁰ Data for the analysis were extracted in February 2009. Due to delays in reporting by employers, our observation period ends in July 2008.

¹¹ Recent empirical literature highlights, that the starting point of the program within the individual welfare spell may be an important determinant for the selection of participating individuals, as well as for the type of program the individual is assigned to, see e.g. Abbring and van den Berg (2003), Fredriksson and Johansson (2008), Hujer and Thomsen (2010), Thomsen (2007), or Thomsen and Walter (2010).

¹² For programs assigned in the third or fourth quarter of the welfare spell, the observation period is less than 12 months due to the time horizon of our data. It amounts to 9 and 6 months, respectively.

¹³ We only estimate the immigrant fixed effects for the first quarter, since in the other quarters the number of program participants is too small.