AFFIRMATIVE ACTION AT UNIVERSITIES INTERNATIONALLY

The debate about affirmative action in the United States, for example, is older than in Brazil. Since 1960, measures have been sought to minimize the inequality of opportunities faced by minorities.

The arguments against affirmative policies are based on the disrespect for meritocracy and the dissociation of race as a determinant for educational performance. Many critics of affirmative action omit that, in North-American history, there has always been positive discrimination⁴ for white men (^{Oliven, 2007}).

In the 1960s, the University of California (UC) was the first to introduce affirmative action in order to promote equal opportunities. However, in 1977, complaints about quality reduction in the university started. Thus, the UC admission process, which combines high school performance with the Scholastic Assessment Test (SAT), becomes more rigorous, so that the admission of students through affirmative actions is reduced (^{Moehlecke, 2004}).

As the first university to revoke race as a criterion for admission to universities in the second half of the 1990s, the UC was able to verify, in 2001, that the representation of black students reached similar levels to those in the 1960s. In 1996, the racial issue at the University of Texas also ended and was replaced by the percentage regime⁵, similarly adopted by UC and in Florida (^{Cortes, 2010}; ^{Dickson, 2006}; ^{Moehlecke, 2004}). The percentage used for admission to UC, 4%, is much lower than that of the University of Texas, 10%, while in Florida, with the program called Florida’s Talented 20 Program, the top 20% students in the class in high school will be admitted to a public state university (^{Hinrichs, 2012}).

The share does not present the same results of affirmative actions for black students as before, but it was able to increase their representativeness in universities (^{Moehlecke, 2004}; ^{Dickson, 2006}). Still on the United States, in Washington and Nebraska, voters voted to end affirmative actions (^{Silva, 2014}; ^{Hinrichs, 2012}).

As in the United States, the establishment of affirmative action in Malaysia has affected interest in attending the university in different ways for both quota and non-quota students. The Malaysian government initiated, in 1969, what was called the New Economic Policy, which, among other aspects (labor, economic)⁶, included the racial composition in access to education and the end of access to universities based on individual performance, that is, the Government tried to guarantee access to higher education for Malaysians in relation to Chinese and Indian minorities in the country (^{Sowell, 2017}).

Preferential entry into both education and employment can generate a feeling of a guaranteed future for Malaysians, reducing interest in better performance among quota holders. If, on the one hand, the Chinese minority was forced to improve its educational and professional performance, since vacancies were scarce, on the other hand the Malaysians felt less pressured. There is also a change in the incentive of those who do not benefit from quotas due to the issue of merit involved, which culminated in the departure of tens of thousands of Chinese people to study in Singapore (^{Horowitz, 1985}).

AFFIRMATIVE ACTIONS AT BRAZILIAN UNIVERSITIES

Considering the specificities of each university, some studies were carried out in order to obtain a more precise analysis of some Brazilian HEI. In this context, at Universidade de Brasília (UnB), Universidade Estadual de Campinas (Unicamp), and Universidade Federal da Bahia (UFBA), through the Differential Performance Index (DPI), it was found that the averages between quota and non-quota students were similar and, therefore, the entrance exam score should not be used solely as a method of predicting student performance in HEI (^{Childs and Stromquist, 2015}). Opposition to the quotas points out that this result is due to the quality of some public schools, such as federal institutes and military schools. Students who attend these institutions raise the group’s average and take advantage of university admission at UFBA (^{Guimarães et al., 2010}).

Considering the first semester of courses at UnB, a study found that quota and non-quota students had insignificant average differences, pointing out that this difference in score is greater in the selection process than during the course (^{Cardoso, 2008}).

At UFBA, non-quota students showed superior performance in 13 of the 48 courses analyzed, using the Performance Coefficient (PC)⁷. Quota students have higher performance in the arts and humanities subjects. However, it is worth noting that this result is closely related to courses considered of low prestige⁸. In addition, students from private schools, regardless of color/race, have better scores and performance than students from public schools (^{Peixoto et al., 2016}; ^{Queiroz and Santos, 2007}).

At UnB, there was also a great representation of quota students in the areas of arts and humanities. There was a large number of quota-enrolled students, however, the effectiveness is higher among non-quota ones (^{Cardoso, 2008}). In 2004, through the Final Argument (FA)⁹, it was found that the demand from quota holders is concentrated in less competitive courses, generating self-selection. In addition, in the most demanded courses, the percentages of mothers of students with higher level of education and of students in private education are higher (^{Velloso, 2005}).

As for the influence of parental education for entering university, when analyzing universities with and without quotas, it was found that it has less influence on universities in which the quota law was implemented (^{Arabage and Souza, 2017}). Furthermore, between 2004 and 2012, in the ENADE cycles, there was a decrease in the influence of parents’ education for a student to enter higher education, implying a change in the socioeconomic profile of higher education students in Brazil (^{Ristoff, 2014}). It is noticed that high demand courses, such as Medicine and Dentistry, are composed mainly of non-quota students, corroborating the result found by ^{Velloso (2005}) and ^{Waltenberg and Carvalho (2012}). ^{Mendes Junior (2014}) notes that, although quota holders at the Universidade do Estado do Rio de Janeiro (UERJ) have a worse performance in terms of Income Ratio, they do better in relation to the graduation and dropout rates of the courses, as they present greater persistence in completing their undergraduate courses.

In state, federal, and private institutions, the higher the prestige of the course, the lower the participation of quota students, according to data from ENADE 2008. Regarding performance by race, some authors point out that black students have inferior performance in public and private institutions compared to white students - 3 and 5%, respectively (^{Waltenberg and Carvalho, 2012}). Other researchers point out that white students obtain higher averages only in highly prestigious careers and, in other careers, the performance of black students is higher (^{Velloso, 2005}). The author states that affirmative actions at UnB doubled the possibility of black students entering higher education. A similar result was found by ^{Cardoso (2008}) for some courses.

In this perspective, the performance of students in ENADE suggests that, in public and private HEI, public school quota holders, by combining this with another affirmative actions, obtain higher scores than those who are not quota holders. In addition, quota students due to low income present a different situation from other quota students, considering that they showed inferior performance to other students in both types of institutions (^{Valente and Berry, 2017}; ^{Gutterres, 2015}).

Other aspects not linked to performance are also analyzed by researchers. At Universidade Federal do Sergipe (UFS), there was an increase in the self-esteem of quota students, although there are tensions of economic nature between quota and non-quota students, as well as with professors (^{Neves, Faro and Schmitz, 2016}).

Clashes between students and professors were also observed in the University Council of the Universidade Federal do Rio Grande do Sul (UFRGS), when discussing compensatory policies, such as the reservation of vacancies in higher education, expanding the debate about the relationship among state, social justice, and democracy (^{Batista, 2015}).

ENTROPY BALANCING WITH MATCHING BY PROPENSITY SCORE

The objective of the quantitative evaluation of public policies is to verify whether the group of students who received any treatment, in this case, quota students (treatment group), differs in any way from the group that did not receive the treatment (control group). The best scenario for making this comparison is that the treatment and control groups are as similar as possible in terms of observed variables, except for the result variable. In the case of this study, the outcome variable is the student’s score in ENADE. Methods such as the nearest neighbor or propensity score are quite common, however, in some situations, these fail to adequately balance all distributions of observed variables (^{Hainmueller and Xu, 2013}).

Introduced by ^{Rosenbaum and Rubin (1983}), the procedure of PEP compares the treatment group in relation to the control one in order to determine a confrontational through the similarity in the probability of receiving the treatment, so that selection biases are mitigated due to adjustments in the difference between groups.

The Entropy Balancing Method corrects these balance flaws, since, in entropy¹⁰, weights¹¹ that satisfy a large set of conditions are assigned to achieve the desired equilibrium level. This method generates weightings to the data, enabling the weighted balance at all times in the distributions of the treatment and control covariate, that is, mean, variance, and asymmetry, respectively, the 1^st, 2^nd, and 3^rd moments. Consequently, the efficient use of weights prevents possible loss of relevant information.

In order to properly balance all covariate distributions and reduce selection bias, that is, to obtain an ideal comparison group to the treatment, entropy balancing was used together with the PEP.

In PEP, control units receive a weight calculated according to the propensity score, which is commonly estimated through a probit or logit regression. If this model is correctly specified, then the estimated weight ensures that the covariate distribution of the weighted control units corresponds to the distribution in the treatment group.

However, in practice, this approach often fails to balance all covariates together. In contrast to other matching methods, Entropy Balancing ensures a high balance of covariates between treatment and control groups, even in small samples.

^{Hainmueller (2012}) developed this method, which allows the weighting of a set of data, such that the distributions of the variables in the reweighted observations satisfy a set of special moment conditions, so that there is an exact balance over the first (mean), second (variance), and third (asymmetry) moments of the distribution of independent variables in both treatment and control groups. This method allows the researcher to specify a desirable equilibrium level for covariates, using a set of conditions associated with the moments of the distribution. The advantage of this method over the logit/probit algorithms lies in the ability to directly implement the exact balance.

The idea behind entropy balancing is to find weights for observations that minimize the metric distance between the treatment and control groups, considering the momentum constraints (mean, variance, and asymmetry), normality and non-negativity.

The method is as follows. Consider a sample with n ₁ observations belonging to the group of treatment and n ₀ students in the control group, which were randomly selected from a population of size N ₁ and N ₀ , respectively (n ₁ ≤ N ₁ en ₀ ≤ N ₀ ). Let D _i ∈ {1,0} be a binary treatment variable, which will take the value of 1 if student i belongs to the treatment, and 0 if he/she belongs to the control group. Let X be a matrix containing the observations of J exogenous pretreatment variables; X _ij corresponds to the value of the j-th covariate of student i, such that, X _i = [X _i1 , X _i2 , ..., X _iJ ] refers to the student’s characteristic vector I and X _j refers to the column vector with j-th characteristic (covariate).

Entropy balancing generalizes the propensity score weighting approach by estimating weights directly from a set of balance constraints exploring the researcher’s knowledge of the sample moments. Consider w _i the entropy balance weight chosen for each control unit, which was found by the following weighting scheme that minimizes the metric entropy distance (Equation 1):

That minimization is subject to balance (Equation 2), normalization (Equation 3) and non-negativity (Equation 4) restrictions

Where:

These weights w _i are weightings given to the units of observations found through this method, which makes the treatment and control groups similar in terms of mean, variance, and asymmetry. The weighting scheme consists of three characteristics. First, the h (.) function is a metric distance chosen from the general classes of minimal discrepancy empirical estimators defined by ^{Read and Cressie (1988}). The authors chose to use the ^{Kullback method (1959}) defined by hwi=wilog⁡wi/qi with estimated weight ω _i and base weight q _i 1/n ₀ .

Initially, the covariate that will be included in the reweighting is chosen. For each covariate, a set of balancing restrictions (Equation 2) is specified to match the timing of the covariate distribution between the reweighted treatment and control groups. Moment constraints can be the mean (first moment), variance (second moment), and asymmetry (third moment). A typical balancing constraint is formulated in such a way that m _r contains the moment of a specific covariate X _j for the treatment group and the moment function for the control group is specified as: criXij=Xijr or criXij=Xij-μjr with mean μj.criXi=mr describes a set of R restrictions imposed on covariate moments in the reweighted control group.

Thus, entropy balancing seeks, for a set of students, w _i weights which minimize Equation 1, entropy distance between W, subject to moment restrictions in Equation 2, normalization restriction in Equation 3, which states that the sum of the weights is equal to 1, and the non-negativity constraint Equation 4, which imposes the absence of negative weights.

MATCHING BY PROPENSITY SCORE WITH MULTIPLE TREATMENTS

Students are able to choose to enter via quotas in the selection process for entering HEI, among the options to which they fit. To find the effect of the heterogeneity of the quota type on the test score in ENADE, a new treatment variable was considered, which will assume multiple values.

The PEP methodology with multiple treatments was used, since the treatment variable is defined as categorical. The methodology, approached by ^{Imbens (2000}) for the PEP, indicates a situation in which the student (i) has multiple treatments due to the different types of quotas.

Let T _i (t) be a treatment function, where t indicates the treatment level for each type of quota,t =1,2,…K ; thus, T _i (0) would be the category in which student i did not enter through quotas.

For the identification of multiple treatments, ^{Rosenbaum and Rubin (1983}) established that the Weak Conditional Independence Assumption (WCIA) requires the interdependence of each treatment with each of the potential results Y(t), as well as the local independence between each result and the treatment of interest, this being independent of D(t), where D(t) is a binary indicator, while T indicates the level of treatment. When considering a student belonging to the treatment, T _i (t), the other categories are disregarded, starting to evaluate T _i (t) as a binary variable. Furthermore, in Equation 5, X corresponds to the vector of pretreatment variables. For example, in this study, there are 5 possibilities of quotas, t = 1,2,3,4,5, as described in Chart 1. Supposing a student i entered university by ethnic-racial criterion (t = 1), then, the other categories are disregarded, starting to evaluate T _i (t) as a binary variable D(t), in which, in this example, D(t) = 1, if the student entered university through ethnic-racial criterion and D(t) = 0, if he/she did not enter through quotas.

In this way, WCIA is equivalent to:

In which Equation 5 indicates that the treatment is orthogonal, that is, it is not correlated with the result variable, conditioned to the pretreatment variables X. As the objective is the estimation of the average of the potential result, E[Y(t)], it must be ensured that the average of the Y _i (t) is represented in the subsample of the observations that receive the treatment D _i (t) = 1, even when D _i (t) = 0, the variable Y _i (t) is always unknown.

Given the conditionality of the explanatory variables (X) and the absence of relevance of other potential results defined by the WCIA, the conditioned average result of the treatment is estimated from conditional averages (Equation 6):

Equation 6 shows that the average of the result variable Y (t) (General score of ENADE) will be calculated based on the averages conditioned to the pre-treatment variables.

Thus, one can define the Generalized Propensity Score (GPS) as the conditional probability of a particular level of treatment, considering the observable variables, which equals the conditional expectation of the binary treatment variable according to ^{Imbens (2000}), which can be represented by Equation 7:

Equation 7 shows the probability that the individual will receive treatment, given the pre-treatment variables. For example, given that the student has completed all or most of high school in a public school, what is the likelihood that he/she will be a quota holder in the university?

Considering the WCIA and the GPS, this assumption can be represented by Equation 8:

Equation 8 is a new assumption that the method makes. Initially, it considers that conditionality occurs in the pre-treatment variables (Equation 5). Then, it now considers that this conditionality occurs in the probability of receiving treatment, considering these variables (Equation 8). Given the GPS for multiple treatments, the overlapping assumption is expressed by Equation 9:

Equations 8 and 9 mean that treatment is independent of potential results and independent of pre-treatment variables.

Changes in the level of treatment imply changes in the conditioning argument. Thus, it is necessary to condition the entire set of K +1 scores, {e(t,x)} ∈ T. K represents the total of treatments observed, in this study, k = 5. K +1 means to consider all treatments plus the group that is not treated.

The multiple treatment methodology is similar to the binary case, in which the propensity score is initially estimated, e(t, x) (Equation 7), in a model with multiple treatments, which in this study means several types of quotas. Then, the effect of the treatment level t on the outcome variable (score in ENADE) is estimated. The effect is estimated through the Kernel pairing¹² for each treatment level in relation to the others. Regarding the individual analysis of the treatment levels, it has D(t) = 1 for students belonging to treatment t, while it has D(t) = 0 for students belonging to another category.

VARIABLES AND DATABASES

In order to verify the performance of quota students at ENADE between 2013 and 2015, a sample was constructed with data referring to the fourth cycle of the exam. Each cycle of the test corresponds to an evaluation triennium¹³, so that, at the end of this, most courses are evaluated.

The data used were made available by the National Institute of Educational Studies and Research Anísio Teixeira (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - INEP). In accordance with the Quota Law No. 12.711, state and private HEI were not considered in the sample.

The following explanatory variables were used: gender, race, age, high school (public or private), employment, parental education, and family income (^{Waltenberg and Carvalho, 2012}; ^{Melguizo and Wainer, 2016}; ^{Arabage and Souza, 2017}; ^{Valente and Berry, 2017}). The binary variable indicates two groups: treated (quota students) and control (students with characteristics similar to quota students, but who are not quota students).

The prestige variable was created based on the courses that obtained an ENEM cut-off score above 700 points at the largest Brazilian universities. The time variable represents the difference between the year of ENADE and the year of entry into the undergraduate course, aiming to capture the impact of permanence on the student’s score. Geographic variables were also considered in the model. Chart 1 describes the variables used.

DESCRIPTIVE ANALYSIS

Table 1 presents the descriptive statistics of the result variable, considering the type of quota. It appears that the public school criterion has the highest average in the test score, 50.22, followed by Combination, 49.85, while others have the lowest average, 41.56.

Table 2 shows the descriptive statistics of the outcome and control variable for quota and non-quota students in federal HEI from 2013 to 2015. Non-quota students have a higher average in the exam’s test score throughout this period of time, and the difference of their grades and the grade of quota students is 1.4 points in 2013, 2 points in 2014, and 1.7 points in 2015.

As expected, quota holders have a greater representation of black students (16.2%), graduates from public high schools (94.4%), and families with income up to 1.5 minimum wages (24.3%). Only 7.7% of non-quota students are black, 48% attended high school in public schools and, on average, 13.8% of non-quota students’ families have an income of up to 1.5 minimum wages. It is noted that quota students have a greater female representation, have parents with lower education, and complete their degrees in less time than non-quota ones.

RESULTS OF THE MODELS

Results of the entropy balancing model

The moment conditions for the distribution of the control covariables are shown in Table 3. Significant differences are observed between the group of treatment (quota holders) and the control group (non-quota holders) in the 3 moments of the distribution. The variables that have a more similar distribution between treatment and control are Age1, since the two groups have observations in the age group between 16 and 25 years, the family income up to 1.5 minimum wages and the dummy indicative of a prestigious course. With the exception of these variables, mean, variance, and asymmetry of all the others are statistically different between groups before balancing by entropy.

Table 3 - Moment conditions of covariates before balancing by entropy. 

Covariates	Treatment			Control			Differences			p-value
	Mean	Variance	Asymmetry	Mean	Variance	Asymmetry	Mean	Variance	Asymmetry
Gender	0.3787	0.2353	0.5002	0.4512	0.2476	0.1962	-0.0752	-0.0123	0.304	0.000
Race	0.1627	0.1362	1.828	0.07748	0.07148	3.161	0.08522	0.06472	-1.333	0.000
Marital Status	0.7552	0.1849	-1.187	0.7979	0.1612	-1.484	-0.0427	0.0237	297	0.000
High School	0.944	0.055284	-3.863	0.481	0.2496	0.07618	0.463	-0.19431	-3.863	0.000
Traditional type	0.7609	0.1819	-1.223	0.8456	0.1306	-1.912	-0.0847	0.0513	689	0.000
Technical type	0.09558	0.08645	2.751	0.07689	0.07098	3.176	0.01869	0.01547	425	0.000
Scholarship	0.4831	0.2497	0.06775	0.4602	0.2484	0.1597	0.0229	0.0013	-0.0919	0.000
Employment	0.2455	0.1852	1.183	0.232	0.1782	1.27	0.0135	0.007	1.181	0.000
Educ_mother1	0.361	0.2307	0.579	0.2065	0.1639	1.45	0.1545	0.0668	-0.871	0.048
Educ_mother2	0.1045	0.09357	2.586	0.2271	0.1755	1.303	-0.1226	-0.08193	1.283	0.000
Educ_father1	0.4577	0.2482	0.17	0.2709	0.1975	1.031	0.1868	0.0507	-1.030	0.007
Educ_father2	0.08658	0.07909	2.94	0.2087	0.1651	1.434	-0.1221	-0.08601	-1.431	0.000
Family income	0.2436	0.1842	1.195	0.1385	0.1193	2.094	0.1051	0.0649	-899	0.483
Age1	0.5396	0.2484	-0.1589	0.5987	0.2403	-0.4027	-0.0591	0.0081	0.2438	0.079
Age2	0.372	0.2336	0.5297	0.3428	0.2253	0.6623	0.0292	0.0083	-0.1326	0.000
Time	3.906	1.495	0.8005	4.3	2.469	1.482	-0.394	-0.974	-1.481	0.000
Prestige	0.0006292	0.0006289	39.83	0.001377	0.001375	26.89	0.00075	-0.00075	12.94	0.285
North	0.1876	0.1524	1.601	0.1357	0.1173	2.128	0.0519	0.0351	-527	0.000
Northeast	0.2821	0.2025	0.9686	0.2785	0.2009	0.9884	0.0036	0.0016	0.7677	0.000
South	0.1714	0.1421	1.744	0.139	0.1197	2.804	0.0324	0.0224	-1.060	0.000
Midwest	0.06554	0.06124	3.511	0.0995	0.0896	2.676	-0.0339	-0.02836	835	0.000

Following ^{Hainmueller (2012}), it is possible to combine Entropy Balancing with PEP, so that the estimation of propensity scores starts to consider the balancing weights. Thus, Graphics 1 and 2 show the difference in the linear distribution of these scores before and after using this algorithm. Observing the adjusted overlap of the two groups, it is proved that the scores were shown to be balanced. In Table 4, after the entropy balance, the differences between the variables associated with the performance in ENADE in the two groups become insignificant, showing themselves to be properly balanced (Graphics 1 and 2).

Source: authors based on ENADE 2013-2015.

Graphic 1 - Propensity Score Matching before entropy balancing. 

Graphic 2 - Propensity Score Matching after entropy balancing. 

Table 4 - Momentary conditions of covariates after balancing by entropy. 

Covariates	Treatment			Control			Differences			p-value
	Mean	Variance	Asymmetry	Mean	Variance	Asymmetry	Mean	Variance	Asymmetry
Gender	0.3787	0.2353	0.5002	0.3787	0.2353	0.5002	0	0	0	0.982
Race	0.1627	0.1362	1.828	0.1627	0.1362	1.828	0	0	0	0.853
Marital status	0.7552	0.1849	-1.187	0.7552	0.1849	-1.187	0	0	0	0.804
High school	0.944	0.05284	-3.863	0.944	0.05284	-3.863	0	0	0	0.722
Traditional type	0.7609	0.1819	-1.223	0.7609	0.1819	-1.223	0	0	0	0.771
Technical type	0.09558	0.08645	2.751	0.09558	0.08645	2.751	0	0	0	0.813
Scholarship	0.4831	0.2497	0.06775	0.4831	0.2497	0.06775	0	0	0	0.971
Employment	0.2455	0.1852	1.183	0.2455	0.1852	1.183	0	0	0	0.960
Educ_mother1	0.361	0.2307	0.579	0.361	0.2307	.579	0	0	0	0.907
Educ_mother2	0.1045	0.09357	2.586	0.1045	0.09357	2.586	0	0	0	0.654
Educ_father1	0.4577	0.2482	0.17	0.4577	0.2482	.17	0	0	0	0.810
Educ_father2	0.08658	0.07909	2.94	0.08659	0.07909	2.94	-0.00001	0	0	0.694
Family Income	0.2436	0.1842	1.195	0.2436	0.1842	1.195	0	0	0	0.831
Age1	0.5396	0.2484	-0.1589	0.5396	0.2484	-0.1589	0	0	0	0.989
Age2	0.372	0.2336	0.5297	0.372	0.2336	0.5297	0	0	0	0.871
Time	3.906	1.495	0.8005	3.906	1.495	0.8005	0	0	0	0.979
Prestige	0.0006292	0.0006289	39.83	0.0006293	0.0006289	39.83	0.0000001	0	0	0.762
North	0.1876	0.1524	1.601	0.1876	0.1524	1.601	0	0	0	0.952
Northeast	0.2821	0.2025	0.9686	0.2821	0.2025	0.9686	0	0	0	0.988
South	0.1714	0.1421	1.744	0.1714	0.1421	1.744	0	0	0	0.938
Midwest	0.06554	0.06124	3.511	0.06554	0.06124	3.511	0	0	0	0.667

The ATE was estimated considering the propensity scores calculated with the weighting by entropy for the result variable (Table 5). It is observed that the effect of quotas on the general test score was negative and significant, with a difference of approximately 4%. Effectively, quota students, taking into account the average of courses at ENADE, obtain a score below that obtained by non-quota students in approximately 2 points.

Table 5 - Impact of quotas on the ENADE score. 

Result Variable	Quota holders	Non-quota holders	ATE	(%)	SD	t-test
Test score ENADE	48.079	50.117	-2.038	-4.06	0.10	-20.37

ATE: average treatment effect on the treated; SD: Standard deviation; ENADE: National Assessment of Student Achievement (Exame Nacional de Desempenho de Estudantes).

This average difference in performance between the two groups represents 0.13 standard deviation, which means that a quota student is, on average, at the 44^th percentile while the non-quota student is, on average, at the 49^th percentile of the score distribution. However, it is necessary to take into account that this differential is impacted differently depending on the type of quota. These heterogeneities show that affirmative actions need to be analyzed by different aspects.

Results of model with multiple treatments

Through the PEP, an attempt is made to mitigate the selection bias caused by the variables observed through the pairing of the groups through the scores estimated in the model, in such a way that it simulates a random experiment. The propensity score was estimated as the probability that the student belongs to one of the categories of the variable quotas. The category of non-quota students was considered as the basis for estimating this model. The results of the multinomial logit model are shown in Table 6. The sign of the coefficients indicates that there are covariates associated, both positively and negatively, with the probability of the discrete variable (dimension types) assuming the defined values (D = 1).

Table 6 - Result of the multinomial logit model. 

Variables	Treatments
	Ethnic-racial		Income		Public school		Combination		Others
	Coefficient	p-value	Coefficient	p-value	Coefficient	p-value	Coefficient	p-value	Coefficient	p-value
Gender	0.0822127	0.080	-0.0391799	0.418	-0.1892341	0.000	-0.1541321	0.000	-0.2690981	0.000
Race	2.42146	0.000	0.0338291	0.649	-0.2010925	0.000	1.163633	0.000	0.3061779	0.000
Marital status	0.2761764	0.000	0.2583822	0.000	0.2453653	0.000	0.2101809	0.000	-0.1574784	0.000
High school	1.735686	0.000	2.517273	0.000	4.288343	0.000	3.331698	0.000	0.6021486	0.000
Traditional type	0.3985074	0.000	-0.0827084	0.284	0.3427349	0.000	0.142501	0.025	-1.13519	0.000
Technical type	0.2281606	0.015	-0.3149657	0.004	0.2892724	0.000	-0.0648523	0.443	-1.110021	0.000
Scholarship	0.018347	0.639	0.1825231	0.000	0.1907278	0.000	0.1859019	0.000	-0.1652407	0.000
Employment	-0.3368237	0.000	-0.2168624	0.001	-0.2528883	0.000	-0.2364539	0.000	0.3402165	0.000
Educ_mother1	0.0187951	0.690	0.1024509	0.077	-0.0249382	0.263	0.0533111	0.237	0.19389	0.000
Educ_mother2	-0.2304234	0.000	-0.433509	0.000	-0.1659937	0.000	-0.3681685	0.000	-0.0539236	0.367
Educ_father1	0.0135826	0.763	-0.2572064	0.004	-0.0358129	0.091	0.0484722	0.625	0.360556	0.000
Educ_father2	-0.0933321	0.139	-0.2572064	0.006	-0.1274693	0.000	-0.2766585	0.000	-0.1238162	0.069
Family Income	0.1562583	0.000	0.595882	0.000	-0.0759462	0.001	0.0328715	0.460	0.0762599	0.052
Age1	0.0689564	0.468	0.3335827	0.003	0.5391591	0.000	0.2952196	0.001	-1.113874	0.000
Age2	0.1750883	0.046	0.0568177	0.588	0.1743214	0.000	0.0099819	0.901	-0.3514717	0.000
Time	-0.0451715	0.000	-0.1473223	0.000	-0.0964815	0.000	-0.1537905	0.000	-0.2394884	0.000
Prestige	-0.1345502	0.751	-12.39588	0.000	-0.5024968	0.198	0.8982139	0.058	-0.8170621	0.274
North	-0.1706847	0.013	-0.3060784	0.000	-0.4895727	0.000	-0.376322	0.000	0.9092588	0.000
Northeast	0.6446868	0.000	-0.1331396	0.030	0.0152833	0.511	-0.2085642	0.000	0.3523564	0.000
South	0.8628803	0.000	0.0765335	0.282	0.3142073	0.000	-0.6270097	0.000	0.0235961	0.699
Midwest	0.8434541	0.000	-0.6612338	0.000	-0.6670581	0.000	-0.9724663	0.000	-0.1493261	0.045
Constant	-6.312464	0.000	-5.820389	0.000	-5.739137	0.000	-5.925265	0.000	-1.637878	0.000

It is observed that boys are associated with a greater probability of being quota holders through the ethnic-racial criterion. For black, single students who went to school at public high schools of the traditional type, the effect was positive and significant¹⁴. A high level of education by the mother and the fact that the student works are associated with a reduction in the probability of entering this quota. As for the age of the student, this indicates that the higher the age, the greater the probability of having entered by this criterion.

The variables indicating marital status, public high school, family income and first age range increase the probability that the student has entered through the income criterion. The students least likely to have entered university through this type of quota are those who studied in technical high schools, in all regions, with the exception of the South, those who work and those who entered a prestigious course. On the other hand, quota holder students with a family income of up to 1.5 minimum wages and aged between 16 and 25 years tend to increase the probability.

In relation to the quota per public school, it is verified that students from public or technical schools, who have scholarships and who are in the South region are more likely to having entered by this criterion. A negative effect, that is, in which the probability of entering through this quota is reduced, is observed for boys, students who work, those with low family income, and those whose parents have higher schooling.

In the combination criterion, the variables that indicate race, public high school and receiving a scholarship, for example, are associated with an increased probability that the student will have admission through this criterion.

Variables such as sex, marital status, traditional and technical high school, scholarship, age, time and the Midwest region, for example, reduce the probability of having entered university through criteria other than the four mentioned above. However, the effect is positive for race, high school, low parental education, and the Northeast region.

In this work, the Kernel matching algorithm, also applied by ^{Heckman, LaLonde and Smith (1999}), was used in the propensity score. This method pairs all the individuals in the sample, assigning lower weights to the observations of the control group that are less similar in the list of observed variables, seeking to make the two groups analogous. It is possible to verify the pairing through the analysis of the balance of the variables used in the estimation through histograms for the groups of treatment and control. The histograms for each type of quota, that is, ethnic-racial, income, public school, combination and others, were generated in Stata Statistical Software and are found in Graphics 3, 4, 5, 6 and 7, in which the distributions have similar propensity scores, indicating that such propensity scores can be paired.

Source: authors based on ENADE 2013-2015.

Graphic 3 - Overlay of propensity score: ethnic-racial. 

Source: authors based on ENADE 2013-2015.

Graphic 4 - Overlay of propensity score: income. 

Source: authors based on ENADE 2013-2015.

Graphic 5 - Overlay of propensity score: public school. 

Source: authors based on ENADE 2013-2015.

Graphic 6 - Overlay of propensity score: combination. 

Source: authors based on ENADE 2013-2015.

Graphic 7 - Overlay of propensity score: others. 

Table 7 shows the results of the ATE, i.e. the difference between average by groups. The estimated ATE suggested for the outcome variable among the types of quotas was performed after estimating the propensity score and removing a portion of the bias attributed by the observed variables.

Table 7 - Impact of multiple treatments on the result variable. 

Quotas		Test score ENADE
Ethnic-racial	Treatment	46.318
	Control	47.759
	ATE	-1.441
		(-3.02%)
	t-test	-4.81
Income	Treatment	45.203
	Control	49.574
	ATE	-4.371
		(-8.82%)
	t-test	-11.97
Public school	Treatment	50.230
	Control	48.680
	ATE	1.550
		(3.18%)
	t-test	11.70
Combination	Treatment	49.825
	Control	48.844
	ATE	0.98
		(2%)
	t-test	3.55
Others	Treatment	41.565
	Control	44.601
	ATE	-3.03
		(-6.8%)
	t-test	-12.92

ENADE: National Assessment of Student Achievement (Exame Nacional de Desempenho de Estudantes); ATE: average treatment effect on the treated.

It appears that the types of ethnic-racial quotas, income and others showed negative and significant effects on the general test score, whose difference between quota and non-quota students is between 1.4 and 4.3 points to the disadvantage of quota students. That is, for these types of quota, non-quota students obtains higher and statistically significant test scores in ENADE in relation to quota students.

On the other hand, students who entered through public school quotas or a combination of quotas obtained a higher performance differential than non-quota students, with a statistically significant difference of approximately 1.5 and 1 point, respectively.

The results presented in Table 7 indicate that, considering individuals with similar characteristics in both groups, students attending federal HEI by racial and income quotas obtain performance below their non-quota peers at the end of the undergraduate course, with a greatest differential impact for the latter type of quota. The matter of income permeates all higher education, since poorer students probably need to work, in addition to studying, thus reducing the time dedicated only to study. In addition, it is possible to observe that quota students coming from the criterion of public schools manage to surpass the performance of their non-quota peers. This result may suggest that there are public schools of higher quality in high school and that they manage to place their best students in a university, although this result cannot be proven with the analysis performed, being, therefore, only a hypothesis.