Abstract

Nonparametric regression is an approach used when the structure of the relationship between the response and the predictor variable is unknown. It tries to estimate the structure of this relationship since there is no predetermined form. The generalized additive model (GAM) and quantile generalized additive (QGAM) model provides an attractive framework for nonparametric regression. The QGAM focuses on the features of the response beyond the central tendency, while the GAM focuses on the mean response. The analysis was done using gam and qgam packages in R, using data set on live-births, fertility-rate and birth-rate, where, live-birth is the response with fertility-rate and birth-rate as the predictors. The spline basis function was used while selecting the smoothing parameter by marginal loss minimization technique. The result shows that the basis dimension used was sufficient. The QGAM results show the effect of the smooth functions on the response variable at 25th, 50th, 75th and 95th quantiles, while the GAM showed only the effect of the predictors on the mean response. The results also reveal that the QGAM have lower Akaike information criterion (AIC) and Generalized cross-validation (GVC) than the GAM, hence producing a better model. It was also observed that the QGAM and the GAM at the 50th quantile had the same R2adj(77%), meaning that both models were able to explain the same percentage of variation in the models, this we attribute to the fact that mean regression and median regression are approximately the same, hence the observation is in agreement with existing literature. The plots reveal that some of the residuals of the GAM were seen to fall outside the confidence band while in QGAM all the residuals fell within the confidence band producing a better smooth.

Keywords: Generalized additive model, Quantile generalized additive model, Backfitting algorithm, Spline basis function, Nonparametric regression, Weighted loss function.

Received: 5 July 2021 / Revised: 19 November 2021 / Accepted: 10 December 2021/ Published: 27 December 2021

Contribution/ Originality

This study is one of the very few studies that have investigated quantile generalized additive model as a robust alternative to generalized additive model. In the study of both models, the work revealed through some comparison criteria that QGAM is a better alternative to GAM and also illustrated this through some graphs.

1. INTRODUCTION

The classical approach for estimating a regression function is the parametric regression estimation, but models with additive nonparametric effects offer a valuable dimension reduction device throughout applied statistics. Parametric regression assumes that the structure of the regression function is known and depends only on some parameters, and uses the data to estimate the (unknown) values of these parameters. In linear regression it is assumed that the regression function is a linear combination of the components of the predictor variable for some unknown parameters. The general linear regression model is a form of parametric regression, where the relationship between the predictor variable ‘x’ and the response variable ‘y’ has some predetermined form with the parameterized relationship between them given as, say;

However, parametric estimates have a big drawback. Regardless of the data, a parametric estimate cannot approximate the regression function better than the best function which has the assumed parametric structure [1].

In contrast to parametric regression, the nonparametric regression comes in when the structure of the relationship between the response and the predictor variable is unknown. Nonparametric regression tries to estimate the structure of the relationship between the response and the predictor variable since there is no predetermined form for the relationship between them. The nonparametric regression methods are simply alternative statistical approaches used when some assumptions valid for parametric regression methods are not met. The non-parametric methods make fewer assumptions; they are more flexible, more robust, and applicable to non-quantitative data. The generalized additive model and quantile generalized additive model provides an attractive framework for nonparametric regression. The quantile generalized additive model focuses on the features of the response beyond the central tendency, while the generalized additive model is focused on the mean response. In this work, we intend to show that the quantile generalized additive model is a robust alternative to the generalized Additive models.

2. GENERALIZED ADDITIVE MODEL (REGRESSION SPLINE) (GAM)

A generalized additive model is a nonparametric technique; it is a generalized linear model with a linear predictor having sum smooth functions of the predictor variable. These models assume that the mean of the response variable depends on an additive predictor through a nonlinear link function, Trevor and Robert [2]. Generalized additive models permit the response probability distribution to be any member of the exponential family of distributions. The structural form of the model is given by Equation 2.

The model allows flexible specification of the dependence of the response variable on the predictors, by specifying the model only in terms of ‘smooth functions’, rather than detailed parametric relationships. Considering a univariate function, we introduce a smooth function of one predictor, given by the form;

A regression procedure can be viewed as a method for estimating how the value of y depends on the values of x1,…,xn. The standard linear regression model assumes the expected value of ‘y’ has a linear form;

The number of “basis” functions depends on the number of inner knots (that is a set of ordered, distinct values of xj) as well as the order of the spline. Specifically, if we let m denote the number of inner knots, the number of ‘basis’ functions will be given as K = p + 1 +m.

Let’s define a quantity;

An estimation procedure for additive models known as backfitting was used; it was introduced by Breiman and Friedman [3]. This method allows the component functions of an additive model to be represented using almost any smoothing or modeling technique but the degree of smoothness of a model is hard to integrate into this technique.

The basic idea behind backfitting is to estimate each smooth component of the additive model by iteratively smoothing partial residuals from the additive model, with respect to the predictor(s) that the smooth relates to. The partial residuals relating to the jth smooth term are the residuals resulting from subtracting all the current model term estimates from the response variable, except for the estimate of jth smooth. Almost any smoothing method (and mixtures of methods) can be employed to estimate the smooths. Here is a more formal description of the backfitting algorithm.

3. QUANTILE GENERALIZED ADDITIVE MODELS (QGAM)

The most popular nonparametric model is the conditional mean regression model. However, compared with a conditional mean function, the conditional quantile regression function, when evaluated at different quantiles, can reveal an entire distributional relationship between the predictor and the response variable. The traditional quantile regression is concerned with the estimation of the τth conditional quantile regression of y for given x which often sets a linear model as:

To estimate of the coefficients, Koenker and Bassett [4], proposed an L1-weighted loss function given as;

Equation 10 is an expanded form of L1-weighted loss function in Equation 8. So that the positive residuals associated with the response which lies above the regression line are assigned weights while the negative residuals associated with observed responses below the regression line are assigned weights ofFor instance When τ=0.7 each positive residual is weighted 7 times that of a negative residual with weight 1−τ=0.3 and so in optimum for every observation above the regression line approximately 7 will be placed below the line. Hence the regression line represents the 0.7 quantile.

Hence the linear program in Equation 3 is analyzed and solved using the standard form;

To achieve this standard form, g must be positive. To achieve this, residuals are decomposed into positive and negative part using slack variables such that:

Equation 14 is the minimization form of the problem as given by Koenker-Roger [5].

4. RESULTS

The analysis was done using a data set on live-births, fertility-rate and birth-rate, where live-birth is the response variable with fertility-rate and bir-thrate as the predictor variables. This analysis was done using the gam and qgam packages in R software.

4.1. Generalized Additive Model results (GAM)

Table-1. Approximate significance of smooth terms.

Smooth terms	edf	Ref.df	F-value	p-value
s(Fertility-Rate)	7.537	8.321	46.34	<2e-16 ***
s(BirthRate)	7.294	8.694	46.09	<2e-16 ***

Table 1 shows that the smooth functions significantly affect the response.

Plots from Generalized Additive Model (GAM)

Figure-1. Residual plot for Fertility-rate

Figure-2. Residual plot for Birth-rate

The Figure 1 & 2 above shows that some of the residual values didn’t fall within the confidence band.

4.2. Quantile Generalized Additive Model results (QGAM)

Table-2. Approximate significance of smooth terms.

25th quantile
Smooth terms	Edf	Ref.df		Chi-square		p-value
s(Fertilityrate)	6.441	7.230		441.7		<2e-16 ***
s(BirthRate)	6.773	8.128		515.8		<2e-16 ***
50th quantile
Smooth terms	Edf	Ref.df		Chi-square		p-value
s(Fertilityrate)	7.126	7.939		559.1		<2e-16 ***
s(BirthRate)	6.863	8.201		580.9		<2e-16 ***
75th quantile
Smooth terms	Edf		Ref.df		Chi-square		p-value
s(Fertilityrate)	7.364		8.117		1442		<2e-16 ***
s(BirthRate)	10.182		11.689		1485		<2e-16 ***
95th quantile
Smooth terms	Edf	Ref.df		Chi-square		p-value
s(Fertilityrate)	6.103	6.891		933.4		<2e-16 ***
s(BirthRate)	9.056	10.469		1061.0		<2e-16 ***

Note: Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1.

From the table we can observe that expected degrees of freedom for QGAM are less wiggly in smoothness than that of gam, because their expected degrees of freedom values are smaller except for the smooth function of birthrate for 75th and 95th quantile. We can also see that all the smooth curves for both gam and QGAM show significant changes in the response.

4.3. Plots Form Quantile Additive Model

Figure 3 & 4 shows that in quantile generalized additive models the residual values fall within the confidence band, producing a better smooth than the GAM model.

Figure-3. Residual plot for Fertility-rate

Figure-4. Residual plot for Birth-rate

Table-3. Comparison Criteria.

Models	R-sq.(adj)	Deviance explained	AIC (Akaike information criterion)	GCV (generalised cross-validation)
GAM	0.77	79%	5095.657	4.2931e+11
QGAM(25th quantile)	0.721	79.4%	5087.255	2513.338
QGAM(50th quantile)	0.77	74.4%	5091.31	2516.059
QGAM(75th quantile)	0.726	81.5%	5086.55	2524.178
QGAM(95th quantile)	0.695	96%*	5074.61*	2511.248*

The results show that QGAM models have lowered AIC and GCV’s than the GAM model and it’s a proof that QGAM model denotes a better performance in comparison with the GAM model. Based on all the models the 95th quantile best fits the model with proportion of deviance explained as 96% and also has the least AIC and GCV denoting the best among all the models (Table 3). It can be said that the outcomes of GAM and QGAM at 50th quantile have shared similar properties in terms of R2adj which is supposed to be because gam uses response based on mean centered value while 50th quantile uses responses based on the median value which is equivalent.

Table-4. Adequacy of the basis dimension.

Model	Smooth functions	k'	K-index	p-value
GAM	s(Fertilityrate)	19	1.24	1.00
GAM	s(BirthRate)	19	1.01	0.54
QGAM(25th quantile)	s(Fertilityrate)	19	1.05	1.00
QGAM(25th quantile)	s(BirthRate)	19	0.85	0.42
QGAM (50th quantile)	s(Fertilityrate)	19	1.24	1.00
QGAM (50th quantile)	s(BirthRate)	19	1.02	0.57
QGAM (75th quantile)	s(Fertilityrate)	19	0.94	0.96
QGAM (75th quantile)	s(BirthRate)	19	0.84	0.56
QGAM (95th quantile)	s(Fertilityrate)	19	0.45	0.40
QGAM (95th quantile)	s(BirthRate)	19	0.43	0.15

Table 4 shows significant p-value which indicates the basis dimension chosen is adequate for all the models. Though for the 75th and 95th quantiles there appears to be a missing pattern left in the residuals because the k-index is lower than 1.

5. CONCLUSION

Motivated by the need to show that the quantile generalized additive model is a robust alternative to generalized additive model. The basic framework, outlined above, represents smooth functions in regression models using spline basis function. Selecting the smoothing parameter by marginal loss minimization was done through the fast stable method of Wood, et al. [8]. The result shows that the basis dimension used was sufficient. The results also show that the expected degrees of freedom (edf) for the QGAM were smaller than that of the GAM except for the smooth functions of birthrate at the 75th and 95th quantiles. The comparison criteria in Table 2 reveals that the qgam models have lower AIC and GVC than the gam model, hence a better model. It was observed that the GAM model and the QGAM at the 50th quantile had the same R2adj(77%), meaning that both models were able to explain the same percentage of variation by the models, this we could attribute to the fact that GAM is based on mean centered value and the QGAM for 50th quantile is based on the median value of the response and in literature mean regression and median regression are approximately the same, hence the observation is in agreement with existing literature. Also from the plots in fig 1-4, we observe that the residuals of the GAM didn’t all fall within the confidence band but for QGAM all the residuals fall within the confidence band producing a better smooth.

Funding: This study received no specific financial support.

Competing Interests: The author declares that there are no conflicts of interests regarding the publication of this paper.

REFERENCES

[1] G. Laszlo, k. Michael, A. krzyzak., and W. Harro, "A distribution-free theory of nonparametric regression," ed New York: Springer-Verlay Inc, 2002, pp. 18-30.

[2] H. Trevor and T. Robert, "Generalized additive models," Statistical Science, vol. 1, pp. 297-318, 1986.

[3] L. Breiman and J. H. Friedman, "Estimating optimal transformations for multiple regression and correlations," Journal of the American Statistical Association, vol. 80, pp. 580–619, 1985.

[4] R. Koenker and G. Bassett, "Regression quantiles," Econometrica, vol. 46, pp. 33-50, 1978.

[5] Koenker-Roger, "Quantile regressions," ed United Kingdom: Cambridge University Press, 2005, pp. 7-14.

[6] M. Fasiolo, S. N. Wood, M. Zaffran, E. Paris, R. Nedellec, and Y. G. Goudé, "Fast calibrated additive quantile regression," Journal of the American Statistical Association, pp. 1-11, 2020.

[7] Koenker-Roger., "Additive models for quantile regression: Model selection and confidence bandaids," Brazilian Journal of Probability and Statistics, vol. 25, pp. 239–262, 2011.

[8] S. N. Wood, N. Pya, and B. Saefken, "Smoothing parameter and model selection for general smooth models (with discussion)," Journal of the American Statistical Association, vol. 111, pp. 1548-1575, 2016.Available at: https://doi.org/10.1080/01621459.2016.1180986.

Views and opinions expressed in this article are the views and opinions of the author(s), International Journal of Mathematical Research shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content.

Index