Misplaced Pages

This is an old revision of this page, as edited by Sleigh (talk | contribs) at 10:53, 11 September 2019 (→Analysis: +link). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

In statistics and econometrics, panel data or longitudinal data are multi-dimensional data involving measurements over time. Panel data contain observations of multiple phenomena obtained over multiple time periods for the same firms or individuals. Not to be confused with market research panel data, which is data collected from a sample of voluntary consumers.

Time series and cross-sectional data can be thought of as special cases of panel data that are in one dimension only (one panel member or individual for the former, one time point for the latter).

A study that uses panel data is called a longitudinal study or panel study.

Example

MRPP balanced panel:		MRPP unbalanced panel:
${\displaystyle {\begin{matrix}\mathrm {person} &\mathrm {year} &\mathrm {income} &\mathrm {age} &\mathrm {sex} \\1&2016&1300&27&1\\1&2017&1600&28&1\\1&2018&2000&29&1\\2&2016&2000&38&2\\2&2017&2300&39&2\\2&2018&2400&40&2\end{matrix}}}$		${\displaystyle {\begin{matrix}\mathrm {person} &\mathrm {year} &\mathrm {income} &\mathrm {age} &\mathrm {sex} \\1&2016&1600&23&1\\1&2017&1500&24&1\\2&2016&1900&41&2\\2&2017&2000&42&2\\2&2018&2100&43&2\\3&2017&3300&34&1\end{matrix}}}$

In the example above, two datasets with a panel structure are shown. Individual characteristics (income, age, sex) are collected for different persons and different years. In the left dataset, two persons (1, 2) are observed every year for three years (2016, 2017, 2018). In the right dataset, three persons (1, 2, 3) are observed two times (person 1), three times (person 2), and one time (person 3), respectively, over three years (2016, 2017, 2018); in particular, person 1 is not observed in year 2018 and person 3 is not observed in 2016 or 2018.

A balanced panel (e.g., the left-hand dataset above) is a dataset in which each panel member (i.e., person) is observed every year. Consequently, if a balanced panel contains N panel members and T periods, the number of observations (n) in the dataset is necessarily n = N×T.

An unbalanced panel (e.g., the right-hand dataset above) is a dataset in which at least one panel member is not observed every period. Therefore, if an unbalanced panel contains N panel members and T periods, then the following strict inequality holds for the number of observations (n) in the dataset: n < N×T.

Both datasets above are structured in the long format, which is where one row holds one observation per time. Another way to structure panel data would be the wide format where one row represents one observational unit for all points in time (for the example, the wide format would have only two (left example) or three (right example) rows of data with additional columns for each time-varying variable (income, age).

Analysis

A panel has the form

${\displaystyle X_{it},\quad i=1,\dots ,N,\quad t=1,\dots ,T,}$

where ${\displaystyle i}$ is the individual dimension and ${\displaystyle t}$ is the time dimension. A general panel data regression model is written as ${\displaystyle y_{it}=\alpha +\beta 'X_{it}+u_{it}.}$ Different assumptions can be made on the precise structure of this general model. Two important models are the fixed effects model and the random effects model.

Consider a generic panel data model:

${\displaystyle y_{it}=\alpha +\beta 'X_{it}+u_{it},}$

${\displaystyle u_{it}=\mu _{i}+v_{it}.}$

${\displaystyle \mu _{i}}$ are individual-specific, time-invariant effects (for example in a panel of countries this could include geography, climate etc.) which are fixed over time., whereas ${\displaystyle v_{it}}$ is a time-varying random component.

If ${\displaystyle \mu _{i}}$ is unobserved, and correlated with at least one of the independent variables, then it will cause omitted variable bias in a standard regression. However, panel data methods, such as the fixed effects estimator or alternatively, the First-difference estimator can be used to control for it.

If ${\displaystyle \mu _{i}}$ is not correlated with any of the independent variables, ordinary least squares linear regression methods can be used to yield unbiased and consistent estimates of the regression parameters. However, because ${\displaystyle \mu _{i}}$ is fixed over time, it will induce serial correlation in the error term of the regression. This means that more efficient estimation techniques are available. Random effects is one such method: it is a special case of feasible generalized least squares which controls for the structure of the serial correlation induced by ${\displaystyle \mu _{i}}$.

Dynamic panel data

Dynamic panel data describes the case where a lag of the dependent variable is used as regressor:

${\displaystyle y_{it}=\alpha +\beta 'X_{it}+\gamma y_{it-1}+u_{it},}$

The presence of the lagged dependent variable violates strict exogeneity, that is, endogeneity may occur. The fixed effect estimator and the first differences estimator both rely on the assumption of strict exogeneity. Hence, if ${\displaystyle u_{i}}$ is believed to be correlated with one of the independent variables, an alternative estimation technique must be used. Instrumental variables or GMM techniques are commonly used in this situation, such as the Arellano–Bond estimator.

Data sets which have a panel design

• Russia Longitudinal Monitoring Survey (RLMS)
• German Socio-Economic Panel (SOEP)
• Household, Income and Labour Dynamics in Australia Survey (HILDA)
• British Household Panel Survey (BHPS)
• Survey of Family Income and Employment (SoFIE)
• Survey of Income and Program Participation (SIPP)
• Lifelong Labour Market Database (LLMDB)
• Longitudinal Internet Studies for the Social sciences (LISS)
• Panel Study of Income Dynamics (PSID)
• Korean Labor and Income Panel Study (KLIPS)
• China Family Panel Studies (CFPS)
• German Family Panel (pairfam)
• National Longitudinal Surveys (NLSY)
• Labour Force Survey (LFS)
• Korean Youth Panel (YP)
• Korean Longitudinal Study of Aging (KLoSA)

Data sets which have a multi-dimensional panel design

See also

• Panel study

Notes

1. Diggle, Peter J.; Heagerty, Patrick; Liang, Kung-Yee; Zeger, Scott L. (2002). Analysis of Longitudinal Data (2nd ed.). Oxford University Press. p. 2. ISBN 0-19-852484-6.
2. Fitzmaurice, Garrett M.; Laird, Nan M.; Ware, James H. (2004). Applied Longitudinal Analysis. Hoboken: John Wiley & Sons. p. 2. ISBN 0-471-21487-6.
3. "The Best of Both Worlds: Panel and Shopper Loyalty Data - IRI". www.iriworldwide.com. Retrieved 2019-09-04.

References

• Baltagi, Badi H. (2008). Econometric Analysis of Panel Data (Fourth ed.). Chichester: John Wiley & Sons. ISBN 978-0-470-51886-1.
• Davies, A.; Lahiri, K. (1995). "A New Framework for Testing Rationality and Measuring Aggregate Shocks Using Panel Data". Journal of Econometrics. 68 (1): 205–227. doi:10.1016/0304-4076(94)01649-K.
• Davies, A.; Lahiri, K. (2000). "Re-examining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts". Analysis of Panels and Limited Dependent Variable Models. Cambridge: Cambridge University Press. pp. 226–254. ISBN 0-521-63169-6.
• Frees, E. (2004). Longitudinal and Panel Data: Analysis and Applications in the Social Sciences. New York: Cambridge University Press. ISBN 0-521-82828-7.
• Hsiao, Cheng (2003). Analysis of Panel Data (Second ed.). New York: Cambridge University Press. ISBN 0-521-52271-4.

External links

• PSID
• KLIPS
• pairfam
• Korea Employment Survey

• Multivariate time series
• Statistical data types
• Panel data
• Mathematical and quantitative methods (economics)