Revision as of 10:30, 12 February 2020 editMwtoews (talk | contribs)Extended confirmed users, Pending changes reviewers, Rollbackers27,455 edits convert latex table to Misplaced Pages sortable table; use language "first" and "second" instead of "left" and "right", as these may not always be side-by-side on narrow screens← Previous edit | Latest revision as of 21:48, 18 August 2024 edit undoAtlasDuane (talk | contribs)Extended confirmed users1,210 edits →Data sets which have a panel design: Removing external and red links.Tag: Visual edit | ||
(31 intermediate revisions by 19 users not shown) | |||
Line 1: | Line 1: | ||
{{Short description|Longitudinal statistical study}} | |||
⚫ | In ] and ], '''panel data''' |
||
{{More footnotes|date=June 2020}} | |||
⚫ | In ] and ], '''panel data''' and '''longitudinal data'''<ref>{{cite book|title=Analysis of Longitudinal Data|url=https://archive.org/details/analysislongitud00digg_730|url-access=limited|last=Diggle|first=Peter J.|last2=Heagerty|first2=Patrick|last3=Liang|first3=Kung-Yee|last4=Zeger|first4=Scott L.|publisher=Oxford University Press|year=2002|isbn=0-19-852484-6|edition=2nd|page=}}</ref><ref>{{cite book|title=Applied Longitudinal Analysis|last=Fitzmaurice|first=Garrett M.|last2=Laird|first2=Nan M.|last3=Ware|first3=James H.|publisher=John Wiley & Sons|year=2004|isbn=0-471-21487-6|location=Hoboken|page=2}}</ref> are both multi-dimensional ] involving measurements over time. Panel data is a subset of longitudinal data where observations are for the same subjects each time. | ||
] and ] can be thought of as special cases of panel data that are in one dimension only (one panel member or individual for the former, one time point for the latter). A literature search often involves time series, cross-sectional, or panel data. Cross-panel data (CPD) is an innovative yet underappreciated source of information in the mathematical and statistical sciences. CPD stands out from other research methods because it vividly illustrates how independent and dependent variables may shift between countries. This panel data collection allows researchers to examine the connection between variables across several cross-sections and time periods and analyze the results of policy actions in other nations.<ref>{{Cite journal |last=Zaman |first=Khalid |date=2023-01-24 |title=A Note on Cross-Panel Data Techniques |url=https://zenodo.org/record/7565625 |journal=Latest Developments in Econometrics |volume=1 |issue=1 |pages=1–7 |doi=10.5281/zenodo.7565625}}</ref> | |||
] and ] data can be thought of as special cases of panel data that are in one dimension only (one panel member or individual for the former, one time point for the latter). | |||
A study that uses panel data is called a ] or |
A study that uses panel data is called a ] or panel study. | ||
==Example== | ==Example== | ||
Line 49: | Line 51: | ||
|} | |} | ||
In the example above, two datasets with a panel structure are shown. Individual characteristics (income, age, sex) are collected for different persons and different years. In the first dataset, two persons (1, 2) are observed every year for three years (2016, 2017, 2018). In the second dataset, three persons (1, 2, 3) are observed two times (person 1), three times (person 2), and one time (person 3), respectively, over three years (2016, 2017, 2018); in particular, person 1 is not observed in year 2018 and person 3 is not observed in 2016 or 2018. | In the '''multiple response permutation procedure''' ('''MRPP''') example above, two datasets with a panel structure are shown and the objective is to test whether there's a significant difference between people in the sample data. Individual characteristics (income, age, sex) are collected for different persons and different years. In the first dataset, two persons (1, 2) are observed every year for three years (2016, 2017, 2018). In the second dataset, three persons (1, 2, 3) are observed two times (person 1), three times (person 2), and one time (person 3), respectively, over three years (2016, 2017, 2018); in particular, person 1 is not observed in year 2018 and person 3 is not observed in 2016 or 2018. | ||
A '''balanced panel''' (e.g., the first dataset above) is a dataset in which ''each'' panel member (i.e., person) is observed ''every'' year. |
A '''balanced panel''' (e.g., the first dataset above) is a dataset in which ''each'' panel member (i.e., person) is observed ''every'' year. Consequently, if a balanced panel contains <math>N</math> panel members and <math>T</math> periods, the number of observations (<math>n</math>) in the dataset is necessarily <math>n = N \cdot T</math>. | ||
An '''unbalanced panel''' (e.g., the second dataset above) is a dataset in which ''at least one'' panel member is not observed every period. |
An '''unbalanced panel''' (e.g., the second dataset above) is a dataset in which ''at least one'' panel member is not observed every period. Therefore, if an unbalanced panel contains <math>N</math> panel members and <math>T</math> periods, then the following strict inequality holds for the number of observations (<math>n</math>) in the dataset: <math>n < N \cdot T</math>. | ||
Both datasets above are structured in the '''long format''', which is where one row holds one observation per time. Another way to structure panel data would be the '''wide format''' where one row represents one observational unit for ''all'' points in time (for the example, the wide format would have only two (first example) or three (second example) rows of data with additional columns for each time-varying variable (income, age). | Both datasets above are structured in the '''long format''', which is where one row holds one observation per time. Another way to structure panel data would be the '''wide format''' where one row represents one observational unit for ''all'' points in time (for the example, the wide format would have only two (first example) or three (second example) rows of data with additional columns for each time-varying variable (income, age). | ||
Line 64: | Line 66: | ||
: <math>X_{it}, \quad i = 1, \dots, N, \quad t = 1, \dots, T, </math> | : <math>X_{it}, \quad i = 1, \dots, N, \quad t = 1, \dots, T, </math> | ||
where <math>i</math> is the individual dimension and <math>t</math> is the time dimension. A general panel data regression model is written as <math>y_{it} = \alpha + \beta' X_{it} + u_{it} |
where <math>i</math> is the individual dimension and <math>t</math> is the time dimension. A general panel data regression model is written as <math>y_{it} = \alpha + \beta' X_{it} + u_{it}</math>. Different assumptions can be made on the precise structure of this general model. Two important models are the ] and the ]. | ||
Different assumptions can be made on the precise structure of this general model. Two important models are the ] and the ]. | |||
Consider a generic panel data model: | Consider a generic panel data model: | ||
Line 73: | Line 74: | ||
: <math>u_{it} = \mu_i + v_{it}.</math> | : <math>u_{it} = \mu_i + v_{it}.</math> | ||
<math>\mu_i</math> are individual-specific, time-invariant effects ( |
<math>\mu_i</math> are individual-specific, time-invariant effects (e.g., in a panel of countries this could include geography, climate, etc.) which are fixed over time, whereas <math>v_{it}</math> is a time-varying random component. | ||
If <math>\mu_i</math> is unobserved, and correlated with at least one of the independent variables, then it will cause omitted variable bias in a standard ] regression. However, panel data methods, such as the fixed effects estimator or alternatively, the ] can be used to control for it. | If <math>\mu_i</math> is unobserved, and correlated with at least one of the independent variables, then it will cause omitted variable bias in a standard ] regression. However, panel data methods, such as the fixed effects estimator or alternatively, the ] can be used to control for it. | ||
Line 83: | Line 84: | ||
Dynamic panel data describes the case where a ] of the dependent variable is used as regressor: | Dynamic panel data describes the case where a ] of the dependent variable is used as regressor: | ||
: <math>y_{it} = \alpha + \beta' X_{it} +\gamma y_{it-1}+ u_{it} |
: <math>y_{it} = \alpha + \beta' X_{it} +\gamma y_{it-1}+ u_{it}.</math> | ||
The presence of the lagged dependent variable violates strict ], that is, ] may occur. The fixed effect estimator and the first differences estimator both rely on the assumption of strict exogeneity. Hence, if <math>u_{i}</math> is believed to be correlated with one of the independent variables, an alternative estimation technique must be used. Instrumental variables or GMM techniques are commonly used in this situation, such as the ]. | The presence of the lagged dependent variable violates strict ], that is, ] may occur. The fixed effect estimator and the first differences estimator both rely on the assumption of strict exogeneity. Hence, if <math>u_{i}</math> is believed to be correlated with one of the independent variables, an alternative estimation technique must be used. Instrumental variables or GMM techniques are commonly used in this situation, such as the ]. | ||
While estimating this we should have the proper information about the instrumental variables. | |||
==Data sets which have a panel design== | ==Data sets which have a panel design== | ||
*] (RLMS) | |||
* ''German'' ] (SOEP) | * ''German'' ] (SOEP) | ||
*] (HILDA) | *] (HILDA) | ||
*] (BHPS) | *] (BHPS) | ||
*] (SoFIE) | |||
*] (SIPP) | *] (SIPP) | ||
*] (LLMDB) | *] (LLMDB) | ||
*) | |||
*] (PSID) | *] (PSID) | ||
*] (KLIPS) | |||
*] (CFPS) | *] (CFPS) | ||
*] (pairfam) | |||
*] (NLSY) | *] (NLSY) | ||
*] (LFS) | *] (LFS) | ||
*] (YP) | |||
*] (KLoSA) | |||
==Data sets which have a multi-dimensional panel design== | ==Data sets which have a multi-dimensional panel design== | ||
{{Main|Multidimensional panel data}} | {{Main|Multidimensional panel data}} | ||
==See also== | |||
*] | |||
==Notes== | ==Notes== | ||
Line 120: | Line 112: | ||
*{{cite book |last=Davies |first=A. |last2=Lahiri |first2=K. |year=2000 |chapter=Re-examining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts |title=Analysis of Panels and Limited Dependent Variable Models |location=Cambridge |publisher=Cambridge University Press |isbn=0-521-63169-6 |pages=226–254 }} | *{{cite book |last=Davies |first=A. |last2=Lahiri |first2=K. |year=2000 |chapter=Re-examining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts |title=Analysis of Panels and Limited Dependent Variable Models |location=Cambridge |publisher=Cambridge University Press |isbn=0-521-63169-6 |pages=226–254 }} | ||
*{{cite book |last=Frees |first=E. |year=2004 |title=Longitudinal and Panel Data: Analysis and Applications in the Social Sciences |location=New York |publisher=Cambridge University Press |isbn=0-521-82828-7 }} | *{{cite book |last=Frees |first=E. |year=2004 |title=Longitudinal and Panel Data: Analysis and Applications in the Social Sciences |location=New York |publisher=Cambridge University Press |isbn=0-521-82828-7 }} | ||
*{{cite book |last=Hsiao |first=Cheng |year=2003 |title=Analysis of Panel Data |location=New York |publisher=Cambridge University Press |edition=Second |isbn=0-521-52271-4 }} | *{{cite book |last=] |first=Cheng |year=2003 |title=Analysis of Panel Data |location=New York |publisher=Cambridge University Press |edition=Second |isbn=0-521-52271-4 }} | ||
==External links== | ==External links== | ||
Line 128: | Line 120: | ||
* | * | ||
⚫ | ] | ||
] | ] | ||
] | ] | ||
⚫ | ] | ||
] | ] |
Latest revision as of 21:48, 18 August 2024
Longitudinal statistical studyThis article includes a list of general references, but it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations. (June 2020) (Learn how and when to remove this message) |
In statistics and econometrics, panel data and longitudinal data are both multi-dimensional data involving measurements over time. Panel data is a subset of longitudinal data where observations are for the same subjects each time.
Time series and cross-sectional data can be thought of as special cases of panel data that are in one dimension only (one panel member or individual for the former, one time point for the latter). A literature search often involves time series, cross-sectional, or panel data. Cross-panel data (CPD) is an innovative yet underappreciated source of information in the mathematical and statistical sciences. CPD stands out from other research methods because it vividly illustrates how independent and dependent variables may shift between countries. This panel data collection allows researchers to examine the connection between variables across several cross-sections and time periods and analyze the results of policy actions in other nations.
A study that uses panel data is called a longitudinal study or panel study.
Example
person | year | income | age | sex |
---|---|---|---|---|
1 | 2016 | 1300 | 27 | 1 |
1 | 2017 | 1600 | 28 | 1 |
1 | 2018 | 2000 | 29 | 1 |
2 | 2016 | 2000 | 38 | 2 |
2 | 2017 | 2300 | 39 | 2 |
2 | 2018 | 2400 | 40 | 2 |
person | year | income | age | sex |
---|---|---|---|---|
1 | 2016 | 1600 | 23 | 1 |
1 | 2017 | 1500 | 24 | 1 |
2 | 2016 | 1900 | 41 | 2 |
2 | 2017 | 2000 | 42 | 2 |
2 | 2018 | 2100 | 43 | 2 |
3 | 2017 | 3300 | 34 | 1 |
In the multiple response permutation procedure (MRPP) example above, two datasets with a panel structure are shown and the objective is to test whether there's a significant difference between people in the sample data. Individual characteristics (income, age, sex) are collected for different persons and different years. In the first dataset, two persons (1, 2) are observed every year for three years (2016, 2017, 2018). In the second dataset, three persons (1, 2, 3) are observed two times (person 1), three times (person 2), and one time (person 3), respectively, over three years (2016, 2017, 2018); in particular, person 1 is not observed in year 2018 and person 3 is not observed in 2016 or 2018.
A balanced panel (e.g., the first dataset above) is a dataset in which each panel member (i.e., person) is observed every year. Consequently, if a balanced panel contains panel members and periods, the number of observations () in the dataset is necessarily .
An unbalanced panel (e.g., the second dataset above) is a dataset in which at least one panel member is not observed every period. Therefore, if an unbalanced panel contains panel members and periods, then the following strict inequality holds for the number of observations () in the dataset: .
Both datasets above are structured in the long format, which is where one row holds one observation per time. Another way to structure panel data would be the wide format where one row represents one observational unit for all points in time (for the example, the wide format would have only two (first example) or three (second example) rows of data with additional columns for each time-varying variable (income, age).
Analysis
Main article: Panel analysisA panel has the form
where is the individual dimension and is the time dimension. A general panel data regression model is written as . Different assumptions can be made on the precise structure of this general model. Two important models are the fixed effects model and the random effects model.
Consider a generic panel data model:
are individual-specific, time-invariant effects (e.g., in a panel of countries this could include geography, climate, etc.) which are fixed over time, whereas is a time-varying random component.
If is unobserved, and correlated with at least one of the independent variables, then it will cause omitted variable bias in a standard OLS regression. However, panel data methods, such as the fixed effects estimator or alternatively, the first-difference estimator can be used to control for it.
If is not correlated with any of the independent variables, ordinary least squares linear regression methods can be used to yield unbiased and consistent estimates of the regression parameters. However, because is fixed over time, it will induce serial correlation in the error term of the regression. This means that more efficient estimation techniques are available. Random effects is one such method: it is a special case of feasible generalized least squares which controls for the structure of the serial correlation induced by .
Dynamic panel data
Dynamic panel data describes the case where a lag of the dependent variable is used as regressor:
The presence of the lagged dependent variable violates strict exogeneity, that is, endogeneity may occur. The fixed effect estimator and the first differences estimator both rely on the assumption of strict exogeneity. Hence, if is believed to be correlated with one of the independent variables, an alternative estimation technique must be used. Instrumental variables or GMM techniques are commonly used in this situation, such as the Arellano–Bond estimator. While estimating this we should have the proper information about the instrumental variables.
Data sets which have a panel design
- German Socio-Economic Panel (SOEP)
- Household, Income and Labour Dynamics in Australia Survey (HILDA)
- British Household Panel Survey (BHPS)
- Survey of Income and Program Participation (SIPP)
- Lifelong Labour Market Database (LLMDB)
- Panel Study of Income Dynamics (PSID)
- China Family Panel Studies (CFPS)
- National Longitudinal Surveys (NLSY)
- Labour Force Survey (LFS)
Data sets which have a multi-dimensional panel design
Main article: Multidimensional panel dataNotes
- Diggle, Peter J.; Heagerty, Patrick; Liang, Kung-Yee; Zeger, Scott L. (2002). Analysis of Longitudinal Data (2nd ed.). Oxford University Press. p. 2. ISBN 0-19-852484-6.
- Fitzmaurice, Garrett M.; Laird, Nan M.; Ware, James H. (2004). Applied Longitudinal Analysis. Hoboken: John Wiley & Sons. p. 2. ISBN 0-471-21487-6.
- Zaman, Khalid (2023-01-24). "A Note on Cross-Panel Data Techniques". Latest Developments in Econometrics. 1 (1): 1–7. doi:10.5281/zenodo.7565625.
References
- Baltagi, Badi H. (2008). Econometric Analysis of Panel Data (Fourth ed.). Chichester: John Wiley & Sons. ISBN 978-0-470-51886-1.
- Davies, A.; Lahiri, K. (1995). "A New Framework for Testing Rationality and Measuring Aggregate Shocks Using Panel Data". Journal of Econometrics. 68 (1): 205–227. doi:10.1016/0304-4076(94)01649-K.
- Davies, A.; Lahiri, K. (2000). "Re-examining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts". Analysis of Panels and Limited Dependent Variable Models. Cambridge: Cambridge University Press. pp. 226–254. ISBN 0-521-63169-6.
- Frees, E. (2004). Longitudinal and Panel Data: Analysis and Applications in the Social Sciences. New York: Cambridge University Press. ISBN 0-521-82828-7.
- Hsiao, Cheng (2003). Analysis of Panel Data (Second ed.). New York: Cambridge University Press. ISBN 0-521-52271-4.