woensdag 21 maart 2018

Composition bias in small samples with skewed distributions

If regressing on wages that are not normally distributed (right-skewed, in general), and comparing groups with very different relative frequencies (like men and women in the STEM-field), added a censoring (e.g. wage floors), then a positive bias for the larger group is likely for small sample sizes.

I demonstrate this below using a simple Monte Carlo simulation.

In a tobit-like fashion, one would like to control for the selection chance, but this is given by gender which is already in the model.

It would be interesting and easy to make this simulation based on actual means, sd, skewness in the (male) population.


clear
cap erase mc.dta
foreach n of numlist 60 80 100 120 140 200 300 500 800 1000 2000 {
forvalues r = 1/500 {
clear
*local n = 1000
set obs `n'
local m = 2000 // mean
local s = 200 // standard deviation (spread)
local f = 0.1 // feminisation
local a = 3 // positive: right skewed (it is possible to compute the skewness metric based on a)
gene g = 1-(runiform()<`f')
gene rn = runiform()
gene d = 2*invnormal(rn)*normal(rn*`a')
gene w = max(1400,`m'+d*`s')
*twoway kdensity w
collapse w, by(g)
gene nsize = `n'
gene c = 1
reshape wide w, i(c) j(g)
list 
cap gene gpg = w1/w0
cap gene gpg = .
cap append using mc.dta
cap save mc.dta, replace
}
}
replace gpg = round(100*gpg-100,.01)
tabstat gpg, by(nsize)
*scatter gpg nsize

exit

dinsdag 2 januari 2018

Betas


Are betas from different regressions comparable? How substantial is a significant effect? Is a zero-effect always the benchmark? These are hard questions, and I have no good answer. The following post will address the difference between algebra and estimation.

Suppose we measure inequality in a region as the difference of a percentile p to the median (in logs). An explanation would be the relative impact of the minimum wage, i.e. its difference to the median. We then have:

p - p50 = fc(mw - p50)

Suppose the function is lineair, so that:

p - p50 = cons + b*(mw - p50) + e

Now suppose the wage distribution is normal with mean M and standard deviation s:

p = M + s*f^(-1)(p)

note that p50 = M in this case (the normal distribution is symmetric)

substitute this in the function above to get:

s*f^(-1)(p) =  cons + b*(mw - p50) + e

and therefore

b = [s*f^(-1)(p) - cons - e] / [mw - p50]

Which may lead to the surprising finding that b is determined and its scale depends on f^(-1), the inverted cumulative normal distribution. This is flawed, because b and e are determined at the same time in an estimation. Using OLS, b will be such that the sum of all squared errors is minimal. The betas are therefore not an algebraic given. Yet, annoyingly, my estimations give betas that are quite in line with f^(-1). This is one thing I don't understand.

woensdag 29 november 2017

Stata command labelmerge to label values from an Excel file

Download the code here:
https://drive.google.com/open?id=0B3AYa3V7lxmoeXhlR2dIT0lNV00

Syntax

labelmerge varname using excelfile.xlsx [, nomerge nolabel]

Your excelfile has one column varname, and the other varnamelabel. Varname could be nace, for instance, and your file could be called nacerev2english.xlsx.

The code adds the label to the values of varname. If you want to disable this, use the option nolabel.
The code also adds the label as a string. If you want to disable this (and gain some speed and space), use the option nomerge.

Don't forget you need -labmask-. Install this first from SSC.

It's as simple as that.

Cheers




From the ado:
___________________

Labelmerge
___________________


The program does two things:
1) it adds a variable that holds the label as a string
2) it assigns the values of that string as variable labels

You can decide to not maintain either of both, but not both as that would be
senseless.


TO DO
-----
1- Perhaps it could be handy to have the option to select a label from different
options (e.g. languages) in the dataset.

2- Maybe trimming some labels is needed if -labmask- doesn't do it already.


HOW SHOULD YOUR LABELFILE LOOK LIKE
-----------------------------------
It should have just two columns:
a) the numeric variable itself, labeled in the main file exactly the same
b) the value labels as a string variable which is labeled as the main variable
with suffix 'label'

The filename of the matrix could be anything, this is to be specified.


WHY IS THIS PROGRAM USEFUL?
---------------------------
Because you can use label files easily between different syntaxes and projects.
It is better than 'label save' which creates do files, because you have the
labels nicely adjustable in a matrix.


INSTALLATION INSTRUCTIONS
-------------------------

You need to have -labmask- installed.

To install: put the contents in the folder

`c(sysdir_personal)'/m/

for example, in Windows this is generally:

c:\ado\personal\m\

- OR -

Install the file through adopath + "folder"
Like this:

adopath + "c:\random_ado_folder\"


donderdag 9 november 2017

Running Python on Mac

Most rookie guides are still to complex for a fairly easy install.

macOS has Python 2.7 pre-installed, you don't need Python 3.X (yet).

Additionally, install (in order):
  • Xcode (including command line tools)
  • Homebrew (a package manager)
  • Sublime Text (or TextWrangler)
Writing the syntax in Sublime Text, you need to select the build system (python), and build (=run) the code. The output is giving in the console. Code is straightforward, but graphic capabilities are poor.

Source: https://www.macworld.co.uk/how-to/mac/coding-with-python-on-mac-3635912/

vrijdag 12 mei 2017

Simulation panel-invariant variable: OLS versus fixed effects

Below is a simulation of the consequence of controlling for fixed effects. In some cases this is desirable, in others not.

In the wage equation below, the wage depends on gender (b = 2), effort (b = 10), and ability (b = 3). There is no unobserved heterogeneity. A straightforward OLS estimation will return the correct b's.

However, as gender and ability are panel-invariant, the fixed effect regression does not yield any effect -- although the effect of effort is unbiased.

On the other hand, say that ability and effort are correlated, and ability is not observed, the estimated beta if effort in OLS would be biased, but not in a fixed effects estimation. The latter, however, would not allow estimating the beta for gender.

Good riddance.



clear
gene byte gender = .
gene float effort = .
gene float wage = .
gene float ability = .
gene long id = .

forvalues i = 1/1000 {
local a = runiform()
local g = runiform() > .5

set obs `=`i'*5'

replace ability = `a' if missing(id)
replace gender = `g' if missing(id)
replace id = `i' if missing(id)
}
replace effort = runiform()
replace wage = 2*gender + 10*effort + 3*ability + .5*runiform()


regress wage gender effort ability
areg wage gender effort ability, abs(id)


/*
. regress wage gender effort ability

      Source |       SS           df       MS      Number of obs   =     5,000
-------------+----------------------------------   F(3, 4996)      >  99999.00
       Model |  50282.7806         3  16760.9269   Prob > F        =    0.0000
    Residual |  104.339671     4,996  .020884642   R-squared       =    0.9979
-------------+----------------------------------   Adj R-squared   =    0.9979
       Total |  50387.1202     4,999  10.0794399   Root MSE        =    .14452

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      gender |   1.999089   .0040916   488.58   0.000     1.991067     2.00711
      effort |   10.00101   .0070513  1418.33   0.000     9.987187    10.01483
     ability |   3.004343   .0071066   422.75   0.000     2.990411    3.018275
       _cons |   .2498592   .0058902    42.42   0.000     .2383118    .2614065
------------------------------------------------------------------------------


end of do-file

. do "/var/folders/p2/2v2ckxtd2794655ypfbmfg9w0000gn/T//SD31177.000000"

. areg wage gender effort ability, abs(id)
note: gender omitted because of collinearity
note: ability omitted because of collinearity

Linear regression, absorbing indicators         Number of obs     =      5,000
                                                F(   1,   3999)   = 1602779.17
                                                Prob > F          =     0.0000
                                                R-squared         =     0.9983
                                                Adj R-squared     =     0.9979
                                                Root MSE          =     0.1450

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      gender |          0  (omitted)
      effort |   10.00054   .0078993  1266.01   0.000     9.985056    10.01603
     ability |          0  (omitted)
       _cons |   2.810143   .0044419   632.65   0.000     2.801435    2.818852
-------------+----------------------------------------------------------------
          id |      F(999, 3999) =    401.808   0.000        (1000 categories)
*/

woensdag 22 maart 2017

Negotiated pay levels

ECB

Aggregate indicator for the Eurozone ("Indicator of negotiated wage rates")
https://sdw.ecb.europa.eu/browse.do?node=9691595

TURI

CAWIE

Eurofound

Eurostat minimum wages


dinsdag 28 februari 2017

Gross domestic product

Introduction

The gross domestic product (GDP) is arguably the most important economic indicator, drawing a picture of total output and economy produces in a given year. The GDP can be approached in three ways:

  1. Production approach
  2. Income approach
  3. Expenditure approach
All three are equivalent but have different components. What is produced is equal to what is earned, and this is equal to what gets spent. I will address the components of each GDP definition, and briefly discuss how to practically deal with it.

Production approach

GDP is the gross value of domestic output of all economic activities (GDP at market prices. This is the value of the total sales of goods and services plus value of changes in the inventories. In the production approach it consists of gross value added (GDP at factor cost) and the value of intermediate consumption (i.e., the cost of material, supplies and services used to produce final goods or services).

Gross value added = gross value of output – value of intermediate consumption.

GDP at factor cost plus indirect taxes less subsidies on products is the "GDP at producer price".

Income approach

GDP = COE + GOS + GMI + [T_PM – S_PM]
compensation of employees (COE) + gross operating surplus (GOS) + gross mixed income (GMI) + taxes less subsidies on production and imports (T_PM – S_PM).

Expenditure approach

Y = C + I + G + (X − M)
consumption (C) + investment (I) + government spending (G) + net exports (X – M)

Often Y is used in this definition instead of GDP.

Gross value added

Gross value added (GVA) is the measure of the value of goods and services produced in an area, industry or sector of an economy, in economics. I.e. of output.

GVA = GDP - intermediate consumption (production approach)
GDP = GVA + taxes on products - subsidies on products
GVA = GDP + subsidies - (direct, sales) taxes

If taxes > subsidies, GDP > GVA, which is the case for Belgium.

Gross value added is used for measuring gross regional domestic product and other measures of the output of entities smaller than a whole economy (for which taxes-subsidies are aggregated).

Over-simplistically, GVA is the grand total of all revenues, from final sales and (net) subsidies, which are incomes into businesses. Those incomes are then used to cover expenses (wages & salaries, dividends), savings (profits, depreciation), and (indirect) taxes.

More on the difference in interpretation between GVA and GDP on Quora.

GDP, GNP, GNI, and HDI

GDP can be contrasted with gross national product (GNP) or gross national income (GNI). The difference is that GDP defines its scope according to location, while GNP/GNI defines its scope according to ownership. In a global context, world GDP and world GNP/GNI are, therefore, equivalent terms. Most often, we care about GDP, but GNI might indicate that an economy is colonized or colonizing.

The Human Development Index (HDI) was created by the United Nations to emphasize that people and their capabilities should be the ultimate criteria for assessing the development of a country, not economic growth alone. The HDI can also be used to question national policy choices, asking how two countries with the same level of GNI per capita can end up with different human development outcomes. These contrasts can stimulate debate about government policy priorities. The Human Development Index (HDI) is a summary measure (geometric mean) of average achievement in key dimensions of human development:

  1. Healtha long and healthy life (life expectancy)
  2. Education (years of schooling)
  3. Standard of living (log of the GNI per capita)

In practice, GDP and the HDI correlate very strongly, although there are notable differences that appear even sharper in other 'happiness' indicators. For instance: if two persons decide to stay at home to clean the house and care for the children, nothing is registered in the GDP. If each cleans the house of the other and pays him or her for the job the same wage, we have precisely the same utility, but two wages are added to GDP. I nevertheless favour the use of GDP as an objective measurement for economic development (in the example given, a market for cleaning opens up), but keeping in mind that we are more interested in utility than in the mere quantity of goods and services provided.

GDP figures from Eurostat

GDP figures can be downloaded from Eurostat (key: namq_10_gdp). Below are a few tips to select the correct indicators:
  • The European System of Accounts (ESA) gives a guideline for the measurement of production in sectors. If you want to have a good laugh, look at what happened in Ireland in 2015: 30% GDP growth because of a change in the guidelines! Depending on the source you will find longer time series in one or the other system, but for recent years in Europe, you should take the most recent ESA.
  • The broadest GDP in nominal terms is 'GDP at market prices'. Mostly you want it to be in EUR or USD, not in the national currency, unless exchange rates are what passionates you.
  • GDP in real term is found in the 'chain linked volume' series. Basically, chain linking output means output is expressed in the prices of the previous year, but as it goes this boils down to using a base year.
Note that ESA/Eurostat use the code B for GDP components using the production approach, code D for components using the income approach, and code P for the expenditure approach (which are extensive).

Together with Sebastien Fontenay I have made the -eurostatuse- command in Stata to fetch data from Eurostat. To make indices, I have made the -reindex- command you find elsewhere on this blog or if you send me an email.