maandag 2 september 2013

A crash course in statistics

I recently managed to explain most of statistics (no, seriously) in about an afternoon using the first chapter of Intermediate Statistics: A Conceptual Course. (Sage, Pelham, B. W. (2012)) and three graphs*. After  many years of working with statistics, I actually believe it should be that easy. Also, at work, I have seen total novices use stats professionally in a matter of weeks. It is simply not that hard!
Of course, I'm not talking about assumptions, time series, endogeneity, robust standard errors, and everything that gets your stats right. I refer to what's needed to understand and report test that will be close enough. After all, a complex GMM regression with bootstrapped standard errors still return a regression coefficient and significance levels.
This would be the content of such a course:
  1. Samples and inferential statistics
  2. Average and variance
  3. The normal distribution (probability, significance)
  4. Chi2 (test and distribution)
  5. Correlations and odds ratios
  6. Regression (linear and nonlinear effects)
  7. Factor analysis
  8. Cluster analysis (similarity)
* One with linear and quadratic relations (for correlations and multiple regression), one with a factor score from two dimensions (also based on correlation), and one with clusters on a plot (similarity).

woensdag 7 augustus 2013

Start to SAS

I hated SAS for years. Because I believe the language is plain ugly, overcomplicated and unintuitive. The main pain: the fact that any DATA or PROC step (what's the difference anyway?) has a closed structure. Also, it's slow. For trial and error programming, SAS is to avoid. I still believe that.

But, sometimes, you just lack the RAM to work in beautiful languages like Stata or R. So you need to use the statistical WMD. To speed up work, there's this golden tip: compress the data sets (compress=binary). More on this here:
http://heuristically.wordpress.com/2010/08/13/make-sas-faster-data-set-compression/

maandag 29 juli 2013

Selection bias

Job quality improves during crisis, wages rise.

Selection bias.

Bad, low wage jobs disappear.

maandag 1 juli 2013

Minimum wages - a logical ground

The economic literature on minimum wages is immense. I won't address this here. Yet there is one simple argument for minimum wages: there exists some kind of wage that is the necessary minimum for a worker to reproduce the workforce. It makes no sense to pay anyone below the level of what is necessary to raise a child. If you did, in one generation your economy would be doomed. This has happened before, as is documented in Marx' Capital.

Of course, if one's market price is below the minimumwage, subsidies may be an option to get this person employed, lowering effectively the taxpayers income.

Here's a random newspaper opinion on the matter: http://www.citylimits.org/news/articles/4571/half-of-recovery-jobs-offer-low-wages-so-raise-them#.UdFhfzsqz64.

dinsdag 7 mei 2013

PGF/TikZ

For a while now I was wondering how on earth those neat vector graphics in textbooks are made. I remember my professor of international economics producing graphs in CorelDraw, which is not the state of the art software I had expected.

Because I'm studying distributions at the moment, I was looking for a LaTeX bell shaped curve. I found the GPF/TikZ package and boy, this is what I needed for so long! Within TeX/LaTeX, it's a fairly recent addition. Its predecessor is PStricks, a postscript based graphical language that can't live in a PDF world.

Unlike much of the LaTeX manuals, the TikZ documentation is very instructive and even starts with 80 pages of tutorials. By the way, up to date manuals can always be found at:


http://texdoc.net/pkg/<packagename>

maandag 6 mei 2013

Reinhart-Rogoff

The best joke in decades is the Reinhart-Rogoff paper in the American Economic Review 2010 special issue (this). Not so much because they messed up stats - everybody does - but because they wouldn't leave their stance. No doubt the couple is more than twice as smart as I am, maybe that's why it's hard to admit. Here's a summary, and this is the Herndon Ash Pollin revelation paper.

I quote some guy called Dennis Schminke on the Wall Street Journal online:

"I spent 29 years of a 38-year career with a major US corporation performing operational and financial analyses. Much of that time included review of the work of peers and subordinates. When PC's and spreadsheets first came along in the early 1980's, the trouble began. My experience...if it's on a spreadsheet, there is a good chance there are errors, especially in the hands of inexperienced or careless users. And all we did with our error is cost our company money. We did not blow up the economy or cause trillion$ to be spent chasing an environmental rabbit."

Anyway, it seems that the US does far worse than thought, if you check alternative stats

Counting lines of code in multiple files

Ever wondered how much work you put in a project? The best way to operationalize is to count (obviously) the number of lines in your code. I'm aware that this is a relative measure: some people need more lines, but you may compare different projects.

I found the following line to type in MS DOS Prompt that counts lines within each of a series of files:

for %G in (*.do) do find /c /v "_+_" %G

Where *.do are the syntax files of the project in the present working directory. You still have to sum all output manually though.

Thanks to Marten Braten.