Plus ça change…
June 9, 2011 § 3 Comments
From Hubert M. Blalock, Jr., “The Real and Unrealized Contributions of Quantitative Sociology.” American Sociological Review 54: 447-460. 1989.:
… one finds a large number of journal articles that briefly discuss the measurement of selected variables, that also admit to the probability of errors, but that then effectively announce to the reader that the subsequent empirical analysis and related interpretations will proceed as if there were absolutely no measurement errors whatsoever!
This is but one illustration of the more general point that methodological ideas are adopted when it is relatively easy and costless to do so, but that they are resisted or totally ignored when it is to the investigator’s vested interest to do so. There is also the related tendency to attempt to substitute sophisticated data analysis techniques for inadequate data collection procedures, which are of course far more costly and time consuming. (p. 450)
The Disgruntled Sociologist notes with some despair that this has not changed much at all over the past 22 years. In fact, the tendency to deploy fancy techniques has increased as it has gotten easier and easier to do so. (Yes, this is your fault, Stata.) And ironically, while data collection has — thanks to on-line data sources — in many respects gotten easier, TDS detects little improvement in the creativity applied to data collection. The vast majority of graduate students today are looking to download their dissertation data.
Blalock then goes for the kill (pp. 457-458):
Sociology is not a high-quality discipline. Over at least the past three decades our undergraduate and graduate applicants have consistently scored near the very bottom on standardized tests, not only with respect to quantitative reasoning scores but verbal reasoning scores as well. [Followed by a long list of examples: undergrad curricula, graduate training, promotion criteria, journal standards.]
Finally, our professional associations, and especially the ASA, also need to face the quality question head on. In the early 1970s, when I first served on ASA Council, “quality” was a dirty word suggesting elitism and an attempt to impose orthodoxy. I even encountered instances where potential journal editors were passed over because it was argued that their standards would be too demanding! …
In the end, ASA policies are influenced rather heavily by those whom we elect to office, particularly those elected to Council and the Publications Committee. I am not too optimistic that “politics” within the ASA will change dramatically over the coming years. If not, it will remain for our leading departments to take our quality problem much more seriously than we have in the past.
Sigh. It would appear that Blalock’s pessimism was well-warranted.