Log-log Paper and a Straight Edge

The TV show Numb3rs started off magnificently, but has been predictably deteriorating. A cop show based on mathematics is a clever idea, and they managed to wring more story ideas out of it than I would have expected, but that’s been accomplished primarily by continually reducing the math content and increasing the amounts of character interplay, police procedural story lines, plus massive firepower.

They also “fine tuned” the ensemble, primarily by getting rid of Sabrina Lloyd’s character, Terry Lake, (photo at left)after Season One. Lloyd is fondly remembered by those of us who were hooked on Sports Night. Her character in Numb3rs was supposed to be a fairly cold and calculating “profiler,” which Lloyd did well, but cold women on TV don’t often go over, and I suspect that Terry Lake got written out of the show for that reason (the “official” explanation seems to be that it was Lloyd’s decision to leave the show, but I’m not buying it).

The replacement profiler on Numb3rs is Megan Reeves, played by Diane Farr (at right), and the first thing they did with her was to show her math illiteracy. She refers to something as “growing exponentially” and Charlie immediately smiles condescendingly and explains what “exponentially” really means: a rate of change that is proportional to size. So the math whiz puts down another dumb blonde. I know, I know, the show is actually better and more subtle than that, but I still have the suspicion that the exchange was supposed to make the new character less threatening, hence more appealing.

Exponential growth, or its twin, exponential decay is common as dirt, or at least the bacteria in dirt, which get to follow exponential growth curves pretty regularly. The typical “S” curve, in fact, is two exponential curves laid onto each other. The first one occurs when some growth phenomenon begins without significant limitation, like the first doublings of yeast in the brew vat. There are so few yeast cells relative to the nutrient that the growth at first doesn’t alter any of the factors that might affect the growth rate.

After time, of course, the yeast begins to run out of nutrients, or begins to be poisoned by their own waste products. At that point you switch to a case where the growth manages to get some fractional distance toward the inevitable limit with each cycle. As the old Firesign Theater bit goes, Antelope Freeway, one mile, Antelope Freeway, one half mile, Antelope Freeway, one quarter mile, Antelope Freeway, one-eighth mile, Antelope Freeway, one sixteenth mile… Xeno’s Paradox as logarithmic decay, in other words, “logarithmic” and “exponential” being two words for the same thing, a constant term added or subtracted to the logarithm of the quantity in question.

At Nasfic in Seattle a few years ago, I was on a panel with an energy consultant who had a good story to tell about the folly of assuming that exponential growth goes on forever. The title of this piece is a quote from that consultant: “…people who believe that you can predict the future with some log-log paper and a straight edge.” This, of course, identified him as an old foggy who remembers when log-log paper was used for that purpose. Now its all spreadsheets and curve fitting, but the principle remains.

My co-panelist told the story of a power company in the Pacific Northwest that did their predictions of electric power demand in the early 1970s and decided on that basis that they should build a nuclear plant. The Pacific Northwest has a lot of hydropower resources, but there came a time when all the easy dams had been built, so they decided that nuclear was the way to go. But nuclear power is pretty capital intensive; it costs a lot of money to build a nuclear plant, though not so much to keep one running. The PN power agency had a regulatory structure that allowed them to pass capital costs through to their customers, so that’s what they did. This raised their rates sufficiently that it effectively suppressed demand growth such that the anticipated growth in demand never occurred and the nuclear plant was an instant white elephant. They’d effectively saturated the market for low cost power and when it got more expensive, well, bye bye exponential growth. It didn’t do much good for the customers and stockholders, either, I’ll bet.

Before I’d ever heard of Moore’s Law, I read a book by Philip Sporn, who noted that electric power generation in the U.S. had doubled every decade in the 20th Century. That was in the early 1970s, just before the doubling stopped.

A goodly amount of both science fiction and futurism is basically just extrapolating growth curves indefinitely, but the really interesting stories are the ones where the exponential growth stops, the whys and wherefores. It takes more insight to do that, of course; it’s a lot easier to just use the log-log paper and the straight edge.

Laffering All the Way

[This is something of a distillation of some discussions from Mark Thoma's Economist's View Blog].

The Laffer Curve is one of the greatest scams of all time. Really. "Please send me your bank account number so I can transfer $50 million in gold to your account" isn't even in the running compared to the Laffer Scam, though the Ponzi scheme might give it a run for the money.

The idea that you can somehow increase tax revenues by cutting tax rates is as seductive as 25% at no risk, though, isn't it? And it's so simple you can explain it to a child, using just the famous bar napkin:



Let's let the Wikipedia supply the basic, simple explanation:

"The curve is most understandable at both extremes of income taxation—zero percent and one-hundred percent—where the government collects no revenue. At one extreme, a 0% tax rate means the government's revenue is, of course, zero. At the other extreme, where there is a 100% tax rate, the government collects zero revenue because (in a "rational" economic model) taxpayers presumably change their behavior in response to the tax rate: either they have no incentive to work or they avoid paying taxes, so the government collects 100% of nothing."

Martin Gardner, in a famous Scientific American article, gave a satirical "neo-Laffer Curve," supposedly to show how fatuous Laffer's arguments were (Gardner's graph is rotated, with tax rates being on the y axis, and revenues on the x axis; the Wikipedia has a redrawn version in the same orientation as Laffer's original):



When I read Gardner's original article I was really pissed, because Gardner's graph fails on basic mathematical terms. A squiggle like Gardner drew isn't a function! So Gardner, the epitome of the punctilious math-head, let himself slide into a basic math error, simply because he was so annoyed with Laffer.

Moreover, he missed the most important mistake in Laffer's analysis. It is simply not true that a 100% tax rate generates no revenue, even in "rational" economics models, because there is no such thing as 100% compliance, among other things.

Consider the illegal drug trade. The criminal and RICO proscriptions on illegal drugs amount to a tax rate that is actually greater than 100%; there have been RICO prosecutions where people's houses and all their financial assets have been seized for a fairly small quantity of drugs, far less than the sale value of the drugs themselves. That's a greater than 100% excise tax. Yet the drug trade continues. Furthermore, various law enforcement agencies obtain substantial funding from RICO seizures, and a number of venues actually have excise tax laws on illegal drugs.

Hey, wait, no fair! exclaim the supply-side enthusiasts. We're only talking about the effects of capital gains taxation on investment and economic growth here. Or they'll claim that it's the effects of high marginal income tax rates on incentives to work or labor specialization.

Well, let's take that last one, which one might think would have a germ of truth to it. I remember a story from the Reader's Digest "Life in these United States" feature from many years ago. How many years ago will become apparent.

A man saw his neighbor out cleaning the leaves from his gutters, The man asked his neighbor why he didn't hire someone to do it, as the neighbor was an affluent professional.

"Well," the neighbor replied, "It would cost at least $10 to hire someone to clean my gutters, and I'm in the 90% tax bracket. So that means I'd have to earn $100 to get that much money after taxes. For $100, I'll do it myself."

It's possible to argue that cleaning gutters was an inefficient use of this man's time. And certainly his forgoing the hiring of it deprived the economic statistics of at least the $10 wage exchange, and possibly the extra hour or of the high income fellow's labor that he'd have surely worked if his marginal tax rate were less (I'm being sarcastic here, in case anyone is in doubt). But there are worse ways of spending a bit of time on the weekend than getting some exercise and seeing the neighborhood from a different viewpoint, to say nothing of the incidental inspection of the roof.

The point here is that even at 90% marginal tax rates, tax revenue was collected, it was not negligible, and the behavioral changes created by the "distortion" weren't necessarily always bad. People do not only work for money; if that were the only motivation in the world, I wouldn't be writing this, would I?

Moreover, by concentrating exclusively on monetary incentives, people tend to devalue the other incentives to labor, such as pride of workmanship and simple civic responsibility. The idea that a 100% tax rate results in zero labor implies that there is no volunteer labor—ever. This is the fallacy of Econ 101 Management.

As for capital gains taxes, well, the effective tax rate on investment in the old Soviet Union was 100%, as the Soviet government owned everything. It's quite true that the USSR did not have a particularly vibrant economy, but it did still have an economy. The government's effective "tax revenues" on invested capital were not zero. It was just that the government was the primary source of investment capital. And please don't anyone try to tell me that we don't have government capital investment in this country. Even ignoring basic infrastructure investment, we have a lot of government investment, from tax credits to outright subsidy and demonstration projects. It's just that most of these investments eventually wind up turning into private wealth, which, as it happens, is what happened eventually to a lot of the wealth that was created in the old USSR, once communism fell and "privatization" began. Then there is the case of the city that built a big baseball stadium so a certain future President could make some money on his investment in a baseball franchise. That was public investment in a private enterprise, no?

But let's say that we're still on the question of would a 100% capital gains tax yield no return. The answer is still yes. If someone wanted to sell their house in order to move to another house, and they had a capital gain on the sale of their house, would they refuse to sell, staying forever in the same house? Some would, but many would not. Many moves are not a matter of choice. They might try to balance the sale vs purchase to render a zero capital gain, but they would not always be successful, and there would be some revenue collected.

The fact is that it is very difficult to come up with realistic scenarios where a 100% tax rate generates zero revenue. The most basic assumption of the Laffer Curve is simply wrong.

Now you may think that I'm spending more time than I should attacking the basic premises and logic of the Laffer Curve and Supply Side Economics. Perhaps I am. Certainly SSE has failed every reasonable empirical test. In every case since the Reagan Administration, tax cuts have caused reductions in tax revenues, and the tax increases that have occurred since then never created the economic contractions that Supply Siders predicted.

Empiricism is tricky stuff, however, and the snake oil salesmen have been peddling the idea that when taxes are cut, all that need happen is for tax revenues to eventually reach what they had been before the cuts—which will generally happen in any economy that is growing over time. Eventually absolute revenues manage to increase, so long as the Supply-Siders don't actually wreck the entire economy, which they haven't managed to do—yet. In any case, such perversions of empiricism damand that a more fundamental attack be made.

The effect of squeezing revenues and increasing "defense" spending (which is to say, paying for wars and military contractor profiteering) over the past 6 years has been to deprive the nation of the investments that governments need to make, on infrastructure, regulatory oversight, education, and all the things that Movement Conservatives hate, because they don't fit into the basically feudal worldview that drives their actions.

And scams like the Laffer Curve need to be attacked at their roots, otherwise they grow back, like a thorny bush of carrion flowers, smelling like rotting meat to attract the flies that it needs to pollinate. And I'm no longer in the mood to cede these twits a single point. I'm no longer willing to give them even an inch.

The Right Formula

In the numerical solution of complex, dynamic systems, you often run into what is called the stiffness problem. If there was ever a sentence designed to lose readers, that may be it. Nevertheless, onward.

Imagine a spring connected to a mass. If you pull on the mass, the spring stretches and exerts a force on the mass. The stronger the force caused by a given extension, the “stiffer” the spring.

Now imagine a lot of different size masses, interconnected by a lot of springs of differing stiffness. Some of the mass-spring combinations will react quickly to any change in the system; other combinations, those with a severe mismatch between mass and spring, will react more slowly. Each combination has what is often called a time constant or a characteristic time for its reaction to change.

When you’re doing a numerical solution of the system (the only option if the system is complex and non-linear), you have to solve the state of the system for one point in time, then for another point somewhat later, a “time step” later, and so on. Some of the mass-spring combinations will require shorter time steps than others, because they have different time constants.

If there are connections between mass-spring combinations of very dissimilar time constants, you have a problem, the stiffness problem in fact. If one part of your system has a time constant of a microsecond, while another has a time constant of an hour, you are going to need billions of time steps to calculate your system for each time step you’d need if you didn’t have that short time step piece to it.

There are some fancy solution algorithms that can deal with the stiffness problem. One of them is called “Gear-Hindemarsh,” and was developed at Lawrence Livermore Labs, originally to facilitate the calculations used in designing thermonuclear weapons. We used it for chemical kinetic calculations in simulating smog chamber experiments. Gear-Hindemarsh is the gold standard for that sort of thing, but it has some problems, especially when you give the system a kick, like turning the lights on or off, or otherwise messing with the boundary conditions in an unsmooth way. Then it becomes pretty inefficient. The first atmospheric smog model produced by Livermore was called LIRAQ, and it spent much of its computing time on the few seconds after sunrise and sunset.

The development group I worked with on the EPA Urban Airshed Model used a different approach to the stiffness problem, called the quasi-steady state approximation. The QSSA makes a few reasonable assumptions, such as the idea that a chemical species that you are treating as being at “steady-state” doesn’t have so much mass that it affects the rest of the system. Imagine an automobile with a bunch of bobble-heads inside. The bobbing of the heads doesn’t affect the behavior of the whole system because their mass is small, relative to the auto itself.

If you react a hydrocarbon with an HO radical, for example, the HO pulls an H off of it to make water plus what is called an acyl radical, a hydrocarbon missing the H. The acyl radical then absorbs an oxygen molecule to form a peroxyacyl radical. This takes a very short time to occur, and we don’t worry about the behavior of the system during the time it takes for the radical to absorb the oxygen. We’ve treated the acyl radical as if it were in steady state.

Of course when there are multiple sources and multiple reaction paths for a QSSA species, the algebra can get more complicated, but it’s not too bad. At least not until the QSSA species begin to react with themselves and each other. In the smog equations, the first place this happened was when we included the reaction of the hydroperoxyl radical, HOO, with itself. That yields oxygen plus hydrogen peroxide (and that’s where the peroxide came from to bleach the guy to death in SunSmoke). The QSSA equations for HOO are quadratic. Fortunately, we have an equation to solve quadratic equations, one we all learned in high school. Unfortunately, it’s the wrong equation.

As you’ll recall (said the character in the pulp novel), the solution to the quadratic equation

aX^2 + bX + c = 0

can be written:

X = -b + [or minus] sqrt(b^2 - 4ac) / 2a

What they don’t usually tell you in high school is what happens when this equation is used on a quadratic that sometimes has the value of “a” as zero. If that happens, we get the dread “divide by zero” condition, and your computer tells you that you’ve just done a Very Bad Thing, and refuses to continue, you naughty person.

It so happens that there is another form of the quadratic equation that they don’t tell you about in high school, or in most colleges, either:

X = 2c / (-b + [or minus] sqrt(b^2 - 4ac))

A little checking tells me that the Wikipedia now gives the alternate formula, no doubt because so many programmers have run into the same problem I did, ‘way back when. I forget exactly where I got the alternate quadratic formula from; it’s penciled into the margins of a handbook I have. Anyway, I used it when I coded the chemistry module for the UAM.

Later, we wound up with more radical-radical cross reactions, and the algebraic QSSA went from quadratic to fifth order. There is no general fifth order solution, so we used a numerical solution called “Newton-Raphson.” I didn’t code the first implementation we did of that, and the program kept blowing up in the QSSA solver. I looked at it and realized that the programmer had used a constant term as the initial value for the Newton-Raphson calculation, and N-R is notoriously sensitive to the initial value. For the best results, you need to start somewhere near the final value. The clever lad that was I realized that if I stripped out all but the HOO quadratic, it was going to be very close to the final value. Using that as the initial value, the N-R calculation usually converged in one or two iterations.

* * *

In 1984, I got very sick. The words “chronic fatigue syndrome” screw up your ability to get health insurance, so I never say that I had CFS on an insurance form, and besides, I was never diagnosed. Nevertheless, I had what was basically a bout of ‘flu that lasted for several years. I was unable to work full time; in 1985, working as a consultant, I averaged maybe 5-10 hours a week working.

I was no longer the go-to guy for working on the kinetics solver, and one person took our module for use in an acid deposition model, Mary, a PhD chemist recently graduated from Cal Tech. She took one look at my “quadratic formula,” saw that it did not conform to what she’d learned in school and replaced it with the “right” version. Of course, it promptly blew up. So she spent the next several weeks putting in all sorts of tests for when the “a” in the formula got too small, switching it over to the linear solution etc. I’m not implying that it took her a lot of effort; she just spent some number of hours over the next few weeks working the bugs out.

When I heard about it, I was, of course, annoyed. It’s one thing to have someone else catch your mistake; it’s quite another when it wasn’t a mistake in the first place.

More recently though, I’ve been working as a technical writer, and I’ve come to understand that I did, in fact, make a mistake. I did not document the tricks I used in the chemical kinetics solver, even the most basic documentation, which is to put comments into the code explaining what was done and why.

I never confronted Mary about the thing in the first place, and she unexpectedly died of a cerebral aneurysm many years ago, so there’s no closure in the cards, unless this essay counts.

What is the Frequency, Kenneth?

There are mathematical proofs, there are empirical tests, and there are good narratives. When push comes to shove, the good story gets more believers.

Last year, at an SF convention, I attended a panel titled, The “Future in Oil,” which was about oil painting, and had three artists on it. One of them wound up talking about the difference between CDs and vinyl records (because a subject like oil painting tends to get a lot of artistic musing). The cherished narrative is that vinyl is better than digital. I did try to set the record straight, but there were true believers in the audience, as well as one tech guy who thought he knew what it was all about (he did not). I don’t try to win unwinnable arguments very often these days, though I do kvetch about it afterwards, hence this essay.

In my own personal history, one of the most fascinating courses I ever took was a graduate level thing “Voice and Image Processing.” It’s been interesting watching the technology of voice, sound, and video over the past thirty years catch up to the things we learned in that course. Add to that the fact that I’ve also had a number of friends and acquaintances who were sound and music nuts, and I will claim some expertise on the subject. Whatever else may be said, I am not uninformed, nor am I lacking for opinions.

So why do so many people think that vinyl LPs are better than CD? We’ll leave aside whatever buried agendas are being worked out (rebellion against the digital age, simple nostalgia, etc.). In fact, I’ll stipulate that many people have had experiences that are consistent with the hypothesis that vinyl is better than digital. Why?

I know of three answers to that question, one kinda obvious and unsurprising, one that gets seriously arcane, and one that’s just counter-intuitive.

Let’s first take the one that’s unsurprising. In the production process for making an LP or CD, there is a thing called the “Master” tape, which is mixed, balanced, and frequency adjusted for the medium. Not surprisingly, CDs and LPs have different frequency responses. In fact, the old LPs had a lot of special things done to the frequency response to compensate the need to put all the sound in those little, inherently noisy, grooves. Then, on playback the “phono” jack on the pre-amplifier had a frequency response curve built into it to undo all those special tweaks. The net result was an RIAA standard “compensation curve” built into the master, to achieve closer to a flat frequency response for the final product.

When CDs first came out, and the record companies realized that they could make tons of money selling baby boomers their favorite music yet again (first on LP, then cassette, then CDs, and some poor guys had the Beatles on 8-track, too). In a few cases, they didn’t bother re-mastering the tapes, and the things sounded just horrible, especially in the high frequencies, which were artificially “bright.” In other cases, they did “digitally re-master” the tapes, but the guys who did it weren’t as good as the guys who had originally done it. So CDs got a reputation as being “harsh.”

Then there was the fact that early CD players weren’t up to properly implement the theory of sound sampling.

To understand what went wrong there, let’s start with the Nyquist Sampling Theorem. Short version: you can perfectly reconstruct any wave form like a sound signal, provided you sample it at a rate that is at least twice the frequency of the highest frequency in the signal. In the above noted course, we actually went through the proof of the NST, and while I can no more reproduce it than you can reproduce an aria by sampling it at ten samples per second, I did once know it well enough to have the sort of belief in it that qualifies as “faith” for my kind.

So CDs use a standard that samples at 44,000 samples per second, which, theoretically can perfectly reproduce sound up to 22 kHz, but actually is set up to go to 20 kHz, because you need to put in a high frequency cut-off.

See, if you try to reproduce sound at something less than the Nyquist rate, you get a phenomenon known as “aliasing,” where the lower frequencies also have an “alias” signal that is frequency shifted up to higher frequencies. This isn’t the same as a harmonic, because the frequency shift is related to the sampling rate, not to the original signal. I’ve heard aliased sound, and it’s one of the most unpleasant things I’ve ever heard, but I can’t think of any analogies for it. It just sounds alien and wrong.

So above 22 kHz in the analog signal that is reconstructed from a CD, you get this horrible noise. You can try to filter it out of the analog signal, but filters are imperfect, so some faint residual alias remains. Worse, real circuits have some slight non-linear responses, non-linear in the mathematical sense, not in the sense of a “linear frequency response.” What a mathematically non-linear effect can do it to take a high frequency signal and leak some of it to a lower frequency. Thus, the aliased signal above 22 kHz could result in some faint distortion down where even hearing-damaged Boomers could hear it.

Again, more for the CD reputation of “harshness.”

The aliasing problem was eventually solved by digital filtering. What later generation CD players do is to use “oversampling.” Basically, the original 44,000 samples per second is converted through interpolation to many more samples (12X oversampling is common). Then, use a digital filter to make damn sure that there aren’t any aliased frequencies in the thing, so when you finally do the digital-to-analog conversion, you only need a pretty good high-frequency cutoff filter. This winds up being both effective and cheap. Win/win.

Finally we come to the strange one. I think I saw it in Science a couple of years ago. I’m not even sure if it spoke to the issue with CDs vs LPs, but the implications were obvious, even if the result was counter-intuitive. Simply put, our perception of sound is such that we hear things more clearly if there is a little bit of low level noise in the mix. I think the speculation was that the noise allows us to “calibrate” our perceptions on the fly. Whatever the reason, it flies in the face of everything in sound reproduction in the past century. Noise is supposed to be your enemy, not your friend.

But the implication is that the slight surface noise that you get on even the best turntable with an LP actually assists your sound perception. Go figure.

In any case, I’ve made CDs of vinyl LPs; not all LPs have been re-issued, and besides, I don’t like buying the same thing twice. I’ve listened to the resultant CDs, and I don’t believe that anyone can tell the difference between the CD and the source LP. Furthermore, there have been many, many, controlled A/B tests of whether or not anyone can tell the difference, and it’s a simple fact that whatever differences there may be, they are imperceptible to human beings.

And if that’s not enough, Frank Zappa looked into the matter and he agreed; CDs can perfectly reproduce vinyl, if you want them to.

None of this will convince anyone who is sure that LPs, vinyl, and analog are better than CDs and bitstreams. Vinyl is “organic” (which is perfectly true in the chemical sense), while CDs and digital are harsh and artificial. And that’s a simple narrative, as opposed to what I’ve just described, which is convoluted (little math joke there) and knurdish.

I’m fine with that; I’d rather be right than win an argument.