history

Statistics in a time of war

John D. Cook shares this interesting piece of history:

During WWII, statistician Abraham Wald was asked to help the British decide where to add armor to their bombers. After analyzing the records, he recommended adding more armor to the places where there was no damage!

This seems backward at first, but Wald realized his data came from bombers that survived. That is, the British were only able to analyze the bombers that returned to England; those that were shot down over enemy territory were not part of their sample. These bombers’ wounds showed where they could afford to be hit. Said another way, the undamaged areas on the survivors showed where the lost planes must have been hit because the planes hit in those areas did not return from their missions.

Wald assumed that the bullets were fired randomly, that no one could accurately aim for a particular part of the bomber. Instead they aimed in the general direction of the plane and sometimes got lucky.

It stories like this one, of a practical application, that make me regret of being a bad student. I think that more of these should be a part of a curriculum.

Food timeline : marshmallows

By accident I came across Food timeline website, which covers the history of some foods. For example, I had no idea about marshmallows:

“Marshmallows are one of the earliest confections know to humankind. Today’s marshmallows come in many forms, from solid…to semi-liquid—to the creme-like or as an ice cream topping. Originally…marshmallows were made from the rood sap of the marsh mallow (Althaea officinalis) plant. It is a genus of herb that is native to parts of Europe, north Africa, and Asia. Marsh mallows grow in marshes and other damp areas…The first marshmallows were made by boiling pieces of the marsh mallow root pulp with sugar until it thickened. After is had thickened, the mixture was strained and cooled. As far back as 2000BC, Egyptians combined the marsh mallow root with honey. The candy was reserved for gods and royalty.

Modern marshmallow confections were first made in France around 1850. This first method of manufacture was expensive and slow because it involved the casting and molding of each marshmallow. French candy makers used the mallow root sap as a binding agent for the egg whites, corn syrup, and water. The fluffy mixture was heated and poured into the corn starch in small molds, forming the marshmallows. At this time, marshmallows were still not mass manufactured. Instead, they were made by confectioners in small stores or candy companies.

By 1900, marshmallows were available for mass consumption, and they were sold in tins as penny candy. Mass production of marshmallows became possible with the invention of the starch mogul system of manufacture in the late 19th century…

In 1955, there were nearly 35 manufacturers of marshmallows in the United States. About this time, Alex Doumak, of Doumak, Inc., patented a new manufacturing method called the extrusion process. This invention changed the history of marshmallow production and is still used today. It now only takes 60 minutes to produce a marshmallow. Today, there are only three manufacturers of marshmallows in the United States, Favorite Brands International (Kraft marshmallows), Doumak, Inc. and Kidded & Company.”

Media landscape after 9/11

GigaOm runs an article on how much media landscape changed since 9/11.

But what strikes me every time I think about September 11 is how much the media landscape — particularly on the web — was transformed by those events, and how very different the world is now when it comes to how we experience real-time news.

When the attack happened, we were still in pre-social network era. No Twitter or Facebook or Google+. And even though quite a few people had blogs, the majority of the news were still coming from the TV and newspapers. For those of you, who don’t remember, most news websites were dead for a day or two immediately after the attack. Slashdot – a popular IT news website which is very much used to having tonnes of traffic was on the edge of collapsing too. Here is their article for this year with a link to the September 11th, 2011 archives.

I remember working in PrimeTel office at the time. I was involved with a project that dealt with video walls and window TV ads in multiple branches of a client’s business. I had a large 40-something-inch plasma TV mounted on a stand next to my desk. I was working on a piece of software that would combine video clips and images into a continuous playlist. I was using sample ads from the client as well as a bunch of landscape photography images for my tests.

Once the attack happened and most of the news sites went down, we established a public folder where all colleagues could drop images and videos they found anywhere on the web and those would get automatically added to the continuous video that was playing on the TV. I remember it was quite something. By the end of the day people from other departments and other floors started to come by to watch it. I remember even the owner of the company came in for a few minutes.

What I couldn’t realize then was how social that thing was. It wasn’t me or anyone else in particular. It was a collective effort of a few people. Each one would come across something and then share it in the public folder. That was very similar to how social networks like Twitter and Facebook distribute things these days. And with the last 10 years, it was proved several times of how well this works.

As Mathew Ingram notes in that GigaOm article:

Now try and think about what it might have been like if September 11 happened today, with ubiquitous smartphones featuring cameras and video and web access. Although cellular networks were overloaded in the aftermath of the attacks, some Blackberry messages got out of the towers — and today, we would almost certainly have gotten a real-time flow of tweets and images and video from people in the towers, at the Pentagon, even on the plane that flew into the ground in Stony Creek, Pennsylvania.

Update: Joe Wilcox of BetaNews also reminds that there was no YouTube back then.

Ten years ago : 9/11

Today is a 10 year anniversary of 9/11. Even though I haven’t been directly affected by the terrorist act 10 years ago, there were still indirect consequences for me and pretty much every other human being on this planet. “War on terror”, military actions in Iraq and Afghanistan, economy defaulting, beefed up security checks in each and every airport, more fear in people and more power in governments – the list can go on.

Bayes theorem history

A fascinating read on the Bayes theorem history:

The German codes, produced by Enigma machines with customizable wheel positions that allowed the codes to be changed rapidly, were considered unbreakable, so nobody was working on them. This attracted Alan Turing to the problem, because he liked solitude. He built a machine that could test different code possibilities, but it was slow. The machine might need four days to test all 336 wheel positions on a particular Enigma code. Until more machines could be built, Turing had to find a way for reducing the burden on the machine.

He used a Bayesian system to guess the letters in an Enigma message, and add more clues as they arrived with new data. With this method he could reduce the number of wheel settings to be tested by his machine from 336 to as few as 18. But soon, Turing realized that he couldn’t compare the probabilities of his hunches without a standard unit of measurement. So, he invented the ‘ban’, defined as “about the smallest change in weight of evidence that is directly perceptible to human intuition.” This unit turned out to be very similar to the bit, the measure of information discovered using Bayes’ Theorem while working for Bell Telephone.

If the whole thing is too much for you, at least read the “Bayes at War” section.