Election time is coming soon.  In a few more weeks, I'll be able to turn on my radio without hearing election ads.  Sweet bliss...

For the most part, I think everyone knows which candidates they will vote for.  You like Obama, you like Romney, or you like neither and have to hold your nose when you vote.  Either way, you probably were decided on your choice months ago.  I know that I was.

The California propositions are a different story.  They are generally confusing, and often deliberately so.  Personally, I am not a fan of the proposition system in general -- a lot of bad laws seem to come from good intentions.  But that is another rant.

When the propositions come up, I like to look at the endorsements coming from different newspapers.  Each newspaper is at least a somewhat disinterested party, and it is the paper's business to understand the issues for each proposition.  This time, I thought that it would be fun to go about it a little more formally.

I went to Wikipedia and took the list of California's largest papers by circulation.  I then visited each paper's website and searched for their endorsements.  (I was not able to find the endorsements for the Orange County Register, Investor's Business Daily, or La Opinión).  The results are summarized below:
 

Proposition:	30	31	32	33	34	35	36	37	38	39	40
Los Angeles Times	YES	no	no	no	YES	no	YES	no	no	YES	YES
San Jose Mercury News	YES	YES	no	no	YES	YES	YES	no	no	YES	YES
San Francisco Chronicle	YES	YES	no	no	YES	YES	YES	no	no	no	YES
San Diego Union-Tribune	no	YES	YES	no	-	YES	YES	no	no	no	YES
Sacramento Bee	YES	no	no	no	YES	no	YES	no	no	YES	YES
The Press-Enterprise	no	no	YES	no	no	no	YES	no	no	YES	YES
Fresno Bee	YES	YES	no	no	no	no	YES	no	no	YES	YES
Los Angeles Daily News	YES	YES	YES	no	YES	YES	YES	no	no	no	YES
Long Beach Press-Telegram	YES	YES	YES	no	YES	YES	YES	no	no	no	YES

There is a surprising amount of consensus between the papers.  For almost half of the propositions, the choice was unanimous.  Here is the breakdown for different issues.

Unanimous for:

• Prop 36: Revise 3 strikes law.
• Prop 40: State senate redistricting.

Strong support:

• Prop 30: Governor's tax increase.
• Prop 31: Legislative reforms.
• Prop 34: End death penalty.

Contentious:

• Prop 32: Paycheck protection.
• Prop 35: Human trafficking.
• Prop 39: Out-of-state corporate tax.

Unanimous against:

• Prop 33: Auto insurance.
• Prop 37: GMO labeling.
• Prop 38: Munger tax plan.

Prop 37 is particularly interesting.  Judging by the number of signs I see around town, it would seem to enjoy strong support among voters.  While there have been a number of seemingly disingenuous ads attacking it, the papers raise some fairly cogent arguments against the proposition.  Even the Santa Cruz Sentinel (not on the list) opposes it:

Citizens would be empowered to sue grocers they believe to be selling unlabeled GE foods, without needing to prove any damages. Clearly, this provision would create even more lawsuits. And who would this benefit? Lawyers. That's what happened after voters in 1986 approved Prop. 65, requiring disclosure of toxic chemicals. The result has been more than 16,000 legal actions. Some were warranted, others were aimed at forcing businesses to settle quickly rather than pay for court costs.
...
The initiative may be well intentioned, but it creates more problems than it solves. Vote no on Proposition 37.

The comments on the article are perhaps predictable:

• "It just goes to show you the hands of the biggest corporations (Monsanto, Dupont, etc) reach far and wide."
• "Who paid for this article to be published? Could it be the no on prop 37 group??"
• "I guess when you don't have the truth on your side, your only option is to confuse people."
• "Absolutely Irresponsible journalism."

Politics as usual. :-)

Why is there an app for that?

I've noticed a disturbing trend lately when I browse the internet with my iPhone or iPad.  It seems like every site that I visit wants me to download their custom iPhone/iPad app.  News sites seem particularly guilty of this quirk.

The main purpose for these apps seems to be storing data offline.  If you have a number of articles cached, you can go read them in your favorite wifi-free park.  Of course, many of these apps seem poorly designed.

Flipboard works quite well, which eliminates any valid reason for a news-related iPhone app that I can dream up.  Since it is now available on Android, it would seem to be much wiser for news sites to focus on creating a quality version of the site for mobile users, and leave the app development to Flipboard.

Also, why don't browsers allow for the same functionality that Flipboard provides?  If users want to be able to review content offline, shouldn't developers be given the tools to create that experience?

Free My Phone

Sprint is a distant third in the iPhone market.  In a somewhat desperate move, they have launched a series of ads about their unlimited data plans.

crickets.

I still have unlimited data with AT&T.  I have not exactly been ecstatic about the service, but I have stayed with them.  Verizon seems to be more popular these days, but AT&T has marginally better coverage in the areas I care about.  More to the point, switching plans is a marginal pain, and I have never seen a reason to bother.

Is unlimited data worthwhile to me?  No.  I'd drop it in a heartbeat if a better offer came by.  So what would it take?

UNLOCK.  MY.  PHONE.

I paid for an expensive phone and I cannot take it with me we when I change networks.  That irritates me immensely.  Sprint, unlock my phone, and I'll switch to your network in a heartbeat.

The Ethics of Piracy

I'm not especially interested in the strict legality of "piracy".  The information age has put the rights of artists and the rights of end users into conflict.  The courts and the legislatures of the world are working on sorting it all out.  Things will change, and a new balance will be established.  I have definite feelings about what the balance should be, but in the meantime, what is the path of the righteous?

I have watched movies and TV shows online for free.  When possible, I try legitimate channels.  I am a fan of Hulu and Netflix.  Some networks make their shows available online.  However, there are times that legitimate channels do not work.  Either the show is not available for free, or there are absurd restrictions.

While I was in France, I was not able to watch several shows online since I was not in the United States.  Why should that matter?  Why should the pilot of the Walking Dead only be available to me if I am in the United States?  Companies are free to make their content available to whomever they wish, though the restriction does seem odd.

I have DVDs in my collection with Chinese subtitles.  I have watched movies on YouTube that are almost certainly not authorized.  Where should I draw the line?

Entertainment companies have been vicious in protecting their cartel.  I have little love for them, and I have no ethical qualms about removing their restrictions for materials that I have purchased.  Nonetheless, my actions have not always been ethical.

Is it ethical to buy DVDs from China?  Is it ethical to watch YouTube videos of copyrighted material?  I believe that the answer is no.

Is it ethical to support a corrupt system that is taking away the rights of its users?  I am not totally sure.  For the time being, it seems the lesser of two evils.

Perhaps the best bet is to limit yourself to free material.  Libraries make a great deal available.  Video rentals are cheap.

We live in dangerous times.  We need to be vigilant for our rights, and respectful of the rights of others.  The convenience of the internet makes it easy to violate both.

I have a Hidden Markov Model... Now What?

I have been working on creating hidden Markov models (HMMs) for computer viruses.  Now that I have them, I'm running into an interesting complication.  Namely, what can I do with them?

With an HMM, you can get the statistical probability for any particular series of observations.  For a very simple case, consider a loaded die.  50% of the time, it will roll a 6.  Otherwise, it will roll a number between 1 and 5 (10% chance each).  Once you have your model built up, you can determine the probability of a series of rolls.

So, continuing the example, pretend that you observe 10 sixes being rolled in succession.  What are the odds that this sequence would have been rolled with the loaded die?  (1/2) ^ 10, or 1 in 1024.  Given these observations, is it likely that you are using the loaded die?

Well...  it depends.  What are the other models?  The probability for the same sequence with a fair die would be 1 in 6,0466,176.  On the other hand, if you suspect that the die might be loaded so that it rolls sixes 90% of the time, the observations fit much better with that model.

My first exposure to HMMs was in linguistics.  I built up two language models for classified advertisements -- one for Spanish and one for English.  By comparing the probabilities of any random classified ad, I could guess fairly easily whether an ad were English or Spanish.  But if it happened to be in French or Vietnamese, my tool would have failed miserably.  (On a side note, one of my friends faced with a similar problem for news stories used a simpler solution -- he counted the number of 'the's and contrasted that to the number of 'el's and 'la's.  I never heard of a single bad identification with his approach.  It goes to show, the sophisticated solution might not always be what is needed).

This raises some interesting questions for me in the context of computer viruses.  HMMs seem to be a compelling option for virus detection, but what do they compare against?  You can imagine a series of models built for different virus families, but what if the file is not a virus?  It does not seem realistic to build a model for 'all benign programs'.  Neither does it seem realistic to build a model for each type of benign program.

There is likely a clean, well-known solution.  I just don't know it yet.

Otherwise, life in Laval has been fun.  My wife has arrived, and we've started to explore the surrounding town together.  The Lavalloise seem to be a little shy about their town.  Compared to Rennes or some of the other larger towns, perhaps Laval is a little sleepy.  But somehow it is very cool to sit and have a glass of wine at the foot of an 800 year old castle.  Coming from the western United States, where 100 years seems like a long time, the history of Laval is amazing.

The Virus War

In ESIEA, I am doing research on metamorphic viruses.  It is a new area for me, so I have been reading up on lots of new material.  I am fascinated at some of the gambits and defenses that are happening in the war between virus writers and antivirus researchers.

In the past week, I have been experimenting with virus construction kits, octave (free version of matlab), and reading reams of papers on computer viruses, hidden Markov models, etc.  I feel like I am going in about 12 directions at once.  But as my master's thesis adviser once told me, "that's research".

A quick history of viruses...

The classic viruses were fairly easy to detect through a method known as "signature detection".  Essentially, virus scanners look for a bit pattern associated with a virus to identify a corrupted file.  This method is still the predominant one, but newer viruses are being designed to evade this method.

"Encrypted viruses" attempt to evade scanners by encrypting the body of the virus.  Typically, this would be done with a XOR operation, so that the same procedure can be used to both encrypt and decrypt the body of the virus.  By itself, this approach is not especially useful -- the virus scanner can still identify the signature of the encryption/decryption code.

"Polymorphic viruses" improve on encrypted viruses by mutating the decrypter function.  A simple version of the signature detection approach will then fail totally.  Except...  Modern scanners will decrypt the virus body, and then scan the virus.  (I am still a little fuzzy on how they know when to decrypt the virus body.)

But polymorphic viruses point the way to a far more interesting approach.  Rather than relying on encryption, "metamorphic viruses" mutate the body of the virus.  This strategy can evade signature detection approaches without relying on encryption.  (Interestingly, DRM systems are apparently exploring this technique to defy reverse engineering efforts).

Detecting metamorphic viruses is fairly challenging.  Fortunately, most of the metamorphic viruses today have not been particularly good.  But some are.  NGVCK (Next Generation Virus Construction Kit) was designed (apparently) as a proof of concept.  It produces harmless, but hard to detect viruses.  (Its last release was in 2002 -- virus scanners might have caught up to it these days).

Current research has been exploring statistical models, especially hidden Markov Models (HMM).  The results seem promising, but the battle is not over.  Some research suggests that attackers could tune the mutations to emulate benign files.  Virus scanners are then left with the unpleasant choice of rejecting benign files or accepting some malicious files (and probably some of both).

Anyway, it is an exciting new realm for me!

An American in Laval

After finishing up a fantastic summer at Mozilla, I hopped on board a plane to France to begin my 3 month odyssey abroad.  I was still exhausted from the all hands meeting at Mozilla.  I woke up at 4am to catch the shuttle to the airport, with a layover in Philly, followed by an hour shuttle from Charles de Gaulle to the train station at Montparnasse, followed by a 2 hour train ride to Laval, finally to arrive at my destination at about noon the following day.  I think I am just finally catching up on sleep now.

I have been in France for almost a week, and I've been overwhelmed by my reception.  The people here have been uniformly friendly, and have gone out of their way to make me feel welcome.  The town of Laval is lovely, and the food has been delicious.

The Saturday market in Laval was overwhelming.  In California, we have farmer's markets, but these are pitifully small compared to Laval.  There was fresh-baked bread, giant tubs of paella, seafood so fresh that it was literally trying to escape, and produce that has to be seen to be believed.  I think the produce section alone would be equivalent to 3 or 4 farmer's markets back home.  I think I will enjoy my time here.

So far, the biggest difference that I have noticed is that there is a sharp divide between work and play.  In the states, we buy huge cups of coffee and take them to go so that we can go back to work.  Half the time, 'work' might consist of Facebook and Farmville, but the pressure to be at our desks is very strong.

In France, cups of coffee are small, and no one gets them to go.  You sit and chat with friends, and when you are finished, you go back to work.  And then you work.  I'm not sure who comes out ahead in terms of production, but I am gaining an appreciation for the French approach.