<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
   <title>aBlog</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/" />
   <link rel="self" type="application/atom+xml" href="http://www.pandamatak.com/people/anand/blog/atom.xml" />
   <id>tag:www.pandamatak.com,2010:/people/anand/blog/2</id>
   <updated>2010-06-11T15:00:37Z</updated>
   <subtitle>Anand Venkataraman&apos;s Blog</subtitle>
   <generator uri="http://www.sixapart.com/movabletype/">Movable Type 3.35</generator>

<entry>
   <title>Can connect to localhost but not to its IP Address?</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2010/06/can_connect_to_localhost_but_n.html" />
   <id>tag:www.pandamatak.com,2010:/people/anand/blog//2.71</id>
   
   <published>2010-06-11T14:34:25Z</published>
   <updated>2010-06-11T15:00:37Z</updated>
   
   <summary>Welcome! If you got here, you probably searched for &quot;Could connect to localhost, but not to its IP address&quot; or some variation of that phrase. That&apos;s how I got here. To writing this post, that is. After unsuccessfully trying many...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Technical" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      Welcome!  If you got here, you probably searched for &quot;Could connect to localhost, but not to its IP address&quot; or some variation of that phrase.

That&apos;s how I got here. To writing this post, that is. After unsuccessfully trying many clever variations of my query theme in finding an answer to this problem, we decided to troubleshoot it ourselves, ended up fixing it, and decided to blog for the benefit of the wider community.

If you&apos;re lucky, (a) our solution works for you and (b) this post shows up in the first few pages of your Google results saving you much time you might have wasted in following dead-ends.

      <![CDATA[Here is the problem - We discovered it first in our hadoop cluster, which had been set up by a contracted sysadmin no longer with us. The namenocde could not talk to any of the datanodes. The logs would show "Connection refused". Debugging with "telnet -d datanode 54311" would surface a misleading error message with setsockopt complaining about lack of permission.  Funnily enough, when logged on to the datanode directly, we could issue a connection request to localhost, or 127.0.0.1 and it would work perfectly, but all connection requests to datanode, its fully qualified domain name, or even its IP address would fail.

Since we weren't running many other services on the datanodes, this problem didn't manifest in other applications. However, a quick check revealed that the problem was general. We could start up sendmail or an echo server and find the same discrepancy between connecting to localhost and its IP address.

Needless to say, as most people on various forums have tried to do, we too tried the obvious thing first - Perhaps an issue with SELinux or firewalls?  We turned both off. But to no avail. That's when we decided to look a bit deeper.

<div style="color: red; font-family:courier-new; font-weight:bold; margin-left:10px;">
  netstat -an | grep 54311
</div>

showed that a listener was attached to 127.0.0.1:54311, which is to the loopback interface, but not to datanode.xyz.com:54311 or its IP-address:54311. Hmmm... how could that be?  We checked the hadoop configuration and it was clear that it bound to the domain name of the host, not localhost. So what gives? That's when we discovered that /etc/hosts was somehow misconfigured on our machines. The culprit, specifically, was this line:

<div style="color: red; font-family:courier-new; font-weight:bold; margin-left:10px;">
127.0.0.1 localhost.localdomain localhost datanode.xyz.com
</div>

If you don't see the problem, it's the fact that datanode.xyz.com was being locally associated to the loopback IP address in this file. So of course, when a program issues a library call to resolve datanode.xyz.com, it's going to resolve to localhost, rather than its IP address, since the /etc/hosts file takes precedence over DNS queries.

Now the fix was clear.  All we had to do was to restore the /etc/hosts file on all our datanodes. Simply replacing

<div style="color: red; font-family:courier-new; font-weight:bold; margin-left:10px;">
127.0.0.1 localhost.localdomain localhost datanode.xyz.com
</div>

with

<div style="color: red; font-family:courier-new; font-weight:bold; margin-left:10px;">
127.0.0.1 localhost.localdomain localhost
</div>

did the trick. If it doesn't for you, then let me know and we'll have another think about this. I hope this helps.

&


]]>
   </content>
</entry>
<entry>
   <title>When is a good time to die? (Farewell 33Across)</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2010/01/when_is_a_good_time_to_die_far.html" />
   <id>tag:www.pandamatak.com,2010:/people/anand/blog//2.69</id>
   
   <published>2010-01-15T01:00:54Z</published>
   <updated>2010-01-17T00:48:29Z</updated>
   
   <summary>&quot;Not to be born is the best thing of all and the next best thing is to die soon after&quot;, said a great Greek playwright once. Almost two and a half millenia later, those powerful words have not lost their...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Happenings" scheme="http://www.sixapart.com/ns/types#category" />
   
   <category term="50" label="33Across" scheme="http://www.sixapart.com/ns/types#tag" />
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA["Not to be born is the best thing of all and the next best thing is to
die soon after", said a great Greek playwright once. Almost two and a
half millenia later, those powerful words have not lost their poignant
profundity. As Arnold Toynbee had observed just a little bit earlier
in our own time, the greatest problem of mankind is spiritual
suffering, and no amount of plumbing and central heating can alleviate
spiritual suffering. Yet, much as I resonate strongly with the ultimately
pessimistic wisdom of Sophocles and Toynbee, I believe we should make
the best of what we have. When it comes to parting, I think Fonzie
packs the most punch (although Nick Romano said it first according to
Richie) "Live fast, die young, leave a good-looking corpse".<p/>

Now that I've been around the block a few times, I think that Fonzie's
counsel applies equally well to people's careers. When's the best time to
leave?  Of course, the answer is different if you are a founder, or
you are treated like one. You would carry what I call the founder's
fardel - The company is as much a part of you as you are of it, and
you relish the burden on your back - Like Hotel California, "you can
check out any time you like, but you can never leave". But for almost
every one else, I think this is what I've learnt: The best time to
leave is when you've done a great job, you've delivered on your
commitment, you're having fun, you're proud of yourself, and you have
successfully and fearlessly made yourself redundant (or almost).<p/>
]]>
      <![CDATA[I agree it is difficult to leave when things are looking
good. Too tempting it is to lay back and enjoy the fruits of hard work
we have poured into an undertaking. The challenge is not unlike a
heart-rending decision to sell off your company when it is on an
upswing rather than on a high plateau. Even the great Yoda recognizes
the immense difficulty in making this decision: "Ready he is, to teach
an apprentice. To let go of his pupil, a greater challenge it will
be".  But deferring the immediate gratification of basking in the
comforting warmth of today's achievements is not only a mark of
maturity, but also a prudent and pragmatic choice. Rosier memory
will offer much needed solace at later and trying times. One would
have to be extremely arrogant to pretend these do not to exist down
the road.<p/>

After the <a href="http://www.pandamatak.com/people/anand/blog/2009/12/from_powerpoint_to_profitabili.html">amazing
progress we made on all fronts</a> over the last months, I feel the
timing is just right for my departure from 33Across. I feel good about
the company; it is poised for great success. I think it has enough
forward momentum to pull through even tough times, except for
extremely grave missteps in strategy. We have together learned, I
hope, important things through experience as we built a fantastic team
and took the company from nebulous notions to power-packed
products. And so I bid a fond farewell to 33Across, a
roller-coaster I am so glad to have gotten on almost two years
ago. Indeed what better time to revive my passion for mobile and
location-based technology than when the Google phone has just out!<p/>

Since I started this note paraphrasing a great man, it's only fitting
that I end it with the words of another great human, this time from
our own time, our own soil, and one whose birthday it is tomorrow. At 33Across, we once
jokingly referred to our ad targeting segments as precision guided
missiles that would take each ad to exactly the person that would want
to see it. Dr King's observation continues to relevant in our new
context also. "May we never have guided missiles and misguided men."<p/>
]]>
   </content>
</entry>
<entry>
   <title>From Powerpoint to Profitability</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2009/12/from_powerpoint_to_profitabili.html" />
   <id>tag:www.pandamatak.com,2009:/people/anand/blog//2.67</id>
   
   <published>2009-12-25T00:36:34Z</published>
   <updated>2010-01-16T16:11:21Z</updated>
   
   <summary>Another fantastic year at 33Across. We have come a long way since our early Power-point days. We have accomplished amazing feats and as Eric said at our recent all-hands, we have continued to hit the ball outta the park. We...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Happenings" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Another fantastic year at 33Across.  We have come a long way since our early Power-point days. We have accomplished amazing feats and as Eric said at our recent all-hands, we have continued to hit the ball outta the park. We have had our ups and our downs, petty squabbles and grand confluences. But in the end, we've all come to truly appreciate what we have.

So here's to a great year in celebration and anticipation.  John, Mike and Sree - We missed you last night and hope you can make the next one.

<div style="text-align: center;">
  <a href="http://www.pandamatak.com/people/anand/blog/33x_dinner-2009-table-800x.jpg">
    <img alt="33x_dinner-2009-table-480x.jpg" src="http://www.pandamatak.com/people/anand/blog/33x_dinner-2009-table-480x.jpg" width="480" height="360" />
  </a>
</div>

Here's the 4-second video that FRC wanted - as one of us said, ultimate proof that the 33Across engineering team always over-delivers :-)

<div style="text-align:center;">

<object width="425" height="344"><param name="movie" value="http://www.youtube.com/v/C1ht_mNJZ0U&hl=en_US&fs=1&"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/C1ht_mNJZ0U&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>

</div>

Happy 2010 all!

&
]]>
      
   </content>
</entry>
<entry>
   <title>Unintentional Poetry</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2009/11/unintentional_poetry.html" />
   <id>tag:www.pandamatak.com,2009:/people/anand/blog//2.65</id>
   
   <published>2009-11-26T18:19:01Z</published>
   <updated>2009-11-27T07:18:16Z</updated>
   
   <summary>Here is the best of totally unintentional communal poetry. It&apos;s written by folk who didn&apos;t have the slightest suspicion that they were contributing to the effort, and shamelessly plagiarised by me. Read on, and sue me if they don&apos;t rival...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Misc" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Here is the best of totally unintentional communal poetry. It's written by folk who didn't have the slightest suspicion that they were contributing to the effort, and shamelessly plagiarised by me.  

Read on, and sue me if they don't rival with the best koans you know.  After you've read the first one, see the picture below to reveal the secret behind the poem, and the names of the authors, themselves unaware of their worthy words.

<div style="color:gold; text-align:center;">
<div style="text-decoration:underline;">Please Tell Me</div><P>

Please tell me it is so
But what will you give me
Such a little firebrand

And I have no mother or father
From that dark day to the present
Far and few, far and few

Look what I pictured on road
At midday
scraped the cupboard
On that little heap of stones

But
You decide, if they die or not
Sorry for your father
Dead or alive

If I were
In the middle of a wood
Reed broke the silence
An inn when he suddenly noticed
You can save more on the goods for solving man's problems

</div>

And here's the secret to the origins of this poem.

]]>
      <![CDATA[<center>
<a href="http://www.pandamatak.com/people/anand/blog/poem-1.jpg"><img alt="poem-1-thumb.jpg" src="http://www.pandamatak.com/people/anand/blog/poem-1-thumb.jpg" width="476" height="411" /></a>
</center>

It's amazing. Don't you agree? And I didn't even have to reorder the lines!

And there's more:

<div style="color:gold; text-align:center;">
<div style="text-decoration:underline;">I Was A Baby</div><P>

I was a baby
Said the Yonghy-Bonghy-Bo

Panic in Atlantic City disco
Please tell me it is so

Quit your day job
Said the Yonghy-Bonghy-Bo

If I had anywhere else to go
In the middle of a wood

Enlarge, expand and strengthen
Said the Yonghy-Bonghy-Bo
</div>

How about this one?

<div style="color:gold; text-align:center;">
<div style="text-decoration:underline;">Is My Card At Yours?</div><P>

Dear
Is my card at yours?
Don't worry
Nor fear the mockery of thy yellow flowers

If I had anywhere else to go
Place where I reside
A good place to make your condition better
I am not deceitful
I shall be one with nature herb and stone
But I knew no one else in the house agreed
And hardly knew what it meant

I was a baby
Arbitrary
Story floor
Toxine destroyer
Have you anything more to add?
If we got nothing lasting in their stead

Now it so happened that on one occasion
Reed had made her promise that she would
Bring electricity back to your love circle!
The voltage will be high
Get a full-mast again
Choose your topic
Become virile like a rabbit!

Bessie came in with a pastry tart for me
Caplet-formed dong-booster
Which they ate with a runcible spoon

When girls fight daily
Bessie will help you pack your things
Indeed she's hot
What else we've got

An inn when he suddenly noticed
It was now late
He set off for home
To thank his generous host
Forget Asking For A Link
Your reply is blocked

</div>

Finally, how about this one?

<div style="color:gold; text-align:center;">
<div style="text-decoration:underline;">I depart as air</div><P>

I depart as air
I shake my locks at the runaway sun
I bequeath myself to the dirt
To grow from the grass I love.
If you want me again
Look for me under your boot soles
You will hardly know
Who I am or what I mean
But I shall be good health to you nevertheless.
Missing me one place?
Search another
I stop somewhere
Waiting for you

</div>

But no - That last real gem was from one of my favourite episodes of Six Feet Under, Season 3, Episode 3, Disc 3 :-)

Feel free to contribute more <em>poetry of dubious value</em>, and wreak havoc on my blog's spam detector!

And on this fine Thanksgiving day of 2009, let me express my heartfelt gratitude to that most unlikely folk of all to receive thanks, those incorrigible spammers, for giving me something rewarm my blog pipe.

&
]]>
   </content>
</entry>
<entry>
   <title>The Four Fours</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/12/the_four_fours.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.60</id>
   
   <published>2008-12-18T03:14:42Z</published>
   <updated>2009-01-15T21:58:03Z</updated>
   
   <summary>This year I&apos;m a volunteer teacher at JLS in Palo Alto to help guide a small number of curious children into the wonders of advanced mathematics over 12 classes in 6 weeks. The program is called &quot;The Number Devil&quot;. It...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Misc" scheme="http://www.sixapart.com/ns/types#category" />
   
   <category term="40" label="four fours" scheme="http://www.sixapart.com/ns/types#tag" />
   <category term="42" label="math game" scheme="http://www.sixapart.com/ns/types#tag" />
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[This year I'm a volunteer teacher at JLS in Palo Alto to help guide a
small number of curious children into the wonders of advanced
mathematics over 12 classes in 6 weeks. The program is called "The
Number Devil".  It uses chapters from the similarly named book by
Hans Magnus Enzensberger as anchor points for discussion during each
class.<p>

Just before the holidays started, the students were to do a warm-up
exercise in preparation for diving into the deeper wonders of Number
Theory in upcoming classes.  Here is the exercise, called the Four
Fours.  Using exactly 4 digits, all of which are fours, and any number
of the arithmetic operators Plus, Minus, Times, Divide,
Exponentiation, Factorial, Square Root, and Parenthesis, derive each
of the numbers from 1 to 50.  E.g. One way you can derive 1 is
obviously 4/4 *4/4.  No doubt there are a number of different ways to
derive each number and the goal is to get students to think about
this, and to come up with expressions that are different from those of
their colleagues.<p>

We, however, decided to turn this little exericse into a game, which
as a matter of fact, also turned out to be a good game to occupy young
minds during the holidays!  Here is how you would play it:<p>

The game has 15 rounds.  Each round lasts 1 minute.  We tried to cover
15 numbers in the game.<p>

Each round consists of:<p>

<ol>
   <li> Host (me): Calling out a different random integer between 1 and 50.
   <li> Players: Each of them has 30 seconds to write an expression
        with exactly 4 4s and the arithmetic operators to get this number.
   <li> We compare answers and points are scored for this round as
        follows:
        <ul>
         <li> Zero points - No expression or invalid expression
              (meaning it doesn't evaluate to the number called)
         <li> M-N points - Otherwise.  M is the number of players and N is the number of other players who have the same expression as you.  If your expression is unique, you get the full M points for the round.
        </ul>
</ol><p>

]]>
      <![CDATA[A great deal of fun though this game was, I was pleasantly surprised
by the number of related meta-issues that we could discuss that laid
the stage for all kinds of advanced concepts one would learn in
grad-school and beyond.  Here is just a sampling of them:<p>

<ul>
 <li> Just how many legal expressions can you ever make (A legal expression 
 is some combination of the 4 fours and the operators that will
 compute correctly, i.e. is syntactically correct)?  What if you
 couldn't use Square Roots, Factorials and Parentheses?  Can you still make
 an infinite number of expressions?  If not exactly how many?<p>

 <li> What is the largest number you can calculate? What if you didn't
 have Factorial?<p>

 <li> How can you tell if two expressions are the same? E.g. If
 John came up with 4*4 * 4/4 = 16 and Mary came up with 4/4 * 4*4 =
 16, how can you tell they were the same?  Notice that it's not always
 this trivial.  This discussion leads to concepts of Normal Forms, how to
 reduce to Normal forms, whether even such reduction is always
 possible, whether unique normal forms exist, and
 how one can algorithmically (and efficiently) compare Normal forms.<p>

 <li> This point, especially, I love because it goes to the root of
 Science, Math, Non-Science and their distinctions! But I've
 completely forgotten the discussion chain that led to this point in
 class!  Suppose John and Mary were to form one expression each as
 above, but instead used a variable for one of the fours.  By
 substituting a value for this variable, each expression would
 evaluate to a certain number. However, it happens to
 be the case that we are unable to reduce
 either expression to Normal form and thus compare them directly.  How
 could we now tell whether the two expressions are in fact the same?
 This brings up all kinds of interesting discussion points -
 Conjectures and Theorems Versus Hypotheses and Theories, Deductive
 and Inductive reasoning, Mathematical Induction, Empirical testing
 and indeed, gives students a real taste of that tantalizing field of
 exploration - Epistemology!<p>

</ul><p>
]]>
   </content>
</entry>
<entry>
   <title>Cool Currency</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/12/a_new_form_of_currency.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.56</id>
   
   <published>2008-12-13T15:52:16Z</published>
   <updated>2009-05-14T16:33:26Z</updated>
   
   <summary> After a long year of Sharon teaching their delightful children, the parents had decided to reward her. On Teacher Appreciation Day, following heart-felt expressions of gratitude and astonishment at what a wonderful job she had done, the head parent,...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Essays" scheme="http://www.sixapart.com/ns/types#category" />
         <category term="Misc" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[<div style="padding-left: 10px; padding-right:10px; color:yellow;">

After a long year of Sharon teaching their <em>delightful</em> children, the parents had decided to reward her. On Teacher Appreciation Day, following heart-felt expressions of gratitude and astonishment at what a wonderful job she had done, the head parent, who had coordinated by collecting contributions towards Sharon's gift, presented her with a brightly colored sealed envelope amidst generous applause.  

Sharon opened her gift with great eagerness and found what she would never have spoiled herself with in her wildest dreams - She was now the proud owner of a certificate bestowing upon her the gratitude of a thousand children in Rwanda, who had just been donated $500 on Sharon's behalf.  

Yes - The head parent, in her infinite wisdom, had decided that the best gift for Sharon would not be a couple of sessions at a posh Palo Alto Spa, or a comfy sofa to relax between classes, or even a Bloomingdale's gift card, but rather a donation to a worthy African cause in Sharon's name.  "What could please Sharon more?" would likely have been the most pressing question in her mind, as she tossed around various options for a gift to proudly decide that someone who spends almost every waking hour with children could only ever be pleased by being told that she had benefited yet more children.

</div>
]]>
      <![CDATA[I don't mean to disparage the gift or to belittle the cause.  But as a recipient, would you rather not that the giver had trusted you to make your own decision? Was there a hint of an insinuation there? Might you have spoiled yourself on a trip to Vegas instead of a worthy donation had you been given cash instead?  And is it wrong if you had?

Much as I love this amazing nation of ours, I cannot but help feeling perplexed at how we have let media drive our culture into pathological consumerism, so much so that today, people seem to have absolutely no respect for the worthiest thing of all - the humble dollar bill - unless it was replaced by some other tangible of supposedly equal or better value.

Since the season of gifts is soon upon us, I feel a great compulsion to get these words out sooner than later, lest we suffer yet again the plight of our poor friend and school teacher. What is the best gift you can give someone? Other than your own personal time and effort - An item such as pottery or jewelry you have made yourself, or your time as you offer to massage tired shoulders - I believe the best proxy for your true intention is pure cash.  If I cannot offer something I personally made or propose to make by my own hands, I'd rather present an envelope stuffed with real cash. It is the sincerest form of appreciation and it says "here is a little of my hard-work for you, in return for all of your hard-work for me."

Anything else is a farce, driven into our heads by malicious monopolies and greedy corporations gunning for our hard-earned dollars.  If you're thinking of a gift for someone this holiday season, why not give them an envelope with cash, and a little printout of this article, or a pointer to this URL?

<div style="color:yellow; padding-left: 10px; padding-right:10px; text-align: center;">

A gift of cash is
The best gift of all
It comes from the heart, no
Matter how small

It says that I care
I love the things you do
You made me want to share my
Hard-earned $$$ with you

</div>

Let me leave you with this fictitious and farcical incident that perhaps better captures my thoughts.

<h2>Scene: Safeway at Middlefield, Palo Alto, CA.</h2>

<div style="text-align=center;">
 <center>
   <img src="/people/anand/blog/images/gift-cards-small.jpg" />
 </center>
</div><p>

<div style='padding-left:10px; padding-right: 10px; color: yellow;'>
"Excuse me, can you change a twenty dollar bill for me please?"

"Sure brother. Tens?"

"Err... no..."

"Fives maybe?"

"Uh, no actually..."

"Well, you better tell me then.  Sure as hell can't do nickels and dimes you know?"

"Well, I was kinda thinking another twenty?"

"Excuse me? That bill real man?  Funny money?  Do I need to call the cops or somethin?"

"Hey hey hey, don't freak out.  Yeah, just what I said.  I want another form of $20, you know, something that isn't as flexible as this one?"

"Dude, am like totally lost man... like, not sure what you mean?"

"I'd want something that I can use to buy $20 worth of stuff, but can't use just anywhere you know.  I want to be able to use it at just one place - like Starbucks.  No no make that Pottery Barn... Hey you know what - let's do Mc Donald's.  I mean I want a 20 dollar bill I can only spend at Mc Donald's.  Nowhere else.  I'd like to swap my government issued spend-anywhere-in-this-beautiful-world mint-condition Jackson for a shiny plastic 20-buck-proxy that no would will accept but Mc Donald's.  Actually, it'd be super if you can make that the Mc Donald's on the corner of El Camino and Ravenswood, but I'll take any Mac."

"Now, why would you want that sir?  Why not keep your Twenty?  You realize it's legal tender, and that your favorite McD will take it anyway?"

"Ah, yeah, sure I know that.  But you see it's not for me.  I want to give it to a friend.  It's a gift for her birthday."

"Yeah?"

"I can't give her the $20 now, can I?  I mean, she'll be able to spend it just about anywhere."

"And that's bad?"

"You bet it is. What if she ends up buying herself some granola bars at Whole-Foods like she always does?  Even worse, she might end up buying a book at Kepler's.  I can't have that. Can I? The truth is, [whisper] I don't trust her you know? She doesn't really know to spend wisely.  I gotta help her out."

"Aah..  Then maybe I can introduce you to this nice box of candy?  It's usually $40, but today only it's on sale for $20, and I can peel the price tag off of it..."

...

</div>

]]>
   </content>
</entry>
<entry>
   <title>Deanimizing - A necessary first step to dehumanizing?</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/08/deanimizing_a_necessary_first.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.55</id>
   
   <published>2008-08-09T20:39:43Z</published>
   <updated>2008-08-10T01:43:36Z</updated>
   
   <summary>Last night, I was at a networking social with the Olympic Opening Ceremony on an enormous screen. But I was there strictly to socialize and catch up with good friends. I could not bring myself to enjoy the opening ceremony,...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Political" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Last night, I was at a networking social with the Olympic Opening Ceremony on an enormous screen.  But I was there strictly to socialize and catch up with good friends.  I could not bring myself to enjoy the opening ceremony, flamboyant and ostentatious as it was, in light of the appalling abuses of human and animal rights in a country where, at the very least, all this money could have been put to much worthier use in improving the human condition!  

People might say that it's my loss for missing this magnificent display of oneness and splendor. But I don't see it.  Fortunately for me, there is no dearth of interesting alternatives in today's world that can equally well and guiltlessly take up my time.  Fortunately, I suffer the ancient Chinese curse.  I live in interesting times.  I must boycott the Olympics to protest in what little way I can against such appalling cruelty as <a href=http://www.peta.org>PeTA</a> exposes.

]]>
      <![CDATA[No doubt there is much cruelty elsewhere too.  Even here on American free soil, most would have seen PeTAs shocking expose of KFC's <a href=http://www.kentuckyfriedcruelty.com/u-georges.asp>red bird syndrome</a> that shows millions of birds end up being scalded to death:<p>

<div style="text-align:center;">
<embed src="http://www.kentuckyfriedcruelty.com/swf/pam_kfc_320.swf" quality="high" pluginspage="http://www.macromedia.com/go/getflashplayer" type="application/x-shockwave-flash" width="335" height="255" allowScriptAccess="always"></embed>
</div>
<br>

But the <a href=http://www.peta.org/feat/ChineseFurFarms/index.asp>anti-fur video</a> took it to a whole new level for me.  I tried not to blog on this topic a second time, but I could not help it. I am still distressed by my memory of the <a href=http://www.pandamatak.com/people/anand/blog/2008/08/skinned_alive_and_tossed_on_to.html>PeTA video</a>.

That a sentient being, much less a human, is even capable of something like this is deeply disturbing.  Despite endless intellectualization that natural variation in neural circuitry is bound to produce innate predispositions to this kind of behavior uncontainable by ethical rationalization, the cold reality of <a href=http://www.pandamatak.com/people/anand/blog/2008/08/skinned_alive_and_tossed_on_to.html>the video</a> scares the shits out of me.  For want of a better word, let me call this deanimization. The way I see it, deanimization, the ability to train oneself to see sentient creatures as unable to feel, is a necessary first step to dehumanization. It is no stretch to the Auschwitz, genocide on a massive scale, to Saddam and his likes, to atrocities of <a href=http://en.wikipedia.org/wiki/Vlad_III_the_Impaler>Vlad the impaler</a> and worse.  One small step to revert to the horrors of the past that we believed were mostly behind us.  Gives me the shits indeed. 

Being the skeptic that I am, I cannot seek solace in the infinite compassion of a claimed superior being.  But I do have hope. One day we will be able to engineer precision genes.  We will have genes for compassion, and for voluntary self-extinction at a time of one's own choosing, despite their evolutionary disvalue and non-viability. Maybe I'll say more about this if I ever complete the article I have tentatively titled "The Jesus quality and the Bhishma faculty."

&

<div style="color:yellow; font-size: 80%;">PS. Please excuse the python-long sentences in this article.  Clearly I can't think straight under distress.</div>
]]>
   </content>
</entry>
<entry>
   <title>Skinned alive and tossed on to a pile of flayed friends?</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/08/skinned_alive_and_tossed_on_to.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.53</id>
   
   <published>2008-08-09T06:08:04Z</published>
   <updated>2008-08-09T23:49:01Z</updated>
   
   <summary>Fuck! I cannot think of any other word. Sorry. I don&apos;t know about you, but I couldn&apos;t watch past the midpoint of this video. I was hoping to vent some in this article, but it doesn&apos;t seem to help! Perhaps...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Political" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Fuck!  I cannot think of any other word.  Sorry.

I don't know about you, but I couldn't watch past the midpoint of this video.  I was hoping to vent some in this article, but it doesn't seem to help!  Perhaps the least I can do to show my support to PeTA (where I am a member) and to other animal rights organizations is to spread the word.

I bet half the people who wear fur will give it up if only they knew what went into making it for them, just as Erwin Schrödinger hypothesizes in his famous essay that half the people who ate meat will give up if only they had to kill the animal themselves!

The following video is not for the weak-hearted.  Click not if you can stomach not.  Instead just read the equally compelling but less shocking text at:  <a href="http://www.peta.org/feat/ChineseFurFarms/index.asp">http://www.peta.org/feat/ChineseFurFarms/index.asp</a>
<br>

<div style="text-align:center;">
<embed src="http://www.peta.org/swf/fur_farm.swf" quality="high" pluginspage="http://www.macromedia.com/go/getflashplayer" type="application/x-shockwave-flash" width="335" height="255" allowScriptAccess="always"></embed><br /><a href="http://www.furisdead.com/pledge-furfree.asp?c=cfsv">Pledge to go fur-free at PETA.org.</a>
</div>

Again, here is a link to the full article at PeTA's site:

<a href="http://www.peta.org/feat/ChineseFurFarms/index.asp">http://www.peta.org/feat/ChineseFurFarms/index.asp</a>

Please support PeTA!

&
]]>
      
   </content>
</entry>
<entry>
   <title>Irrational decisions in the workplace</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/07/irrational_decisions_in_the_wo.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.51</id>
   
   <published>2008-07-16T05:46:38Z</published>
   <updated>2008-07-16T08:54:59Z</updated>
   
   <summary>Ori Brafman asked on Linkedin: What are your best examples of irrational decision-making in the workplace? I answered thusly, though not to his point: I believe, perhaps irrationally :-), that we all make two kinds of decisions regularly in our...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Forum Posts" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Ori Brafman asked on <a href=http://www.linkedin.com>Linkedin</a>:  <font color=cyan>What are your best examples of irrational decision-making in the workplace?</font>

I answered thusly, though not to his point:

I believe, perhaps irrationally :-), that we all make two kinds of decisions regularly in our life. Well-founded, data-driven and reasoned decisions are the most common and form the first kind. The much rarer second kind is driven by instinct, gut feeling and perhaps no more than a deep sense of passion and belief in the long-term success of what appears, on all immediate counts, to be unfounded.
]]>
      <![CDATA[Interestingly, as I say in my blog blurb on <a href="http://www.pandamatak.com/people/anand/blog/1995/01/scrabble_idealism_and_careers.html">Scrabble, Idealism and Careers,</a> the former kind of decisions (well-reasoned ones) leads to steady incremental success with high likelihood. This decision-making process is just a calculus. We learn it in business schools. Every CEO should be at least competent in it.  On the other hand, the latter kind of decisions (instinctive ones) is what leads to major revolutions and fundamental game changes in the playing field. It takes vision, and is often perplexing and hard-to-explain to most. This is what distinguishes true visionaries from the rest. 

The flip side of this equation, of course, is that like all laws of Nature, the high yield of the latter kind of decision making tends to be balanced with a correspondingly diminished likelihood of success. But yet, if not for these, what excruciating tedium life would become to bear! In science and art, at least, there is some evidence that beautiful and penetrating insights often come from instinct rather than by "treading the groove", as Robert Kanigel says eloquently in his biography of a mathematician. 

I have certainly made irrational decisions in my life. I look forward to and welcome the opportunity to make each one. I savor their taste and bask in the warmth of the fire in my belly that fuels them. When successful, these are the kinds of decisions people write about, and in the end, we ourselves might look back at and treasure.
]]>
   </content>
</entry>
<entry>
   <title>The kiss you can remember</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/07/the_kiss_you_can_remember.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.50</id>
   
   <published>2008-07-16T05:10:56Z</published>
   <updated>2008-07-18T18:47:58Z</updated>
   
   <summary>How often do you google yourself? Other than for reasons for plain vanity, I think it important to do it often enough to ensure your online identity isn&apos;t hijacked. Since I have multiple online aliases, some of which I share...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Forum Posts" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      How often do you google yourself? Other than for reasons for plain vanity, I think it important to do it often enough to ensure your online identity isn&apos;t hijacked.  Since I have multiple online aliases, some of which I share with others, I have an automatic system that googles my online avatars every once in a while, reporting any &quot;anomalies&quot;.

Imagine my surprise when I found that I had, unbeknownst to myself, given some lame-ass kissing advice on an eHarmony forum. After several email messages to the moderator, I decided to take matters into my own hands and just made a half-decent post myself, pointing out that I if I were to post, I&apos;d have posted something that at least took a few minutes for me to pen, like the poem below:

      <![CDATA[<div style="color:yellow; text-align:center;">
<div style="text-decoration:underline;">THE KISS YOU CAN REMEMBER</div><P>
The kiss you can remember<br>
Is not a real kiss<br>
Just as the circle you can see<br>
Is not a true circle<br>
The perfect kiss is as pleasant dream<br>
Pure experience untainted<br>
By awful reality of tangibles<br>
That everywhere surrounds us<br>
You must emerge from it<br>
With but vague recall<br>
Too fleeting to capture<br>
Too slippery to grasp<br>
Too pristine to bind<br>
By fetters of finite memory<br>
You will remember starting<br>
But should you bethink anything after<br>
Then, my friend, I'm afraid<br>
You must try again<br>
And again<br>
Until mindlessness becomes you<p>

<font color=cyan>Copyright (c) 2008, Anand Venkataraman</font>
</div>

<font color=yellow>
I find it interesting that much bandwidth is spent describing what is a purely experiential thing. Indeed, words mean nothing here, and it is an irony of life that we often waste the most words in talking precisely about things over which words are so poignantly powerless!"
</font>

]]>
   </content>
</entry>
<entry>
   <title>Calculating the median of a MySql table column</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/02/calculating_the_median_of_a_my.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.49</id>
   
   <published>2008-02-22T14:52:38Z</published>
   <updated>2008-02-23T17:44:50Z</updated>
   
   <summary>My staff are amazingly efficient at automating the various slices and dices of data for me. They like to do it in SQL scripts, MySql in specific, and as a consequence, the data I want to look at is automatically...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Technical" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[My staff are amazingly efficient at automating the various slices and dices of data for me.  They like to do it in SQL scripts, MySql in specific, and as a consequence, the data I want to look at is automatically generated every night into various MySql tables.  I, in turn, have my own scripts, and ye old console, to extract the information I want.  However, one issue that bugged me early on was getting at the median of a set of numbers.  <p>
]]>
      <![CDATA[When analyzing data that has been collected for you, it is often useful to look at the median rather than the mean whenever you suspect the presence of outliers. As an example, a single McMansion valued at $12M can easily throw the mean home price significantly off in a small neighborhood.  But the median is unlikely to change much.  Simply put, the median is the middle number, or the average of the middle two numbers in an ordered list of numbers.  How does one get it in MySql?<p>

Looking for a simple answer to this turned out to be an interesting exercise; so interesting, in fact, that I had many times asked this question of potential hires (If you're a potential hire, note that I don't ask this question any more).  More often than not, if a candidate doesn't know of a canonical answer, their approach to finding a solution tells me far more than just how to get the median in MySql.<p>

On a Linux command line, you want something like:<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
  echo "SELECT MEDIAN(foo) FROM bar" | mysql -p mydb
</PRE>
</div>

analogous to the<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
  echo "SELECT AVG(foo) FROM bar" | mysql -p mydb
</PRE>
</div>

But alas, such a function is conspicuously missing in MySql.  Part of the reason is probably to protect from gross inefficiencies injected by naive DBAs and programmers.  If the column of interest is not indexed, then a median computation necessarily involves a sorting (which is log-linear in complexity).  It is not uncommon for uninformed programmers to make unjustifiably liberal use of seemingly simple statements that hide an enormous amount of complexity.  (For this reason, powerful high-level programming languages and tools that provide simple abstractions for complex algorithms should only ever be used by well-educated programmers who know what they are doing and what it will cost).  But returning to the issue at hand, and knowing now that MySql doesn't provide a native function call for it, how does one calculate the median?  There are several ways, and many good ones are described in the <a href="http://dev.mysql.com/doc/refman/5.0/en/group-by-functions.html">MySql Forum</a>.  But none is the pretty one-liner you want.  <p>

The answer to this question is what potentially distinguishes a good engineer or scientist from a good manager.  A good manager is focused on getting the job done, and does not care too much about exactly how it is done as long as she is confident it's being done efficiently.  A good scientist or engineer is naturally curious, and wants to solve the problem a particular way just to know whether it can be done or not.  The good engineer will most likely come up with one of the many excellent answers on the MySql forum.  A good manager will likely say, "Is there a reason it has to be done in MySql?  'Cos I don't know the answer off the top of my head, but I'm sure I can find it soon enough if I browse a few minutes.  If you just want the median efficiently, and don't care how I get it, I'd probably just use some combination of MySql and Perl/php/whatever:<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
 <PRE>
  echo SELECT foo FROM bar ORDER BY foo \
    | mysql -N -p mydb \
    | perl -e 'use POSIX qw(floor ceil);  
                @f = <>; 
                ($a,$b) = (floor(@f/2-0.5), ceil(@f/2-0.5));
                $median = ($f[$a]+$f[$b])/2; 
                print "$median\n";
      '
 </PRE>
</div>

Here is the line-by-line breakdown:<p>

<ol>
  <li>SELECT:  The column must be ordered (doesn't matter which way).  If the column is not indexed, it can be sorted externally using the Linux sort program with the same or similar complexity.  Note the -N option to mysql which tells it to not print the column header.
  <li> Perl: Perl and its POSIX floor() and ceil() functions calculate the middle two rows in such a way that they are the same for a table with an odd-number of rows.  If the table has 5 rows for example, then the first number, obtaining by flooring 2.0 is 2 and the second number, obtained by lifting 2.0 is also 2, which is the middle row in a 5 row Perl array with rows (0,1,2,3,4).  If the table had 4 rows, then the first number will be floor(1.5) = 1 and the second number will be ceil(1.5) which is 2.  In either case, the average of the numbers in these two rows gives the correct median.
</ol>

If you simply want to use the perl script as a unix command called median, you can grab the program from <a href="http://pandamatak.com/people/anand/xfer/median">here for presorted input</a> or <a href="http://pandamatak.com/people/anand/xfer/median-unsorted">here for unsorted input</a>.  With these tucked away in your personal bin directory, you can simply issue:<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
  echo "SELECT foo FROM bar order by foo" | mysql -p mydb -N | median
</PRE>
</div>

or<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
  echo "SELECT foo FROM bar" | mysql -p mydb -N | median-unsorted
</PRE>
</div>

If inside a Perl or Mod_perl script using Perl DBI, one could use the following fragment:<p>

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
  use DBI;
  use POSIX qw(floor ceil);
  ...
  my $sth = dbh->prepare('SELECT foo FROM bar ORDER BY foo");
  $sth->execute();
  my @f = $sth->fetchrow_array();
  my($a,$b) = (floor(@f/2-0.5), ceil(@f/2-0.5));
  my $median = ($f[$a]+$f[$b])/2;
  ...
</PRE>
</div>

Of course, if the MySql table is indexed, one may also get at the median in a two step procedure completely inside of a MySql script, if that's what is needed - first selecting the total number of rows with "COUNT(*)" and then extracting from the ordered column the average from the middle or two middle rows since we know which these are, for example:<p>

<div style="font-size:80%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
   SELECT AVG(foo) FROM (SELECT foo FROM bar ORDER BY foo limit m,n) as x
</PRE>
</div>
where you know m to identify the row number of the first of the 1 or 2 middle rows and n to be either 1 or 2 depending upon whether the total number of rows is odd or even respectively.<p>

Finally, lest you worry about the memory requirements imposed by sucking in an entire column into a Perl or other such array let me reassure you that (a) Perl's implementation of in-memory arrays is very efficient, (b) you can easily read in arrays with millions of rows without taking up more than a few tens of megabytes of memory and (c) if, in fact, your table had many billions of rows, you should be able to get a very good approximation to your true median by sampling every nth row (Why?).  This last point is especially relevant to scalable data analysis. You can calculate your answer as a median of medians by farming out computations using such paradigms as <a href="http://en.wikipedia.org/wiki/MapReduce">MapReduce</a>.
]]>
   </content>
</entry>
<entry>
   <title>How to browse the Internet perfectly anonymously</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/02/how_to_browse_the_internet_per.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.47</id>
   
   <published>2008-02-21T05:31:02Z</published>
   <updated>2010-06-09T12:55:50Z</updated>
   
   <summary>Related website: Pippini.com &quot;Tell me Mr. Jones,&quot; said the public prosecutor, &quot;All the evidence points to the fact that you were indeed contemplating the murder of your lover on that night. Were you not? Remember that you are under oath,...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Misc" scheme="http://www.sixapart.com/ns/types#category" />
         <category term="Technical" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[<div style="text-align:center; font-size: 90%;">Related website: <a href="http://pippini.com">Pippini.com</a></div>

<div style="color:yellow;padding: 5px;">
"Tell me Mr. Jones," said the public prosecutor, "All the
evidence points to the fact that you were indeed contemplating
the murder of your lover on that night. Were you not? Remember
that you are under oath, and that your polygraph results have
betrayed your innermost thoughts. You had the motive, ability
and, in your judgment, sufficient anonymity to carry out the
macabre act with impunity. And I admit that were it not for this
mass of incontrovertible evidence we uncovered by following your
intent, you might well have succeeded in pulling the wool over
our eyes! Is this not what you are thinking?"  </div>

There it is. Fragment from a mystery novel as it were, when you
gauge your own mental reaction as you read the passage, you will
find it to be a clear example: Most reasonable people would not
think twice about sanctioning techniques, including the
administration of polygraph tests, that authorities may use to
probe into and conclude on the conents of that innermost
repository of our private thoughts - our mind.  Yet, it appears
unreasonable to most of us if authorities claim to have the
ability, much less to use it, to determine probable intent in
criminal behavior from one's behavior on the Internet. Is the
Internet that different in people's perceptions as a place
where absolute anonymity may be enjoyed regardless of disposition
and intent?  Indeed, do people honestly believe it is ok to probe
one's mind, but not one's online behavior?<p>

]]>
      <![CDATA[I can only conjecture that the reason for this allowance stems
directly from the perceived unreliability of today's mind-probe
techniques.  All said, the polygraph is not a hundred percent
reliable, and the public prosecutor's browbeat even less so.  But
what most of us don't realize is that in our acceptance of the
admittedly inaccurate interrogation techniques we have set the
precedent. With time, mind-probing techniques could get
better. Polygraphs will become more accurate, and it is not
inconceivable that in a hypothetical future world, there will exist
humane and accurate mind mapping methods that play back the very
memories a suspect hopes to conceal.  Scarcely anyone would lift
a finger in protest against this most intense intrusion of one's
privacy.<p>

Now I'm not arguing that even our humane interrogation techniques
are immoral and therefore should be abolished.  Nor am I calling
for greater cooperation from web-site owners in sharing sensitive
information with the government.  I'm only pointing out the
poignant inconsistency in people's perceptions of what is
allowable and what is not. Indeed, it is a rather paradoxical
inconsistency, for on the one hand we sometimes allow the system to
probe what we cherish as our most private and sacred sanctuaries,
but on the other, we clamor with our loudest voices when the
same system attempts to investigate information on the Internet,
arguably many times removed from our mind-cores!<p>

The righteousness (or lack of it) of various investigative
procedures is not my concern here. But I am interested in
exploring what persons who are genuinely concerned about the loss
of their privacy might do to thwart authoritarian attempts to
invade it.  After all, do we not assume that it is natural for a
suspect to attempt concealment despite being badgered at the
stand?  And do we not think it reasonable that polygraph test takers have at
least tried to learn techniques to beat it?  Why else would we
find so much humor in George Costanza's parting advice to
Seinfeld, who has to take the polygraph to prove he doesn't like
Melrose place?  "Remember," he says "It's not a lie if you
believe it."  That is is the perfect analogue to wiping your
tracks clean on the Internet. A probe will discover nothing.<p>

So having said that, we now ask: Is it possible to have an
absolutely anonymous Internet experience - a veritable
no-strings-attached one-night-stand with your browser?  I confess
it scares me a little to know that there are people who might
want this - I mean, concern for privacy and a desire to enjoy an
unsnooped life by making it "hard" for people to track you is one
thing.  But going out of one's way to make it practically
impossible to associate one's online activity with oneself is
borderline creepy.  Who would want to do this?<p>

The fact, however, is that most people are at the other end of
the spectrum.  They are naive enough to not know even to clear
their browser histories and cookies when they leave their
stations.  Over the last day or so, we quickly put together a website using entirely free resources
on an old PC (Clearly, with more resources, the scope for
tracking and profiling users is vastly greater).  The page shows how much information
about users is likely logged by all the websites they visit.
(Mar 2009 - The page is no longer active after we let the domain name expire)<p>

Having been in online advertising long enough and being aware as I am
of the information that we unwittingly make available to humongously hungry
hosts out there, I confess to still being mildly unsettled to
see all of the information distilled and presented in one place like
this.  But yet, I typically settle for the most common precautions one
should take, like clearing browser caches periodically, and refreshing
DHCP addresses once in a while.  If I wanted to be <em>absolutely</em>
anonymous, the effort involved is simply too much and is not worth it
for most of us.  Nevertheless, here are three precautions such a
person might be inclined to take, and actions authorities might assume
to address attempts at attaining absolute anonymity.<p>

<ol>

  <li> Not use one's own Internet connection.  As the data and
  discussion at Pippini (decommissioned now, sorry :-) shows, an IP
  address is attached to the user or the limited number of people in
  his or her domicile. It is easy to track someone down if the number
  of people to examine is small.<p>

  <li> Not use free Internet connections in wireless hotspots on one's
  own computer. WiFi routers need to assign you an Internet address
  and they can be configured to log MAC addresses of computers that
  were assigned specific IP addresses at specific times.  MAC
  addresses are unique and instantly identify a computer.  One could
  presumably buy a wireless access card using cash (not a credit
  card), and destroy it after malicious use.  This would mean that the
  MAC address can be tracked as far as the vendor who sold it, but no
  further.  For this reason, authorities may require that checkout
  counters have recorded video surveillance to photograph customers as
  they purchase goods.<p>

  <li> Use one of the many unoccupied computers at a paid public
  Internet cafe and pay in cash. For this reason, authorities may
  require paid public Internet cafes to insist on viewing and noting
  government issued photo-IDs of users.  Further, they might
  also require that users be assigned to specified stations.<p>

</ol>

These are suggestions one might do well to keep in mind, especially if
he or she is considering running for President.<p>
]]>
   </content>
</entry>
<entry>
   <title>Governor Schwarzenegger helps to make Bay Area housing affordable</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/01/governer_shwarzenegger_helps_t.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.41</id>
   
   <published>2008-01-10T23:50:09Z</published>
   <updated>2008-02-05T06:29:11Z</updated>
   
   <summary>In what may be Governor Schwarzenegger&apos;s most sympathetic move yet to make housing more affordable for potential home-buyers in California, he does what he can to encourage young Californians to migrate away from the state, thus setting the stage for...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Misc" scheme="http://www.sixapart.com/ns/types#category" />
         <category term="Political" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[In what may be Governor Schwarzenegger's most sympathetic move yet to make housing more affordable for potential home-buyers in California, he does what he can to encourage young Californians to migrate away from the state, thus setting the stage for home-prices to spiral downwards.

<div style="font-size:90%; padding:10px; color:cyan;">
<a href=http://www.sfgate.com/cgi-bin/article.cgi?f=/n/a/2008/01/10/financial/f113022S86.DTL>
http://www.sfgate.com/cgi-bin/article.cgi?f=/n/a/2008/01/10/financial/f113022S86.DTL
</a>
</div>

With already ailing education budgets slashed even further, convicted criminals let loose to terrorize innocent citizens and 20% of state parks closed, the Governor hopes that more productive Californians will be encouraged to seek domicile in Oregon and nearby states. This calculated and clever move will enable property prices eventually to reach levels where the 22,000 inmates, who will be in dire need of good housing after their early releases, can afford them.

]]>
      
   </content>
</entry>
<entry>
   <title>On Data Analysis</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2008/01/on_data_analysis.html" />
   <id>tag:www.pandamatak.com,2008:/people/anand/blog//2.40</id>
   
   <published>2008-01-08T15:38:54Z</published>
   <updated>2008-01-09T03:41:40Z</updated>
   
   <summary>Forget OLAP cubes and pivot tables! As is the case most of the time, 90% of the insight can be gleaned from very simple data plots. To be sure, the remaining 10% does shed valuable new light, but is also...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Technical" scheme="http://www.sixapart.com/ns/types#category" />
   
   <category term="25" label="Useful utility script" scheme="http://www.sixapart.com/ns/types#tag" />
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      Forget OLAP cubes and pivot tables!  As is the case most of the time, 90% of the insight can be gleaned from very simple data plots.  To be sure, the remaining 10% does shed valuable new light, but is also orders of magnitude relatively more tedious to distill.

One of my favorite tools for performing quick sanity checks on data and even for inferring high-level trends is a histogram; it simply partitions your data points into a fixed number of buckets, with each bucket holding points that fall within a given range.  The resultant bucket sizes are then available to eye-ball, often plotted as bars whose lengths are proportional to the number of elements in the corresponding buckets.

      <![CDATA[When your data is generated from Unix/Linux scripts as is typically the case with most LAMP based systems, migrating it into CSV tables to histogram within Excel, or even firing up your free copy of OpenOffice is a cumbersome overhead.  A simple script will often suffice to generate compelling histograms.  

Here is one such script I had written a long time ago.  I have used it so many times that I feel that someone else, somewhere else, is bound to benefit from it too.

<div style="font-size:90%; padding:10px; color:cyan;">
<a href=http://www.pandamatak.com/people/anand/xfer/histo>http://www.pandamatak.com/people/anand/xfer/histo</a>
</div>

As an example of its usage, here is the command line and output for generating a histogram over a set of 10K Gaussian random numbers.  (As a by-product, observe the cool trick exploiting the Central Limit Theorem to  generate a normally distributed random number using an awk one-liner.  Normalized Gaussian randoms are useful to deliberately add controlled noise to a process, for example in selecting the top 3 (with variety) out of a ranked list of 10 ads to show on a publication.

<div style="font-size:90%; padding:10px; font-family: 'Courier'; color:yellow;">
<PRE>
$ gawk 'BEGIN {for(i=0;i<1e4;i++)print rand()+rand()+rand()}' \
   | histo -stars -scale 50 -interval 0.15
# NumSamples = 10000; Max = 2.94838; Min = 0.0360424
# Mean = 1.5030500454; Variance = 0.253799739614845; SD = 0.503785410283828
# Each * represents a count of 50
     0.0360 - 0.1860 [    13]: *
     0.1860 - 0.3360 [    58]: **
     0.3360 - 0.4860 [   125]: ***
     0.4860 - 0.6360 [   250]: *****
     0.6360 - 0.7860 [   374]: ********
     0.7860 - 0.9360 [   552]: ************
     0.9360 - 1.0860 [   758]: ****************
     1.0860 - 1.2360 [   942]: *******************
     1.2360 - 1.3860 [  1026]: *********************
     1.3860 - 1.5360 [  1130]: ***********************
     1.5360 - 1.6860 [  1145]: ***********************
     1.6860 - 1.8360 [  1006]: *********************
     1.8360 - 1.9860 [   854]: ******************
     1.9860 - 2.1360 [   652]: **************
     2.1360 - 2.2860 [   501]: ***********
     2.2860 - 2.4360 [   306]: *******
     2.4360 - 2.5860 [   174]: ****
     2.5860 - 2.7360 [   108]: ***
     2.7360 - 2.8860 [    25]: *
     2.8860 - 3.0360 [     1]: *
</PRE>
</div>

Enjoy!

&
]]>
   </content>
</entry>
<entry>
   <title>Fixing Fontconfig on Linux with strace (Unable to load default config file)</title>
   <link rel="alternate" type="text/html" href="http://www.pandamatak.com/people/anand/blog/2007/08/fixing_fontconfig_on_linux_wit.html" />
   <id>tag:www.pandamatak.com,2007:/people/anand/blog//2.36</id>
   
   <published>2007-08-18T16:15:56Z</published>
   <updated>2007-08-18T20:25:42Z</updated>
   
   <summary>Today, Linux is a beautiful thing and I typically waste no opportunity to evangelize it to friends considering a new desktop purchase. I mean, if the average user today calculates how much of his or her computer time is spent...</summary>
   <author>
      <name>&amp;</name>
      <uri>http://pandamatak.com/people/anand</uri>
   </author>
         <category term="Technical" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en" xml:base="http://www.pandamatak.com/people/anand/blog/">
      <![CDATA[Today, Linux is a beautiful thing and I typically waste no opportunity to evangelize it to friends considering a new desktop purchase.  I mean, if the average user today calculates how much of his or her computer time is spent either on-line or within office applications, it seems  like common-sense to buy a Linux desktop with Firefox and Open Office rather than buy expensive Microsoft products.  Homer Simpson captures the sentiment best in the opening line of the latest <a href=http://www.imdb.com/title/tt0462538/>Bart Simpson flick</a>.  Why would someone pay hard-earned cash to get something we can get for free?  We're all giant suckers!

]]>
      <![CDATA[One would naturally expect that it makes even more sense for the developer community to go the Linux way, since they tend to have greater familiarity with the ins and outs of the operating system.  And this is mostly true, but yet I am intensely frustrated by problems in a few sloppily developed open-source programs that are part of the official Linux distributions.  I ran into one such problem this weekend, and wasted almost an hour trying, unsuccessfully, to find a solution through Google.  The problem was with fontconfig, part of Linux's font system.  When I upgraded from RedHat to Fedora Core 6 (Fedora 7 has the same problem), all of a sudden, I had ended up breaking the font-system for my developers.  No matter how many times I tried to reinstall font-config by the book and no matter which version I used, I hit the same wall.

Calls to the library would print the ominous message:

<div style="font-family:arial-narrow,sans-serif; padding-left: 10px; color: yellow;"> 
  Fontconfig error: Cannot load default config file
</div>

Fortunately, Linux comes armed with a rich toolbox that lets you diagnose and, in many case, fix problems yourself.  One such tool is strace, which prints the sequence of system-calls issued by a program.  So after my hour of unsuccessful searches with Google, I decided to go to the next level of troubleshooting and strace my problem. I ran a stripped down version of the failed command sent to me by my developer:

<div style="font-family:arial-narrow,sans-serif; padding-left: 10px; color: yellow;"> 
   strace convert label:hello-world /tmp/x.jpg
</div>

Looking through the output, the root of the problem was clear:

<div style="font-family:arial-narrow,sans-serif; padding-left: 10px; color: yellow;"> 
...<br>
access("/usr/local/etc/fonts/fonts.conf", R_OK) = -1 ENOENT (No such file or directory)<br>
write(2, "Fontconfig error: ", 18Fontconfig error: )      = 18<br>
write(2, "Cannot load default config file", 31Cannot load default config file) = 31<br>
...
</div>

Aha! There is the culprit.  I had no idea why the system would look in /usr/local/etc/fonts, when I had done everything in my power, as per the documentation, to indicate that the config file lived in /etc/fonts.  But I was already at the limit of my patience with this problem and too tired to look through what was probably badly designed code.  So I decided to apply the simple fix:

<div style="font-family:arial-narrow,sans-serif; padding-left: 10px; color: yellow;"> 
  ln -sf /etc/fonts /usr/local/etc/fonts
</div>

I now run convert and it runs, with output as clean as a whistle:

So here's another reason to love Linux. It may come with sloppy programs that are yet to mature.  But it does provide you the tools necessary, in most cases, to get around these issues while the program designers work to find a long-term solution.

&
]]>
   </content>
</entry>

</feed>
