Jcrop creates an interface to crop an image. However, actually creating a new, cropped image file is obviously beyond the scope of a client-side plugin. It may also be the most challenging part of implementing such a process in your web application (now that you've found Jcrop).

   <?php
   if (isset($_POST['i'])) {
     $data=$_POST['i'];
     $filteredData=substr($data, strpos($data, ",")+1);
     $decodedData=base64_decode($filteredData);
     $fp = fopen('/dev/null', 'wb');
     fwrite($fp, $decodedData);
     fclose($fp);
     echo "Upload successful\n";
   } else {
     echo "No uploaded data\n";
   }
   ?>

TODO

• Progress meter while reading the image file
• Error checks of various kinds
• Bells and whistles as needed
• Better smoothing/interpolation when shrinking a large image into the div and cropping a small part of it.

In today's agile world, software projects typically demand strategic choices over a plethora of possible solutions to most problems. Some solutions are appropriate for some problems and some aren't. It's important to choose a solution for all the right reasons, and it's doubly important to not choose it for all the wrong reasons. Human factors complicate this issue because inappropriate solutions to some problems are appropriate and good solutions to other problems. And these "other" problems are sometimes faced by reputed technology companies, who lend "celebrity" status to the solutions. This leads the young engineer astray, and coerces them into thinking that just because Google or Microsoft have found great success with a particular paradigm, they ought to as well. Further, within most companies, there are bound to be a number of non-engineering execs whose bandwidth is almost completely soaked up by non-engineering matters. It is consequently easy to get their buy-in for adopting such technologies using that unfortunately untrue magic phrase, an egregiously extant engineering enchantment and the mother of all modus ponens if I can call it that: If it is good enough for Google, it ought to be good enough for us". It behooves every company to have at least one theoretically and practically savvy engineer to protect precisely against this kind of thing happening. And to support decisions that are data-driven, rather than driven by subjective matters such as coolness and esthetics. At worst, data-driven decisions are at least likely to prevent general resentment among staff - one person's sense of beauty may not always be another's. On the other hand, a truly creative idea or proposal is bound to bubble up and find general support in the numbers. Those that shirk experimentation, calculation and testing are the prophets of faith that every organization should strive to avoid hiring into engineering, in preference to those that support the voice of empirical reason.

In this article, I want to focus on two specific cool technologies that frequently go hand-in-hand: Google's Map-reduce (e.g. Hadoop), and Cloud Computing (e.g. EC2). For whatever reasons possibly including those I've mentioned above, temptation is high among many engineers today to use these sledge hammers to crack nuts. As an illustrative case in point, I'd like to take one of our typical problems and talk about how we address it at 33Across. Our customers often express amazement at how we manage to process their gargantuan data sets and produce results in short order, seemingly with ease. We are often asked probing questions that try to get us to divulge our core data-processing techniques. Needless to say, I won't be describing any of our proprietary procedures or secret sauces here. But a great deal can be said without going into such matters. Most of what we have done to get where we are at is simply to follow published information you can find in any decent computing journal on occasion, but most of the time in good Computing-101 texts. Before we dive into using a framework that has any overhead associated with it, we always do quick back-of-the-envelope calculations (not unlike Rapleaf's nice analysis of whether to host or cloud) to determine if it will really be worth it.

]]> 33Across, we analyze graphs - massive graphs that encode link information from social networks. One can immediately imagine the scale of the problem. Take Facebook as an example. By last count, Facebook has about 200 million users, with an average of 120 friends per user. But temper this observation with some business sense and practicality - the more sane observation according to Sheryl Sandberg is that the average number of connections that actually matter is more likely about 20. Say we want to perform calculations on this graph of 200 million persons with about 20 friends per person on averge. People will be represented as nodes in the graph and connections between people will be represented as the edges between the nodes. In the context of Facebook, we're talking about a graph with 200M nodes and about 20 edges per node. I don't think there is any doubt that such a graph would be considered reasonably large by most standards. And when we tell folk that our team routinely processes graphs of this magnitude, the first reaction is almost inevitably one of awe, followed by an enquiry as to whether we process it using Hadoop. The answer is no. In truth, although most people consider our graph to be large, it is nowhere even near as large as a truly large graph, by which I mean something like the Internet or the human brain. And so, as it turns out, our problem is very different in scale from, say, computing ranks on every document on the web. In fact, our graph is small by comparison, and we can afford to do most of our calculations on this graph in memory and in real-time. We are often able to get away with efficiently processing, within each of many simultaneous threads, the edges of any particular node in sequence. To be sure, one can indeed Hadoop this process by clever definitions of functions that will do the kind of recursive aggregation for the quantities that interest us. But we want to see if it is necessary at all, and for the foreseeable future at least.

What does it take to store the entire useful Facebook graph in memory on a single machine? And what would the specifications of such a machine be? Obviously we aren't talking super-computers here, but something you can buy off the shelf from Dell, or your local Fry's. Here's how we do our back-of-the-envelope calculations - Say that you want to store the graph as a vector of nodes, where the i'th element of your vector is actually node i. Having the luxury of being able to scan and compute on edges sequentially in our algorithms means that we can simply store the edge-lists for our nodes as singly linked lists of destination nodes. Actually we use a map rather than a singly linked list, but to keep the argument simple, assume this data structure to be a list for now. Also ignore, for now, the quantities we compute on the edges and the nodes themselves. Consider only the graph, which consists of nodes, edges and the normalized weight of each edge. How much space does it take to store the Facebook graph in RAM? Well, 200 million is an order of magnitude less than 4 billion, which is approximately the largest value storable in a long (four-byte) integer data type (32 bits means 2³² bit patterns), so we can comfortably assume that our graph nodes can be indexed by an unsigned long integer. This means that each element of our singly linked list, which ought to contain a destination node, a weight on the edge, and a pointer to its sibling will occupy exactly 16 bytes (pointers on a 64-bit Dell PowerEdge are 8 bytes in length, and we use a single floating point number (4 bytes) to store an edge's weight. We're talking about 200M × 20 edges = 4B edges in all. At 16 bytes per edge, we're looking at approximately 64 billion bytes (that's 64 GB of RAM). Even if you add in the overhead of a sentinel node to each of our linked lists, and the constant cost of storing 200M start pointers in a vector, you're still looking at a maximum of under 70G to store the entire graph in memory.

Now, I want to point out that today, computers with this much main memory are not only available, they're even relatively inexpensive. While companies like Virident pride themselves in building eco-friendly green machines with gigantic amounts (over 256GB per blade) of solid-state RAM having comparable read-latencies as DIMMs, for only slightly more modest tasks than processing Facebook's entire graph, one can even get away with far cheaper solutions. A dual-Xeon Quad-core Dell PowerEdge 1950 III, which comes in a slim 1U form factor, complete with cable management cages and rack-mount rails can be purchased for under about $3K, especially if you're on good terms with the company rep :-). Quoting from Dell's website, a "2-processor Quad Core Intel® Xeon® X5470, 2X6MB Cache, 3.33GHz, 1333MHz FSB" PowerEdge 1950 III (1U form factor) costs about $4K, and adding in 32G of RAM from Kingston still prices the machine at just slightly over $5K. There you have it - 8 computing cores, and 32G of RAM - allowing for the OS and other house-keeping programs, that's still enough juice to load a graph with almost two billion edges (just under half the size of Facebook). Finally, if you really need it, you can, for not a whole lot more money, actually buy 2U servers with up to 192G of RAM and 16 cores, allowing you to store the entire Facebook graph and still leave much room to spare.

Now, it's all well and good that you can store such graph in memory on a single machine in your garage, but how efficiently can you process it? Evidently very efficiently. Let's take edge weight normalization as an example. I pick this because it's an operation that needs to touch every edge on the graph, and touch each edge twice. It is not an operation that can be optimized much by most compilers. While you can calculate the number of nanoseconds it will take to perform each touch and go from published FLOPS specs on the hardware, we resort to extrapolating results from simple empirical tests. On the Dell we cited above, it takes about 25 nanoseconds for a single touch and go (accumulating the edge weight into a normalizer). A second touch and go will be required to divide by this normalizer, and so we're talking approximately 50 nanoseconds per edge. At this rate, normalizing a graph of 1B edges ought to take 50 × 10^-9 × 1B = 50 seconds (less than a minute). Normalization can be efficiently parallelized over the nodes in the graph, and so if you run 8 concurrent threads, you're only looking at slightly over 6 seconds to normalize the entire graph. And that's less time than spent waiting for some social network pages to load!

Lest this article be misinterpreted as an advocacy against using the cloud or map-reduce, let me clarify by explicitly stating that's not the case. As Don Knuth observes piercingly, premature optimization is the root of all evil. Map-reduce is an optimization technique that buys efficiency using a clever divide-and-conquer approach. But to know what to divide, and how, and whether at all, one needs to first measure, estimate and calculate using tried and trusted techniques. Most of the time, a bigger home is not the right answer to clutter in a small one. An engineer who cannot write efficient code on a single computer most likely won't be able to write efficient code in the cloud. Only, it will be much harder to tell. And so in summary, let me leave you with these parting thoughts - One must first learn to walk properly on the ground before trying to fly in the clouds. ]]> tag:www.pandamatak.com,2008:/people/anand/blog//2.60 2008-12-18T03:14:42Z 2009-01-15T21:58:03Z

Just before the holidays started, the students were to do a warm-up exercise in preparation for diving into the deeper wonders of Number Theory in upcoming classes. Here is the exercise, called the Four Fours. Using exactly 4 digits, all of which are fours, and any number of the arithmetic operators Plus, Minus, Times, Divide, Exponentiation, Factorial, Square Root, and Parenthesis, derive each of the numbers from 1 to 50. E.g. One way you can derive 1 is obviously 4/4 *4/4. No doubt there are a number of different ways to derive each number and the goal is to get students to think about this, and to come up with expressions that are different from those of their colleagues.

We, however, decided to turn this little exericse into a game, which as a matter of fact, also turned out to be a good game to occupy young minds during the holidays! Here is how you would play it:

The game has 15 rounds. Each round lasts 1 minute. We tried to cover 15 numbers in the game.

Each round consists of:

1. Host (me): Calling out a different random integer between 1 and 50.
2. Players: Each of them has 30 seconds to write an expression with exactly 4 4s and the arithmetic operators to get this number.
3. We compare answers and points are scored for this round as follows:
   • Zero points - No expression or invalid expression (meaning it doesn't evaluate to the number called)
   • M-N points - Otherwise. M is the number of players and N is the number of other players who have the same expression as you. If your expression is unique, you get the full M points for the round.

]]>

• Just how many legal expressions can you ever make (A legal expression is some combination of the 4 fours and the operators that will compute correctly, i.e. is syntactically correct)? What if you couldn't use Square Roots, Factorials and Parentheses? Can you still make an infinite number of expressions? If not exactly how many?

• What is the largest number you can calculate? What if you didn't have Factorial?

• How can you tell if two expressions are the same? E.g. If John came up with 4*4 * 4/4 = 16 and Mary came up with 4/4 * 4*4 = 16, how can you tell they were the same? Notice that it's not always this trivial. This discussion leads to concepts of Normal Forms, how to reduce to Normal forms, whether even such reduction is always possible, whether unique normal forms exist, and how one can algorithmically (and efficiently) compare Normal forms.

• This point, especially, I love because it goes to the root of Science, Math, Non-Science and their distinctions! But I've completely forgotten the discussion chain that led to this point in class! Suppose John and Mary were to form one expression each as above, but instead used a variable for one of the fours. By substituting a value for this variable, each expression would evaluate to a certain number. However, it happens to be the case that we are unable to reduce either expression to Normal form and thus compare them directly. How could we now tell whether the two expressions are in fact the same? This brings up all kinds of interesting discussion points - Conjectures and Theorems Versus Hypotheses and Theories, Deductive and Inductive reasoning, Mathematical Induction, Empirical testing and indeed, gives students a real taste of that tantalizing field of exploration - Epistemology!

]]> tag:www.pandamatak.com,2008:/people/anand/blog//2.56 2008-12-13T15:52:16Z 2009-05-14T16:33:26Z

After a long year of Sharon teaching their delightful children, the parents had decided to reward her. On Teacher Appreciation Day, following heart-felt expressions of gratitude and astonishment at what a wonderful job she had done, the head parent,...

& http://pandamatak.com/people/anand After a long year of Sharon teaching their delightful children, the parents had decided to reward her. On Teacher Appreciation Day, following heart-felt expressions of gratitude and astonishment at what a wonderful job she had done, the head parent, who had coordinated by collecting contributions towards Sharon's gift, presented her with a brightly colored sealed envelope amidst generous applause. Sharon opened her gift with great eagerness and found what she would never have spoiled herself with in her wildest dreams - She was now the proud owner of a certificate bestowing upon her the gratitude of a thousand children in Rwanda, who had just been donated $500 on Sharon's behalf. Yes - The head parent, in her infinite wisdom, had decided that the best gift for Sharon would not be a couple of sessions at a posh Palo Alto Spa, or a comfy sofa to relax between classes, or even a Bloomingdale's gift card, but rather a donation to a worthy African cause in Sharon's name. "What could please Sharon more?" would likely have been the most pressing question in her mind, as she tossed around various options for a gift to proudly decide that someone who spends almost every waking hour with children could only ever be pleased by being told that she had benefited yet more children. ]]> A gift of cash is The best gift of all It comes from the heart, no Matter how small It says that I care I love the things you do You made me want to share my Hard-earned $$$ with you Let me leave you with this fictitious and farcical incident that perhaps better captures my thoughts.

Scene: Safeway at Middlefield, Palo Alto, CA.

"Excuse me, can you change a twenty dollar bill for me please?" "Sure brother. Tens?" "Err... no..." "Fives maybe?" "Uh, no actually..." "Well, you better tell me then. Sure as hell can't do nickels and dimes you know?" "Well, I was kinda thinking another twenty?" "Excuse me? That bill real man? Funny money? Do I need to call the cops or somethin?" "Hey hey hey, don't freak out. Yeah, just what I said. I want another form of $20, you know, something that isn't as flexible as this one?" "Dude, am like totally lost man... like, not sure what you mean?" "I'd want something that I can use to buy $20 worth of stuff, but can't use just anywhere you know. I want to be able to use it at just one place - like Starbucks. No no make that Pottery Barn... Hey you know what - let's do Mc Donald's. I mean I want a 20 dollar bill I can only spend at Mc Donald's. Nowhere else. I'd like to swap my government issued spend-anywhere-in-this-beautiful-world mint-condition Jackson for a shiny plastic 20-buck-proxy that no would will accept but Mc Donald's. Actually, it'd be super if you can make that the Mc Donald's on the corner of El Camino and Ravenswood, but I'll take any Mac." "Now, why would you want that sir? Why not keep your Twenty? You realize it's legal tender, and that your favorite McD will take it anyway?" "Ah, yeah, sure I know that. But you see it's not for me. I want to give it to a friend. It's a gift for her birthday." "Yeah?" "I can't give her the $20 now, can I? I mean, she'll be able to spend it just about anywhere." "And that's bad?" "You bet it is. What if she ends up buying herself some granola bars at Whole-Foods like she always does? Even worse, she might end up buying a book at Kepler's. I can't have that. Can I? The truth is, [whisper] I don't trust her you know? She doesn't really know to spend wisely. I gotta help her out." "Aah.. Then maybe I can introduce you to this nice box of candy? It's usually $40, but today only it's on sale for $20, and I can peel the price tag off of it..." ...

]]>