BIG Mistake

Dear Readers,

Today's blog is about Information Technology, namely about Big Data and what I think of several Big Data projects that are running out there.

Data the real Asset

In an earlier Blog you read about my views on why many software projects are a liability, if you want to jog your memory go to http://thombrem.blogspot.in/2012/11/software-asset-or-liability.html 

So if I think software is a liability you may ask, what is an asset on your project? Undoubtedly it is the DATA. Data is used to make decisions such as 

• Where to go next (new products and services) 
• What processes within your organization need automation or change
• Who's slacking
• What data is NOT being captured
• Why your business is sucking. etc.

These decisions are key to success of your enterprise or government or any other goal oriented outfit. Organizations who want to build assets should take technology with a gain of salt. They should ask tough questions like:

• What data will be generated by a project?
• Will it be analyzed?
• Will the information extracted from it be useful and actionable?
• OR Is a process going to be automated by this application?

In other words, will Life be easier and more fun after this project? ;) 

So your IT consultant may tell you that Scala or Ruby is better than Java or C++. DO NOT get carried away by these fallacies. Keep the focus on Data and Analysis of Data. Everything else is a fools paradise in IT. And as you will learn constantly: Anything that does not make sense goes away eventually! Good riddance!

Brains Vs Big Data

So now that you know you need to focus on Data. Perhaps you will start a Big Data project. Analyse all your customer buying patterns, Study their social media behaviour and unleash cloud computing to compute their changing behaviour and tastes. Will you become a successful enterprise? Maybe not! but you will surely end up spending many million dollars on your IT before you get this Epiphany. 

Data is useful but ALL data may not be useful. Big data aims at collecting ALL data available blindly unleashing Analytics on it and producing 'intelligence'. I am not doubting that a correctly implemented BIG data project will succeed. BUT, here again I must ask you to stop and think! 

Can the same or near-same results be produced by other means? 

YES! 

Can it be done at a fraction of the cost? 

YES!

How?

Sampling

Representative sampling is one method that immediately comes to mind. I am sure there are many more ways to find out the very same intelligence without Analysis of Absolutely ALL the Data. 

e.g. the Sampling Theorem (by Nyquist-Shannon) states that if the frequency of ANY waveform is B hertz, this waveform can be reconstructed by sampling at 2B hertz! Bloody brilliant! Now applying this to your data (time series) and figure out how many sample points you will need to construct a perfect model. This is the beauty of theory and brains Vs Brawn.

Pattern Novelty

Many times you find patterns that are repeating over time, over and over again. (No change). This is likely to happen in your BIG data projects as well. e.g. Everybody already knows Christmas season is big sales or that the next tropical storm will trigger panic buying. Unless there is likely to be Pattern Novelty in your Domain, I feel you should not be spending hard earned cash on Big data projects. In other words, The Big data project should have lasting value.

Conclusion

Smart organizations harness brains and new ideas not Brawn (Big Data). If you start a big data project with a long lasting goal in mind you will probably succeed. But if you jump into Big Data because "Everyone else is doing it" you will be going with the Flow. And as I like to say..."If you go with the flow you will end up down the drain"

+Milind K Thombre

(comments Welcome)

Fall of the Talibarbarians

Dear readers,

Today I have coined the term "Talibarbarians" to refer to people who believe in the use of force to achieve their ends. As you know these traits are not culture specific and such elements exist in each and every civilization throughout the world today.

Success of a culture

In my analysis, any culture eventually acquires attributes that it reveres most. This happens by Darwin's principle of natural selection. Each successive generation goes closer to the goal as they mate with partners with the 'desired' characteristics. Different cultures emerged due to isolation in ancient times. For example, the Chinese revered wisdom and industriousness, Jewish civilization revered Cleverness and Business Acumen, Hindus revered Intellect and Europeans revered Power.

Ancient Hindus realised that success as a civilization will only be guaranteed by honing ALL of the following attributes:

• Intellect
• Military Might
• Material Wealth
• Skill and operational excellence.

No attributes were less important than others. (e.g. a Civilization that is wise, intelligent, skilled and rich will fall to another if it is not defended) The Hindus created the caste system to differentiate, specialize and hone these distinct traits and became a largely successful civilization for a long time. Marriage was strictly allowed within a caste itself. (This endogamous practice continues in Modern day India, though looked down upon by the West) Though the caste system exists to this day there were several notable exceptions to it even in ancient India. (DhanaNanda the sweeper king or the Peshwas who were Brahmin rulers in India to name a couple)

The rise of Intelligence

With the passage of time intellect became more and more important and continues its march forward as perhaps the most important attribute towards human evolution. Hence people with intellect were and are still revered in modern society. Civilizations that promote military might (violence as a means to achieve ends) or rampant materialism without intellect or skills without any of the above are headed for failure in my opinion. Many attributes became obsolete in the industrial age due to the introduction of technology e.g. Physical strength which was an important attribute is now rendered obsolete by modern weapons. Materialism has stood the test of time so far, but excessive greed by a few will create social discord. The "Occupy wall street" movement is an excellent example in modern times. Skills with no application of intelligence will create a country of clerks. Afterall a large section of the software community in India are nothing but code coolies.

Mediocrity

Why should one man not pursue all of the above? You may but then you will be a "Jack of all trades". Someone who lacks greatness in anything, a truly mediocre person. This is the epidemic that plagues the West today. Most of the West has given up the caste system and embraced 'equality', become truly mediocre. America realised the fallacy later and began importing the best minds from every culture into its country. However many of those elected in power continue to remain representatives of a (largely) mediocre culture that promotes shameless materialism and use of military force to achieve ends. However, there is still hope there as more of those who are 'imported' (H1) become voting citizens and vote wisely in future elections. After all only a baboon would use a gun instead of a vote or a blog. (pen)

Joke

I will take your leave with this joke that I thought up today. Back when the world was created the good lord decided to divvy up the goodies amongst different peoples.

The Chinese went first and asked for wisdom and industriousness, Granted said the lord.
The Arabs asked for Oil and guns to guard the oil. Granted said the lord.
The Europeans asked for Technology and money. Granted said the lord
The Jews asked for cleverness and Gold, Granted said the lord
The Hindus came last and they said Lord give us your brain and we will store it in our DNA. Granted said the lord.

Ever since the Chinese are working hard for the Europeans. The Europeans and Arabs are busy killing each other for oil. The jews are collecting all the gold, God has gone silent and and the Hindus are showing the rest of the world just who God is!

That's all for now folks!

Regards
+Milind Thombre

(comments welcome)

Management Wisdom from the Hindu Scriptures

Excerpt fom http://en.wikipedia.org/wiki/Bhrigu

Many great sages gathered at the bank of river Sarasvati to participate in Maha yagya organized at that time. Maharishi Bhrigu was also present there. All the great saints and sages could not decide that out of the Trinity Lord Vishnu, Brahma and Shiv who is pre-eminent and to whom should they offer Pradhanta (Master) of that yagya. With the consent of all the great saints present there, it was decided that Maharishi Bhrigu will test and decide who was pre-eminent.

Upon being entrusted with the task Maharishi Bhrigu decided to test Lord Brahma first. He went on to see Lord Brahma in Brahmalok. On reaching Brahmalok Maharishi displayed utter disrespect to Lord Brahma on purpose. Lord Brahma got angry and wanted to punish Maharishi but Maha Saraswati, wife of Lord Brahma saved Maharishi from his anger. Angry with disrespect, Maharishi Bhrigu cursed Lord Brahma that no one will worship Brahma in Kaliyug. To this day, there are very few temples devoted to Lord Brahma (the notable exception being the Brahma Temple at Pushkar).

Maharishi Bhrigu then decided to visit Lord Shiva at Kailash Parvat. On reaching Kailash Parvat, Nandi stopped him from going inside because at that time, Shiva and Parvati were sporting fun. Bhrigu curses Lord Shiva to be only worshipped in Linga form.

Then in order to test Lord Vishnu, Maharishi reached Vaikunth Dham. He entered the Dham without Lord Vishnu's permission and saw that the Lord was resting at that time. Maharishi asked him to wake up, but Lord was in deep sleep. On seeing no reaction from Lord, Maharishi hit Lord Vishnu on his chest (that strike by Maharishi Bhrigu left a foot print on Lord's chest and that foot print is known as "Shri Vatsa"). Lord Vishnu got up after the strike and realized what has happened. On realizing that Maharishi had hit him with his foot, Lord asked him, “Maharishi, are you hurt in your foot? My chest is strong but your foot is not so strong". Seeing the decorum of Lord Vishnu, Bhrigu was pleased and declared him superior amongst the Tridev.

My 2 cents

As People Managers all of us have a choice:

1. Be creative and worked up but cranky with your reportees, get called the "innovator", but never liked by anybody and quickly forgotten in this day and age (Kalayuga).
2. Be aggressive and destructive and be feared and respected by your reportees, be known as the dick head manager!
3.  Be relaxed and tolerant of your reportees tantrums, forgive their occasional transgressions and be understanding of their emotional turmoil, and you will be WORSHIPPED!

So What kind of Manager do you want to be?

Regards

+Milind Thombre

Felt like a Camel Ride!

This is a true incident that yours truly experienced way back in 2001. It was just post the 911 incident.

Flying back to the US via Dubai-Frankfurt-Cincinnati on an +Emirates flight (was the cheapest ticket at the time) .Personally it was a troubling time for me and I was reading a book on Hindu philosophy titled "The Art of Man Making" for solace. This book is a series of essays on the Bhagvad Gita.

Emirates decided to divert the flight via Jeddah. I continued reading my book on the flight. But when the flight landed in Jeddah an irate co-passenger started yelling and pointing me out to the air hostess. I was puzzled till he started pointing at the saffron covered book in my hand. The well meaning guy sitting next to me quietly advised me to fasten my seat belt and not get off the flight in any circumstances. I did so.

A long ordeal followed.

1. The Arab looking air hostess asked (just)me if I would like to get off the flight. I politely refused. At this point the guy sitting next to me muttered something about the Geneva Convention to her. She went away, only to come back later. At this point i was getting very alarmed!
2. Meanwhile the fasten seat belt sign went off. The hostess  then informed me that the sign was off. I said "OK" and continued to wear my seat belt.
3. After 5 minutes the flight was not getting off the ground and i realised that I was causing the delay! She emerged again and asked (only me) if i wanted something to drink. I politely refused and then added:
4. " You have the right to remain silent, anything you say may be used against you in the court of law in the USA where i am flying to. I request you not to speak to me for the remaining part of this flight." 
5. Following this an armed guard wearing a jet black uniform got on the flight. The hostess pointed me out to him. I continued to wear my seat belt he came over and said " Salam wale kum". "Wale kum asalam" I replied. He asked me what i was reading. I showed him the book. He proceeded  to ask me  if there is anything anti-Muslim in the book. I told him the word "Muslim" does not figure in the book. So how can the book be anti-muslim?
6. He then asked me angrily if "Muslim" does not figure in the book what kind of book is this? I told him "I'm trying to find out myself with a laugh" i told him it is a book about philosphy and not religion.

The guard left the flight and the plane took off. (Phew!) I refused food and drink throughout the flight. When the flight landed, as is customary the flight attendants and the pilots (white and Arab) were at the door to see us off at Frankfurt. The white pilot was asking everyone how was the flight? To which i loudly replied 

" I was rough and bumpy" "Felt like a Camel Ride!" The white pilot tried hard to control his laughter!

Love and Peace to all.

Milind Thombre

(comments welcome)

Death

Yesterday a friend and batchmate +Sunil Ivan Gomes had a massive heart and died. I am deeply shocked that a guy around 39-40 should suffer this untimely a death! Sunil RIP. Today's blog is dedicated to you

Innovation

A lot of people in the software industry who are doing the same thing for years together talk about innovation. "In-novate" is derived from "Novel" which means new. This requires original thought, not copying and repetition. To build skills requires a discipline and rigour, but once you have built the skills, if you keep on doing the same thing with those skills, that is NOT innovation. That is Specialization and "specialization is for insects"

So what exactly is innovation when it comes to the software industry? Let me give you an example from my life.

Years ago i ventured out trying to build India's first Analytics product (Nash Data Miner). We failed but the experience enriched me in many ways. This was not innovation as we were following in the giant footsteps of +SAS, however, I came in touch with several people who wanted to do new things and were all fired up! Needless to say several of them were Fresh graduates and yet un-destroyed by the Indian IT industry's un-innovative regimen! Though the idea was not new to the world, it was new to India and certainly new to me. However, I toyed with several innovations when it came to solving my practical problems. This was the first time i was dealing with 16 odd software engineers at once, all writing code at varying speeds and with varying quality.

I decided to use the Co-operative model for software development! this was an innovation (new to world) I was inspired by Mr. Kurien the milk man of India. he set up +Amul Milk which was based on the co-operative model, and i decided to use something similar for software!

Which meant :

1. Developers would not need to be physically present at one place most of the time
2. They would be evaluated by their work alone. (and paid in proportion)
3. They would bring in their code which would get evaluated by automated tools (similar to Amuls milk quality and quantity evaluation tools)
4. The modular code would then be built with the rest of the engine
5. There would be metrics for code quality as well as quantity (LOC)
6. There would be deductions for bugs, mistakes and such.

This was and still is a fine example of innovation according to me.

Why do most people lack it? And why innovators are often rebels.

People lack innovative thinking because they are programmed by society. They are "beaten into shape" till the programming becomes hard-wired. Wake up, Go to work, lunch, tea, work and so on. Only a die hard rebel is truly able to question this programming, challenge it and rise above and destroy anyone who tries to program him/her. This requires supreme belief in ones own abilities and the ability to swim against the flow. As I like to say "if you go with the flow, you end up down the drain"

How do you spot an innovator in robot-town-hell ?

• Someone who is building new skills all the time (so if you get a guy with the headline "16 years of Java experience" he's not it)
• Someone with a die-hard attitude ( this is required to fight the programming) (team players can leave the room now)
• Someone who has ideas and is willing to work hard to make them happen. (bye bye work life balancers)
• Someone who does not care to be judged by society, leave alone companies and managers.
• Thinker as well as Doer.

That's all for now folks!

Regards

+Milind Thombre

(comments welcome)