Big Data - The next big thing to come... or is it already here?

This is the trailer from a movie called "Moneyball", which is based on the true story of how a low budget baseball team was formed from players who were picked according to a statistical analysis of many players. This is an example of "big data" application. Big data is the systematic analysis of enormous amounts of information and extraction of knowledge (or wisdom) from this an analysis.

Data are being recorded more than ever. Internet in all forms, mobile phones, sensors, are recording continuously and in digital - standardized format all kinds of data. Heart beat rate of runners, traffic in the cities, searches in the web, weather, news, emotions (through social media), location of smartphones, and many other details that there is no point to list now. Recoding is not producing any knowledge. This amount of information is not even manageable by humans alone. This is the playground of computerized processors! Humans are inventing algorithms that analyze with the help of computers the available data. Of course within the limits of our current imagination.

Data are stored though and are available for ever, with the new technologies of the "cloud". They will be there for generations to come. Computers are starting to learn! Because simply... they have unlimited time to look into and combine all this information. And they are way faster than humans in analyzing huge amounts of raw data. And when a new algorithm is available they can look back at all the stored data with the new "eye" made available and enrich their knowledge. So the possibilities are only expanding.

This is a trend that cannot be reversed. Simply because our lives are improving through this procedure. Some examples:

• Less traffic or plane accidents
• New health procedures, like epidemic analysis and reactions (Personal Monitoring Tech: Invasion of the Body Trackers)
• More convenience in everyday life (for example, location services and personalized information on demand and in real time)
• Longer weather forecasts
• Superhuman fast stock market reactions and complex financial services
• Better pricing and distribution of products at retail (super markets for example can plan their stock according to weather forecast, or customer behavior analysis!)
• Urban crime control through analysis of events in correlation to various parameters like historical arrest patterns, paydays, sporting events, rainfall and holidays!

And for all these you do not need to conduct a survey as in the past. Most likely all the data anyone will ever need are there already. You just have to think of the best way to "dig" in them and extract the knowledge you need.

Of course, as written in a very enlightening New York Times article: "Data is tamed and understood using computer and mathematical models. These models, like metaphors in literature, are explanatory simplifications. They are useful for understanding, but they have their limits. A model might spot a correlation and draw a statistical inference that is unfair or discriminatory, based on online searches, affecting the products, bank loans and health insurance a person is offered, privacy advocates warn." But even this kind of problems can be solved much faster than in the past. Remember... data are there for ever and they only get more and more!

The main issue that must be addresses and resolved is access to the available data. I believe that recording is not bad for anyone, as long as the access to the records are open to anyone! This is the only way that true wisdom can be extracted from all this information. This the "Open Data" movement which is affecting all of us!  The Guardian has been encouraging this movement for some time now.  This a society issue and in my opinion it should be ranking very high in our priorities for the future. I would even call it a political issue!

Of course, there’s a major shortage of analytical talent. According to the McKinsey Global Institute (MGI), this is already an issue in the private sector. MGI predicts that in the US alone there will be an annual shortage of graduates in deep analytical fields of 140,000 to 190,000 by 2018. A very good article on this subject is from Linda Rosencrance of the Spotfire Blogging Team "The Promise and Challenge of Using Big Data to Address World Problems".

Big data in the end is all about wisdom and evolution! And obviously this cannot be left to the hands of few or only in the processing power of computers!

Why do I run?

Running is an activity which seems, for the majority of people, a very unpleasant thing to do. Involving a lot of sweating, some physical discomfort with the heart beating more intensely, risk of injury for the joints and muscles, and of course all of these things are happening during our free time, at least for 99,9% of us runners!

Running is primarily about covering distances, with your own physical ability. Secondarily, speed may get in the way, how fast one can cover a given distance. But first comes always the fact that the runner must be able to cover the distance. Two elements are highlighted here: covering the distance and doing so only by using your body.

In sports, people tend to reveal their real character because of the simplicity of the situation. The connection between effort invested and the result returned is very straightforward and linear. No tricks can be applied and no shortcuts exist to be taken. Your way of thinking, your physical capability will get you to the destination. Even when doing a casual run, the runner is able to feel his body and observe his performance. And this activity requires, after a few minutes in it, the full attention of the participant. No distractions are allowed. It is absorbing our body and mind.

So running is a "back to basics" activity. Regular running will change ones life in many aspects. A morning run will make you feel fully charged for the day ahead, an evening run will relieve most of the stress. Eating will become more pleasant, digestion will be easier, and above all, at night when time comes to go to sleep, your body will be needing it. There is a notable side effect involving our hormones and the feeling of happiness we get out of most of our runs. Also it is a character shaping activity by testing our determination, discipline and capability to achieve our goals by overcoming some simple physical difficulties. Long distance running (whatever "long" is for each one of us) will test our mind and character against some very primitive and elemental instincts, because we will experience feelings of discomfort, exhaustion, maybe pain, hunger, thirst. While running each one will address the emotions and difficulties according to his character and mentality. And there are no external factors to assist you.

About myself: I run because while on the move I have this unique feeling that I can hear my self without any obstructions! I see my thoughts and emotions. I feel the earth rolling under my feet and I have this unique sense of flow. Over time, I realized that every run (especially long distance ones) includes a life of its' own, with a great deal of self awareness, but also realization of the characters of those running next to me. There is no better way to get to know people. Including myself!

Why do you run?

Happy New Year to everyone! May this year be the one you run the most you have ever run.