The 10 Commandments for data driven leaders

The 10 Commandments for data driven leaders
The 10 Commandments for data driven leaders

Couple of year’s back I was invited to a boxing match between two world-class boxers. The match was not that important that day, but a conversation that followed with one of the coach was important. It was the importance of footwork and the emphases on game as a big rumble of strategy, tactical execution and attained skill-set. The coach explained it as a set of rules that the boxers have to keep in mind to keep his thinking straight. The more you protect the rules, the more the rule protects you. It was a mesmerizing experience and stuck with me since. I have ushered the experience to few executives and had as much fun explaining it as I had experiencing it.

Today’s leaders, in the world of big data analytics, need to act much like those boxers. While leaders are taking head-on with one of the world top business challenges, leaders need to stick to certain strategic principals that will prepare them to be agile and take any challenge head-on and come out winner. I have tried capturing 10 most used lessons that I have heard and pitched through my conversation with various leaderships. Think of it as 10 commandments that I hold dear:

1. Thou shalt not do it alone:
However strange it may sound but it is real. Leaders need to understand that it takes a village to install the right data analytics strategies in place. Leaders should consider it to be a team effort and not work in any isolation. Some of the best implementation of data analytics strategies has used a council model to put together a right team of leaders to identify and deploy best data analytics strategies to help business reap benefits of being data driven.

2. Thou shalt walk the walk
Yes, understanding the journey from bits to insight is important. A supporter should be the biggest implementer of the strategy as well. It is important for the leadership to not just define strategies but also see them in action. Understanding how data analytics works and how an insight is generated from raw data, how teams are embracing analytics will help leadership in extrapolating some of the “not very evident” use cases that will help create a win-win strategy.

3. Thou shalt embrace analytics from the top
Yes, this is not a very hard thing to explain someone but surprisingly we do hear horror stories of this stage completely being ignored. While we see many preachers on the top level management, but most of senior leadership still primarily bank of gut based decision making which is as Ludacris as it is scary. Leadership should embrace sophisticated capabilities to help them become data driven and not gut based political tainted decision makers. It is important that top leadership should be open towards strategies embracing data driven to their decision process.

4. Thou shalt not ignore the culture
I often find myself joking around that its not big data that is the problem but big culture. Which is to much degree a true and consistent statement. There are very few companies, which has really nailed down the cultural adoption aspect for being data driven. Creating a culture of embracing data analytics in businesses will remove any gut based and politically motivated decisions and replaced it with data driven approaches. This is by far the most challenging stage. It requires slow changes to promote and appreciate data driven decisions to help teams adopts such practices within their circles. If done right, this opportunity has the best shot at inducing change.

5. Thou shalt translate bits to analytics
Understanding data is not a business friendly act. Which is why most of the business shies away from being data driven. Great business builds right translators to help businesses understand value of analytics by translating it to insights that businesses could value. If done right, this rule will make selling a data driven strategy to leadership a cakewalk. Business leadership would easily understand how analytics is generating insights and thereby appreciate it more.

6. Thou shalt deal from the problem first
Another great rule for generating quick wins to justify any investment in data driven strategy. It is important for business to see the value first before embracing data analytics practices. Starting with problem first will shorten the value generation cycle and create opportunities for achieving some quick wins. Therefore, it would be a good idea to start with a business problems first as many times as possible and eventually a good strategy will help proactively identify problem worth looking as well.

7. Thou shalt grow the analytics club
As with creating a culture, there should be a need for creating a wide and deep council / clubs where people could share their best practices, learn the latest/greatest and exchange opportunities. These innovation hubs will expedite the cultural adoption and help the spread of best practices. If done right, this rule will expedite the creation of center of excellence within businesses without spending too much time and resources. This is the most organic and viral way to grow data driven culture within company.

8. Thou shalt build a center of excellence
Yes, this rule surely needs its own space in this discussion. Having a center of excellence is the key to having a data driven business. While a good strategy many times accidently give rise to some flavor of a center of excellence, a good rule of thumb requires a dedicated effort toward creating a center of excellence. A well-designed center of excellence will help achieve a faster data driven decision cycles and rapid identification of business opportunities. A well-designed center of excellence will pay for itself rather quickly than anyone anticipates.

9. Thou shalt not ignore strategy
While this and many other related article points towards the importance of data analytics driven process to help run a business but that does not trump the importance of a good strategy. As Henry Ford rightly pointed out: “If I had asked people what they wanted, they would have said faster horses”, this is true for data driven learning as well. Still a good rounded strategy when combined with a good data analytics process forms the best way to go data driven.

10. Thou shalt not run it as a spring but a marathon
And lastly, it is important to understand that a good execution is a constant and consistent set of operations. Surely, there are lots of successes along the way but still it should be treated as a long and consistent journey towards creating a data driven organization. While this is the most understood rule, this is the most ignored rule as well. We all are bounded by our urge to deliver fast results that we many time forget the bigger picture that we are suppose to achieve and understanding this rule will help us stay focused towards long term vision.

So, while the road being data driven is painful and long, but keeping a grip on these 10 rules will help leaders win their fight and at the least put-up a fight worth tuning in for.

Source: The 10 Commandments for data driven leaders by v1shal

The Pitfalls of Using Predictive Models

I joined my friend’s fantasy football league this past season. I was skeptical to join at first. My friend’s league had been together for 7 years, each participant with deep knowledge about nearly all the NFL players and the game. I, on the other hand, have not followed NFL football for nearly 20 years and had only a superficial knowledge of only the popular NFL players. Given my lack of knowledge, I thought that I could use predictive modeling to help me pick my fantasy team during the draft.

Predictive Modeling

Predictive modeling is a process by which a statistical model is chosen to best predict the probability of an outcome. The use of predictive modeling was illustrated in the excellent movie, Moneyball, in which Billy Beane, the manager of the Oakland Athletics used predictive modeling to select the players for his team. Working under a limited budget compared to other teams in the league, his predictive model identified baseball players who were undervalued by the other teams yet were predicted to get runs/points. Those would be the players he selected. As a measure of Billy Beane’s success, the Athletics, in 2006, were ranked 24th of 30 major league teams in player salaries but had the 5th-best regular-season record.

We used Yahoo!’s Fantasy Football service as the platform for our league. The site provide a predicted overall score (the total score expected across the entire regular season) for each NFL player. I used this predicted score as the basis for my draft selection, selecting the top player (most points in his position) each time it was my turn in the draft. I continued to use Yahoo!’s predictions each week when selecting my team lineup for that week’s games.

My Performance

Our league included 12 teams. Out of 12, I came in last place. No matter what metric of performance we used to rank the teams at the end of the season, my team was the worst. My team had the lowest Win/Loss ratio. My team scored the fewest points. My team gave up the most points. Although I did not expect to win the league, I did not expect to get last place. Where did I go wrong? Here are two reasons that could explain my poor performance.

1. The Model Provides Poor Predictions

The value of a predictive model is measured by how well it predicts an outcome. I did (do) not know how well Yahoo!’s model predicts actual total scores (but something I am going to calculate next season). It is quite possible that the correlation between predicted and actual end-0f-season point totals is low. Even if this correlation was non-zero, the model still may not provide much forecasting power on which to base player selection decisions. Even a correlation of .50 (rather good for predicting any type of behavior) means that you can only explain 25% of the variance in end-of-season points. Perhaps my use of Yahoo!’s predicted scores did not help me select the best players because they were simply not good at predicting actual end-of-season scores.

Yahoo!’s predictions of player’s end-of-season point totals is offered with no additional background information about what variables they use or how those variables are combined to make their predictions. Information about how the model was built (subjectively, statistically, both) would help users evaluate the quality of the model. Were the selection and weighting of variables chosen to maximize the predictive power? Were key player variables excluded (under-specified model) in the development of the predictive model?

2. I Lack Content Knowledge

My lack of knowledge about the NFL players could have impacted my performance in two ways: 1) by  limiting my ability to use Yahoo!’s predictions correctly; 2) by limiting my ability to augment Yahoo!’s model with unique, nuanced information about players/games. As I continue to play fantasy football, I expect my performance will improve based solely on the information I learn through watching games.

Predictive Modeling and Customer Experience Management

We use predictive modeling in customer experience management (CEM) to describe the likelihood that customers will engage in certain types of loyalty behaviors (e.g., advocate, open up more of their wallet, stay for a long time). Specifically, using CEM-related data, we create models to predict the level of customer loyalty given different types of interactions with and different levels of satisfaction with the customer experience. We know that both attitudinal and operational metrics impact customer loyalty. A predictive model of customer loyalty that includes important variables maximizes the value (predictive power) of your model.  These predictive models are used to create different what/if scenarios to understand how improvements across different touch points will impact customer loyalty.


When executives make decisions about how to improve customer loyalty, they rely on many sources of information, including personal knowledge, past experiences of similar situations and even predictive models of expected performance. While I admit to relying on the use of  statistical modeling in my decision-making process, I showed how my sole reliance on this type of information did not lead to any success. My experience in playing fantasy football illustrated the importance of:

  1. Knowing how well the model predicts the outcome of interest. How well does your predictive model explain customer loyalty?
  2. Knowing the variables in the model. Does the model include a comprehensive list of variables to predict the outcome? A simple list of all the variables will do. Ensure the measures used to create the model are reliable and valid. Poor metrics will limit both the predictive power of your model and the value of it to your company.
  3. Knowing how the model was developed. Was the model built on a statistical basis? If and when judgement was introduced into the model, when was it introduced?

Bonus: Commentary on the NFL Commentators

I watched many football games on TV over the season. I saw a lot of receivers make spectacular catches. I witnessed many running backs make unbelievable runs. Even though I reestablished my love for the sport, I, unfortunately, had to suffer through the NFL commentators’ remarks about the game, during the game.  Besides stating the obvious (e.g., “To win, they’ll have to score more points than the other team.”), the commentators might be using subliminal messaging/imagery to to attract a different segment of the viewing public. Here is my top 10 list of actual remarks made by NFL commentators this past season:

10. They “double-teamed” him.
9. It’s all about the penetration.
8. He came into his own.
7. He takes advantage of the opportunity he gets.
6. You just ride that guy.
5. Watch him squirt through this hole.
4. That’s an impressive sack.
3. That guy’s got really good ball skills.
2. You cannot advance a muff.
1. Comes right up the A gap.

Go local sports team!!!!

Source: The Pitfalls of Using Predictive Models

Feb 22, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)


Pacman  Source

[ AnalyticsWeek BYTES]

>> 5 Brands That Are Using the Internet of Things in Awesome Ways by analyticsweekpick

>> SDN and network function virtualization market worth $ 45.13 billion by 2020 by analyticsweekpick

>> Can Big Data Help You Plan Your Commute? by analyticsweekpick

Wanna write? Click Here


 Artificial Intelligence: 35,000 Jobs on offer with state’s own data … – Economic Times Under  Data Science

 Network hardware components scrutinized as apps move to cloud – TechTarget Under  Cloud

 IoT Knits a Factory Together – EE Times Under  IOT

More NEWS ? Click Here


R, ggplot, and Simple Linear Regression


Begin to use R and ggplot while learning the basics of linear regression… more


On Intelligence


Jeff Hawkins, the man who created the PalmPilot, Treo smart phone, and other handheld devices, has reshaped our relationship to computers. Now he stands ready to revolutionize both neuroscience and computing in one strok… more


Save yourself from zombie apocalypse from unscalable models
One living and breathing zombie in today’s analytical models is the pulsating absence of error bars. Not every model is scalable or holds ground with increasing data. Error bars that is tagged to almost every models should be duly calibrated. As business models rake in more data the error bars keep it sensible and in check. If error bars are not accounted for, we will make our models susceptible to failure leading us to halloween that we never wants to see.


Q:Explain what a long-tailed distribution is and provide three examples of relevant phenomena that have long tails. Why are they important in classification and regression problems?
A: * In long tailed distributions, a high frequency population is followed by a low frequency population, which gradually tails off asymptotically
* Rule of thumb: majority of occurrences (more than half, and when Pareto principles applies, 80%) are accounted for by the first 20% items in the distribution
* The least frequently occurring 80% of items are more important as a proportion of the total population
* Zipf’s law, Pareto distribution, power laws

1) Natural language
– Given some corpus of natural language – The frequency of any word is inversely proportional to its rank in the frequency table
– The most frequent word will occur twice as often as the second most frequent, three times as often as the third most frequent…
– The” accounts for 7% of all word occurrences (70000 over 1 million)
– ‘of” accounts for 3.5%, followed by ‘and”…
– Only 135 vocabulary items are needed to account for half the English corpus!

2. Allocation of wealth among individuals: the larger portion of the wealth of any society is controlled by a smaller percentage of the people

3. File size distribution of Internet Traffic

Additional: Hard disk error rates, values of oil reserves in a field (a few large fields, many small ones), sizes of sand particles, sizes of meteorites

Importance in classification and regression problems:
– Skewed distribution
– Which metrics to use? Accuracy paradox (classification), F-score, AUC
– Issue when using models that make assumptions on the linearity (linear regression): need to apply a monotone transformation on the data (logarithm, square root, sigmoid function…)
– Issue when sampling: your data becomes even more unbalanced! Using of stratified sampling of random sampling, SMOTE (‘Synthetic Minority Over-sampling Technique”, NV Chawla) or anomaly detection approach



@AnalyticsWeek Keynote: The CMO isn't satisfied: Judah Phillips

 @AnalyticsWeek Keynote: The CMO isn’t satisfied: Judah Phillips

Subscribe to  Youtube


Everybody gets so much information all day long that they lose their common sense. – Gertrude Stein


#BigData @AnalyticsWeek #FutureOfData #Podcast with  John Young, @Epsilonmktg

 #BigData @AnalyticsWeek #FutureOfData #Podcast with John Young, @Epsilonmktg


iTunes  GooglePlay


Data production will be 44 times greater in 2020 than it was in 2009.

Sourced from: Analytics.CLUB #WEB Newsletter

Big data offers big benefits but privacy concerns must be addressed

Big data is big business. It enables firms to gather huge reams of information to help provide better insights and so make better decisions. But with great power comes great responsibility.

With ever more data being generated, collected, harvested and processed, issues around data privacy and protection will only grow.

Hackers’ eyes will light up at the potential for accessing such vast troves of data, while in the UK fines await from the Information Commissioner’s Office (ICO) for anyone who falls foul of the Data Protection Act (DPA) by allowing unauthorised access to personal information processed via big data systems.

Follow the rules
No doubt this is why the ICO released its first big data guidance document in late July. “Big data is not a game that is played by different rules,” said Steve Wood, the ICO’s head of policy delivery, as he unveiled the report.

The report acknowledged that, while many big data projects do not use personal information, for example weather projects, many gather data on people from social media sites, loyalty cards and sensors in clinical trials.

Wood advised firms embarking on such projects to familiarise themselves with the UK’s data laws to ensure they know their responsibilities for keeping this information secure.

The ICO also issued guidance on data collection, including keeping data secure and encrypted, ensuring it is not kept for longer than necessary and that the data collected is, “adequate, relevant and not excessive”.

This last line is especially pertinent. With so much information being created it is easy to gather huge amounts of data, but without any clear reason for doing so.

Mark Brown, director of information security at consultancy EY, urged firms to be aware of this pitfall as it could store up problems.

“A lot of companies have viewed big data as ‘let’s collect as much data as possible so we’ve got it there to use it.’ But, with the prevalence of breaches we are seeing, data needs to be managed correctly,” he said.

Brown splits this management into two areas that need consideration for data protection and data use: consumer trust and operational efficiency, which complement one another.

“Consumer trust is based upon reputation and you can’t maintain brand reputation if you’re losing data,” he explained.

“But if you understand the risk and compliance issues you can create an information architecture that offers enhanced operational efficiency. Put these together and your business can grow.”

Silver bullets
To get this right security must play an important part of any big data strategy. This is especially important as many firms will turn to cloud-based systems for the storage and processing of big data.

Raj Samani, EMEA chief technology officer at McAfee, part of Intel Security, advised companies to pay particular attention to this challenge, as outsourcing data to a cloud provider does not absolve a company of data protection responsibility.

“You need to do your due diligence. You are legally obliged for the data, even if hosted by the cloud provider, so you need to ensure they have appropriate security in place,” he warned.

For those managing data themselves, big data security best practice follows similar security requirements as those applied to ‘normal’ data, noted Samani, who argued that firms need to employ tools to protect themselves, and their data, as much as possible.

“People will say there is no silver bullet, but I disagree. There are, it’s just that not every threat is a werewolf,” he said.

“You can use encryption, malware detection, white-listing, signature-based detection, undertake regular scanning and patching and so on. There is no such thing as 100 percent security, but it’s about reducing the risk level.”

Show your working
If firms get this right it should give some peace of mind people can hand over their data with confidence. This, though, leads to another issue: explaining what the data will be used for.

The ICO urges companies to be open and clear on this, citing Channel 4’s use of a YouTube video to explain how it gathers and uses data as a good example of how to do this, rather than posting dense and dull privacy notices.

“The fact that there are poorly written privacy notices does not remove the responsibility on organisations to explain to customers what they are doing,” the ICO report noted.

This is an important part of the future of big data. If people become savvier about their personal data, questioning who is gathering it, for what purpose and how it is secured, it will become harder for organisations to gather the big data they desire.

Samani believes that while at present people seem unaware of the value of their data, this could change. “The perceived value of personal data has never been so low, but at the same time the value of data is increasing,” he said.

“If people realised this they would start to make better decisions about when to release data and when it may be unsafe to do so. We may get to a world were people begin to preserve their personal data because they realise it has monetary value.”

Target’s targeting works too well
Heeding all this advice could save businesses and consumers from some tricky big data situations, as the following example involving US retailer Target shows.

The firm used data analytics on its customer database to work out when female customers appeared to be pregnant based on browsing and buying habits.

To enhance customer loyalty, Target would then send coupons regarding certain items, including those that would appeal to expectant mothers as their due date approached.

One day, though, an irate father confronted a store manager when his daughter was sent the booklet, claiming the inclusion of baby items was inappropriate. The store manager apologised and rang the man a few days later to reiterate the apology.

However, the man then admitted that his daughter was indeed pregnant, a fact she had to reveal owing to the situation. It was a win for Target’s analytics system, but not the ideal human or emotional outcome.

Target realised that such a blatant sales approach could make customers uneasy so the retailer changed its strategy to subtly include such promotions alongside everyday items.

The case demonstrates the power of data and the importance of considering everything from security and data protection to privacy issues for any firm embracing big data.

A version of this article first appeared in the V3 Big Data Definitive Guide available for iOS, Android and on the web.

Originally posted via “Big data offers big benefits but privacy concerns must be addressed”

Originally Posted at: Big data offers big benefits but privacy concerns must be addressed

Big Data: Would number geeks make better football managers?

Charles Reep was a retired RAF Wing Commander who loved football.

Specifically, Swindon Town. And it ached to see them losing – something the team made a habit of in the 1949/50 season.

So frustrated was Wing Cdr Reep with one particular performance, that for the second half he pulled out his notepad and started making notes on the players – their movements, their positions, the shape of their play. He identified small changes that he thought could help the team grab a few more goals.

He was decades ahead of his time.

Now, behind the biggest football teams in the world, lies a sophisticated system of data gathering, metrics and number-crunching. Success on the pitch – and on the balance sheet – is increasingly becoming about algorithms.

The richest 20 clubs in the world bring in combined revenues of 5.4bn euros ($7.4bn, £4.5bn), according to consultancy firm Deloitte. And increasingly, data is being seen as crucial to maximising that potential income by getting the most from football’s prized investments – the players.

Hoof it!

Data and football have had a strained relationship over the years.

Back in the 1950s, Swindon didn’t have much time for Wing Cdr Reep’s approach. But west London side Brentford did.

ProzoneProzone’s software offers real-time match tracking, and is used by over 300 clubs worldwide

The club was facing a relegation battle. Wing Cdr Reep was taken on as an advisor – and with his counsel, the team turned their fortunes around and were safe from relegation at the close of the season.

A triumph, you would think – but his approach, despite the measurable success, drew considerable scorn.

His data suggested that most goals were scored from fewer than three direct passes, and he therefore recommended the widely-despised “long-ball” game.

In other words, the ugliest type of football imaginable. Hoof the ball forward, hope you get a lucky break, and poke it into the net.

“Unfortunately it kind of brought statistics and football into disrepute,” says Chris Anderson, author of The Numbers Game, an analytical and historical look at the use of data in football.

“Because people pooh-poohed the idea of the long ball game in football and thought it responsible for the England team not doing nearly as well as they should have for all these years.”

Leg sensors

Wing Cdr Reep passed away in 2002. Were he alive today, he would likely be a welcome guest at German football club TSG Hoffenheim, where the “big data” revolution is changing everything about how they prepare for a match.

Through a partnership with SAP – which specialises in handling “big data” for business – the club has incorporated real-time data measurements into its training schedule.

“It’s a very new way of training,” says Stefan Lacher, head of technology at SAP.

SAP data at HoffenheimThe data can be analysed in real-time by data experts – and training schedules can be adapted

“The entire training area becomes accessible virtually by putting trackers on everything that’s important – on the goals, on the posts. Every player gets several of them – one on each shinpad – and the ball of course has a sensor as well.

“If you train for just 10 minutes with 10 players and three balls – it produces more than seven million data points, which we can then process in real time.”

SAP’s software is able to crunch that data, and suggest tweaks that each individual player can make.

“It’s about better understanding the strengths and weaknesses of the players,” Mr Lacher says, “and spending more time working on the weaknesses and making better use of the strengths.

“It’s moving from gut feeling to facts and figures.”

Career-threatening injury

But it’s in the boardroom where football data has an even more critical role to play in the success of the team, says Dr Paul Neilson from football technology specialists Prozone.

“One of the most important things within elite sport is making sure your players are available for training and matches as much as possible, and that is about mitigating injury risks,” he says.

“If you’re doing that you should be able to reduce the risk of physical overload, and reduce the risk of injury.

SAP data at HoffenheimThe data can be relayed to players so they can work on their weaknesses

“When you’re paying players as much as players get paid, it’s very important to make sure they’re on the pitch as much as possible.”

Non-playing players is a massive financial concern for football clubs. The famous example is the case of Jonathan Woodgate, who left Newcastle United in 2004 to join Spanish giants Real Madrid – for a tasty £13.4m.

Plagued by injury, Woodgate played for Real just nine times before leaving in 2007. That’s just under £1.5m per game – without his weekly wages taken into consideration.

Prozone’s research lab wants to reduce this risk for clubs by using data to analyse body movements and spot, before a physio can, where future injuries may occur.

In young players, analysis of movement can also provide an early warning system for those who may develop career-threatening injuries.

Collecting this data is a sophisticated task. Prozone’s approach relies on a complex network of cameras fitted around the stadium, picking up player movements from several angles at once.

Two worlds

Football managers and coaches like to think it’s their instinct, not geeky data, that gets results. And so, uptake of data analysis in football has been a slow process.

“Football, particularly in the UK, can be a little bit conservative,” says Dr Neilson.

“You look at rugby, and the head coach/manager will often be in the stand for all the game and be surrounded by data and technology and video analysis.

SAP sensor
Players at Hoffenheim attach sensors to their kit to monitor their movements

“Compare that with football and the manager is still very much in the dugout, trying to affect the players personally, in terms of instructions and shouting – and very much being part of the sometimes chaotic nature of football.”

This culture clash means there are no managers that prowl the touchline with a tablet – yet. But behind the scenes it’s a very different picture.

Prozone provides intricate data for more than 300 football clubs around the world, including every team in the lucrative English Premier League.

But to make sense of it all requires talent – and Dr Neilson believes that soon, fans will come to admire – or despise – their club’s data scientist in the same way they treat the manager now.

“In a typical football club you have technical people like your sports analysis staff, or sports science staff. They are very analytical, very objective and process driven.

“At the opposite end of the scale you have the decision makers – the chief executive who writes the cheques, the manager that makes the weekly decision in terms of team selection.

“The challenge is connecting those two worlds – so the decision makers trust in that data.”

Sadly, Wing Cdr Reep didn’t live to see the true appreciation of his craft.

And to this day, his long-ball philosophy is criticised by many who say that his data collection was far too primitive to come to such sweeping conclusions.

But nevertheless, his work pioneered what has become a cornerstone of the modern, beautiful game.

Somewhere, in the not-so-distant future, at a football club losing three-nil at home – the fans are chanting “you’re getting sacked in the morning”. Not at the manager, but at the man with the big data.

Follow Dave Lee on Twitter @DaveLeeBBC

Originally posted via “Big Data: Would number geeks make better football managers?”


Feb 15, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)


Tour of Accounting  Source

[ AnalyticsWeek BYTES]

>> Why Your Company Should Use Data Science to Make Better Decisions by analyticsweekpick

>> Put big data to work with Cortana Analytics by analyticsweekpick

>> Predictive Workforce Analytics Studies: Do Development Programs Help Increase Performance Over Time? by groberts

Wanna write? Click Here


 UBS loses senior quant risk specialists as juniorisation hits the … – eFinancialCareers Under  Risk Analytics

 Machine Learning And Behavioral Biometrics: A Match Made In Heaven – Forbes Under  Machine Learning

 Microsoft and SAP both commit to running SAP HANA on Azure internally – ZDNet Under  Hybrid Cloud

More NEWS ? Click Here


Machine Learning


6.867 is an introductory course on machine learning which gives an overview of many concepts, techniques, and algorithms in machine learning, beginning with topics such as classification and linear regression and ending … more


Hypothesis Testing: A Visual Introduction To Statistical Significance


Statistical significance is a way of determining if an outcome occurred by random chance, or did something cause that outcome to be different than the expected baseline. Statistical significance calculations find their … more


Data Analytics Success Starts with Empowerment
Being Data Driven is not as much of a tech challenge as it is an adoption challenge. Adoption has it’s root in cultural DNA of any organization. Great data driven organizations rungs the data driven culture into the corporate DNA. A culture of connection, interactions, sharing and collaboration is what it takes to be data driven. Its about being empowered more than its about being educated.


Q:Provide examples of machine-to-machine communications?
A: Telemedicine
– Heart patients wear specialized monitor which gather information regarding heart state
– The collected data is sent to an electronic implanted device which sends back electric shocks to the patient for correcting incorrect rhythms

Product restocking
– Vending machines are capable of messaging the distributor whenever an item is running out of stock



#BigData @AnalyticsWeek #FutureOfData with Jon Gibs(@jonathangibs) @L2_Digital

 #BigData @AnalyticsWeek #FutureOfData with Jon Gibs(@jonathangibs) @L2_Digital

Subscribe to  Youtube


Data beats emotions. – Sean Rad, founder of


@TimothyChou on World of #IOT & Its #Future Part 1 #FutureOfData #Podcast

 @TimothyChou on World of #IOT & Its #Future Part 1 #FutureOfData #Podcast


iTunes  GooglePlay


The Hadoop (open source software for distributed computing) market is forecast to grow at a compound annual growth rate 58% surpassing $1 billion by 2020.

Sourced from: Analytics.CLUB #WEB Newsletter

Rethink Big Analytics to handle BigData

Rethinking Big Analytics to handle BigData
Rethinking Big Analytics to handle BigData

Big data is growing. Shocked? Sure, you must have heard about it again and again, to the point it sounds like another lame statement. What you will be amazed to see is how few are actually doing anything about it. The fun begins when we start digging deeper into what constitutes to be the real problem, is it the capability to handle data, data channel, query data, data prep or the big daddy, analytics. Hmm.. How about calling it Big Analytics.

Let me put it in another way: “Big Data is a Big Analytics Problem”. Sounds strange? – per CIO Insight “81% of IT professionals around the globe agree that better analytics is central to the enterprise’s big data problems”

Before I get to the point in the talk, let us take a quick detour to current bottleneck that analytics world is facing, that could potentially restrict it from growing:

1. Human problem with Analytics: I know you must be already feeling the connection here. It’s you, it’s me, and it’s us. We all love our analytics to death and we hate sharing it. We share it with people we have established analytics intimacy with. Your work buddies, team, boss etc. But there is a problem. What if the main gravy that you could learn from stays outside your reach? How will you ever find out what others are doing? I am sure you must be thinking, I am the cutting edge guy. Sure, I feel that all the time as well, but what if, there is something out there that we could learn from. What if you are pushed to stand in front of datasets that some other guy in some other team has already mastered, how will you scale. Problem, right? Yes, completely human driven analytics starts to fall apart from this point.

2. AI problem with Analytics: Now let’s get to artificial intelligence, alias machine-learning, alias that si-fi s#$t. Now don’t jump on me yet. I agree that AIis never meant to completely out take analytics. We have created a restricted container for our machine learning AI friends. Their task goes in, results come out. They are patted on their backs, life moves on. But there is a problem, human deal with most of pressing analytics that is directly impacting the businesses. So, AI alone will not sustain, and there will always be a need for human analysts, to make sense of AI friends. So, sci-fi is incomplete to handle growing demand as well.

3. Politics, perception and whole 9 yards: Yes, I love this part; I have gottenyelled at and a few times shown the door. Many believe analytics is a sacred entity and guarded with utmost care and should obey to company politics and perceptions. Smelling funny? Don’t worry, if things still appear rosy. Our brains are wired to accommodate the politics, regulations and compliance. I am not against it either, but having your analytics capabilities enslaved by some paralytic limitation could really restrict you from growing. So, make sure you understand what the restrictions are and how much you could let it float. Remember, there is always learning across that closed wall.

4. Current Deliveries Vs Long Term Vision: Yay, appraisal talks! We are all compensated for our current and short-term deliveries, that plague analytics as well. Who will be evaluated in pursuing something that may show its color in couple of year if not the next? Yuck! Believe it or not, it is what is another bottleneck that current analytics standard face. We need to build analytics strategies and models accommodating changing business needs and environment variables. Scalable methodical analytical models will help you tame the growing data needs and help in grasping bigger picture which most of the time is missing from the equation. All you need to do is combine long term and short term analytics and give your analytics strategy a hard and long view. Maybe, you could find a magic sauce that will help your business stay afloat on awesome analytics.

5. We are doing it, look at our BigData tools: First, Bravo! Getting infrastructure right and committed to your bigdata need is a huge step most of the laggards are still crying about. So, I commend you for crossing that chasm. But, remember, bigdata tools are not substantial to solve your big-data needs. In fact, bigdata tools and capabilities will make available more data and cleaner data at shorter durations for you to play with. Which in no ways mimics the speed at which at we do analytics. Our analytics are pretty much enslaved to our current needs. There is a strong chance that things will fall through the crack and might never get noticed. Current tools and capabilities are not talking about urgency of revamping your analytics strategy.

Almost done with my rant. So, what’s the conclusion? Have a strategy that will help you grow your analytics to handle the growing data. Just by buying boxes, infrastructure layer and other toys to handle your data deluge might not provide a complete and rounded solution. You still have to deal with your traditional analytics and change it to reflect ROA / ROI on your awesome big-data toy investments. So, up your sleeves and get cranking on drafting a scalable analytics framework. Want some idea? Hush.. You heard it here first; let’s call it “collaborative analytics [™]”. So, what is collaborative analytics? If you’ve read the blog, you already know it. Are you still curious? Wait for our follow-up blog. Till then happy analyzing!

Source: Rethink Big Analytics to handle BigData by v1shal

Feb 08, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)


Data security  Source

[ AnalyticsWeek BYTES]

>> Feb 01, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..) by admin

>> Jul 27, 17: #AnalyticsClub #Newsletter (Events, Tips, News & more..) by admin

>> Businesses Say Big Data Is Changing Markets, But What’s Holding Back Adoption? by analyticsweekpick

Wanna write? Click Here


 Best Trade-off Point Algorithm for Efficient Resource Provisioning in Hadoop – Under  Hadoop

 Virtualization efforts move full-speed ahead at Scania – SiliconANGLE News (blog) Under  Virtualization

 Microsoft adds cloud security to keep out hackers and government snoops – Colorado Springs Gazette Under  Cloud Security

More NEWS ? Click Here


Artificial Intelligence


This course includes interactive demonstrations which are intended to stimulate interest and to help students gain intuition about how artificial intelligence methods work under a variety of circumstances…. more


Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking


Written by renowned data science experts Foster Provost and Tom Fawcett, Data Science for Business introduces the fundamental principles of data science, and walks you through the “data-analytic thinking” necessary for e… more


Grow at the speed of collaboration
A research by Cornerstone On Demand pointed out the need for better collaboration within workforce, and data analytics domain is no different. A rapidly changing and growing industry like data analytics is very difficult to catchup by isolated workforce. A good collaborative work-environment facilitate better flow of ideas, improved team dynamics, rapid learning, and increasing ability to cut through the noise. So, embrace collaborative team dynamics.


Q:Explain selection bias (with regard to a dataset, not variable selection). Why is it important? How can data management procedures such as missing data handling make it worse?
A: * Selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved
– Sampling bias: systematic error due to a non-random sample of a population causing some members to be less likely to be included than others
– Time interval: a trial may terminated early at an extreme value (ethical reasons), but the extreme value is likely to be reached by the variable with the largest variance, even if all the variables have similar means
– Data: “cherry picking”, when specific subsets of the data are chosen to support a conclusion (citing examples of plane crashes as evidence of airline flight being unsafe, while the far more common example of flights that complete safely)
– Studies: performing experiments and reporting only the most favorable results
– Can lead to unaccurate or even erroneous conclusions
– Statistical methods can generally not overcome it

Why data handling make it worse?
– Example: individuals who know or suspect that they are HIV positive are less likely to participate in HIV surveys
– Missing data handling will increase this effect as it’s based on most HIV negative
-Prevalence estimates will be unaccurate



#FutureOfData Podcast: Peter Morgan, CEO, Deep Learning Partnership

 #FutureOfData Podcast: Peter Morgan, CEO, Deep Learning Partnership

Subscribe to  Youtube


Data are becoming the new raw material of business. – Craig Mundie


Using Analytics to build A #BigData #Workforce

 Using Analytics to build A #BigData #Workforce


iTunes  GooglePlay


Facebook users send on average 31.25 million messages and view 2.77 million videos every minute.

Sourced from: Analytics.CLUB #WEB Newsletter

For the airline industry, big data is cleared for take-off

Tracking bags, personalizing offers, boosting loyalty, and optimizing operations are all goals of a renewed data-driven approach by major airlines.

When a customer checks into a flight with United Airlines UAL -0.87% , there is typically an array of potential add-on offers to navigate through: flight upgrades, access to the airline’s United Club, and more.

Under United’s old “collect and analyze” approach to data, the airline would use information about customers’ choices about those items, in aggregated fashion to “see what the most successful products were, and market with those [insights] in mind,” said Scott Wilson, the company’s vice president of e-commerce and merchandising.

That approach has changed. As of the beginning of this year, “collect, detect, act” is United’s new data-focused mantra, and it’s changing the way the airline serves its customers.

“Now we look at who the customer is and his or her propensity to buy certain products,” Wilson explained. More than 150 variables about that customer—prior purchases and previous destinations among them—are now assessed in real time to determine an individual’s likely actions, rather than an aggregated group of customers.

The result, delivered in about 200 milliseconds later, is a dynamically generated offer tailored to the individual. Its terms, on-screen layout, copy, and other elements will vary based on an individual’s collected data. For United, the refined approach led to an increase in year-over-year ancillary revenue of more than 15 percent, he said.

‘Airlines evolved big data’

Welcome to the big data era in the airline industry, which in many ways was one of its earliest participants.

“Airlines are awash in data, much of it unstructured,” said Bob Mann, an industry analyst with R.W. Mann & Co. But only recently have airlines been able to use big-data techniques “to solve, among other objectives, how to recognize and enhance customer value, and how to cultivate high-value customers,” he said.

“Airlines have always been very good at collecting data, but they haven’t always been good at using it,” United’s Wilson said. Now that the costs of storing and processing data have dropped—even as airlines collect more and more of it—it’s becoming easier for a company to act on it. At United, roughly a terabyte of customer data is floating around at any given time within its systems. “We don’t keep it all,” Wilson said. “We have to be selective about what we grab.” For the data that is selected, a real-time decision engine does the crunching to turn it into something useful.

It starts at the baggage carousel

One area in which the effects of big data technology are visible is in the handling of customers’ luggage. “We have over a number of years invested millions of dollars in baggage tracking,” said Paul Skrbec, a spokesman withDelta Air Lines. “That was one of those core, behind-the-scenes services for our customers.”

Millions of bags are checked each year with Delta DAL -1.46% —a total of 130 million are projected for 2014, Skrbec said—and “every customer has had the experience of boarding a plane after checking their bag and wondering if it was there.”

Through the use of hand-held baggage scanners used at passenger check-in, “we’ve had all this tracking data available,” Skrbec said. But “one of the things we realized about two years ago is that customers would benefit from having that information.”

Which is why Delta was the first major airline to launch an application allowing customers to track their bags from their mobile devices, he said. Spanning the iOS, Google Android, BlackBerry and Windows Phone mobile operating systems, the free app has been downloaded more than 11 million times.

In search of new revenue streams

It’s a similar story at Southwest Airlines  LUV -1.43% , which is using big data to determine which new customer services to implement.

“Southwest uses aggregated, anonymous customer data to promote products, services, and featured offers to customers on multiple channels, devices, and websites including,” said Dan Landson, a company spokesman. “By observing and looking into customer behaviors and actions online, we are better suited to offer our travelers the best rates and experiences possible. We also use this data to support the evolving relationships with our customers.”

For example, “we look at the city pairs that are being searched to help us determine what type of service we should have on a specific route,” Landson said.

The payoff? “Our customer and loyalty segments grow year-over-year,” Landson said. “We believe that intelligent, data-based targeting has a lot to do with that growth.”

‘$1 million per week’

The benefits of a data-focused approach may be easy to understand, but execution is another matter entirely. For most airlines, the first problem lies in “bringing together all sorts of disparate silos of passenger information—booking information from transaction systems, web and mobile behavior (including searches, visits, abandoned carts), email data, customer service info, etc.—to create a single, consolidated view of the customer,” said Allyson Pelletier, vice president of marketing with Boxever, which offers a marketing platform focused on putting big data to work for the travel industry.

“Armed with this information, and the resulting insights, they can then take specific action that helps them convert more visitors on-site, secure more revenue, or increase loyalty across any channel,” Pelletier said.

At Norwegian airline Wideroe, for example, a single customer view “enables agents in the call center to understand the full history of the customer—not just the customer service history, but also their recent visits to the website or promotional emails they’ve opened,” she explained.  “After they solve the customer service issue at hand, they’re in a powerful position to then recommend the most appropriate ancillary service—driving add-on revenue—or offer a complimentary upgrade, thereby driving loyalty.”

Insights garnered from a single customer view can also drive personalized messaging into various communications channels, and email is a popular starting place, Pelletier noted.

“One of our largest clients in Europe uses Boxever to understand abandoned carts and then trigger personalized emails to the abandoners,” she said. “They reported back subsequent bookings of $1 million per week from these communications.”

Boxever also cites a 21 percent reduction in customer-acquisition costs on paid media “by understanding who the customer was, where they came from and whether or not they were already a customer,” said Dave O’Flanagan, the company’s chief executive. “This way they could start to move those customers away from expensive acquisition channels to retention channels, like email, which is much cheaper.” There is also potential for a 17 percent uplift in conversion on ancillary cross-sells, such as adding hotel or car to a booking, he added.

‘Few companies are really leveraging big data’

Exciting though those benefits may be, there’s an even bigger pool of potential payoffs remaining untouched. “Surprisingly few [airline] companies are really leveraging big data today,” O’Flanagan said.

Indeed, “I’ve not seen a single major airline with an integrated ‘big data’ business solution, nor an airline with a plan to integrate such a program,” said Richard Eastman, founder and president of The Eastman Group, which builds travel software.

That depends on how one defines big data, however. “The airlines will tell you they ‘have it all’ without really knowing or understanding what ‘big data’ really is,” Eastman said. “Airline managements remain so focused on selling seats with their existing inventory systems that they have ignored buyer information needs as well as the tools that would enable them to reach out to buyers and travelers to serve those needs—let alone, reach buyers at decision-making moments.”

Marketing, flight operations and crew operations are all areas of rich opportunity, O’Flanagan said.

“I think there’s still a huge unmet need in the marketing and customer experience area,” he said. “Companies like Google are trying to be the ultimate assistant with technologies like Google Now. I think there’s a huge opportunity for airlines to create a helpful travel assistant that knows what I need before I do by combining data with mobile—helping people through airports, in-destination, right throughout the whole travel journey.

“Imagine a travel application that knows where I am, that I’m traveling with my family and that the weather is bad on our beach holiday. It could start to offer alternative itineraries close by that are family-friendly and not weather-dependent. These are truly valuable things an airline could do for me if they could use big data effectively and join the dots between me, my travel experience and environmental factors affecting that.

Originally posted via “For the airline industry, big data is cleared for take-off”

Originally Posted at: For the airline industry, big data is cleared for take-off