What Big Data Analytics Professionals Want From IT

Big Data is where the paying jobs are. An update posted this week by Forbes Contributor Louis Columbus indicates that the “The [median] advertised salary for technical professionals with Big Data expertise is $104,850 net of bonuses and additional compensation.” And the demand is significant. He points out that IBM alone advertised more than 2300 such jobs in the past 12 months.

Many of these positions are in IT. That’s great for IT people, right? Not necessarily.

One of the many challenges associated with Big Data is the challenge of making some money from it. That often means taking advantage of data analytics (which comes in many varieties, including data mining, data science, web analytics or even good old statistics). For IT professionals who haven’t learned to cooperate with data analysts, these roles are a real test of skills and cultural adaptability.

In the course of more than a decade guiding organizations beginning or expanding the use of analytics, I have spoken with thousands of analysts and business managers. The majority of these people share a common complaint: they are unsatisfied with the level of support they are able to get from their IT organizations.

They view IT as a hindrance, an obstacle, something to work around. Many of them resort to secretive maneuvers, often violating policy to avoid cooperation, or even contact, with their own IT staff.

These analysts are not doing themselves any favors. Often, they end up working with datasets that are poorly prepared, undocumented and easily lost. Some report spending as much as 80-90% of their time on data preparation.
Photo courtesy of Wikipedia
Photo courtesy of Wikipedia

As analysts grow in numbers and influence, IT professionals must adapt their needs. In order to support data analysts well, you must understand what they need.

Analysts need large volumes of data.

Many roles call for the use of just a bit of data at a time. A customer service staff member may handle just one transaction at any given moment, or perhaps one customer’s transaction history. But analysts are constantly examining groups of transactions, groups of customers. It is common for analysts to work with data for thousands, even millions of individuals in a single analysis.

Analysts need detailed raw data.

Never dispose of data without a good reason. “Nobody will ever need that” is not a good reason. “All they’ll need is a roll-up” is not a good reason. I am routinely forced to tell business people that it is not possible to perform the analysis they desire because they have not saved historic data, or have kept only aggregated data.

Analysts need access to data during their working hours.

Data is the analyst’s raw material. No access to data, no work getting done. Overnight batch jobs do not cut it in the 21st century. Data analysis today is an interactive process.

Analysts need data that is current, complete, consistent and correct.

Data analysts spend most of their time on data prep. They would much prefer to spend most of their time on data analysis.

Analysts need data that is organized appropriately.

In most cases, data for analysts should be organized in flat tables, with rows representing individual people, accounts or transactions. This may not be the best approach for other uses, so you’ll need a way to bridge that gap.

Analysts need lots of computing power.

They often don’t know how to estimate their own requirements. This is your opportunity to be a hero, by acting as a go-between with vendor sales or tech support staff to scope those requirements.

Analysts need plenty of storage space.

Nobody wants to waste time or risk destroying something useful while culling files for lack of space.

Analysts need tools that can perform the analysis methods appropriate for the business requirements.

You may not always be able to procure each analyst’s favorite products, but you can always open the process for input and make an effort to understand concerns.

Analysts need help to organize, document and share their work.

Analysts produce valuable intellectual assets every day, but they aren’t very good at managing them. Make yourself a hero by making it easy to protect and exploit those assets.

Yes, there are discussions to be had, details to be worked out. Yet understanding these few points, and accepting them, is enough to make you a stand-out IT professional in the eyes of data analysts and the executives who look to them for business insights.

To read the original article on Forbes, click here.

Source: What Big Data Analytics Professionals Want From IT

Apr 26, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Data Accuracy  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> Nate-Silvering Small Data Leads to Internet Service Provider (ISP) industry insights by bobehayes

>> A Data-Driven Solution to the Data Scientist Shortage by analyticsweekpick

>> What Insurance companies could do to save others from eating their lunch by d3eksha

Wanna write? Click Here

[ NEWS BYTES]

>>
 Sell smarter: How data insights can help your team meet sales targets – EuropeanCEO Under  Talent Analytics

>>
 SAS Boosts AI Capabilities Of Upcoming Release Of Its Business Analytics Platform – CRN Under  Business Analytics

>>
 What can be done about the data science skills gap? – Siliconrepublic.com Under  Business Analytics

More NEWS ? Click Here

[ FEATURED COURSE]

Machine Learning

image

6.867 is an introductory course on machine learning which gives an overview of many concepts, techniques, and algorithms in machine learning, beginning with topics such as classification and linear regression and ending … more

[ FEATURED READ]

On Intelligence

image

Jeff Hawkins, the man who created the PalmPilot, Treo smart phone, and other handheld devices, has reshaped our relationship to computers. Now he stands ready to revolutionize both neuroscience and computing in one strok… more

[ TIPS & TRICKS OF THE WEEK]

Data Analytics Success Starts with Empowerment
Being Data Driven is not as much of a tech challenge as it is an adoption challenge. Adoption has it’s root in cultural DNA of any organization. Great data driven organizations rungs the data driven culture into the corporate DNA. A culture of connection, interactions, sharing and collaboration is what it takes to be data driven. Its about being empowered more than its about being educated.

[ DATA SCIENCE Q&A]

Q:Give examples of data that does not have a Gaussian distribution, nor log-normal?
A: * Allocation of wealth among individuals
* Values of oil reserves among oil fields (many small ones, a small number of large ones)

Source

[ VIDEO OF THE WEEK]

@Schmarzo @DellEMC on Ingredients of healthy #DataScience practice #FutureOfData #Podcast

 @Schmarzo @DellEMC on Ingredients of healthy #DataScience practice #FutureOfData #Podcast

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

Hiding within those mounds of data is knowledge that could change the life of a patient, or change the world. – Atul Butte, Stanford

[ PODCAST OF THE WEEK]

Using Analytics to build A #BigData #Workforce

 Using Analytics to build A #BigData #Workforce

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

Data is growing faster than ever before and by the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet.

Sourced from: Analytics.CLUB #WEB Newsletter

@AngelaZutavern & @JoshDSullivan @BoozAllen discussed Mathematical Corporation #FutureOfData – Playcast – Data Analytics Leadership Playbook Podcast

@AngelaZutavern & @JoshDSullivan, @BoozAllen discussing Mathematical Corporation #FutureOfData
@AngelaZutavern & @JoshDSullivan, @BoozAllen discussing Mathematical Corporation #FutureOfData

In this conversation Angela & Joshua from BoozAllen sat down with Vishal to discuss the Mathematical Corporation. They discussed some of the stories, challenges and opportunities facing any corporation on their journey to Mathematical corporation.

Their Book: The Mathematical Organization is out to help business stay data driven and competitive. You could download the book @ http://amzn.to/2hNsoaH

THE MATHEMATICAL CORPORATION: Where Machine Intelligence and Human Ingenuity Achieve the Impossible (PublicAffairs; June 6, 2017), by Booz Allen Hamilton machine intelligence experts Joshua Sullivan and Angela Zutavern, is the first book to show business leaders how to compete in this new era: by combining the mathematical smarts of machines with the intellect of visionary leaders.

About the Guest
DR. JOSH SULLIVAN is SVP at Booz Allen Hamilton. One of the world’s leading experts in data science and machine intelligence, he was among the first people to hold the title “data scientist.” He has appeared on CNBC.

ANGELA ZUTAVERN is VP at Booz Allen Hamilton and pioneered the application of machine intelligence to leadership and strategy. Together, they’re radically transforming how Fortune 500 companies, nonprofits and major government agencies perform by helping leaders shatter long-held constraints and reveal hidden truths in their organizations. They live in Washington, D.C.

Source by v1shal

Apr 19, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Insights  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> Caterpillar digs in to data analytics—investing in hot startup Uptake by analyticsweekpick

>> Predicting the Future and Shaping Strategy with HR Analytics by analyticsweekpick

>> List of VC firms in Boston by v1shal

Wanna write? Click Here

[ NEWS BYTES]

>>
 Consumers Energy wind farm powering GM plants, Switch data center – Crain’s Detroit Business Under  Data Center

>>
 Why deploy desktop and application virtualization? – TechTarget Under  Virtualization

>>
 What Small Businesses Should Know About AWS Machine Learning – Business News Daily Under  Machine Learning

More NEWS ? Click Here

[ FEATURED COURSE]

Process Mining: Data science in Action

image

Process mining is the missing link between model-based process analysis and data-oriented analysis techniques. Through concrete data sets and easy to use software the course provides data science knowledge that can be ap… more

[ FEATURED READ]

The Signal and the Noise: Why So Many Predictions Fail–but Some Don’t

image

People love statistics. Statistics, however, do not always love them back. The Signal and the Noise, Nate Silver’s brilliant and elegant tour of the modern science-slash-art of forecasting, shows what happens when Big Da… more

[ TIPS & TRICKS OF THE WEEK]

Save yourself from zombie apocalypse from unscalable models
One living and breathing zombie in today’s analytical models is the pulsating absence of error bars. Not every model is scalable or holds ground with increasing data. Error bars that is tagged to almost every models should be duly calibrated. As business models rake in more data the error bars keep it sensible and in check. If error bars are not accounted for, we will make our models susceptible to failure leading us to halloween that we never wants to see.

[ DATA SCIENCE Q&A]

Q:Explain the difference between “long” and “wide” format data. Why would you use one or the other?
A: * Long: one column containing the values and another column listing the context of the value Fam_id year fam_inc

* Wide: each different variable in a separate column
Fam_id fam_inc96 fam_inc97 fam_inc98

Long Vs Wide:
– Data manipulations are much easier when data is in the wide format: summarize, filter
– Program requirements

Source

[ VIDEO OF THE WEEK]

RShiny Tutorial: Turning Big Data into Business Applications

 RShiny Tutorial: Turning Big Data into Business Applications

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

Information is the oil of the 21st century, and analytics is the combustion engine. – Peter Sondergaard

[ PODCAST OF THE WEEK]

Understanding Data Analytics in Information Security with @JayJarome, @BitSight

 Understanding Data Analytics in Information Security with @JayJarome, @BitSight

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

Poor data can cost businesses 20%–35% of their operating revenue.

Sourced from: Analytics.CLUB #WEB Newsletter

Inovalon’s Next Generation Big Data Platform Solution Achieves NCQA Measure Certification

Big Data Processing and Advanced Analytics Deliver Unprecedented Speed and Real-Time Insights for Improved Clinical and Quality Outcomes.

BOWIE, Md., March 19, 2015 (GLOBE NEWSWIRE) — Inovalon (Nasdaq:INOV), a leading technology company providing advanced cloud-based data analytics and data-driven intervention platforms to the healthcare industry, today announced that on March 16, 2015, the next generation of big data platform for Healthcare Effectiveness Data and Information Set (HEDIS®) quality measurement and reporting, Inovalon’s Quality Spectrum Insight (QSI®-XL) solution, received National Committee for Quality Assurance (NCQA) Measure Certification for HEDIS® 2015.

The QSI®-XL platform, the core analytics engine within Inovalon’s HEDIS Advantage™ solution, leverages big data processing with the industry’s most robust analytics platform available, further enhancing the industry-leading solution utilized by more than two-thirds of the nation’s outsourced quality measurement initiatives. Inovalon’s advanced technology architecture delivers unprecedented processing times that enable accelerated decision-making and speed-to-impact to inform quality analysis and improvement programs – achieving dramatic improvements in data aggregation simplicity and greater than 15 times processing speed capabilities as compared to the company’s previously available industry-leading technologies.

“Inovalon’s advanced cloud-based technologies continue to transform the healthcare industry,” said Joseph Rostock, chief technology officer at Inovalon. “As the industry shifts to quality- and value-based outcome models, the ability to efficiently and rapidly integrate and analyze large sets of data has become increasingly critical for healthcare organizations to be successful. Inovalon drives superior value for our clients by providing the timely, granular insights that health plans, Accountable Care Organizations, provider organizations, and employer groups need to improve quality initiatives, while solving the challenges of volume and disparate sources of healthcare and patient information.”

Prior to the rise in quality-driven care, the aggregation and analysis of quality measurement data was predominantly undertaken on an annual basis – submitted to oversight bodies, state, and federal regulatory agencies to fulfill licensing and accreditation requirements. With the dramatic rise in performance-based incentives across the U.S. healthcare landscape, programs such as the Five-Star Quality Rating System for Medicare Advantage plans, state Medicaid quality programs such as New York’s Quality Assurance Reporting Requirements (QARR) program, the Affordable Care Act’s Quality Rating System (QRS) program, Medicare Shared Savings Program for Accountable Care Organizations (ACOs), and a host of other programs, the need for advanced sophistication in quality data aggregation and analysis has risen significantly. These programs now drive billions of dollars in performance incentives. As a result, adjacent constituents of the incentivized market are becoming similarly subject to the same market forces through shared-risk and share-savings agreements. The ultimate impact is broadening the influence of quality- and value-based incentives to extend beyond the health plan marketplace to include hospitals, providers, pharma/life sciences, and even device manufacturers.

Further emphasizing the speed and significance of the market’s transition to quality- and value-based incentives within healthcare, on January 26, 2015, the U.S. Health and Human Services (HHS) announced its intention to transition 50 percent of its payments (or approximately $215 billion) to quality-based payments in Medicare by 2018, with even higher percentage goals in the years shortly thereafter.

“Providing the industry’s most robust and widely used quality analytics platform positions Inovalon well to serve its clients and the market in the rapidly expanding need for quality data aggregation and analytics,” said Dan Rizzo, Inovalon’s chief innovation officer. “We believe that the ability for Inovalon to receive massive amounts of data easily within its data lake repository, and process this data in near real-time, providing clients not only with up-to-date insights into their patients, provider and population quality status, but also to serve as a critical element of their quality improvement program, is truly unique in the marketplace.”

Built upon Inovalon’s QSI® engine, Inovalon’s HEDIS Advantage™ platform performs the analytics for more than 1200 Interactive Data Submission System (IDSS) submissions annually for leading health plans across the U.S. to the NCQA, the Centers for Medicare and Medicaid Services (CMS), the Utilization Review Accreditation Commission (URAC), and other state regulatory agencies. The majority of HEDIS Advantage™ users access these capabilities via Inovalon’s cloud-based platform, with the balance of users leveraging installed software versions. As of the date of this release, QSI®-XL is already in place and operating with healthcare organization’s NCQA HEDIS® reporting and quality improvement programs. Going forward, both QSI® and QSI®-XL solutions will be offered by Inovalon with clients selecting their desired platform based on dataset size and compute speed needs.

In addition to supporting client needs in quality data aggregation and analytics, the QSI®-XL engine will replace QSCL®, Inovalon’s prior-generation big data quality measurement platform, as a critical component of Inovalon’s predictive analytics platform, Star Advantage®, which identifies future gaps in quality and optimal venue, timing, modality, and content for resolving such identified gaps in quality through the utilization of Inovalon’s data-driven intervention platform. With Inovalon’s arsenal of cloud-based quality analytics tools, clients have the ability to uniquely understand and improve their quality score in near-real time – a capability that has demonstrated improved quality rates by 300 percent compared to populations unaided by such a technology.

The QSI®-XL engine additionally enables the application of customer analytical modules, created through Inovalon’s proprietary Quality Spectrum Flowchart Designer (QSFD®), across massive datasets at dramatically accelerated speeds. QSFD® allows non-technologists to design proprietary algorithms through easy-to-use “drag and drop” graphical interfaces, compiling logic routines for application within the cloud, minutes after creation. These capabilities allow a diverse array of data-users, from internal product designers to device manufactures and pharma/life science researchers, to design unique analysis and apply them against Inovalon’s large-scale datasets in seconds and minutes.

In addition to the NCQA Measure Certification received by Inovalon’s QSI®-XL big data quality analytics engine on March 16th, 2015, Inovalon’s Quality Spectrum Insight (QSI®) software engine (version 18) has also received NCQA Measure Certification for HEDIS® 2015. Inovalon’s QSI® software engine has now received NCQA’s Measure Certification for HEDIS® every year, since 2001. Going forward, both QSI® and QSI®-XL solutions will be offered to Inovalon’s clients.

NCQA is an independent, not-for-profit organization dedicated to improving health care quality. NCQA accredits and certifies a wide range of health care organizations and manages the evolution of HEDIS®, the performance measurement tool used by more than 90 percent of the nation’s health plans. HEDIS® is a registered trademark of the NCQA.

About Inovalon

Inovalon is a leading technology company that combines advanced cloud-based data analytics and data-driven intervention platforms to achieve meaningful insight and impact in clinical and quality outcomes, utilization, and financial performance across the healthcare landscape. Inovalon’s unique achievement of value is delivered through the effective progression of Turning Data into Insight, and Insight into Action®. Large proprietary datasets, advanced integration technologies, sophisticated predictive analytics, data-driven intervention platforms, and deep subject matter expertise deliver a seamless, end-to-end capability that brings the benefits of big data and large-scale analytics to the point of care. Driven by data, Inovalon uniquely identifies gaps in care, quality, data integrity, and financial performance – while bringing to bear the unique capabilities to resolve them. Providing technology that supports hundreds of healthcare organizations in 98.2% of U.S. counties and Puerto Rico, Inovalon’s cloud-based analytical and data-driven intervention platforms are informed by data pertaining to more than 754,000 physicians, 248,000 clinical facilities, and more than 120 million Americans providing a powerful solution suite that drives high-value impact, improving quality and economics for health plans, ACOs, hospitals, physicians, consumers and pharma/life-sciences researchers. For more information, visitwww.inovalon.com.

CONTACT: Inovalon
         Kim E. Collins
         4321 Collington Road
         Bowie, Maryland 20716
         Phone: 301-809-4000
         kimecollins@inovalon.com
         
         Greenough, on behalf of Inovalon
         Andrea LePain
         Phone: 617-275-6526
         alepain@greenoughcom.com

Originally posted via "Inovalon's Next Generation Big Data Platform Solution Achieves NCQA Measure Certification"

Originally Posted at: Inovalon’s Next Generation Big Data Platform Solution Achieves NCQA Measure Certification by analyticsweekpick

Why big data matters to Boeing, and what it means for your next flight

he name Boeing typically conjures images of airplanes, defense systems and space capsules.

But with 8,000 to 10,000 sensors on each aircraft Boeing manufactures, the world’s largest aerospace company is piloting itself into a new realm, one where it harnesses massive amounts of data to improve airplane fuel efficiency, speed up delivery of parts and even help pick better flight paths so you can get to your destination on time.

Harish Rao, director of business analytics at Boeing, offered a rare glimpse into the company’s big data efforts at the Technology Alliance’s Insights to Impactconference in Seattle on Tuesday.

According to Rao, the analysis of data — along with a “lean” development process — is becoming an important component of Boeing.

“We have been a really process driven company, but we are really starting to transform more into a data and insight culture,” said Rao. For example, Boeing is working alongside customers like Alaska Airlines to provide a full-package of real-time data analytics, offering everything from how to optimize fuel management to the best routes to fly during certain weather conditions.

“On a plane where we have 8,000 sensors capturing the 8,000 data points per second … if we extrapolate that for more than 5,000 planes … and optimizing that and providing sort of real-time optimization, (that) is where there is a huge benefit for our customers,” said Rao. “But it is also a great opportunity for our company as far as a revenue generation standpoint.”

In total, Rao said Boeing is sitting on a treasure trove of about 100 Petabytes of data, and now the company is looking to unlock that in new ways, benefitting its carrier customers and future flyers.

Harish Rao
Harish Rao

Of course, when you’re dealing with the life and death scenario of flight, lean development principles and the use of data to make key decisions takes on added weight. Asked what keeps him awake at night, Rao noted that is is the “accuracy of the predictions” that Boeing’s data scientists are working on.

He contrasted the importance of the Boeing predictions with those developed at online retailers, noting that if you don’t happen to buy a recommended book “nothing happens.”

“But, if I say you can fly another 2,000 miles, and the fuel ends in 1,000 miles, well, guess what? The plane will crash, and everyone will die,” said Rao. “That is a very different scenario.”

Certainly, the life or death situations faced by Boeing make for an interesting challenge when it comes to making sense of data. While some non-aerospace companies may take an 80-20 approach to developing and testing new ideas — hoping 80 percent turn into a success — Boeing employs a “99-1 approach” on its new ideas. As data scientists at Boeing mine information, he said the key is to strive to “get very accurate results and yet be agile and nimble, because typically they don’t go hand-in-hand.”

That mentality represents a huge culture change at Boeing, which was founded in 1916. Rao said that the mindset really started to shift in earnest about two years ago at the company, adding that it is not “easy to change the culture or the direction.” He said most of the data science analysis that his team is working on takes place in a six to eight week timeframe, compared to more than a year previously.

“Value is being realized on a very short time frame,” he said.

Even though companies like Facebook or Google or Amazon.com can move faster, Boeing said it is working hard to embrace a data-driven culture, one which mirrors that of a startup. Now, he said Boeing often proactively looks to “understand what the data is telling us” — rather than waiting for a customer to bring up a problem, forcing engineers to find a solution.

For example, Boeing is working hard with its carrier partners to make sure that the airplanes are constantly in flight, since the biggest loss for an airline occurs when it is parked at a gate or sitting in a maintenance hangar.

“The best time for an airline is when it is in the air. If it could be in the air for 24 hours, they’d probably prefer it, but unfortunately that is not a reality,” he said. “From that standpoint, it is really optimizing the route, so you can make sure that there is minimal downtime, as well as the data can tell us, based on what some of their other competitors are flying in that area, what is the connection they want to look at or the weather patterns they have. We have data coming in from a variety of these different avenues, and we package them up and provide a valuable insight. But the airlines that actually take that, deploy it … and really use it on a daily basis, you can clearly see them coming out as significant winners, almost to the tune of 10 percent to 20 percent improvement in margins.”

As it relates to the comfort of passengers on commercial flights, Rao said that airlines are making strides in how they utilize data to enhance the customer experience.

“They are starting to look at, based on the data, coming up with complete personalization,” he said. “Really understanding from the customer who has put in their flight information It is almost an experience, so the types of channels, the types of music, the variety of information that is available, for say an international flight, is all customized based on your behavior or your profile. Some airlines try to order food you might want, so if you like Sushi or something else. We have the data, and we share the data. It is just about how best do you use it.”

John Cook is GeekWire’s co-founder and editor, a veteran reporter and the longest-serving journalist on the Pacific Northwest tech startup beat. Follow him @johnhcookand email john@geekwire.com.

Originally posted via “Why big data matters to Boeing, and what it means for your next flight”

Originally Posted at: Why big data matters to Boeing, and what it means for your next flight

Apr 12, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Weak data  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> Automating Data Modeling for the Internet of Things: Accelerating Transformation and Data Preparation by jelaniharper

>> The Blueprint for Becoming Data Driven: Data Quality by jelaniharper

>> Why every flight you take is obsessively monitored by analyticsweekpick

Wanna write? Click Here

[ NEWS BYTES]

>>
 Macy’s earnings-driven stock spike was costly for traders fearing the retail apocalypse – Business Insider Australia Under  Financial Analytics

>>
 The Industry Pulse – DSNews.com Under  Risk Analytics

>>
 How to make machine learning systems work in the network – TechTarget Under  Machine Learning

More NEWS ? Click Here

[ FEATURED COURSE]

CS229 – Machine Learning

image

This course provides a broad introduction to machine learning and statistical pattern recognition. … more

[ FEATURED READ]

The Industries of the Future

image

The New York Times bestseller, from leading innovation expert Alec Ross, a “fascinating vision” (Forbes) of what’s next for the world and how to navigate the changes the future will bring…. more

[ TIPS & TRICKS OF THE WEEK]

Data aids, not replace judgement
Data is a tool and means to help build a consensus to facilitate human decision-making but not replace it. Analysis converts data into information, information via context leads to insight. Insights lead to decision making which ultimately leads to outcomes that brings value. So, data is just the start, context and intuition plays a role.

[ DATA SCIENCE Q&A]

Q:What is collinearity and what to do with it? How to remove multicollinearity?
A: Collinearity/Multicollinearity:
– In multiple regression: when two or more variables are highly correlated
– They provide redundant information
– In case of perfect multicollinearity: ?=(XTX)?1XTy doesn’t exist, the design matrix isn’t invertible
– It doesn’t affect the model as a whole, doesn’t bias results
– The standard errors of the regression coefficients of the affected variables tend to be large
– The test of hypothesis that the coefficient is equal to zero may lead to a failure to reject a false null hypothesis of no effect of the explanatory (Type II error)
– Leads to overfitting

Remove multicollinearity:
– Drop some of affected variables
– Principal component regression: gives uncorrelated predictors
– Combine the affected variables
– Ridge regression
– Partial least square regression

Detection of multicollinearity:
– Large changes in the individual coefficients when a predictor variable is added or deleted
– Insignificant regression coefficients for the affected predictors but a rejection of the joint
hypothesis that those coefficients are all zero (F-test)
– VIF: the ratio of variances of the coefficient when fitting the full model divided by the variance of the coefficient when fitted on its own
– rule of thumb: VIF>5 indicates multicollinearity
– Correlation matrix, but correlation is a bivariate relationship whereas multicollinearity is multivariate

Source

[ VIDEO OF THE WEEK]

#BigData @AnalyticsWeek #FutureOfData #Podcast with Eloy Sasot, News Corp

 #BigData @AnalyticsWeek #FutureOfData #Podcast with Eloy Sasot, News Corp

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

It is a capital mistake to theorize before one has data. Insensibly, one begins to twist the facts to suit theories, instead of theories to

[ PODCAST OF THE WEEK]

Ashok Srivastava(@aerotrekker @intuit) on Winning the Art of #DataScience #FutureOfData #Podcast

 Ashok Srivastava(@aerotrekker @intuit) on Winning the Art of #DataScience #FutureOfData #Podcast

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

According to estimates, the volume of business data worldwide, across all companies, doubles every 1.2 years.

Sourced from: Analytics.CLUB #WEB Newsletter

What Insurance companies could do to save others from eating their lunch

What Insurance companies could do to save others from eating their lunchHistorically, insurance is considered as a slow paced industry as it is heavily regulated and is not one of the first followers in technology adoption. Insurance as an industry is facing unprecedented changes in the market. In addition to the macro economic factors like low GDP growth, more natural disasters over last year, slow employment growth etc, consolidation and integration of channels of distribution of insurance has caused a stir in the marketplace. The changes in the consumer expectations, technological advancements and competitive pressure (low cost of entry for other players) has further exacerbated this pressure. If the leading companies in the business are not able to change, they might lose out to the competition.
Big insurers are trying to preserve profitability and increase market share.
Current market condition and changing regulations have sent other closely aligned businesses like banks, retailers and credit card companies to expand their offerings to include insurance products. With lower barrier to entry on the technological front, many smaller companies have also joined the insurance club. This has further complicated the competitive landscape.

So, now, it is all the more crucial for large insurance providers to be innovative and agile. As with everything, they also could save themselves from this turmoil. Few recommendations that could help them are:

Listing 5 important changes that could save big insurance companies and help them ride smoothly with the tide:

Innovate Fast:
As stated above, this is one of the handicap for most insurance companies. Some companies are faring better than others but still there is an opportunity to improve. Innovation is a key driver that keeps companies ahead of the curve, helps them get the best tools to get the job done, find ways to do things optimally and at lower costs. So, it is  important for insurance companies to build a system that could help them learn, deploy and validate innovations fast. There is no magic bullet as to how that system could be built. It takes sincere commitment from leadership, and synchronized effort from internal trained staff to pursue and strategize innovation. Once done right this program could easily pay for itself.

Reaching out where most of eyes are:
It is not a surprise that businesses wants to stay as close to their customers as possible. Another challenge with industries like insurance is the number of times customers interact with their product or service. For insurance, it is mostly limited to purchase, renewal, cancellations and accidents. So, insurance companies should build use cases that brings them closer to users/prospects and increases the frequency of those interactions. It means building or partnering with apps, social initiatives and lifestyle businesses. TFor example – Partner with social media apps, social gaming apps , build strategic alliance with banks and bundle services to engage more often with the consumers. This will not only engage the consumers and create credibility and trust, but also provide ease to the consumers by providing customized bundled solution at the places where they hang out.

Investing in building strong customer experience that delivers consistent value:
As stated earlier, staying closer to customer is the most important thing for businesses like insurance. Banking and other similar services and products set a high bar on customer experience as a differentiation. This should not come as a surprise as almost every insurance campaign has a central slogan of being a “customer centric” organization. So, they should invest every bit into improving customer experience. This not only includes providing WOW service today but also learning from it and improvising it. Insurance companies should implement a robust and scalable “Voice of Customer” program. This will help companies understand customer expectation and align closer to it. So, it is a win-win area for both business and customers and therefore, should be given due attention.

Listen hard and act fast:
It resonates pretty well with some of previous suggestions. To innovate, to provide extraordinary service and to build a stronger edge ,it is extremely important for companies to listen hard and act quickly. This is often part of innovation initiative where systems are build, frameworks are placed that facilitates quick listening.  Customer feedback could come from direct channels like surveys or indirect ones like logs. But every bit of information helps to understand the expectation. Being savvier at this implies moving organizations quickly to resonate with customer expectations. There is no better key to success than having a business cater to its customer’s expectation. So, this results in improving retention, NPS score, lead generation and loyalty. Something every business is craving for in this economy.

Better data management:
Insurance industry is still plagued with legacy systems, that are not easier to mine and that do not provide data in real time for any analysis and action. Having stayed in business for years means that they have loads of data that is gold. So, harnessing it should not come as a surprise. Its important for big companies to manage their data. Data management is crucial in this age of big-data and should be taken seriously by insurance companies. Nothing is better than learning about strengths, weaknesses, opportunities and threats in real time using data and without investing too much effort into direct engagement with customers. Data captured could be direct or indirect data from customers and it could be used for predictive modeling, identifying product-market fit, understand loyalty, key business drivers and  other drivers impacting business.

So, with the right leadership and clear mindset, a sustainable strategy could be carved that will help big insurance giants to stay on top of their technological curve and stay close to their customer despite being a 800 pound gorilla in the room.

 

Source: What Insurance companies could do to save others from eating their lunch

@TimothyChou on World of #IOT & Its #Future Part 1 #FutureOfData #Podcast

[youtube https://www.youtube.com/watch?v=ezNX6XYozIc]

In this first part of two part podcast @TimothyChou discussed the Internet of Things landscape. He laid out how internet has always been about internet of things and not internet of people. He sheds light on internet of things as it is spread across themes of things, connect, collect, learn and do workflows. He builds an interesting case about achieving precision to introduction optimality.

 

Timothy’s Recommended Read:
Life 3.0: Being Human in the Age of Artificial Intelligence by Max Tegmark http://amzn.to/2Cidyhy
Zone to Win: Organizing to Compete in an Age of Disruption Paperback by Geoffrey A. Moore http://amzn.to/2Hd5zpv

Podcast Link:
iTunes: http://math.im/itunes
GooglePlay: http://math.im/gplay

Timothy’s BIO:
Timothy Chou has his career spanning through academia, successful (and not so successful) startups and large corporations. He was one of only a few people to hold the President title at Oracle. As President of Oracle On Demand he grew the cloud business from it’s very beginning. Today that business is over $2B. He wrote about the move of applications to the cloud in 2004 in his first book, “The End of Software”. Today he serves on the board of Blackbaud, a nearly $700M vertical application cloud service company.

After earning his PhD in EE at the University of Illinois he went to work for Tandem Computers, one of the original Silicon Valley startups. Had he understood stock options he would have joined earlier. He’s invested in and been a contributor to a number of other startups, some you’ve heard of like Webex, and others you’ve never heard of but were sold to companies like Cisco and Oracle. Today he is focused on several new ventures in cloud computing, machine learning and the Internet of Things.

About #Podcast:
#FutureOfData podcast is a conversation starter to bring leaders, influencers and lead practitioners to come on show and discuss their journey in creating the data driven future.

Wanna Join?
If you or any you know wants to join in,
Register your interest @ http://play.analyticsweek.com/guest/

Want to sponsor?
Email us @ info@analyticsweek.com

Keywords:
#FutureOfData #DataAnalytics #Leadership #Podcast #BigData #Strategy

Source by admin