Month: April 2020

Reading Time: 3 minutesIn today’s dynamic and forward-moving tech-world, technologies and processes are getting obsolete faster than ever. What assumes the role of these discarded technologies is Artificial Intelligence, which is sometimes also the reason behind certain technologies and processes being phased out earlier than expected.

Software development is one such area that is getting affected by Artificial Intelligence and Machine Learning concepts. Let’s have a look at how AI is transforming software development to make it more agile, effective, and error-free.

Core areas of Software Engineering that AI will transform

– Design: Understanding the clients’ need and replicating it into a software design is a long-drawn process which requires multiple alternatives and re-iterations to finally land at a satisfactory and effective design. Artificial Intelligence is reducing this effort multi-fold by understanding and learning about the client’s needs and requirements and previously fuzzy areas such as brand personality to suggest a base design after scout millions of combinations. This virtual associate can offer opportune direction on the most proficient method to do design projects.

– Testing: Software testing is a critical process and takes up a lot of human hours before a fully functional and a (relatively) error-free software is handed over to a client. AI helps mitigate repetitive and trivial manual tests and gives more time to create new automated tests with sophisticated features. AI tools can create test information, judge its authenticity, and examines the scope of derived information for better test management. Artificial Intelligence, if trained correctly can ensure an error-free software and fast delivery of the project.

– GUI Testing: As important as it is, it is tough to carry out GUI testing due to the lack of dedicated GUI testing tools and techniques. GUI testing as of today is a bone-breaking process where test engineers have to manually create test cases, identify the scenarios to check, determine the timeline to check these scenarios, and ultimately evaluate if the GUI software is optimally tested. Moreover, even for the slightest modification to the software, the test suite needs to be adequately changed again for re-testing and ensuring the software is error-free. Test engineers are now banking upon AI to simplify and speed up this highly complicated and tedious job.

Source: 5 Ways AI is being applied in Software Development by administrator

The Wizard of Oz was a favorite childhood movie of mine, and one of my favorite characters was the Cowardly Lion. As a young child, I liked him because he was animated and goofy; as I got older, I was intrigued at the paradox of lacking courage to obtain courage. It’s a self-supporting dysfunction.

Many data scientists show the same absence of courage. Fortunately, there is something you can do to mitigate this expensive problem.

Aha, that’s what’s missing

Courage is one of those unexpected ingredients of data science that usually becomes an unpleasant aha moment for leaders in the middle of strategy implementation. Courage is the fuel for experimentation and, without experimentation, there is no data science.

But it’s more than experimentation in the scientific sense. To succeed, programmers need to take risks with code; mathematicians need to reinvent math; and data artists need to color outside of the lines. Intellectually, data scientists know this, but their personality doesn’t generally support taking risks, so they don’t.

One of my college professors said that MBA graduates don’t start their own business because they know better — the same theory follows with data scientists. They have the unique ability to calculate the risk of taking any route, and they’re smart enough to know when something probably won’t work. Data scientists also hate it when something they try doesn’t work (especially when there are others around to witness them get the wrong answer), so they don’t attempt to answer the question. You need to neutralize the downside.

The power of undo

Lack of courage (or fear) is the anxiety attached to an anticipated downside. People who are afraid of heights aren’t literally afraid of altitude — they’re afraid of what might happen if they suddenly lose altitude. For a data scientist, there’s a psychological fear of being wrong; however, there are some fundamental, tactical fears that must be addressed first — the strongest of which is losing work. You must install a failsafe mechanism that I’ll generally classify as the power of undo.

The power of undo comes in many forms and will be specific to your team, though here is some general advice: To mitigate the fear of losing work, you must work with your team on ways to step back when experimentation goes wrong. This means getting serious about testing infrastructure and software change control. You should always have automated methods of testing system. You should also build functionality in small increments, so the most recent working version is from yesterday rather than six months ago. The confidence of having yesterday’s build as a failsafe goes a long way in mustering the courage to experiment.

The courage to be courageous

Although the power of undo is very important, it pales in comparison to consequences that reinforce experimentation and all the other behaviors that demonstrate courage. This is where most leaders get it wrong.

Leaders spend most of their time and energy in setup (change control system, automated test harness, team building) and then very little time in reinforcement mechanisms. And yet, research shows that only 20% of behavioral influence comes from the antecedent (i.e., what prompts the behavior) and 80% comes from the consequence (i.e., what reinforces the behavior). So spend some upfront time on the power of undo, but then focus the majority of your effort on reinforcing courageous behavior.

To do that, be clear about the behaviors you expect. Remember, courage cannot be directly observed, but the behaviors that demonstrate courage can.

Is your team experimenting or playing it safe? Are they attempting bold leaps in functionality, or are they inching their way to the solution? Are they restoring from previous builds? Normally, reverting to a previous build has a negative effect, so it’s important that you flip this around by making it a positive effect.

Instead of criticizing the team for wasting time on an idea that didn’t work out, celebrate them for making a bold move. Don’t forget that once in a while those bold moves pay off and wind up saving months of development time.

Summary

Courage is unnatural for a data scientist, let alone a team of data scientists, so it must be nurtured and developed. It took a special medallion from the Wizard of Oz for the Cowardly Lion to find his courage. Your medallion of courage is the power of undo and the positive consequences that reinforce risky behaviors. Offer this to your data science team, and welcome the brave new world.

Originally posted at: http://www.techrepublic.com/article/what-the-cowardly-lion-and-data-scientists-have-in-common/

Originally Posted at: What the Cowardly Lion and data scientists have in common by analyticsweekpick

We’ve all dreamed of going to bed one day and waking up the next with superpowers – stronger, faster and even perhaps with the ability to fly.  Yesterday that is exactly what happened to Tom Reilly and the people at Cloudera and Hortonworks.   On October 2^nd they went to bed as two rivals vying for leadership in the big data space. In the morning they woke up as Cloudera 2.0, a $700M firm, with a clear leadership position.  “From the edge to AI”…to infinity and beyond!  The acquisition has made them bigger, stronger and faster. 

Like any good movie, however, the drama is just getting started, innovation in the cloud, big data, IoT and machine learning is simply exploding, transforming our world over and over, faster and faster.  And of course, there are strong villains, new emerging threats and a host of frenemies to navigate.

What’s in Store for Cloudera and Hortonworks 2.0

Overall, this is great news for customers, the Hadoop ecosystem and the future of the market.  Both company’s customers can now sleep at night knowing that the pace of innovation from Cloudera 2.0 will continue and accelerate.  Combining the Cloudera and Hortonworks technologies means that instead of having to pick one stack or the other, now customers can have the best of both worlds. The statement from their press release “From the Edge to AI” really sums up how complementary some of the investments that Hortonworks made in IoT complement Cloudera’s investments in machine learning.  From an ecosystem and innovation perspective, we’ll see fewer competing Apache projects with much stronger investments.  This can only mean better experiences for any user of big data open source technologies.

At the same time, it’s no secret how much our world is changing with innovation coming in so many shapes and sizes.  This is the world that Cloudera 2.0 must navigate.  Today, winning in the cloud is quite simply a matter of survival.  That is just as true for the new Cloudera as it is for every single company in every industry in the world.  The difference is that Cloudera will be competing with a wide range of cloud-native companies both big and small that are experiencing explosive growth.  Carving out their place in this emerging world will be critical.

The company has so many of the right pieces including connectivity, computing, and machine learning.  Their challenge will be, making all of it simple to adopt in the cloud while continuing to generate business outcomes. Today we are seeing strong growth from cloud data warehouses like Amazon Redshift, Snowflake, Azure SQL Data Warehouse and Google Big Query.  Apache Spark and service players like Databricks and Qubole are also seeing strong growth.  Cloudera now has decisions to make on how they approach this ecosystem and they choose to compete with and who they choose to complement.

What’s In Store for the Cloud Players

For the cloud platforms like AWS, Azure, and Google, this recent merger is also a win.  The better the cloud services are that run on their platforms, the more benefits joint customers will get and the more they will grow their usage of these cloud platforms.  There is obviously a question of who will win, for example, EMR, Databricks or Cloudera 2.0, but at the end of the day the major cloud players will win either way as more and more data, and more and more insight runs through the cloud.

Talend’s Take

From a Talend perspective, this recent move is great news.  At Talend, we are helping our customers modernize their data stacks.  Talend helps stitch together data, computing platforms, databases, machine learning services to shorten the time to insight. 

Ultimately, we are excited to partner with Cloudera to help customers around the world leverage this new union.  For our customers, this partnership means a greater level of alignment for product roadmaps and more tightly integrated products. Also, as the rate of innovation accelerates from Cloudera, our support for what we call “dynamic distributions” means that customers will be able to instantly adopt that innovation even without upgrading Talend.  For Talend, this type of acquisition also reinforces the value of having portable data integration pipelines that can be built for one technology stack and can then quickly move to other stacks.  For Talend and Cloudera 2.0 customers, this means that as they move to the future, unified Cloudera platform, it will be seamless for them to adopt the latest technology regardless of whether they were originally Cloudera or Hortonworks customers. 

You have to hand it to Tom Reilly and the teams at both Cloudera and Hortonworks.  They’ve given themselves a much stronger position to compete in the market at a time when people saw their positions in the market eroding.  It’s going to be really interesting to see what they do with the projected $125 million in annualized cost savings.  They will have a lot of dry powder to invest in or acquire innovation.  They are going to have a breadth in offerings, expertise and customer base that will allow them to do things that no one else in the market can do. 

The post Cloudera 2.0: Cloudera and Hortonworks Merge to form a Big Data Super Power appeared first on Talend Real-Time Open Source Data Integration Software.

Source by analyticsweekpick

I’m excited to announce the first look of the results of a “Global Wireless Satisfaction Survey.” Conducted in collaboration with Mob4Hire and myself in Feb 2010, the unprecedented 111-country survey analyzes the impact of mobile apps on operator’s churn (# of new customers acquired minus # of existing customers lost), as well as many dimensions of the app ecosystem as it relates to mobile user behavior and satisfaction.

You can have a look at the official press release here: http://www.prweb.com/releases/2010/03/prweb3727914.htm

The 8-page Executive Summary is available at Slideshare by clicking here or see below.

This research from this survey is significant from two standpoints:

1. It demonstrates that the mobile apps and ecosystem that network operators provide users are extremely important to their customer loyalty around the globe: 75% of people say mobile apps are important when choosing their new operator.
2. It demonstrates a great use of Mob4Hire’s distributed community of more than 40,000 people in 146 countries on 364 network operators.

We will be releasing additional results of the survey in coming weeks. Stay tuned.

Source: Business Over Broadway and Mob4Hire release first look at “Global Wireless Satisfaction Survey” by bobehayes

The title comes from Ronald Reagan, sometimes known as “The Great Communicator.” If he knew anything, he knew that simplicity was foundational when reaching a broad audience.

That doesn’t stop people from finding ways to overcomplicate their data visualizations. Take this example from The Data Viz Catalogue. It is called a Radial Bar Chart.

Radial Bar Chart

To their credit, The Data Viz Catalogue explains “While they look cool, the problem with Radial Bar Charts is that the bar lengths can be misinterpreted. Each bar on the outside gets relatively longer to the last, even if they represent the same value.”

Radial bars may be the simplest chart option for something, but not when showing trending values. Here, the data is crying out for a simple time-based bar chart.

 

The urge toward over-complication can be seen even for people who otherwise know better. As a fan of watching the Premier League, I often track match results with this graphic on ESPN.com.

It is two-thirds good. Can you see which part could be simpler? 

The table is the best option for comparing multiple metrics with different scales. Showing this as a bar chart would make it look like the fouls is far more important than any other game statistic.

 

Another simple visualization: The black and red bar combined with big value labels make it obvious who has the most possession.

 

Here’s where we run into problems — maybe driven by a need to fill-up extra space or desire to add some visual variety. The designer chose this boxes-inside-of-boxes concept to show shots on goal. Most of all, the use of area is misleading. The area of the boxes does not correspond with the associated value. It is actually the length (or height) that represents the shot value. By combining length and height into area, the visual is exaggerating differences. It is also confusing to interpret with the unlabeled two color shades in the boxes.

 

Here I am, having to explain how to interpret the chart. And I’m losing you.

Source

Reading Time: 10 minutesNatural Language Processing, abbreviated as NLP helps the machines understand and analyse natural languages. It is an automated process to extract required information from data by applying machine learning algorithms. 

While applying for job roles that deal with Natural Language Processing, it is often not clear to the applicants the kind of questions that the interviewer might ask. Apart from learning the basics of NLP, it is important to prepare specifically for the interviews. Here is a list of most common NLP interview questions and answers that you might face during an interview. 

1. Which of the following techniques can be used for keyword normalization, the process of converting a keyword into its base form?

a. Lemmatization

b. Soundex

c. Cosine Similarity

Source