What are Decision Trees in Machine Learning (Classification And Regression)

by | Dec 26, 2019 | Uncategorized | 0 comments

All Premium Themes And WEBSITE Utilities Tools You Ever Need! Greatest 100% Free Bonuses With Any Purchase.

Greatest CYBER MONDAY SALES with Bonuses are offered to following date: Get Started For Free!
Purchase Any Product Today! Premium Bonuses More Than $10,997 Will Be Emailed To You To Keep Even Just For Trying It Out.
Click Here To See Greatest Bonuses

and Try Out Any Today!

Here’s the deal.. if you buy any product(s) Linked from this sitewww.Knowledge-Easy.com including Clickbank products, as long as not Google’s product ads, I am gonna Send ALL to you absolutely FREE!. That’s right, you WILL OWN ALL THE PRODUCTS, for Now, just follow these instructions:

1. Order the product(s) you want by click here and select the Top Product, Top Skill you like on this site ..

2. Automatically send you bonuses or simply send me your receipt to consultingadvantages@yahoo.com Or just Enter name and your email in the form at the Bonus Details.

3. I will validate your purchases. AND Send Themes, ALL 50 Greatests Plus The Ultimate Marketing Weapon & “WEBMASTER’S SURVIVAL KIT” to you include ALL Others are YOURS to keep even you return your purchase. No Questions Asked! High Classic Guaranteed for you! Download All Items At One Place.

That’s it !

*Also Unconditionally, NO RISK WHAT SO EVER with Any Product you buy this website,

60 Days Money Back Guarantee,

IF NOT HAPPY FOR ANY REASON, FUL REFUND, No Questions Asked!

Download Instantly in Hands Top Rated today!

Remember, you really have nothing to lose if the item you purchased is not right for you! Keep All The Bonuses.

Super Premium Bonuses Are Limited Time Only!

Day(s)

:

Hour(s)

:

Minute(s)

:

Second(s)

Get Paid To Use Facebook, Twitter and YouTube
Online Social Media Jobs Pay $25 - $50/Hour.
No Experience Required. Work At Home, $316/day!
View 1000s of companies hiring writers now!
Order Now!

MOST POPULAR

*****
Customer Support Chat Job: $25/hr
Chat On Twitter Job - $25/hr
Get Paid to chat with customers on
a business’s Twitter account.
Try Free Now!
Get Paid To Review Apps On Phone
Want to get paid $810 per week online?
Get Paid To Review Perfect Apps Weekly.
Order Now!
Look For REAL Online Job?
Get Paid To Write Articles $200/day
View 1000s of companies hiring writers now!
Try-Out Free Now!

How To Develop Your Skill For Great Success And Happiness Including Become CPA? | Additional special tips From Admin

Talent Progression is actually the number 1 essential and chief consideration of getting real achieving success in all of professions as anyone witnessed in the modern culture as well as in Around the world. For that reason fortuitous to talk about together with you in the right after with regards to what productive Competence Progression is; the best way or what strategies we deliver the results to realize wishes and gradually one will probably deliver the results with what someone prefers to conduct every day regarding a full daily life. Is it so terrific if you are in a position to establish resourcefully and obtain being successful in everything that you dreamed, steered for, picky and worked well very hard just about every single day and without doubt you turn into a CPA, Attorney, an master of a considerable manufacturer or even a medical professionsal who may remarkably play a role awesome support and values to other individuals, who many, any modern culture and community clearly esteemed and respected. I can's believe that I can help others to be leading expert level who seem to will lead sizeable systems and elimination values to society and communities right now. How completely happy are you if you develop into one just like so with your personal name on the title? I have landed at SUCCESS and defeat all the hard areas which is passing the CPA examinations to be CPA. What's more, we will also handle what are the risks, or various complications that is perhaps on the approach and the simplest way I have in person experienced all of them and will indicate you tips on how to overcome them. | From Admin and Read More at Cont'.

What are Decision Trees in Machine Learning (Classification And Regression)

Machine Learning is an interdisciplinary field of study and is a sub-domain of Artificial Intelligence. It gives computers the ability to learn and infer from a huge amount of homogeneous data, without having to be programmed explicitly.

Introduction to Machine Learning and types of Machine Learning

Types of Machine Learning: Machine Learning can broadly be classified into three types:

Supervised machine learning

Unsupervised machine learning

Reinforcement machine learning

Classification is the process of determination/prediction of the category to which a data-point may belong to. It is the process by which a Supervised Learning Algorithm learns to draw inference from the features of a given dataset and predict which class or group or category does the particular data point belongs to.

Example of Classification: Let’s assume that we are given a few images of handwritten digits (0-9). The problem statement is to “teach” the machine to classify correctly which image corresponds to which digit. A small sample of the dataset is given below:

Example of Classification

The machine has to be thus trained, such that, when given an input of any such hand-written digit, it has to correctly classify the digits and mention which digit the image represents. This is classed classification of hand-written digits.

Looking at another example which is not image-based, we have 2D data (x1 and x2) which is plotted in the form of a graph shown below.

2D data example of Classification in Machine Learning

The red and green dots represent two different classes or categories of data. The main goal of the classifier is that given one such “dot” of unknown class, based on its “features”, the algorithm should be able to correctly classify if that dot belongs to the red or green class. This is also shown by the line going through the middle, which correctly classifies the majority of the dots.

Applications of Classification: Listed below are some of the real-world applications of classification Algorithms.

Regression is also a type of supervised learning. Unlike classification, it does not predict the class of the given data. Instead, it predicts the corresponding values of a given dataset based on the “features” it encounters.

Example of Regression: For this, we will look at a dataset consisting of California Housing Prices. The contents of this dataset are shown below.

Regression

Here, there are several columns. Each of the columns shows the “features” based on which the machine learning algorithm predicts the housing price (shown by yellow highlight). The primary goal of the regression algorithm is that, given the features of a given house, it should be able to correctly estimate the price of the house. This is called a regression problem. It is similar to curve fitting and is often confused with the same.

Applications of Regression: Listed below are some of the real-world applications of regression Algorithms.

In order to get started with Decision Trees, it is important to understand the basic building blocks of decision trees. Hence, we start building the concepts slowly with some basic theory.

Definition: It is a commonly used concept in Information Theory and is a measure of “purity” of an arbitrary collection of information.

Mathematical Equation:
Entropy Formula

Here, given a collection S, containing positive and negative examples, the Entropy of S is given by the above equation, where, p represents the probability of occurrence of that example in the given data.

In a more generalized form, Entropy is given by the following equation:

Generalised form of Entropy equation

Example: As an example, a sample S is taken, which contains 14 data samples and includes 9 positive and 5 negative samples. The same is denoted by the mathematical notion: [9+, 5­­–].

Thus, Entropy of the given sample can be calculated as follows:
Entropy Calculation

Definition: With the knowledge of Entropy, the amount of relevant information that is gained form a given random sample size can be calculated and is known as Information Gain.

Mathematical Equation:

Gain formula with Entropy

Here, the Gain (S, A) is the Information Gain of an attribute A relative to a sample S. The Values(A) is a set of all possible values for attribute A.

Example: As an example, let’s assume S is a collection of 14 training-examples. Here, in this example, we will consider the Attribute to be Wind and the values of that corresponding attribute will be Weak and Strong. In addition to the previous example information, we will assume that out of the previously mentioned 9 positives and 5 negative samples, 6 positive and 2 negative samples have the value of the attribute Wind=Weak, and the remaining have Wind=Strong. Thus, under such a circumstance, the information gained by the attribute Wind is shown below.

Finding Gain value from Entropy equations

Introduction: Since we have the basic building blocks out of the way, let’s try to understand what exactly is a Decision Tree. As the name suggests, it is a Tree which is developed based on certain decisions taken by the algorithm in accordance with the given data that it has been trained on.

In simple words, a Decision Tree uses the features in the given data to perform Supervised Learning and develop a tree-like structure (data structure) whose branches are developed in such a way that given the feature-set, the decision tree can predict the expected output relatively accurately.

Example:  Let us look at the structure of a decision tree. For this, we will take up an example dataset called the “PlayTennis” dataset. A sample of the dataset is shown below.

Play Tennis Data set used to look at the structure of a Decision Tree.

In summary, the target of the model is to predict if the weather conditions are suitable to play tennis or not, as guided by the dataset shown above.

As it can be seen in the dataset, it contains certain information (features) for each day. In this, we have the feature-attributes: Outlook, Temperature, Humidity and Wind and the target-attribute PlayTennis. Each of these attributes can take up certain values, for example, the attribute Outlook has the values Sunny, Rain and Overcast.

With a clear idea of the dataset, jumping a bit forward, let us look at the structure of the learned Decision Tree as developed from the above dataset.

The structure of the learned Decision Tree as developed from the "PlayTennis" Data Set

As shown above, it can clearly be seen that, given certain values for each of the attributes, the learned decision tree is capable of giving a clear answer as to whether the weather is suitable for Tennis or not.

Algorithm: With the overall intuition of decision trees, let us look at the formal Algorithm:

Connecting the dots: Since the overall idea of decision trees have been explained, let’s try to figure out how Entropy and Information Gain fits into this entire process.

Entropy (E) is used to calculate Information Gain, which is used to identify which attribute of a given dataset provides the highest amount of information. The attribute which provides the highest amount of information for the given dataset is considered to have more contribution towards the outcome of the classifier and hence is given the higher priority in the tree.

For Example, considering the PlayTennis Example, if we calculate the Information Gain for two corresponding Attributes: Humidity and Wind, we would find that Humidity plays a more important role in deciding whether to play tennis or not. Hence, in this case, Humidity is considered as a better classifier. The detailed calculation is shown in the figure below:

Calculating the Information Gain for 2 corresponding attributes Humidity and Wind.

With the basic idea out of the way, let’s look at where decision trees can be used:

Listed below are some of the advantages of Decision Trees:

Listed below are some of the limitations of Decision Trees:

With the theory out of the way, let’s look at the practical implementation of decision tree classifiers and regressors.

In order to conduct classification, a diabetes dataset from Kaggle has been used. It can be downloaded here.

Example Data set for Classification in Machine Learning

The highlighted column is the target value that the model is expected to predict, given the parameters.

The Decision Tree generated from the Dataset by Classification method in Machine Learning.

In order to conduct classification, a diabetes dataset from Kaggle has been used. It can be downloaded here.

The graph that is thus generated is shown below. Here we can clearly see that for a simple dataset  when we used max_depth=5 (green), the model started to overfit and learned the patterns of the noise along with the sine wave. Such kinds of models do not perform well. Meanwhile, for max_depth=3 (blue), it has fitted the dataset in a better way when compared to the other one.

Decision Tree by Regression in Machine Learning.

Conclusion

In this article, we tried to build an intuition, by starting from the basics of the theory behind the working of a decision tree classifier. However, covering every aspect of detail is beyond the scope of this article. Hence, it is suggested to go through this book to dive deeper into the specifics. Further, moving on, the code snippets introduces the “Hello World” of how to use both, real-world data and artificially generated data to train a Decision Tree model and predict using the same. This will allow any novice to get an overall balanced theoretical and practical idea about the workings of Classification and Regression Trees and their implementation.

  • Supervised Learning: If the available dataset has predefined features and labels, on which the machine learning models are trained, then the type of learning is known as Supervised Machine Learning. Supervised Machine Learning Models can broadly be classified into two sub-parts: Classification and Regression. These have been discussed further in detail.

  • Unsupervised Learning: If the available dataset has predefined features but lacks labels, then the Machine Learning algorithms perform operations on this data to assign labels to it or to reduce the dimensionality of the data. There are several types of Unsupervised Learning Models, the most common of them being: Principal Component Analysis (PCA) and Clustering.

  • Reinforcement Learning: Reinforcement Learning is a more advanced type of learning, where, the model learns from “Experience”. Here, features and labels are not clearly defined. The model is just given a “Situation” and is rewarded or penalized based on the “Outcome”. The model thus learns to optimize the “Situation” to maximize the Rewards and hence improves the “Outcome” with “Experience”.

  • Face Recognition: Face recognition finds its applications in our smartphones and any other place with Biometric security. Face Recognition is nothing but face detection followed by classification. The classification algorithm determines if the face in the image matches with the registered user or not.

  • Medical Image Classification: Given the data of patients, a model that is well trained is often used to classify if the patient has a malignant tumor (cancer), heart ailments, fractures, etc.

  • Stock Market Prediction: Regression algorithms are used to predict the future price of stocks based on certain past features like time of the day or festival time, etc. Stock Market Prediction also falls under a subdomain of study called Time Series Analysis.

  • Object Detection Algorithms: Object Detection is the process of detection of the location of a given object in an image or video. This process returns the coordinates of the pixel values stating the location of the object in the image. These coordinates are determined by using regression algorithms alongside classification.

  • ID3(Samples, Target_attribute, Attributes):
  • Create a root node for the Tree
  • If all the Samples are positive, Return a single-node tree Root, with label = +
  • If all the Samples are negative, Return a single-node tree Root, with label = –
  • If Attribute is empty, Return the single-node tree Root with the label = Most common value of the Target_attribute among the Samples.
  • Otherwise:

    • A ← the attribute from Attributes that best classifies the Samples
    • The decision attribute for Root ← A
    • For each possible value of A:
      • Add a new tree branch below Root, corresponding to the test A = vi
      • Let the Samplesvi be the subset of Samples that have value vi for A
      • If Samplesvi is empty:
        • Then below the new branch add a leaf node with the label = most common value of Target_attribute in the samples
        • Else below the new branch add the subtree:
        • ID3(Samplesvi, Target_attribute, Attributes – {A})
  • A ← the attribute from Attributes that best classifies the Samples

  • The decision attribute for Root ← A
  • For each possible value of A:
    • Add a new tree branch below Root, corresponding to the test A = vi
    • Let the Samplesvi be the subset of Samples that have value vi for A
    • If Samplesvi is empty:
      • Then below the new branch add a leaf node with the label = most common value of Target_attribute in the samples
      • Else below the new branch add the subtree:
      • ID3(Samplesvi, Target_attribute, Attributes – {A})
  • Add a new tree branch below Root, corresponding to the test A = vi
  • Let the Samplesvi be the subset of Samples that have value vi for A
  • If Samplesvi is empty:
    • Then below the new branch add a leaf node with the label = most common value of Target_attribute in the samples
    • Else below the new branch add the subtree:
    • ID3(Samplesvi, Target_attribute, Attributes – {A})
  • Then below the new branch add a leaf node with the label = most common value of Target_attribute in the samples
  • Else below the new branch add the subtree:
  • ID3(Samplesvi, Target_attribute, Attributes – {A})
  • End
  • Return Root
  • Select a flight to travel: Decision trees are very good at classification and hence can be used to select which flight would yield the best “bang-for-the-buck”. There are a lot of parameters to consider, such as if the flight is connecting or non-stop, or how reliable is the service record of the given airliner, etc.

  • Selecting alternative products: Often in companies, it is important to determine which product will be more profitable at launch. Given the sales attributes such as market conditions, competition, price, availability of raw materials, demand, etc. a Decision Tree classifier can be used to accurately determine which of the products would maximize the profits.

  • Sentiment Analysis: Sentiment Analysis is the determination of the overall opinion of a given piece of text and is especially used to determine if the writer’s comment towards a given product/service is positive, neutral or negative. Decision trees are very versatile classifiers and are used for sentiment analysis in many Natural Language Processing (NLP) applications.

  • Energy Consumption: It is very important for electricity supply boards to correctly predict the amount of energy consumption in the near future for a particular region. This is to make sure that un-used power can be diverted towards an area with a higher demand to keep a regular and uninterrupted supply of power throughout the grid. Decision Trees are often used to determine which region is expected to require more or less power in the up-coming time-frame.

  • Fault Diagnosis: In the Engineering domain, one of the widely used applications of decision trees is the determination of faults. In the case of load-bearing rotatory machines, it is important to determine which of the component(s) have failed and which ones can directly or indirectly be affected by the failure. This is determined by a set of measurements that are taken. Unfortunately, there are numerous measurements to take and among them, there are some measurements which are not relevant to the detection of the fault. A Decision Tree classifier can be used to quickly determine which of these measurements are relevant in the determination of the fault.

  • Comprehensive: Another significant advantage of a decision tree is that it forces the algorithm to take into consideration all the possible outcomes of a decision and traces each path to a conclusion.

  • Specific: The output of decision trees is very specific and reduces uncertainty in the prediction. Hence, they are considered as really good classifiers.

  • Easy to use: Decision Trees are one of the simplest, yet most versatile algorithms in Machine Learning. It is based on simple math and no complex formulas. They are easy to visualize, understand and explain.

  • Versatile: A lot of business problems can be solved using Decision Trees. They find their applications in the field of Engineering, Management, Medicine, etc. basically, any situation where data is available and a decision needs to be taken in uncertain conditions.

  • Resistant to data abnormalities: Data is never perfect and there are always many abnormalities in the dataset. Some of the most common abnormalities are outliers, missing data and noise. While most Machine Learning algorithms fail with even a minor set of abnormalities, Decision Trees are very resilient and is able to handle a fair percentage of such abnormalities quite well without altering the results.

  • Visualization of the decision taken: Often in Machine Learning models, data scientists struggle to reason as to why a certain model is giving a certain set of outputs. Unfortunately, for most of the algorithms, it is not possible to clearly determine and visualize the actual process of classification that leads to the final outcome. However, decision trees are very easy to visualize. Once the tree is trained, it can be visualized and the programmer can see exactly how and why the conclusion was reached. It is also easy to explain the outcome to a non-technical team with the “tree” type visualization. This is why many organizations prefer to use decision trees over other Machine Learning Algorithms.

  • Sensitivity to hyperparameter tuning: Decision Trees are very sensitive to hyperparameter tuning. Hyperparameters are those parameters which are in control of the programmer and can be tuned to get better performance out of a given model. Unfortunately, the output of a decision tree can vary drastically if the hyperparameters are inaccurately tuned.

  • Overfitting: Decision trees are prone to overfitting. Overfitting is a concept where the model learns the data too well and hence performs well on training dataset but fails to perform on testing dataset. Decision trees are prone to overfitting if the breadth and depth of the tree is set to very high for a simpler dataset.

  • Underfitting: Similar to overfitting, decision trees are also prone to underfitting. Underfitting is a concept where the model is too simple for it to learn the dataset effectively. Decision tree suffers from underfitting if the breadth and depth of the model or the number of nodes are set too low. This does not allow the model to fit the data properly and hence fails to learn.

  • The initial step for any data science application is data visualization. Hence, the dataset is shown below:
  • Load the Libraries. We will be using pandas to load and manipulate data. Sklearn is used for applying Machine Learning models on the data.
  • Load the data. Pandas is used to read the data from the CSV.
  • Feature Selection: The relevant features are selected for the classification.

  • splitting the data: The dataset needs to be split into training and testing data. The training data is used to train the model, while the testing data is used to test the model’s performance on unseen data.

  • Building the decision tree. These few lines initialize, train and predict on the dataset.
  • The model’s accuracy is evaluated by using Sklearn’s metrics library. 
  • This will generate the decision tree that is shown in the following image
  • For this example, we will generate a Numpy Array which will simulate a scatter plot resembling a sine wave with a few randomly added noise elements. 
  • This time we create two regression models to experiment and see how overfitting looks like for a decision tree. Hence, we initialize the two Decision Tree Regression objects and train them on the given data.
  • After fitting the model, we predict on a custom test dataset and plot the results to see how it performed.

    • Research & References of What are Decision Trees in Machine Learning (Classification And Regression)|A&C Accounting And Tax Services
    Source

    facebookShare on Facebook
    TwitterTweet
    FollowFollow us

    Send your purchase information or ask a question here!

    4 + 6 =

    Welcome To Knowledge-Easy Management Sound Tips and Thank You Very Much! Have a great day!

    From Admin and Read More here. A note for you if you pursue CPA licence, KEEP PRACTICE with the MANY WONDER HELPS I showed you. Make sure to check your works after solving simulations. If a Cashflow statement or your consolidation statement is balanced, you know you pass right after sitting for the exams. I hope my information are great and helpful. Implement them. They worked for me. Hey.... turn gray hair to black also guys. Do not forget HEALTH? Competency Development is the number 1 very important and primary component of having authentic achieving success in all procedures as you actually watched in this the community and even in Across the world. Thus privileged to look at with everyone in the soon after in relation to exactly what effective Competence Expansion is;. how or what methods we deliver the results to get wishes and eventually one could job with what the person prefers to accomplish just about every single daytime with regard to a whole everyday living. Is it so very good if you are equipped to produce proficiently and obtain achievement in just what you thought, planned for, encouraged and did wonders very hard each and every working day and without doubt you turned into a CPA, Attorney, an holder of a substantial manufacturer or possibly even a health care professional who are able to seriously contribute wonderful guidance and principles to many people, who many, any culture and neighborhood surely shown admiration for and respected. I can's imagine I can help others to be major high quality level who will lead major choices and pain relief valuations to society and communities nowadays. How cheerful are you if you end up one like so with your own name on the headline? I have landed at SUCCESS and get over most of the really hard parts which is passing the CPA tests to be CPA. On top of that, we will also include what are the hurdles, or other sorts of difficulties that may just be on your current means and the best way I have in person experienced them and will certainly clearly show you the right way to overcome them.

    0 Comments

    Submit a Comment

    Business Best Sellers

     

    Get Paid To Use Facebook, Twitter and YouTube
    Online Social Media Jobs Pay $25 - $50/Hour.
    No Experience Required. Work At Home, $316/day!
    View 1000s of companies hiring writers now!
    Order Now!

     

    MOST POPULAR

    *****

    Customer Support Chat Job: $25/hr
    Chat On Twitter Job - $25/hr
    Get Paid to chat with customers on
    a business’s Twitter account.
    Try Free Now!

     

    Get Paid To Review Apps On Phone
    Want to get paid $810 per week online?
    Get Paid To Review Perfect Apps Weekly.
    Order Now!

    Look For REAL Online Job?
    Get Paid To Write Articles $200/day
    View 1000s of companies hiring writers now!
    Try-Out Free Now!

     
     
    error: Content is protected !!