SHORT QUESTIONS ON DECISION TREE

 

SHORT QUESTIONS ON DECISION TREE

1. What is the main purpose of a decision tree in machine learning?

A decision tree is a tool that helps make decisions by showing possible outcomes step-by-step. It works for both classification (sorting things into categories) and regression (predicting numbers). The tree splits the data into smaller groups based on tests on different features. Each internal node is a question about a feature, and each leaf node is the final decision or prediction.

2. What criterion is commonly used to split nodes in a decision tree?

The most common ways to decide where to split a node in a decision tree are Gini impurity and Information Gain (which uses entropy). These measures show how well a split separates the data into different classes, with the goal of making each group as pure (or similar) as possible.

3. What is a leaf node in a decision tree?

A leaf node, also called a terminal node, is the final point at the end of a path in a decision tree. It represents the model’s ultimate output or decision after all the splits. For classification tasks, the leaf node gives the predicted class or category. For regression tasks, it provides the predicted numerical value. This is where the decision-making process in the tree finishes.

4. What is overfitting in the context of decision trees?

Overfitting happens when a decision tree learns the training data too closely, even picking up on random noise and outliers. This creates a very complex tree that works great on the training data but doesn’t do well on new, unseen data. In other words, the tree fails to generalize and makes poor predictions outside of the data it was trained on.

5. How can you prevent a decision tree from overfitting?

Overfitting can be avoided using different techniques. These include pruning, which means cutting off unnecessary branches of the tree; setting a maximum depth to limit how deep the tree can grow; requiring a minimum number of samples in each leaf or split to avoid making decisions based on very few points; or using ensemble methods like Random Forests, which combine many trees to improve accuracy and reduce overfitting.

6. Is a decision tree suitable for both classification and regression?

Yes, decision trees can be used for both classification and regression. In classification, the tree predicts which category or class a data point belongs to. In regression, the tree predicts a continuous number by calculating the average of the values in the leaf nodes.

7. What is the difference between Gini impurity and entropy?

Both Entropy and Gini impurity are ways to measure how good a split is when building a decision tree. Entropy, from information theory, measures how mixed or uncertain the data is after the split—higher entropy means more disorder. Gini impurity measures how often a randomly chosen data point would be wrongly classified if it were assigned a label based on the split. Gini impurity is usually faster to calculate and is often preferred in practice because of its simplicity.