Estimated reading time: 6 minutes
8. Naive Bayes
Naive Bayes is a probabilistic classification algorithm based on Bayes’ theorem, assuming independence between features.
Use Cases:
- Text classification.
- Spam filtering.
- Sentiment analysis.
Sample Data:
import numpy as np
# Features (Word count 1, Word count 2)
text_counts = np.array([[2, 1], [0, 3], [1, 0], [3, 2]])
# Target (Class)
text_labels = np.array([0, 1, 0, 1])
text_counts represents the frequency of two words in different documents, and text_labels are the corresponding document classes.
Code Implementation:
from sklearn.naive_bayes import GaussianNB
import numpy as np
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y = np.array([0, 1, 0, 1])
model = GaussianNB()
model.fit(X, y)
new_X = np.array([[2, 3]])
predicted_class = model.predict(new_X)
predicted_probability = model.predict_proba(new_X)
print(f"Predicted class for [2, 3]: {predicted_class}")
print(f"Predicted probabilities for [2, 3]: {predicted_probability}")
Code Explanation:
from sklearn.naive_bayes import GaussianNB: Imports the GaussianNB class, suitable for continuous features. For discrete features like word counts,MultinomialNBorBernoulliNBmight be more appropriate.X: Independent features.y: Target variable.model = GaussianNB(): Initializes a Gaussian Naive Bayes model.model.fit(X, y): Trains the model.model.predict(new_X): Predicts the class label for new data.model.predict_proba(new_X): Predicts the probability of each class for new data.
9. Gradient Boosting Machines (GBM)
Gradient Boosting Machines are an ensemble learning method that builds multiple decision trees sequentially, with each new tree trying to correct the errors made by the previous ones.
Use Cases:
- Predictive modeling on structured data.
- Ranking problems.
- Risk assessment.
Sample Data:
import numpy as np
# Features (Feature 1, Feature 2, Feature 3)
feature_data = np.array([[1, 0.1, 5], [2, 0.5, 3], [3, 0.2, 7], [4, 0.8, 2]])
# Target (Continuous value for regression, class for classification)
target_values = np.array([10, 25, 15, 30])
feature_data contains multiple features, and target_values are the corresponding target variable (for regression in this example).
Code Implementation (Regression Example):
from sklearn.ensemble import GradientBoostingRegressor
import numpy as np
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y = np.array([10, 15, 20, 25])
model = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1, max_depth=3)
model.fit(X, y)
new_X = np.array([[8, 9]])
predicted_value = model.predict(new_X)
print(f"Predicted value for [8, 9]: {predicted_value}")
Code Explanation:
from sklearn.ensemble import GradientBoostingRegressor: Imports the GradientBoostingRegressor class for regression tasks. For classification, useGradientBoostingClassifier.n_estimators=100: The number of boosting stages to perform.learning_rate=0.1: Shrinks the contribution of each tree to prevent overfitting.max_depth=3: The maximum depth of the individual regression estimators.- The rest of the steps are similar to other regression algorithms.
10. Artificial Neural Networks (ANN) / Multi-layer Perceptron (MLP)
Artificial Neural Networks are a set of algorithms modeled loosely after the human brain, designed to recognize patterns. Multi-layer Perceptron is a type of feedforward ANN.
Use Cases:
- Image recognition.
- Natural Language Processing.
- Time series forecasting.
Sample Data:
import numpy as np
# Features (Feature 1, Feature 2)
input_features = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
# Target (Binary output for classification)
output_labels = np.array([0, 1, 1, 0]) # Example for XOR
input_features are the input to the neural network, and output_labels are the target classes.
Code Implementation (Classification Example):
from sklearn.neural_network import MLPClassifier
import numpy as np
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([0, 1, 1, 0])
model = MLPClassifier(hidden_layer_sizes=(2,), activation='relu', solver='adam', max_iter=200)
model.fit(X, y)
new_X = np.array([[1, 0]])
predicted_class = model.predict(new_X)
predicted_probability = model.predict_proba(new_X)
print(f"Predicted class for [1, 0]: {predicted_class}")
print(f"Predicted probabilities for [1, 0]: {predicted_probability}")
Code Explanation:
from sklearn.neural_network import MLPClassifier: Imports the MLPClassifier class for classification. For regression, useMLPRegressor.hidden_layer_sizes=(2,): Defines a single hidden layer with 2 neurons. You can add more layers and neurons.activation='relu': The activation function used by the neurons. Other options include ‘tanh’, ‘logistic’.solver='adam': The optimization algorithm used for training the network. Other options include ‘lbfgs’, ‘sgd’.max_iter=200: Maximum number of iterations for training.- The rest of the steps are similar to other classification algorithms. Note that neural networks often require more data and careful tuning of hyperparameters.
11. Association Rule Mining (Apriori Algorithm)
Apriori is an algorithm for frequent itemset mining and association rule learning over transactional databases.
Use Cases:
- Market basket analysis (e.g., what items are frequently bought together?).
- Recommendation systems.
- Web usage mining.
Sample Data:
# Sample transactional data
transactions = [
['milk', 'bread', 'butter'],
['bread', 'butter'],
['milk', 'butter'],
['milk', 'bread', 'eggs'],
['milk', 'bread', 'butter', 'eggs'],
]
transactions is a list of lists, where each inner list represents a transaction containing items.
Code Implementation:
from mlxtend.frequent_patterns import apriori
from mlxtend.frequent_patterns import association_rules
import pandas as pd
# Sample transactional data
transactions = [
['milk', 'bread', 'butter'],
['bread', 'butter'],
['milk', 'butter'],
['milk', 'bread', 'eggs'],
['milk', 'bread', 'butter', 'eggs'],
]
# Convert transaction data into a one-hot encoded DataFrame
from mlxtend.preprocessing import TransactionEncoder
te = TransactionEncoder()
te_array = te.fit(transactions).transform(transactions)
df = pd.DataFrame(te_array, columns=te.columns_)
# Find frequent itemsets
frequent_itemsets = apriori(df, min_support=0.4, use_colnames=True)
print("Frequent Itemsets:\n", frequent_itemsets)
# Generate association rules
rules = association_rules(frequent_itemsets, metric="confidence", min_threshold=0.6)
print("\nAssociation Rules:\n", rules)
Code Explanation:
from mlxtend.frequent_patterns import aprioriandassociation_rules: Imports the necessary functions from themlxtendlibrary. Make sure you have it installed (`pip install mlxtend pandas`).import pandas as pd: Imports the pandas library for data manipulation.- The sample
transactionsdata is defined. TransactionEncoderis used to convert the list of transactions into a one-hot encoded pandas DataFrame, which is required by theapriorifunction.apriori(df, min_support=0.4, use_colnames=True): Runs the Apriori algorithm to find frequent itemsets with a minimum support of 40%.use_colnames=Trueensures that item names are used instead of column indices.association_rules(frequent_itemsets, metric="confidence", min_threshold=0.6): Generates association rules from the frequent itemsets with a minimum confidence of 60%. Other metrics like ‘lift’ can also be used.
Leave a Reply