Objective: -
The goal of this challenge is to build system to detect hate speech.
There is no legal definition of hate speech because people’s opinions cannot easily be classified as hateful or offensive. Nevertheless, the United Nations defines hate speech as any type of verbal, written or behavioural communication that can attack or use discriminatory language regarding a person or a group of people based on their identity based on religion, ethnicity, nationality, race, colour, ancestry, gender or any other identity factor.
Hope you now have understood what hate speech is. Social media platforms need to detect hate speech and prevent it from going viral or ban it at the right time.
Dataset: -
The dataset I’m using for the hate speech detection task is downloaded from Kaggle. This dataset was originally collected from Twitter and contains the following columns:
index
count
hate_speech
offensive_language
neither
class
tweet
Step 1: Import all the required libraries
Pandas : In computer programming, pandas is a software library written for the Python programming language for data manipulation and analysis and storing in a proper way. In particular, it offers data structures and operations for manipulating numerical tables and time series
Sklearn : Scikit-learn (formerly scikits.learn) is a free software machine learning library for the Python programming language. It features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy. The library is built upon the SciPy (Scientific Python) that must be installed before you can use scikit-learn.
Pickle : Python pickle module is used for serializing and de-serializing a Python object structure. Pickling is a way to convert a python object (list, dict, etc.) into a character stream. The idea is that this character stream contains all the information necessary to reconstruct the object in another python script.
Seaborn : Seaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.
Matplotlib : Matplotlib is a plotting library for the Python programming language and its numerical mathematics extension NumPy. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits like Tkinter, wxPython, Qt, or GTK.
#Loading libraries
import pandas as pd
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
import sklearn.linear_model
import sklearn
import pickle
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import OneHotEncoder
from matplotlib.pyplot import figure
import matplotlib.pyplot as plt
from sklearn.metrics import mean_absolute_percentage_error
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
from sklearn.preprocessing import scale
from sklearn.linear_model import LinearRegression, Ridge, RidgeCV, Lasso, LassoCV
from sklearn.model_selection import KFold, cross_val_score, train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.metrics import mean_absolute_error
from sklearn.decomposition import PCA
from sklearn.feature_extraction import text
from sklearn.metrics.pairwise import cosine_similarity
import re
import nltk
stemmer = nltk.SnowballStemmer("english")
from nltk.corpus import stopwords
import string
stopword=set(stopwords.words('english'))
import warnings
warnings.filterwarnings('ignore')
Step 2 : Read dataset and basic details of dataset
Goal:- In this step we are going to read the dataset, view the dataset and analysis the basic details like total number of rows and columns, what are the column data types and see to need to create new column or not.
In this stage we are going to read our problem dataset and have a look on it.
#loading the dataset
try:
df = pd.read_csv('C:/Users/YAJENDRA/Documents/final notebooks/Hate Speech Detection/Data/twitter.csv')#Path for the file
print('Data read done successfully...')
except (FileNotFoundError, IOError):
print("Wrong file or file path")
Data read done successfully...
# To view the content inside the dataset we can use the head() method that returns a specified number of rows, string from the top.
# The head() method returns the first 5 rows if a number is not specified.
df.head()
Step3: Data Preprocessing
Why need of Data Preprocessing?
Preprocessing data is an important step for data analysis. The following are some benefits of preprocessing data:
It improves accuracy and reliability. Preprocessing data removes missing or inconsistent data values resulting from human or computer error, which can improve the accuracy and quality of a dataset, making it more reliable.
It makes data consistent. When collecting data, it’s possible to have data duplicates, and discarding them during preprocessing can ensure the data values for analysis are consistent, which helps produce accurate results.
It increases the data’s algorithm readability. Preprocessing enhances the data’s quality and makes it easier for machine learning algorithms to read, use, and interpret it.
After we read the data, we can look at the data using:
# count the total number of rows and columns.
print ('The train data has {0} rows and {1} columns'.format(df.shape[0],df.shape[1]))
The train data has 24783 rows and 7 columns
# lets drop Unnamed: 0 as it's not useful in anyway
df = df.drop(['Unnamed: 0'], axis =1)
Axis are defined for arrays with more than one dimension. A 2-dimensional array has two corresponding axes: the first running vertically downwards across rows (axis 0) and the second running horizontally across columns (axis 1).
The df.value_counts() method counts the number of types of values a particular column contains.
df.shape
(24783, 6)
The df.shape method shows the shape of the dataset.
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 24783 entries, 0 to 24782
Data columns (total 6 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 count 24783 non-null int64
1 hate_speech 24783 non-null int64
2 offensive_language 24783 non-null int64
3 neither 24783 non-null int64
4 class 24783 non-null int64
5 tweet 24783 non-null object
dtypes: int64(5), object(1)
memory usage: 1.1+ MB
The df.info() method prints information about a DataFrame including the index dtype and columns, non-null values and memory usage.
df.iloc[1]
count 3
hate_speech 0
offensive_language 3
neither 0
class 1
tweet !!!!! RT @mleew17: boy dats cold...tyga dwn ba...
Name: 1, dtype: object
df.iloc[ ] is primarily integer position based (from 0 to length-1 of the axis), but may also be used with a boolean array. The iloc property gets, or sets, the value(s) of the specified indexes.
Data Type Check for every column
Why data type check is required?
Data type check helps us with understanding what type of variables our dataset contains. It helps us with identifying whether to keep that variable or not. If the dataset contains contiguous data, then only float and integer type variables will be beneficial and if we have to classify any value then categorical variables will be beneficial.
objects_cols = ['object']
objects_lst = list(df.select_dtypes(include=objects_cols).columns)
print("Total number of categorical columns are ", len(objects_lst))
print("There names are as follows: ", objects_lst)
Total number of categorical columns are 1
There names are as follows: ['tweet']
int64_cols = ['int64']
int64_lst = list(df.select_dtypes(include=int64_cols).columns)
print("Total number of numerical columns are ", len(int64_lst))
print("There names are as follows: ", int64_lst)
Total number of numerical columns are 5
There names are as follows: ['count', 'hate_speech', 'offensive_language', 'neither', 'class']
float64_cols = [‘float64’] float64_lst = list(df.select_dtypes(include=float64_cols).columns)print(“Total number of float64 columns are “, len(float64_lst)) print(“There name are as follow: “, float64_lst)
Step 2 Insights: -
- We have total 6 features where 1 of them are object type and 5 are int type.
After this step we have to calculate various evaluation parameters which will help us in cleaning and analysing the data more accurately.
Step 3: Descriptive Analysis
Goal/Purpose: Finding the data distribution of the features. Visualization helps to understand data and also to explain the data to another person.
Things we are going to do in this step:
Mean
Median
Mode
Standard Deviation
Variance
Null Values
NaN Values
Min value
Max value
Count Value
Skewness
df.describe()
The df.describe() method returns description of the data in the DataFrame. If the DataFrame contains numerical data, the description contains these information for each column: count — The number of not-empty values. mean — The average (mean) value.
Measure the variability of data of the dataset
Variability describes how far apart data points lie from each other and from the center of a distribution.
1. Standard Deviation
The standard deviation is the average amount of variability in your dataset.
It tells you, on average, how far each data point lies from the mean. The larger the standard deviation, the more variable the data set is and if zero variance then there is no variability in the dataset that means there no use of that dataset.
So, it helps in understanding the measurements when the data is distributed. The more the data is distributed, the greater will be the standard deviation of that data.Here, you as an individual can determine which company is beneficial in long term. But, if you didn’t know the SD you would have choosen a wrong compnay for you.
df.std()
count 0.883060
hate_speech 0.631851
offensive_language 1.399459
neither 1.113299
class 0.462089
dtype: float64
We can also understand the standard deviation using the below function.
def std_cal(df,float64_lst):
cols = ['normal_value', 'zero_value']
zero_value = 0
normal_value = 0
for value in float64_lst:
rs = round(df[value].std(),6)
if rs > 0:
normal_value = normal_value + 1
elif rs == 0:
zero_value = zero_value + 1
std_total_df = pd.DataFrame([[normal_value, zero_value]], columns=cols)
return std_total_df
std_cal(df, int64_lst)
int64_cols = ['int64']
int64_lst = list(df.select_dtypes(include=int64_cols).columns)
std_cal(df,int64_lst)
zero_value -> is the zero variance and when then there is no variability in the dataset that means there no use of that dataset.
2. Variance
The variance is the average of squared deviations from the mean. A deviation from the mean is how far a score lies from the mean.
Variance is the square of the standard deviation. This means that the units of variance are much larger than those of a typical value of a data set.
Why do we used Variance ?
By Squairng the number we get non-negative computation i.e. Disperson cannot be negative. The presence of variance is very important in your dataset because this will allow the model to learn about the different patterns hidden in the data
df.var()
count 0.779795
hate_speech 0.399236
offensive_language 1.958487
neither 1.239434
class 0.213526
dtype: float64
We can also understand the Variance using the below function.
zero_cols = []
def var_cal(df,float64_lst):
cols = ['normal_value', 'zero_value']
zero_value = 0
normal_value = 0
for value in float64_lst:
rs = round(df[value].var(),6)
if rs > 0:
normal_value = normal_value + 1
elif rs == 0:
zero_value = zero_value + 1
zero_cols.append(value)
var_total_df = pd.DataFrame([[normal_value, zero_value]], columns=cols)
return var_total_df
var_cal(df, int64_lst)
var_cal(df, int64_lst)
zero_value -> Zero variance means that there is no difference in the data values, which means that they are all the same.
Measure central tendency
A measure of central tendency is a single value that attempts to describe a set of data by identifying the central position within that set of data. As such, measures of central tendency are sometimes called measures of central location. They are also classed as summary statistics.
Mean — The average value. Median — The mid point value. Mode — The most common value.
The mean is the arithmetic average, and it is probably the measure of central tendency that you are most familiar.
Why do we calculate mean?
The mean is used to summarize a data set. It is a measure of the center of a data set.
df.mean()
count 3.243473
hate_speech 0.280515
offensive_language 2.413711
neither 0.549247
class 1.110277
dtype: float64
We can also understand the mean using the below function.
def mean_cal(df,int64_lst):
cols = ['normal_value', 'zero_value']
zero_value = 0
normal_value = 0
for value in int64_lst:
rs = round(df[value].mean(),6)
if rs > 0:
normal_value = normal_value + 1
elif rs == 0:
zero_value = zero_value + 1
mean_total_df = pd.DataFrame([[normal_value, zero_value]], columns=cols)
return mean_total_df
mean_cal(df, int64_lst)
mean_cal(df,int64_lst)
zero_value -> that the mean of a paticular column is zero, which isn’t usefull in anyway and need to be drop.
The median is the middle value. It is the value that splits the dataset in half.The median of a dataset is the value that, assuming the dataset is ordered from smallest to largest, falls in the middle. If there are an even number of values in a dataset, the middle two values are the median.
Why do we calculate median ?
By comparing the median to the mean, you can get an idea of the distribution of a dataset. When the mean and the median are the same, the dataset is more or less evenly distributed from the lowest to highest values.The median will depict that the patient below median is Malignent and above that are Benign.
df.median()
count 3.0
hate_speech 0.0
offensive_language 3.0
neither 0.0
class 1.0
dtype: float64
We can also understand the median using the below function.
def median_cal(df,int64_lst):
cols = ['normal_value', 'zero_value']
zero_value = 0
normal_value = 0
for value in int64_lst:
rs = round(df[value].mean(),6)
if rs > 0:
normal_value = normal_value + 1
elif rs == 0:
zero_value = zero_value + 1
median_total_df = pd.DataFrame([[normal_value, zero_value]], columns=cols)
return median_total_df
median_cal(df, int64_lst)
median_cal(df, int64_lst)
zero_value -> that the median of a paticular column is zero which isn’t usefull in anyway and need to be drop.
3. Mode
The mode is the value that occurs the most frequently in your data set. On a bar chart, the mode is the highest bar. If the data have multiple values that are tied for occurring the most frequently, you have a multimodal distribution. If no value repeats, the data do not have a mode.
Why do we calculate mode ?
The mode can be used to summarize categorical variables, while the mean and median can be calculated only for numeric variables. This is the main advantage of the mode as a measure of central tendency. It’s also useful for discrete variables and for continuous variables when they are expressed as intervals.
df.mode()
def mode_cal(df,int64_lst):
cols = ['normal_value', 'zero_value', 'string_value']
zero_value = 0
normal_value = 0
string_value = 0
for value in int64_lst:
rs = df[value].mode()[0]
if isinstance(rs, str):
string_value = string_value + 1
else:
if rs > 0:
normal_value = normal_value + 1
elif rs == 0:
zero_value = zero_value + 1
mode_total_df = pd.DataFrame([[normal_value, zero_value, string_value]], columns=cols)
return mode_total_df
mode_cal(df, list(df.columns))
zero_value -> that the mode of a paticular column is zero which isn’t usefull in anyway and need to be drop.
Null and Nan values
- Null Values
A null value in a relational database is used when the value in a column is unknown or missing. A null is neither an empty string (for character or datetime data types) nor a zero value (for numeric data types).
df.isnull().sum()
count 0
hate_speech 0
offensive_language 0
neither 0
class 0
tweet 0
dtype: int64
As we notice that there are no null values in our dataset.
- Nan Values
NaN, standing for Not a Number, is a member of a numeric data type that can be interpreted as a value that is undefined or unrepresentable, especially in floating-point arithmetic.
df.isna().sum()
count 0
hate_speech 0
offensive_language 0
neither 0
class 0
tweet 0
dtype: int64
As we notice that there are no nan values in our dataset.
# We have many ways to fill Null/Nan Values as below:
mean -> average value (for numerical)
mode -> most repeated value (for categorical)
Another way to remove null and nan values is to use the method “df.dropna(inplace=True)”.
Count of unique occurences of every value in all categorical value
for value in objects_lst:
print(f"{value:{10}} {df[value].value_counts()}")
tweet !!! RT @mayasolovely: As a woman you shouldn't complain about cleaning up your house. & as a man you should always take the trash out... 1
RT @MurderHigashi: I cannot be worried 'bout no bitch. 1
RT @MrPerfect_22: Most of these niggas go broke for these hoes !! 1
RT @MrPOONsoaker: Foh RT @Thotcho: @MrPOONsoaker lmao 6′0 but I’m hella skinny so bitches don’t like me 1
RT @MrNationWide: fake eyelashes are okay if they look natural, but some of you bitches look like you gonna take flight if you blink too fa… 1
..
But she can shade the shit out a bitch RT @axolROSE: #YourWifeCantRead 1
But she a bad bad bitch make you late for it. 1
But rich is a light skin coon. So it's expected. 1
But not prettier than Amber's. RT @YumiYoko: Her pussy is pretty 1
~~Ruffled | Ntac Eileen Dahlia - Beautiful color combination of pink, orange, yellow & white. A Coll http://t.co/H0dYEBvnZB 1
Name: tweet, Length: 24783, dtype: int64
Skewness
Skewness is a measure of the asymmetry of a distribution. A distribution is asymmetrical when its left and right side are not mirror images. A distribution can have right (or positive), left (or negative), or zero skewness
Why do we calculate Skewness ?
Skewness gives the direction of the outliers if it is right-skewed, most of the outliers are present on the right side of the distribution while if it is left-skewed, most of the outliers will present on the left side of the distribution
Below is the function to calculate skewness.
def right_nor_left(df, int64_lst):
temp_skewness = ['column', 'skewness_value', 'skewness (+ve or -ve)']
temp_skewness_values = []
temp_total = ["positive (+ve) skewed", "normal distrbution" , "negative (-ve) skewed"]
positive = 0
negative = 0
normal = 0
for value in int64_lst:
rs = round(df[value].skew(),4)
if rs > 0:
temp_skewness_values.append([value,rs , "positive (+ve) skewed"])
positive = positive + 1
elif rs == 0:
temp_skewness_values.append([value,rs,"normal distrbution"])
normal = normal + 1
elif rs < 0:
temp_skewness_values.append([value,rs, "negative (-ve) skewed"])
negative = negative + 1
skewness_df = pd.DataFrame(temp_skewness_values, columns=temp_skewness)
skewness_total_df = pd.DataFrame([[positive, normal, negative]], columns=temp_total)
return skewness_df, skewness_total_df
float64_cols = ['float64','int64']
float64_lst_col = list(df.select_dtypes(include=float64_cols).columns)
skew_df,skew_total_df = right_nor_left(df, float64_lst_col)
skew_df
skew_total_df
We notice with the above results that we have following details:
- 5 columns is positive skewed.
Step 3 Insights: -
With the statistical analysis we have found that the data has 5 columns with +ve skewness
Statistical analysis is little difficult to understand at one glance so to make it more understandable we will perform visulatization on the data which will help us to understand the process easily.
Why we are calculating all these metrics?
Mean / Median /Mode/ Variance /Standard Deviation are all very basic but very important concept of statistics used in data science. Almost all the machine learning algorithm uses these concepts in data preprocessing steps. These concepts are part of descriptive statistics where we basically used to describe and understand the data for features in Machine learning
df
We will add a new column to this dataset as labels which will contain the values as:
Hate Speech
Offensive Language
No Hate and Offensive
df["labels"] = df["class"].map({0: "Hate Speech",
1: "Offensive Language",
2: "No Hate and Offensive"})
df
Now we will only select the tweet and labels columns for the rest of the task of training a hate speech detection model:
df = df[["tweet", "labels"]]
df
Now we will create a function to clean the texts in the tweet column:
def clean(text):
text = str(text).lower()
text = re.sub('\[.*?\]', '', text)
text = re.sub('https?://\S+|www\.\S+', '', text)
text = re.sub('<.*?>+', '', text)
text = re.sub('[%s]' % re.escape(string.punctuation), '', text)
text = re.sub('\n', '', text)
text = re.sub('\w*\d\w*', '', text)
text = [word for word in text.split(' ') if word not in stopword]
text=" ".join(text)
text = [stemmer.stem(word) for word in text.split(' ')]
text=" ".join(text)
return text
df["tweet"] = df["tweet"].apply(clean)
df
Step 2: Data Preparation
Goal:-
Tasks we are going to in this step:
Now we will separate the target variable and feature columns in two different dataframe and will check the shape of the dataset for validation purpose.
Split dataset into train and test dataset.
Model Training
1. Now we will separate the target variable and feature columns in two different dataframe and will check the shape of the dataset for validation purpose.
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
# Separate target and feature column in X and y variable
# X will be the features
X = np.array(df["tweet"])
# y will be the target variable
y = np.array(df["labels"])
X
array([' rt mayasolov woman shouldnt complain clean hous amp man alway take trash',
' rt boy dat coldtyga dwn bad cuffin dat hoe place',
' rt urkindofbrand dawg rt ever fuck bitch start cri confus shit',
..., 'young buck wanna eat dat nigguh like aint fuckin dis',
'youu got wild bitch tellin lie',
'ruffl ntac eileen dahlia beauti color combin pink orang yellow amp white coll '],
dtype=object)
y
array(['No Hate and Offensive', 'Offensive Language',
'Offensive Language', ..., 'Offensive Language',
'Offensive Language', 'No Hate and Offensive'], dtype=object)
# Check the shape of X and y variable
X.shape, y.shape
((24783,), (24783,))
2. Split dataset into train and test dataset.
cv = CountVectorizer()
X = cv.fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.33,
random_state=42)
Insights: -
Train test split technique is used to estimate the performance of machine learning algorithms which are used to make predictions on data not used to train the model.It is a fast and easy procedure to perform, the results of which allow you to compare the performance of machine learning algorithms for your predictive modeling problem. Although simple to use and interpret, there are times when the procedure should not be used, such as when you have a small dataset and situations where additional configuration is required, such as when it is used for classification and the dataset is not balanced.
In the next step we will train our model on the basis of our training and testing data.
Step 3: Model Training
Goal: In this step we are going to train our dataset ondifferent algorithms.
In this step we are going to train our dataset on different classification algorithms. As we know that our target variable is in discrete format so we have to apply classification algorithm. Target variable is a category like filtering.In our dataset we have the outcome variable or Dependent variable i.e Y having only three set of values that is 0: “Hate Speech”, 1: “Offensive Language”,2: “No Hate and Offensive”, So we will use Classification algorithm.
Algorithms we are going to use in this step
Bernoulli Naive Bayes
Linear SVC
Decision Tree Classifier
K-fold cross validation is a procedure used to estimate the skill of the model on new data. There are common tactics that you can use to select the value of k for your dataset. There are commonly used variations on cross-validation, such as stratified and repeated, that are available in scikit-learn
# Define kfold with 10 split
cv1 = KFold(n_splits=10, shuffle=True, random_state=42)
The goal of cross-validation is to test the model’s ability to predict new data that was not used in estimating it, in order to flag problems like overfitting or selection bias and to give an insight on how the model will generalize to an independent dataset (i.e., an unknown dataset, for instance from a real problem).
1. Bernoulli Naive Bayes
Naive Bayes is a supervised machine learning algorithm to predict the probability of different classes based on numerous attributes. It indicates the likelihood of occurrence of an event. Naive Bayes is also known as conditional probability. Bernoulli Naive Bayes is a part of the Naive Bayes family. It is based on the Bernoulli Distribution and accepts only binary values, i.e., 0 or 1. If the features of the dataset are binary, then we can assume that Bernoulli Naive Bayes is the algorithm to be used.
Train set cross-validation
#Using Bernoulli Naive Bayes algorithm to the Training Set
from sklearn.naive_bayes import BernoulliNB
BNB = BernoulliNB()
BNB.fit(X_train, y_train)
BernoulliNB()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
BernoulliNB
BernoulliNB()
#Accuracy check of trainig data
#Get R2 score
BNB.score(X_train, y_train)
0.8550951577933028
#Accuracy check of trainig data
#Get R2 score
BNB.score(X_train, y_train)
0.8550951577933028
Prediction
Now we will perform prediction on the dataset using Bernoulli Naive Bayes.
# Predict the values on X_test_scaled dataset
y_predicted = BNB.predict(X_test)
Various parameters are calculated for analysing the predictions.
- Confusion Matrix 2) Classification Report 3) Accuracy Score 4) Precision Score 5) Recall Score 6) F1 Score
Confusion Matrix
A confusion matrix presents a table layout of the different outcomes of the prediction and results of a classification problem and helps visualize its outcomes. It plots a table of all the predicted and actual values of a classifier.
This diagram helps in understanding the concept of confusion matrix.
# Constructing the confusion matrix.
from sklearn.metrics import confusion_matrix
#confusion matrix btw y_test and y_predicted
cm = confusion_matrix(y_test,y_predicted)
Evaluating all kinds of evaluating parameters.
Classification Report :
A classification report is a performance evaluation metric in machine learning. It is used to show the precision, recall, F1 Score, and support of your trained classification model.
F1_score :
The F1 score is a machine learning metric that can be used in classification models.
Precision_score :
The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The best value is 1 and the worst value is 0.
Recall_score :
Recall score is used to measure the model performance in terms of measuring the count of true positives in a correct manner out of all the actual positive values. Precision-Recall score is a useful measure of success of prediction when the classes are very imbalanced.
# Evaluating the classifier
# printing every score of the classifier
# scoring in anything
from sklearn.metrics import classification_report
from sklearn.metrics import f1_score, accuracy_score, precision_score,recall_score
from sklearn.metrics import confusion_matrix
print("The model used is Bernoulli Naive Bayes")
l_acc = accuracy_score(y_test, y_predicted)
print("\nThe accuracy is: {}".format(l_acc))
prec = precision_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The precision is: {}".format(prec))
rec = recall_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The recall is: {}".format(rec))
f1 = f1_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The F1-Score is: {}".format(f1))
c1 = classification_report(y_test, y_predicted)
print("Classification Report is:")
print()
print(c1)
The model used is Bernoulli Naive Bayes
The accuracy is: 0.8267514366059421
The precision is: 0.5625011041163845
The recall is: 0.44543607947147223
The F1-Score is: 0.4640794339137708
Classification Report is:
precision recall f1-score support
Hate Speech 0.00 0.00 0.00 465
No Hate and Offensive 0.86 0.34 0.49 1379
Offensive Language 0.82 0.99 0.90 6335
accuracy 0.83 8179
macro avg 0.56 0.45 0.46 8179
weighted avg 0.78 0.83 0.78 8179
2. Linear SVC
The objective of a Linear SVC (Support Vector Classifier) is to fit to the data you provide, returning a “best fit” hyperplane that divides, or categorizes, your data. From there, after getting the hyperplane, you can then feed some features to your classifier to see what the “predicted” class is. This makes this specific algorithm rather suitable for our uses, though you can use this for many situations.
#Using KNeighborsClassifier Method of neighbors class to use Nearest Neighbor algorithm
from sklearn.svm import LinearSVC
classifier = LinearSVC()
classifier.fit(X_train, y_train)
LinearSVC()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
LinearSVC
LinearSVC()
#Accuracy check of trainig data
#Get R2 score
classifier.score(X_train, y_train)
0.9931341845338473
#Accuracy check of trainig data
#Get R2 score
classifier.score(X_train, y_train)
0.9931341845338473
Prediction
Now we will perform prediction on the dataset using Linear SVC.
# Predict the values on X_test_scaled dataset
y_predicted = classifier.predict(X_test)
Various parameters are calculated for analysing the predictions.
- Confusion Matrix 2) Classification Report 3) Accuracy Score 4) Precision Score 5) Recall Score 6) F1 Score
Confusion Matrix
A confusion matrix presents a table layout of the different outcomes of the prediction and results of a classification problem and helps visualize its outcomes. It plots a table of all the predicted and actual values of a classifier.
# Constructing the confusion matrix.
from sklearn.metrics import confusion_matrix
#confusion matrix btw y_test and y_predicted
cm = confusion_matrix(y_test,y_predicted)
Evaluating all kinds of evaluating parameters.
# Evaluating the classifier
# printing every score of the classifier
# scoring in anything
from sklearn.metrics import classification_report
from sklearn.metrics import f1_score, accuracy_score, precision_score,recall_score
from sklearn.metrics import confusion_matrix
print("The model used is Bernoulli Naive Bayes")
k_acc = accuracy_score(y_test, y_predicted)
print("\nThe accuracy is: {}".format(l_acc))
prec = precision_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The precision is: {}".format(prec))
rec = recall_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The recall is: {}".format(rec))
f1 = f1_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The F1-Score is: {}".format(f1))
c1 = classification_report(y_test, y_predicted)
print("Classification Report is:")
print()
print(c1)
The model used is Bernoulli Naive Bayes
The accuracy is: 0.8267514366059421
The precision is: 0.7132852369937952
The recall is: 0.6822767748141855
The F1-Score is: 0.695917161343886
Classification Report is:
precision recall f1-score support
Hate Speech 0.39 0.31 0.34 465
No Hate and Offensive 0.83 0.80 0.81 1379
Offensive Language 0.92 0.94 0.93 6335
accuracy 0.88 8179
macro avg 0.71 0.68 0.70 8179
weighted avg 0.87 0.88 0.88 8179
3. Decision Tree Classifier
Decision tree is a type of supervised learning algorithm that can be used for both regression and classification problems. The algorithm uses training data to create rules that can be represented by a tree structure. Like any other tree representation, it has a root node, internal nodes, and leaf nodes.
The main advantage of the decision tree classifier is its ability to using different feature subsets and decision rules at different stages of classification.
DecisionTreeClassifier is a class capable of performing multi-class classification on a dataset. In case that there are multiple classes with the same and highest probability, the classifier will predict the class with the lowest index amongst those classes.
#Using DecisonTreeClassifier Algorithm to the Training Set
from sklearn.tree import DecisionTreeClassifier
DTC = DecisionTreeClassifier()
DTC.fit(X_train,y_train)
DecisionTreeClassifier()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
DecisionTreeClassifier
DecisionTreeClassifier()
#Accuracy check of trainig data
#Get R2 score
DTC.score(X_train, y_train)
0.9996386412912551
#Accuracy check of trainig data
#Get R2 score
DTC.score(X_test, y_test)
0.8755349064677833
Prediction
Now we will perform prediction on the dataset using Linear Regression.
# Predict the values on X_test_scaled dataset
y_predicted = DTC.predict(X_test)
Various parameters are calculated for analysing the predictions.
- Confusion Matrix 2) Classification Report 3) Accuracy Score 4) Precision Score 5) Recall Score 6) F1 Score
Confusion Matrix
A confusion matrix presents a table layout of the different outcomes of the prediction and results of a classification problem and helps visualize its outcomes. It plots a table of all the predicted and actual values of a classifier.
# Constructing the confusion matrix.
from sklearn.metrics import confusion_matrix
#confusion matrix btw y_test and y_predicted
cm = confusion_matrix(y_test,y_predicted)
Evaluating all kinds of evaluating parameters.
# Evaluating the classifier
# printing every score of the classifier
# scoring in anything
from sklearn.metrics import classification_report
from sklearn.metrics import f1_score, accuracy_score, precision_score,recall_score
from sklearn.metrics import confusion_matrix
print("The model used is Decision Tree Classifier ")
r_acc = accuracy_score(y_test, y_predicted)
print("\nThe accuracy is: {}".format(l_acc))
prec = precision_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The precision is: {}".format(prec))
rec = recall_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The recall is: {}".format(rec))
f1 = f1_score(y_test, y_predicted,average="macro", pos_label="Hate Speech")
print("The F1-Score is: {}".format(f1))
c1 = classification_report(y_test, y_predicted)
print("Classification Report is:")
print()
print(c1)
The model used is Decision Tree Classifier
The accuracy is: 0.8267514366059421
The precision is: 0.7011886754292281
The recall is: 0.6917310333516896
The F1-Score is: 0.6961275501343908
Classification Report is:
precision recall f1-score support
Hate Speech 0.37 0.33 0.35 465
No Hate and Offensive 0.82 0.82 0.82 1379
Offensive Language 0.92 0.93 0.93 6335
accuracy 0.88 8179
macro avg 0.70 0.69 0.70 8179
weighted avg 0.87 0.88 0.87 8179
Insight:-
cal_metric=pd.DataFrame([l_acc,k_acc,r_acc],columns=["Score in percentage"])
cal_metric.index=['Bernoulli Naive Bayes','LinearSVC','DecisionTree ']
cal_metric
As you can see with our Bernoulli Naive Bayes(87.83%) we are getting a better result even for the recall (0.6991 or 69.91%) which is the most tricky part.
So we gonna save our model with DecisionTree Classifier.
Step 4: Save Model
Goal:- In this step we are going to save our model in pickel format file.
import pickle
pickle.dump(BNB, open("HATE_SPEECH_Bernoulli Naive Bayes.pkl", 'wb'))
import pickle
pickle.dump(classifier, open("HATE_SPEECH_linearSVC.pkl", 'wb'))
import pickle
pickle.dump(DTC, open("HATE_SPEECH_DecisionTreeClassifier.pkl", 'wb'))
import pickle
def model_prediction(features):
features = cv.transform([features]).toarray()
pickled_model = pickle.load(open('HATE_SPEECH_DecisionTreeClassifier.pkl', 'rb'))
Message = str(list(pickled_model.predict(features)))
return str(f'The Message is {Message}')
Message = "Cows lose their jobs as milk prices drop"
Message_2 = "Let's unite and kill all the people who are protesting against the government"
model_prediction(Message_2)
"The Message is ['Hate Speech']"
model_prediction(Message)
"The Message is ['No Hate and Offensive']"
Conclusion
After observing the problem statement we have build an efficient model to overcome it. So this is how you can train a machine learning model for the task of detecting hate speech by using the Python programming language. Hate speech is one of the serious issues we see on social media platforms like Twitter and Facebook daily. Most of the posts containing hate speech can be found in the accounts of people with political views.
Checkout whole project codehere(github repo).
🚀 Unlock Your Dream Job with HiDevs Community!
🔍 Seeking the perfect job?HiDevs Community is your gateway to career success in the tech industry. Explore free expert courses, job-seeking support, and career transformation tips.
💼 We offer an upskill program in Gen AI, Data Science, Machine Learning, and assist startups in adopting Gen AI at minimal development costs.
🆓 Best of all, everything we offer is completely free! We are dedicated to helping society.
Book free of cost 1:1 mentorship on any topic of your choice —topmate
✨ We dedicate over 30 minutes to each applicant’s resume, LinkedIn profile, mock interview, and upskill program. If you’d like our guidance, check out our services here
💡 Join us now, and turbocharge your career!
Deepak Chawla LinkedIn
Vijendra Singh LinkedIn
Yajendra Prajapati LinkedIn
YouTube Channel
Instagram Page
HiDevs LinkedIn
Project Youtube Link