International Journal of Computer Engineering and Applications, Volume- XIV, Issue - Special Issue, June 2020, www.ijcea.com ISSN 2321-3469 NCRTCEA- 2020

IMPLEMENTATION OF SIGMOID FUNCTION IN LOGISTIC REGRESSION

Dr. G. N. R. PRASAD Sr. Asst. Professor, Dept. of MCA, CBIT, Hyderabad – 75. E-Mail : [email protected]

ABSTRACT Regression is a form of machine learning where we try to predict a continuous value based on some variables. It is a form of supervised learning where a model is taught using some features from existing data. Around 60% of the world’s classification problems can be solved by using the logistic regression algorithm. It predicts the probability of occurrence of a binary outcome using a logic function. It is a special case of linear regression as it predicts the probabilities of outcome using log function. There are many examples where we can use logistic regression for example; it can be used for fraud detection, spam detection, cancer detection, etc using relevant datasets. Sigmoid is a mathematical function having a characteristic that can take any real value and map it to between 0 to 1 shaped like the letter “S”. The sigmoid function also called the logistic function gives an ‘S’ shaped curve that can take any real-valued number and map it into a value between 0 and 1. If the outcome of the sigmoid function is more than 0.5 then we classify that label as class 1 or positive class and if it is less than 0.5 than we can classify it to negative class or label as class 0. If the curve goes to positive infinity, y predicted will become 1, and if the curve goes to negative infinity, y predicted will become 0. In this paper, I explain how to use the sigmoid function to convert the outcome into categorical value and implementation of sigmoid function using python language.

Keywords : Regression, Classification, Sigmoid Function;

Dr. G. N. R. PRASAD 1 IMPLEMENTATION OF SIGMOID FUNCTION IN LOGISTIC REGRESSION

1.0 INTRODUCTION

A sigmoid function is a type of activation function, and more specifically defined as a squashing function. Squashing functions limit the output to a range between 0 and 1, making these functions useful in the prediction of probabilities.

The name Sigmoidal comes from the Greek letter Sigma, and when graphed, appears as a sloping “S” across the Y-axis. A sigmoidal function is a type of logistic function and purely refers to any function that retains the “S” shape, such as tanh(x). Where a traditional sigmoidal function exists between 0 and 1. On its own, a sigmoidal function is also differentiable, meaning we can find the slope of the sigmoid curve, at any two points.

Classification is one of the biggest problems machine learning explores. Where we used polynomial regression to predict values in a continuous output space, logistic regression is an algorithm for discrete regression, or classification, problems. In the process of machine learning algorithms, Logistic regression is the most famous machine learning algorithm after linear regression. In a lot of ways, logistic regression and linear regression are similar. But, the main difference lies in what they are utilized for. Generally, Linear regression algorithms are used to predict or forecast values but logistic regression is used for classification assignments. There are many classification tasks done routinely by people. For example, classifying whether an email is a spam or not, classifying whether a tumor is malignant or benign, classifying whether a website is fraudulent or not, classifying whether a corona virus is effected to the person or not etc. These are typical examples where machine learning algorithms can make our lives a lot easier. 2.0 LITERATURE COLLECTION

2.1 Sigmoid Function

Logistic regression algorithm also uses a linear equation with independent predictors to predict a value. The predicted value can be anywhere between negative infinity to positive infinity. We need the output of the algorithm to be class variable, i.e 0-no, 1-yes. Therefore, we are squashing the output of the linear equation into a range of [0,1]. To squash the predicted value between 0 and 1, we use the sigmoid function.

General linear regression model

Y = β0 + β1X1 + β2X2 + Ɛ

Here, in the above x is an Independent variable and y is dependent variable.

The variables can be divided into Continuous and Categorical. The continuous provides the result of consumption or time spent value. While the Categorical provides a Boolean value Yes/ No , True / False or 1/0. Means the categorical provides a binary value.

Some of the continuous variables are : Age , income, Height. These values are numerical values.

The categorical values are : City, State.

The simple example of binary outcomes are

Dr. G. N. R. PRASAD 2 International Journal of Computer Engineering and Applications, Volume- XIV, Issue - Special Issue, June 2020, www.ijcea.com ISSN 2321-3469 NCRTCEA- 2020 The bank provides a loan or not to a person, an individual transaction fraudulent or not, Which people are more likely to vote against a new law. Which customers are more likely to buy a new product.

The Data required to know the customers’ subscription: A sample data of 1,000 random customers from a given city is collected. The question is to know they subscribe to a magazine.

The age is a factor. How age influences the likelihood of subscription and gender may also influence likelihood of subscription. The end result subscribe is a Boolean value indicates a customer has subscribed to the magazine.

The subscribe = β0 + β1 age + Ɛ

Squashed output-h : We take the output(z) of the linear equation and give to the function g(x) which returns a squashed value h, the value h will lie in the range of 0 to 1. To understand how sigmoid function squashes the values within the range, let’s visualize the graph of the sigmoid function.

Fig : Sigmoid Function Graph

Sigmoid Function graph becomes asymptote to y=1 for positive values of x and becomes asymptote to y=0 for negative values of x.

Cost Function : Since we are trying to predict class values, we cannot use the same cost function used in linear regression algorithm. Therefore, we use a logarithmic loss function to calculate the cost for misclassifying.

2.2 ROLE OF SIGMOID IN APPLICATIONS

In machine learning, in the event that we will in general get familiar with a connection between certain highlights and a paired component then we utilize a sigmoid function at the yield layer. As the output ranges between 0 and 1, we can set a decision boundary and determine whether the label was 0 or 1. Also, they were used in hidden layers of Artificial Neural Networks. Sigmoid produces an activation based on its inputs which is then multiplied by the weights of the succeeding layer to produce further activations. If a greater positive value is intercepted by Sigmoid, it gives a fully saturated firing of 1. In the case of small negative value, a firing of 0 is produced. Hence, it produces an activation value based on a threshold. Also, since the output is between 0 and 1, its output could be interpreted as a probability for a particular class.

Dr. G. N. R. PRASAD 3 IMPLEMENTATION OF SIGMOID FUNCTION IN LOGISTIC REGRESSION

3.0 IMPLEMENTATION

Numpy is the main and the most used package for scientific computing in Python. It is maintained by a large community. It is a logistic function which gives an ‘S’ shaped curve that can take any real-valued number and map it into a value between 0 and 1. If the curve goes to positive infinity, y predicted will become 1, and if the curve goes to negative infinity, y predicted will become 0. If the output of the sigmoid function is more than 0.5, we can classify the outcome as 1 or YES, and if it is less than 0.5, we can classify it as 0 or NO. For example: If the output is 0.75, we can say in terms of probability that there is a 75 percent chance that patient will suffer from cancer. The sigmoid activation function is:

The sample code to implement Sigmoid Function is : #import section from matplotlib import pylab import pylab as plt import numpy as np #sigmoid = lambda x: 1 / (1 + np.exp(-x)) def sigmoid(x): return (1 / (1 + np.exp(-x))) mySamples = [] mySigmoid = [] x = plt.linspace(-10,10,10) y = plt.linspace(-10,10,100) plt.plot(x, sigmoid(x), 'r', label='linspace(-10,10,10)') plt.plot(y, sigmoid(y), 'b', label='linspace(-10,10,100)') plt.grid() plt.title('Sigmoid Function') plt.suptitle('Sigmoid') plt.legend(loc='lower right') plt.text(4, 0.8, r'$\sigma(x)=\frac{1}{1+e^{-x}}$', fontsize=15) plt.gca().xaxis.set_major_locator(plt.MultipleLocator(1)) plt.gca().yaxis.set_major_locator(plt.MultipleLocator(0.1)) # plt.plot(x) plt.xlabel('X Axis') plt.ylabel('Y Axis') plt.show()

Dr. G. N. R. PRASAD 4 International Journal of Computer Engineering and Applications, Volume- XIV, Issue - Special Issue, June 2020, www.ijcea.com ISSN 2321-3469 NCRTCEA- 2020

Fig : Result of the program (Sigmoid(x))

4.0 CONCLUSION

Logistic regression is a simple algorithm that can be used for binary/multivariate classification tasks. Sigmoidal functions are frequently used in machine learning, specifically in the testing of artificial neural networks, as a way of understanding the output of a node or “neuron.” For example, a neural network may attempt to find a desired solution given a set of inputs. A sigmoidal function will determine the output and that output will be used as the input for the following node. This process will repeat until the solution to the original problem is found.

REFERENCES

[1] https://hackernoon.com/introduction-to-machine-learning-algorithms-logistic-regression- cbdd82d81a36 [2] https://www.kaggle.com/jeppbautista/logistic-regression-from-scratch-python [3] https://deepai.org/machine-learning-glossary-and-terms/sigmoid-function [4] http://www.holehouse.org/mlclass/06_Logistic_Regression.html [5] http://machinelearningmastery.com/logistic-regression-tutorial-for-machine-learning [6] https://scilab.io/machine-learning-logistic-regression-tutorial/ [7] https://github.com/perborgen/LogisticRegression/blob/master/logistic.py [8] http://neuralnetworksanddeeplearning.com/chap3.html [9] http://math.stackexchange.com/questions/78575/derivative-of-sigmoid-function-sigma-x-frac11e-x [10] https://en.wikipedia.org/wiki/Monotoniconotonic_function [11]Rajesh Kumar Tiwari and G. Sahoo “Microsoft Excel Files: A Steganographic Carrier File”, International Journal of Digital Crime and Forensics, IGI Publications, USA, Vol. 3 No. 1, 2011, pp- 37- 52. [12] Rajesh Kumar Tiwari and G. Sahoo, “A Novel Methodology for Data Hiding in PDF File”, Information Security Journal: A Global Perspective, Taylor & Francis Publication, Vol. 20, No. 1, 2011, pp- 1-13. [13] Rajesh Kumar Tiwari and G. Sahoo “Microsoft Power Point Files: A Secure Steganographic Carrier File”, International Journal of Digital Crime and Forensics, IGI Publications, USA, Vol. 3 No. 4, 2011, pp- 17-29. [14] http://scikit-learn.org/stable/modules/linear_model.html#logistic-regression> [15] https://towardsdatascience.com/comparison-of-activation-functions-for-deep-neural-networks- 706ac4284c8a

Dr. G. N. R. PRASAD 5