From a mathematical standpoint, the three pillars of data science that we need to grasp relatively well are linear algebra, statistics, and optimization, which are employed in almost all data science techniques. A solid foundation in linear algebra is required to comprehend optimization principles.

What exactly is Optimization?

According to Wikipedia, optimization is a task in which you maximize or minimize a real function by methodically selecting input values from a permitted set and determining the function's value. We constantly look for the best answer when we talk about optimization. So, suppose someone is interested in a functional form (e.g., f(x)) and is seeking to discover the optimal solution for this functional form. What does "best" mean now? He may be said to be interested in either reducing or enhancing this functional form.

Components of an Optimization Problem

  • The objective function(f(x)): The first element is an objective function f(x) that we are attempting to maximize or decrease. In general, we are discussing minimization difficulties. This is because we may turn a maximizing issue with f(x) into a minimization problem with -f (x). So we can look at minimization issues without losing generality.

 

  • Decision variables(x): The decision variables are the variables that we may use to minimize the function. As a result, we write this as min f. (x).

  • Constraints(a ≤ x ≤ b): The constraint, which simply limits this x to a set, is the third component.

 

As a result, anytime you look at an optimization problem, you should search for these three components.

 

Depending on the purpose functions, decision variables, and restrictions are used:

  1. If the decision variable (x) is continuous:

If a variable x has an unlimited number of values, it is said to be continuous. In this example, x can have an unlimited number of values ranging from -2 to 2. (-2, 2)

min f(x), x ∈ (-2, 2)

 

  • Linear programming problem: This sort of problem exists when the decision variable (x) is continuous, the objective function (f) is linear, and all restrictions are also linear. 

 

  • Nonlinear programming problem: This sort of problem exists if the decision variable(x) stays continuous and if either the objective function(f) or the restrictions are non-linear. As a result, a non-linear programming problem occurs when either the aim or the constraints become non-linear.

 

Learn this technical concept with the help of the best Machine learning course in Delhi

  1. If the decision variable (x) is an integer variable:

 Then any values with a fractional component of 0 (zero), such as -3, -2, 1, 0, 10, 100, are integers.

 

  • Linear integer programming problem: A linear integer programming problem exists when the decision variable (x) is an integer variable, the objective function (f) is linear, and all constraints are linear. As a result, the decision variables in this example are integers, the objective function is linear, and the constraints are likewise linear.

 

  • Nonlinear integer programming problem: If the decision variable(x) stays integer, the problem is known as a non-linear integer programming problem; but, if either the objective function(f) or the constraints are non-linear, the problem is known as a non-linear integer programming problem. As a result, a non-linear programming problem occurs when either the aim or the constraints become non-linear.

 

  • Binary integer programming problem: If the decision variable(x) can only accept binary values such as 0 and 1, then it's called a binary integer programming problem."

 

min f(x), x ∈ [0, 1]

 

  1. If the decision variable(x) is a mixed variable: 

If we combine both a continuous and an integer variable, we get a mixed variable.


min f(x1, x2), x1 ∈ [0, 1, 2, 3] and x2 ∈ (-2, 2)

 

  • Mixed-integer linear programming problem: A mixed-integer linear programming problem exists when the decision variable (x) is a mixed variable, the objective function (f) is linear, and all constraints are linear. As a result, the choice variables are mixed in this example, the objective function is linear, and the constraints are likewise linear.

 

  • Mixed-integer nonlinear programming problem: If either the objective function(f) or the constraints are non-linear, this issue is known as a mixed-integer nonlinear programming problem. As a result, a non-linear programming problem occurs when either the aim or the constraints become non-linear.

 

If you want to learn more about other data science techniques, Learnbay offers rigorous training in its Data science course in Delhi, for working professionals. It has 12+ real-time projects along with placement assistance.