Nonconvex Vector Optimization and Optimality Conditions for Proper Efficiency

In this paper, we consider, a new nonlinear scalarization function in vector spaces which is a generalization of the oriented distance function. Using the algebraic type of closure, which is called vector closure, we introduce the algebraic boundary of a set, without assuming any topology, in our context. Furthermore, some properties of this algebraic boundary set are given and present the concept of the oriented distance function via this set in the concept of vector optimization. We further investigate Q-proper efficiency in a real vector space, where Q is some nonempty (not necessarily convex) set. The necessary and sufficient conditions for Q-proper efficient solutions of nonconvex optimization problems are obtained via the scalarization technique. The scalarization technique relies on the use of two different scalarization functions, the oriented distance function and nonconvex separation function, which allow us to characterize the Q-proper efficiency in vector optimization with and without constraints.

Efficiency is one of the most important concepts in vector optimization. This concept has been studied in many papers [1,2,4,5,8]. Kuhn and Tucker and Geoffrion [16,17], introduced the concept of proper efficiency. Since then, different definitions of proper efficient points have been introduced by the other authors. Wang and Li [18,19] studied the Benson and Borwein proper efficiency in finite-dimensional Euclidean spaces. Borwein [20] has proposed a definition for extending Geoffrion's concept of proper efficiency to the vector maximization problem in which the domination cone could be any nontrivial, closed convex cone.
Adán and Novo [1,2,8] used the vector closure to define the concept of Benson proper efficiency of vector optimization problems and they proved scalarization theorems. Also, they investigated weak and proper efficiency of vector optimization problems with generalized convex set-valued maps involving relative algebraic interior and vector closure of ordering cone in linear spaces. Ha [21] presented the notion of Q-minimal solution of vector optimization problems via topological concepts, where Q is some nonempty open (not necessarily convex) cone. Q-minimal points were characterized by the Hiriart-Urruty function.
Scalarization techniques play a vital role in sketching the numerical algorithms and duality results [5,7,8,10,12,22,23]. During the last three decades, many authors have been interested in extending scalarization approaches in vector optimization. Nonlinear scalarization approaches have been widely used as efficient methods to study several optimization problems in recent years.
In vector optimization, two types of nonlinear scalarization functions are most widely used, the Gerstewitz function [24] and the oriented distance function [21]. The Gerstewitz function is a nonlinear scalarization function most commonly used in optimization problems with vector-valued or set-valued maps [5,13,15,22,25]. This function was introduced by different names such as the Gerstewitz function, nonlinear scalarization function, shortage function, and smallest strictly monotonic function, [25][26][27][28]. The properties of the Gerstewitz function in a topological vector space with a closed convex (solid) cone, have been studied in [26][27][28][29]. Hernández and Rodríguez-Marín [30] presented an extension of the Gerstewitz function and characterized some topological properties to obtain a nonconvex scalarization and optimality conditions for set-valued optimization problems. The nonconvex separation functional in real linear spaces without considering a topology has been presented by La Torre, Popovici, and Rocca [31,32] . They showed that weakly cone-convex vector-valued functions can be characterized in terms of weakly convexity and weakly quasiconvexity of the Gerstewitz scalarization functions. The authors in [13,15] extend the Gerstewitz function from the topological spaces to real linear via algebraic concepts.
Beside the Gerstewitz function, the oriented distance function is a common scalarization function in vector spaces introduced by Hiriart-Urruty [33]. The oriented distance function has been used to study well-posedness and stability for vector optimization problems in [35][36][37][38]. The generalized version of the oriented distance function introduced by Crespi et al. [35] can be used to characterize optimality conditions of set-valued vector optimization.
In this paper, we propose a new definition of distance function by using vector closure. For this a new definition of the boundary set is presented which can be used to define the new form of the oriented distance function. The aim of this work is to provide a necessary and sufficient conditions of Q-Global Borwein Vectorial Proper Efficient (Q-GBOV) solutions in a real vector space. We use algebraic concepts such as algebraic interior and vectorial closure to define and characterize Q-GBOV. The necessary and sufficient conditions for Q-proper efficient solutions of nonconvex optimization problems are obtained via scalarization by oriented distance function and nonlinear scalarization function in a vector space. As the reader sees, some arguments developed for Global Borwein's proper efficiency are still valid for Q-GBOV. Some results in this paper, are the generalization of several results given in [1,2,5,8,13,39].
The remainder of the paper is organized as follows: In Section 2, we introduce an algebraic boundary set in a vector space and study its properties. We discuss the notion of Q-proper efficiency where Q is not necessarily a convex set in Section 3. Section 4 is devoted to the scalarization functions; We describe a new nonlinear scalarization functions and explain how to use these functions to obtain optimality conditions. Finally, constrained problems in real vector spaces have been discussed in section 5. We use the results of previous sections to obtain optimality conditions for the constrained problems without convexity assumption. The results of this paper can be also used to develop a vector optimization on vector spaces, which can be applied to any numerical and theoretical scalar optimization.

Preliminaries
Throughout the paper, X and Y are real spaces and A is a subset of X. Furthermore, we consider K ⊆ Y be a pointed convex proper cone which introduces a partial order on Y by the equivalence The algebraic interior of A and the vectorial closure of A are denoted by cor (A) and v cl(A), respectively and these are defined as follows [1] in addition cor (K) + K = cor (K) and cor (cor (K)) = cor (K) for solid nontrivial convex cone K [8].
For each q ∈ Y , q-vector closure of A in the direction q is denoted by v cl q (A) and define as follows: In fact it can be shown that v cl q (A) = {x ∈ X : ∃λ n ≥ 0, λ n → 0 ; x + λ n q ∈ A, ∀n ∈ N}. Obviously, for a free disposal Q. The following proposition shows that there is e ∈ cor K such that v cl e (Q) + (0, +∞)e = cor (Q).
Proposition 2.1. [15] Suppose that Q is free disposal with respect to an algebraic solid convex cone where e ∈ cor (K).
The boundary of a subset A of a topological space X is the set of points which can be approached both from A and from the outside of A. More precisely, it is the set of points in the closure of A not belonging to the interior of A. The algebraic boundary of a set A in a vector space can be defined by using algebraic type of interior and closure. which is the set of points in the vectorial closure of A not belonging to the algebraic interior of A.
It is clear that bd(tA) = tbd(A) for t > 0 because v cl(tA) = tv cl(A) and cor (tA) = tcor (A), Proof. Assume by contradiction that x ∈ v cl(Y \Q) and x ∈ cor (Q). x ∈ v cl(Y \Q) implies that for all λ > 0 there exist x ∈ X and λ ∈ [0, λ ] such that On the other hand, for x ∈ cor (Q) we have If we consider x = x and λ = λ then we can write

Proper Efficiency
Definition of Global Borwein vectorial proper efficient solutions in vector spaces, for the first time, Let Y and Z be two real spaces that are partially ordered by nontrivial ordering convex cones K and M, respectively. Let f : X → Y and g : X → Z be two maps on X.
Consider the following unconstrained and constrained problems: and the following vector optimization problem: where the feasible set S can be either If Q is a solid set and 0 / ∈ cor (Q), then x 0 is called Q-Weak Proper Eficient solution (Q-WEF) of (P) when It is easy to see that if x 0 is a Q-GBOV for (P), then x 0 is also a (Q-EF) and a (Q-WEF) for (P).

Scalarization
In this section, we will present the necessary and sufficient optimality conditions for Q-Global Borwein Vectorial Proper Efficient solutions of vector optimization problems. A useful approach for solving a vector problem is to reduce it to a scalar problem. In general, scalarization means the replacement of a vector optimization problem by a suitable scalar problem which tends to be an optimization problem with a real valued objective function. The main idea of this section obtained from [13,15]. In [13] the Gerstewitz function is generated by a general convex cone in a real space and the authors investigated some properties of this function such as sub-additive and positively homogeneous. However, similar to [15], in this section we consider the nonconvex separation function which is an extension of the Gerstewitz function and generated by a subset of a linear space instead of a convex cone. The main properties of the nonconvex separation functional were extended from the topological framework to the linear setting via suitable algebraic counterparts [15].   In Theorem 4.3, we prove h e Q is sub-additive whenever Q is closed under addition. This theorem will be used in the sequel.
, for all y 1 , y 2 ∈ Y , except for these make it indeterminate form ∞ − ∞.
Proof. From definition of h e Q given in ( 4.1) and Lemma 3 in [15], we have We can use the fact that h e v cle (Q) = h e Q to obtain Then obviously, and this yields h e Q (y 1 + y 2 ) ≤ h e Q (y 1 ) + h e Q (y 2 ). For a set Q ⊂ Y let the oriented distance function Q : Y → R ∪ {±∞} be defined as . One can find the main properties of the oriented distance function in topological spaces in [33,40].
However, here we recall them for conveniences. If y ∈ cor (Q), then there exists a sequence λ n → 0 such that y ∈ v cl(λ n Q), thus we get d(y , Q) = 0 and then Q (y ) < 0. Also, if d(y , Q) = 0, then y ∈ cor (Q). Therefore, we can write y ∈ cor (Q) if and only if Q (y ) < 0. Moreover, Q (y ) > 0 if and only if y ∈ v cl(Q). Since bd(Y \Q) = bd(Q) and bd(Q) = v cl(Q)\cor (Q), we have Q (y ) = 0 if and only if y ∈ bd(Q). Furthermore, it is obvious that Proof. Consider y ∈ Y . Thus d(ty , Q) = i nf {λ ≥ 0; ty ∈ v cl(λQ)}, (4.2) Therefore, In the following theorem, we use the Gerstewitz function to obtain the sufficient condition for Q-GBOV.
Thus, by theorem 4.1, one has . By definition of vectorial closure, there exist x ∈ Y and a sequence of positive real numbers λ n such that λ n → 0 and Therefore, there are sequences α n ≥ 0 and y n ∈ f (S) such that y + λ n x = α n (y n − f (x 0 )).
It is obvious that there exist an n ∈ N such that α n > 0. Since h e Q (y n − f (x 0 )) ≥ 0, then it implies that h e Q (y + λ n x) ≥ 0. and 0 ≤ h e Q (y + λ n x) ≤ h e Q (y ) + λ n h e Q (x).
By taking limit n → ∞, we have h e Q (y ) ≥ 0.
Now, by applying Theorem 4.1, we conclude that y ∈ Y \((−∞, 0)e − v cl e (Q)). On the other hand, for q ∈ cor (Q) there exists λ > 0 such that It means that Therefore, Since Q is an algebraically close set, we have which implies that Thus, x 0 is a Q-GBOV for (p).
The following theorems state the necessary condition for a point to become a Q-GBOV for problem (P). In Theorem 4.6, we use the nonconvex separation function to obtain the necessary condition while in Theorem 4.7, the oriented distance function has been used. We would like to point out that the oriented distance function is a simple tool to work, thus optimality conditions can be obtained with simple calculations by using the properties of the oriented distance function without any condition on the set Q.
Theorem 4.6. Let ∅ = Q ⊂ Y is free disposal with respect to an algebraic solid convex cone K.
Suppose that there exists e ∈ cor (K) such that x 0 is a Q-GBOV for problem (P), then x 0 satisfies the following condition Proof. Let us suppose x 0 does not satisfy the condition and we have for some x ∈ S. Hence, from Theorem 4.1 we get On the other hand, since Q is free disposal with respect to K, then by Proposition 2.1, one has Therefore, But as one can see, this contradicts the definition of Q-GBOV.
Theorem 4.7. Let e ∈ cor K and x 0 ∈ S such that x 0 is a Q−GBOV for (P), then Proof. By contrary suppose that for e ∈ cor K there exists x ∈ S such that Thus, we can write Furthermore, we have which contradicts the assumption that x 0 is Q−GBoV. Therefore, From Theorems 4.5 and 4.6 we conclude the following corollary.
In the following theorem, we present necessary and sufficient conditions of Q-WEF for the problem (P).
Theorem 4.8. Let us assume x 0 ∈ S. The point x 0 is a Q-WEF if and only if Proof. Assume that x 0 is not Q-WEF. Then one has Hence, there exists f (x) ∈ f (S) such that Then this proofs the necessary condition. The sufficient condition follows easily.

Scalarization and constrained problems
Constrained problems in real vector spaces were originally studied in [5,8]. In the following theorem, we discuss Q-GBOV in corresponding problems.
Theorem 5.1. In a constrained vector optimization problem, assume that e ∈ cor (K), ∅ = Q ⊂ Y be an algebraically open set which is free disposal with respect to the algebraic solid convex cone K, and closed under the addition that 0 ∈ v cl(Q). Let the convex cones K and M are pointed and the Slater constraint qualification holds, Assume that T (g(x 0 )) = 0 for x 0 ∈ S and T ∈ Γ. If x 0 is a Q-GBOV for problem given in 5.1, then x 0 is a Q-GBOV for (CP).

Conflicts of Interest:
The author(s) declare that there are no conflicts of interest regarding the publication of this paper.