Estimation of different entropies via Abel–Gontscharoff Green functions and Fink’s identity using Jensen type functionals

Abstract

In this work, we estimated the different entropies like Shannon entropy, Rényi divergences, Csiszár divergence by using Jensen’s type functionals. The Zipf’s–Mandelbrot law and hybrid Zipf’s–Mandelbrot law are used to estimate the Shannon entropy. The Abel–Gontscharoff Green functions and Fink’s Identity are used to construct new inequalities and generalized them for m-convex function.

Citation

Khan, K.A., Niaz, T., Pečarić, Đ. and Pečarić, J. (2020), "Estimation of different entropies via Abel–Gontscharoff Green functions and Fink’s identity using Jensen type functionals", Arab Journal of Mathematical Sciences, Vol. 26 No. 1/2, pp. 15-39. https://doi.org/10.1016/j.ajmsc.2018.12.002

1. Introduction and preliminary results

In recent years many researchers generalized different inequalities using different identities involving green functions, for example in [24] Nasir et al. generalized the Popoviciu inequality using Mongomery identity along with the new green function. Also in [25] Niaz et al. used Fink’s identity along with new Abel–Gontscharoff type Green functions for ‘two point right focal’ to generalize the refinement of Jensen inequality.

The most commonly used words, the largest cities of countries, income of billionaire can be described in terms of Zipf’s law. The f-divergence means the distance between two probability distributions by making an average value, which is weighted by a specified function. As f-divergence, there are other probability distributions like Csiszár f-divergence [11,12], some special case of which is Kullback–Leibler-divergence used to find the appropriate distance between the probability distributions (see [20,21]). The notion of distance is stronger than divergence because it gives the properties of symmetry and triangle inequalities. Probability theory has application in many fields and the divergence between probability distribution has many applications in these fields.

Many natural phenomena like distribution of wealth and income in a society, distribution of face book likes, distribution of football goals follow power law distribution (Zipf’s Law). Like above phenomena, distribution of city sizes also follows Power Law distribution. Auerbach [3] first time gave the idea that the distribution of city size can be well approximated with the help of Pareto distribution (Power Law distribution). This idea was well refined by many researchers but Zipf [32] worked significantly in this field. The distribution of city sizes is investigated by many scholars of the urban economics, like Rosen and Resnick [29], Black and Henderson [4], Ioannides and Overman [19], Soo [30], Anderson and Ge [2] and Bosker et al. [5]. Zipf’s law states that: “The rank of cities with a certain number of inhabitants varies proportional to the city sizes with some negative exponent, say that is close to unit”. In other words, Zipf’s Law states that the product of city sizes and their ranks appear roughly constant. This indicates that the population of the second largest city is one half of the population of the largest city and the third largest city equal to the one third of the population of the largest city and the population of nth city is 1n of the largest city population. This rule is called rank, size rule and also named as Zipf’s Law. Hence Zip’s Law not only shows that the city size distribution follows the Pareto distribution, but also shows that the estimated value of the shape parameter is equal to unity.

In [18] L. Horváth et al. introduced some new functionals based on the f-divergence functionals and obtained some estimates for the new functionals. They obtained f-divergence and Rényi divergence by applying a cyclic refinement of Jensen’s inequality. They also construct some new inequalities for Rényi and Shannon entropies and used Zipf–Mandelbrot law to illustrate the results.

The inequalities involving higher order convexity are used by many physicists in higher dimension problems since the founding of higher order convexity by T. Popoviciu (see [27, p. 15]). It is quite interesting fact that there are some results that are true for convex functions but when we discuss them in higher order convexity they do not remain valid.

In [27, p. 16], the following criteria are given to check the m-convexity of the function.

If f(m) exists, then f is m-convex if and only if f(m)≥0.

In recent years many researchers have generalized the inequalities for m-convex functions; like S. I. Butt et al. generalized the Popoviciu inequality for m-convex function using Taylor’s formula, Lidstone polynomial, Montgomery identity, Fink’s identity, Abel–Gontscharoff interpolation and Hermite interpolating polynomial (see [6–10]).

Since many years Jensen’s inequality has of great interest. The researchers have given the refinement of Jensen’s inequality by defining some new functions (see [16,17]). Like many researchers L. Horváth and J. Pečarić in [14,17], see also [15, p. 26], gave a refinement of Jensen’s inequality for convex function. They defined some essential notions to prove the refinement given as follows:

Let X be a set, and:

P(X) := Power set of X,

|X| :=Number of elements of X,

ℕ :=Set of natural numbers with 0.

Consider q≥1 and r≥2 be fixed integers. Define the functions

Fr,S :{1,…,q}r→{1,…,q}r−1 1≤S≤r,

Fr :{1,…,q}r→P({1,…,q}r−1),

and

Tr :P({1,…,q}r)→P({1,…,q}r−1),

by

Fr,S(i1,…,ir) := (i1,i2,…,iS−1,iS+1,…,ir) 1≤S≤r,

Fr(i1,…,ir) = ⋃S=1r{Fr,S(i1,…,ir)},

and

Tr(I)={φ,I=φ;⋃(i1,…,ir)∈IFr(i1,…,ir),I≠φ.}

Next let the function

αr,i : {1,…,q}r→ℕ 1≤i≤q

defined by

αr,i (i1,…,ir) is the number of occurrences of i in the sequence (i1,…,ir).

For each I∈P ({1,…,q}r) let

αI,i  := ∑(i1,…ir) ∈ Iαr,i(i1,…,ir) 1≤i≤q.

(H1) Let n,m be fixed positive integers such that n≥1, m≥2 and let Im be a subset of {1,…,n}m such that

αIm,i≥1 1≤i≤n.

Introduce the sets Il⊂{1,…,n}l(m−1≥l≥1) inductively by

Il−1  := Tl(Il) m≥l≥2.

Obviously the sets I1={1,…,n}, by (H1) and this insures that αI1,i=1(1≤i≤n). From (H1) we have αIl,i≥1(m−1≥l≥1,1≤i≤n).

For m≥l≥2, and for any (j1,…,jl−1)∈Il−1, let

HIl(j1,…,jl−1) :={((i1,…,il),k)×{1,…,l}|Fl,k(i1,…,il)=(j1,…,jl−1)}.

With the help of these sets they define the functions ηIm,l : Il→ℕ(m≥l≥1) inductively by

ηIm,m(i1,…,im) :=1 (i1,…,im)∈Im;

ηIm,l−1(j1,…,jl−1) := ∑((i1,…il),k)∈HIl(j1,…,jl−1)ηIm,l(i1,…,il).

They define some special expressions for 1≤l≤m, as follows

Am,l=Am,l(Im,x1,…,xn,p1,…,pn; f) := (m−1)!(l−1)!∑(i1,…il)∈IlηIm,l(i1,…,il)×(∑j=1lpijαIm,ij)f(∑j=1lpijαIm,ij xij∑j=1lpijαIm,ij)

and prove the following theorem.

Theorem 1.1.

Assume (H1), and let f : I→R be a convex function where I⊂ℝ is an interval. If x1,…,xn∈I and p1,…,pn are positive real numbers such that ∑S=1npS=1, then

(1)f(∑S=1npSxS)≤Am,m≤Am,m−1≤⋯≤Am,2≤Am,1=∑S=1npSf(xS).

We define the following functionals by taking the differences of refinement of Jensen’s inequality given in (1).

(2) Θ1(f)=Am,r−f(∑S=1npSxS), r=1,…,m,

(3)Θ2(f)=Am,r−Am,k, 1≤r<k≤m.

Under the assumptions of Theorem 1.1, we have

(4)Θi(f)≥0, i=1,2.

Inequalities (4) are reversed if f is concave on I.

In [26], the green function G:[α1,α2]×[α1,α2]→R is defined as

(5)G(u,v)={(u−α2)(v−α1)α2−α1,α1≤v≤u;(v−α2)(u−α1)α2−α1,u≤v≤α2.

The function G is convex with respect to v and due to symmetry also convex with respect to u. One can also note that G is continuous function.

In [31] it is given that any function f : [α1,α2]→R, such that f∈C2([α1,α2]) can be written as

(6)f(u)=α2−uα2−α1f(α1)+u−α1α2−α1f(α2)+∫α2α1G(u,v)f″(v)dv.

2. Inequalities for Csiszár divergence

In [11,12] Csiszár introduced the following notion.

Definition 1.

Let f : ℝ+→ℝ+ be a convex function, let r=(r1,…,rn) and q=(q1,…,qn) be positive probability distributions. Then f-divergence functional is defined by

(7) If(r, q) :=∑i=1nqif(riqi).

And he stated that by defining

(8) f(0) := limx→0+f(x); 0f(00) := 0; 0f(a0) := limx→0+xf(a0), a>0,

we can also use the nonnegative probability distributions as well.

In [18], L. Horv´ath, et al. gave the following functional based on the previous definition.

Definition 2.

Let I⊂ℝ be an interval and let f : I→ℝ be a function, let r=(r1,…,rn)∈Rn and q=(q1,…,qn)∈(0,∞)n such that

rSqS∈I, S=1,…,n.

Then they define the sum I^f(r, q) as

(9)I^f(r, q):=∑S=1nqSf(rSqS).

We apply Theorem 1.1 to I^f(r, q)

Theorem 2.1.

Assume (H1), let I⊂ℝ be an interval and let r=(r1,…,rn) and q=(q1,…,qn) are in (0,∞)n such that

rSqS∈I, S=1,…,n.

(i) If  f : I→ℝ is a convex function, then

I^f(r, q) = ∑S=1nqSf(rSqS)=Am,1[1]≥Am,2[1]≥⋯≥Am,m−1[1]≥Am,m[1]

(10)≥f(∑S=1nrS∑S=1nqS)∑S=1nqS.

where

(11) Am,l[1]=(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)f(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)

If f is a concave function, then inequality signs in (10) are reversed.

(ii) If f:I→R is a function such that x→xf(x)(x∈I) is convex, then

(12)(∑S=1nrS) f(∑S=1nrS∑S=1nqS)≤ Am,m[2]≤Am,m−1[2]≤⋯≤Am,2[2]≤Am,1[2] = ∑S=1nrS f(rSqS)=I^idf(r, q)

where

Am,l[2] = (m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)× f(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)

Proof.

• (i) Consider pS=qS∑S=1n qS and xS=rSqS in Theorem 1.1, we have

(13)f(∑S=1nqS∑S=1nqSrSqS)≤⋯≤(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il) × (∑j=1lqij∑S=1nqSαIm,ij)f(∑j=1lqij∑i=1nqiαIm,ijrijqij∑j=1lqij∑i=1nqiαIm,ij) ≤…≤∑S=1nqS∑i=1nqSf(rSqS)

And taking the sum ∑S=1nqi we have (10).

• (ii) Using  f :=idf (where “id” is the identity function) in Theorem 1.1, we have

  (14)∑S=1npSxSf(∑S=1npSxS)≤⋯≤(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)×(∑j=1lpijαIm,ij) (∑j=1l pijαIm,ij xij∑j=1l pijαIm,ij) f (∑j=1l pijαIm,ij xij∑j=1l pijαIm,ij)≤…≤∑S=1npSxSf(xS)

Now on using pS=qS∑S=1nqS and xS=rSqS,S=1,…,n, we get

(15)∑S=1nqS∑S=1nqSrSqSf(∑S=1nqS∑S=1nqSrSqS)≤⋯≤(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)× (∑j=1lqij∑S=1nqSαIm,ij)(∑j=1lqij∑S=1nqSαIm,ijrijqij∑j=1lqij∑S=1nqSαIm,ij)f(∑j=1lqij∑S=1nqSαIm,ijrijqij∑j=1lqij∑S=1nqSαIm,ij) ≤∑S=1nqS∑S=1nqSrSqSf(rSqS)

On taking sum ∑S=1nqS on both sides, we get (12). □

3. Inequalities for Shannon Entropy

Definition 3

(See [18]). The Shannon entropy of positive probability distribution r=(r1,…,rn) is defined by

(16) S:=−∑S=1nrSlog(rS).

Corollary 3.1.

Assume (H1).

(i) If q=(q1,…,qn)∈(0,∞)n, and the base of log is greater than 1, then

(17) S≤Am,m[3]≤Am,m−1[3]≤⋯≤Am,2[3]≤Am,1[3]=log(n∑S=1nqS)∑S=1nqS,

where

(18) Am,l[3]=−(m−1)!!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)log(∑j=1lqijαIm,ij).

If the base of log is between 0 and 1, then inequality signs in (17) are reversed.

(ii) If q=(q1,…,qn) is a positive probability distribution and the base of log is greater than 1, then we have the estimates for the Shannon entropy of q

(19) S≤Am,m[4]≤Am,m−1[4]≤⋯≤Am,2[4]≤Am,1[4]=log (n),

where

Am,l[4]=−(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)log(∑j=1lqijαIm,ij).

Proof.

(i) Using  f := log and r=(1,…,1) in Theorem 2.1 (i), we get (17).

(ii) It is the special case of (i). □

Definition 4

(See [18])

The Kullback–Leibler divergence between the positive probability distribution r=(r1,…,rn) and q=(q1,…,qn) is defined by

(20)D(r, q):=∑S=1nrilog(riqi).

Corollary 3.2.

Assume (H1).

(i) Let r=(r1,…,rn)∈(0,∞)n and q:=(q1,…,qn)∈(0,∞)n. If the base of log is greater than 1, then

(21) ∑S=1nrSlog(∑S=1nrS∑S=1nqS)≤Am,m[5]≤Am,m−1[5]≤⋯≤Am,2[5]≤Am,1[5] = ∑S=1nrSlog(rSqS)=D(r, q),

where

Am,l[5] = (m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)×log(∑j=1lrijαIm,ij∑j=1lqijαIm,ij).

If the base of log is between 0 and 1, then inequality in (21) is reversed.

(ii) If r and q are positive probability distributions, and the base of l is greater than 1, then we have

(22) D(r, q)=Am,1[6]≥Am,2[6]≥⋯≥Am,m−1[6]≥Am,m[6]≥0,

where

Am,l[6] = (m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lqijαIm,ij)(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)×log(∑j=1lrijαIm,ij∑j=1lqijαIm,ij)

If the base of log is between 0 and 1, then inequality signs in (22) are reversed.

Proof.

• (i) On taking f:=log in Theorem 2.1 (ii), we get (21).

• (ii) Since r and q are positive probability distributions therefore ∑S=1nrS=∑S=1nqS=1, so the smallest term in (21) is given as

(23) ∑S=1nrSlog(∑S=1nrS∑S=1nqS)=0.

Hence for positive probability distribution r and q the (21) will become (22). □

4. Inequalities for Rényi Divergence and Entropy

The Rényi divergence and entropy come from [28].

Definition 5.

Let r = (r1,…,rn) and q:=(q1,…,qn) be positive probability distributions, and let λ≥0, λ≠1.

• (a) The Rényi divergence of order λ is defined by

(24) Dλ(r, q):=1λ−1log(∑i=1nqi(riqi)λ).

• (b) The Rényi entropy of order λ of r is defined by

(25) Hλ(r):=11−λlog(∑i=1nriλ).

The Rényi divergence and the Rényi entropy can also be extended to non-negative probability distributions. If λ→1 in (24), we have the Kullback–Leibler divergence, and if λ→1 in (25), then we have the Shannon entropy. In the next two results, inequalities can be found for the Rényi divergence.

Theorem 4.1.

Assume (H1), let r=(r1,…,rn) and q=(q1,…,qn) are probability distributions.

(i) If 0≤λ≤μ such that λ,μ≠1, and the base of log is greater than 1, then

(26) Dλ(r, q)≤Am,m[7]≤Am,m−1[7]≤⋯≤Am,2[7]≤Am,1[7]=Dμ(r, q),

where

Am,l[7] = 1μ−1log((m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×(∑j=1lrijαIm,ij (rijqij)λ−1∑j=1lrijαIm,ij)μ−1λ−1)

The reverse inequalities hold in (26) if the base of log is between 0 and 1.

(ii) If 1<μ and the base of log is greater than 1, then

(27)D1(r, q) = D(r, q)=∑S=1nrSlog(rSqS)≤Am,m[8]≤Am,m−1[8]≤⋯≤Am,2[8]≤Am,1[8] = Dμ(r, q),

where

Am,l[8] = ≤1μ−1log((m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×exp((μ−1)∑j=1lrijαIm,ijlog(rijqij)∑j=1lrijαIm,ij))

here the base of exp is the same as the base of log, and the reverse inequalities hold if the base of log is between 0 and 1.

(iii) If 0≤λ<1, and the base of log is greater than 1, then

(28)Dλ(r, q)≤Am,m[9]≤Am,m−1[9]≤⋯≤Am,2[9]≤Am,1[9]=D1(r, q),

where

(29) Am,l[9] = 1λ−1(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)

Proof.

By applying Theorem 1.1 with I=(0,∞),  f:(0,∞)→R,  f(t) = tμ−1λ−1

pS:=rS, xS:=(rSqS)λ−1, S=1,…,n,

we have

(30) (∑S=1nqS(rSqS)λ)μ−1λ−1=(∑S=1nrS(rSqS)λ)μ−1λ−1≤…≤(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)μ−1λ−1≤…≤∑S=1nrS((rSqS)λ−1)μ−1λ−1

if either 0≤λ<1<β or 1<λ≤μ, and the reverse inequality in (30) holds if 0≤λ≤β<1. By raising to power 1μ−1, we have from all

(31) (∑S=1nqS(rSqS)λ)1λ−1≤…≤((m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)μ−1λ−1)1μ−1≤…≤(∑S=1nrS((rSqS)λ−1)μ−1λ−1)1μ−1 = (∑S=1nqS(rSqS)μ)1μ−1

Since log is increasing if the base of log is greater than 1, it now follows (26). If the base of log is between 0 and 1, then log is decreasing and therefore inequality in (26) is reversed. If λ=1 and β=1, we have (ii) and (iii) respectively by taking limit, when λ goes to 1. □

Theorem 4.2.

Assume (H1), let r=(r1,…,rn) and q=(q1,…,qn) are probability distributions. If either 0≤λ<1 and the base of log is greater than 1, or 1<λ and the base of log is between 0 and 1, then

(32)1∑S=1nqS(rSqS)λ∑S=1nqS(rSqS)λlog(rSqS)=Am,1[10]≤Am,2[10]≤⋯≤Am,m−1[10]≤Am,m[10]≤Dλ(r,q)≤Am,m[11]≤Am,m[11]≤⋯≤Am,2[11]≤Am,1[11]=D1(r,q)

where

Am,m[10] = 1(λ−1)∑S=1nqS(rSqS)λ(m−1)!(l−1)!∑(i1,…il)∈IlηIm,l(i1,…il)×(∑j=1lrijαIm,ij(rijqij)λ−1)log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)andAm,m[11] = 1λ−1(m−1)!(l−1)!∑(i1,…il)∈IlηIm,l(i1,…il)(∑j=1lrijαIm,ij)×log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij).

The inequalities in (32) are reversed if either 0≤λ<1 and the base of log is between 0 and 1, or 1<λ and the base of l is greater than 1.

Proof.

We prove only the case when 0≤λ<1 and the base of log is greater than 1 and the other cases can be proved similarly. Since 1λ−1<0 and the function log is concave then choose I=(0,∞), f:=log, pS=rS, xS:=(rSqS)λ−1 in Theorem 1.1, we have

(33) Dλ(r, q)=1λ−1log(∑S=1nqS(rSqS)λ)=1λ−1log(∑S=1nrS(rSqS)λ−1)≤⋯≤1λ−1(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)≤⋯≤1λ−1∑S=1nrSlog((rSqS)λ−1)=∑S=1nrSlog(rSqS)=D1(r, q)

and this gives the upper bound for Dλ(r, q).

Since the base of log is greater than 1, the function x ↦ xf(x) (x>0) is convex therefore 11−λ<0 and Theorem 1.1 gives

(34) Dλ(r, q)=1λ−1log(∑S=1nqS(rSqS)λ)=1λ−1(∑S=1nqS(rSqS)λ)(∑S=1nqS(rSqS)λ)log(∑S=1nqS(rSqS)λ)≥⋯≥1λ−1(∑S=1nqS(rSqS)λ)(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij) = 1λ−1(∑S=1nqS(rSqS)λ)(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij(rijqij)λ−1)log(∑j=1lrijαIm,ij(rijqij)λ−1∑j=1lrijαIm,ij)≥⋯≥1λ−1∑S=1nrS(rSqS)λ−1log(rSqS)λ−11∑S=1nrS(rSqS)λ−1 = 1∑S=1nqS(rSqS)λ∑S=1nqS(rSqS)λlog(rSqS)

which give the lower bound of Dλ(r, q). □

By using Theorems 4.1, 4.2 and Definition 5, some inequalities of Rényi entropy are obtained. Let 1n=(1n,…,1n) be a discrete probability distribution.

Corollary 4.3.

Assume (H1), let r=(r1,…,rn) and q=(q1,…,qn) are positive probability distributions.

• (i) If 0≤λ≤μ, λ,μ≠1, and the base of log is greater than 1, then

(35)Hλ(r)=log(n)−Dλ(r,1n)≥Am,m[12]≥Am,m[12]≥⋯Am,2[12]≥Am,1[12]=Hμ(r),

where

Am,l[12]=11−μlog((m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)×(∑j=1lrijαIm,ij)×(∑j=1lrijλαIm,ij∑j=1lrijαIm,ij)μ−1λ−1).

The reverse inequalities hold in (35) if the base of log is between 0 and 1.

• (ii) If  1<μ and base of log is greater than 1, then

(36) S=−∑S=1npilog(pi)≥Am,m[13]≥Am,m−1[13]≥⋯≥Am,2[13]≥Am,1[13]=Hμ(r)

where

Am,l[13] = log(n)+11−μlog((m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×exp((μ−1)∑j=1lrijαIm,ijlog(nrij)∑j=1lrijαIm,ij)),

the base of exp is the same as the base of log. The inequalities in (36) are reversed if the base of log is between 0 and 1.

• (iii) If 0≤λ<1, and the base of log is greater than 1, then

(37) Hλ(r)≥Am,m[14]≥Am,m−1[14]≥⋯≥Am,2[14]≤Am,1[14]=S,

where

(38) Am,m[14]=11−λ(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)×log(∑j=1lrijλαIm,ij∑j=1lrijαIm,ij).

The inequalities in (37) are reversed if the base of log is between 0 and 1.

Proof.

• (i) Suppose q=1n then from (24), we have

(39)Dλ(r, q)=1λ−1log(∑S=1nnλ−1rSλ)=log(n)+1λ−1log(∑S=1nrSλ),

therefore we have

(40) Hλ(r)=log(n)−Dλ(r,1n).

Now using Theorem 4.1 (i) and (40), we get

(41)Hλ(r)=log(n)−Dλ(r,1n)≥⋯≥log(n)−1μ−1×log(nμ−1(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)×(∑j=1lrijαIm,ij)(∑j=1lrijλαIm,ij∑j=1lrijαIm,ij)μ−1λ−1)≥⋯≥log(n)−Dμ(r, q)=Hμ(r),

(ii) and (iii) can be proved similarly. □

Corollary 4.4.

Assume (H1) and let r=(r1,…,rn) and q=(q1,…,qn) are positive probability distributions.

If either 0≤λ<1 and the base of log is greater than 1, or 1<λ and the base of log is between 0 and 1, then

(42) −1∑S=1nrSλ∑S=1nrSλlog(rS) = Am,1[15]≥Am,2[15]≥⋯≥Am,m−1[15]≥Am,m[15]≥Hλ(r)≥Am,m[16]≥Am,m−1[16]≥⋯Am,2[16]≥Am,1[16]=H(r),

where

Am,l[15] = 1(λ-1)∑S=1nrSλ(m-1)!(l-1)!∑(i1,…,il)ÎIlηIm,l(i1,…,il)(∑j=1lrijλαIm,ij)log(nλ-1∑j=1lrijλαIm,ij∑j=1lrijαIm,ij)andAm,1[16] = 11-λ(m-1)!(l-1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1lrijαIm,ij)log(∑j=1lrijλαIm,ij∑j=1lrijαIm,ij).

The inequalities in (42) are reversed if either 0≤λ<1 and the base of log is between 0 and 1, or 1<λ and the base of log is greater than 1.

Proof.

The proof is similar to Corollary 4.3 by using Theorem 4.2. □

5. Inequalities by using Zipf–Mandelbrot law

In probability theory and statistics, the Zipf–Mandelbrot law is a distribution. It is a power law distribution on ranked data, named after the linguist G. K. Zipf who suggests a simpler distribution called Zipf’s law. The Zipf’s law is defined as follows (see [32]).

Definition 6.

Let N be a number of elements, S be their rank and t be the value of exponent characterizing the distribution. Zipf’s law then predicts that out of a population of N elements, the normalized frequency of element of rank S, f(S,N,t) is

(43)f(S,N,t)=1St∑j=1N 1jt.

The Zipf–Mandelbrot law is defined as follows (see [22]).

Definition 7.

Zipf–Mandelbrot law is a discrete probability distribution depending on three parameters N∈{1,2,…,},q∈[0,∞) and t>0, and is defined by

(44)f(S;N,q,t):=1(S+q)tHN,q,t, S=1,…,N,

where

(45)HN,q,t=∑j=1N1(j+q)t.

If the total mass of the law is taken over all N, then for q≥0, t>1, S∈N, density function of Zipf–Mandelbrot law becomes

(46)f(S;q,t)=1(S+q)tHq,t,

where

(47)Hq,t=∑j=1∞1(j+q)t.

For q=0, the Zipf–Mandelbrot law (44) becomes Zipf’s law (43).

Conclusion 5.1.

Assume (H1), let r be a Zipf–Mandelbrot law, by Corollary 4.3 (iii), we get: If 0≤λ<1, and the base of log is greater than 1, then

(48)Hλ(r)=11−λlog(1HN,q,tλ∑S=1n1(S+q)λS)≥⋯≥11−λ(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)(∑j=1l1αIm,ij(ij+q)HN.q,t)log(1HN,q,tλ−1∑j=1l1αIm,ij(ij−q)λS∑j=1l1αIm,ij(ij−q)S)≥⋯≥tHN,q,t∑S=1Nlog(S+q)(S+q)t+log(HN,q,t)=S.

The inequalities in (48) are reversed if the base of log is between 0 and 1.

Conclusion 5.2.

Assume (H1), let r1 and r2 be the Zipf–Mandelbort law with parameters N∈{1,2,…}, q1,q2∈[0,∞) and S1,S2>0, respectively, then from Corollary 3.2 (ii), we have if the base of l is greater than 1, then

(49) D¯(r1,r2)=∑S=1n1(S+q1)t1HN,q1,t1log((S+q2)t2HN,q2,t2(S+q1)t1HN,q2,t1)≥⋯≥(m−1)!(l−1)!∑(i1,…,il)∈IlηIm,l(i1,…,il)×(∑j=1l1(ij+q2)t2HN,q2,t2αIm,ij)(∑j=1l1(ij+q1)t1HN,q1,t1αIm,ij∑j=1l1(ij+q2)t2HN,q2,t2αIm,ij)×log(∑j=1l1(ij+q1)t1HN,q1,t1αIm,ij∑j=1l1(ij+q2)t2HN,q2,t2αIm,ij)≥⋯≥0.

The inequalities in (49) are reversed if the base of l is between 0 and 1.

6. Shannon entropy, Zipf–Mandelbrot law and hybrid Zipf–Mandelbrot law

Here we maximize the Shannon entropy using method of Lagrange multiplier under some equations constraints and get the Zipf–Mandelbrot law.

Theorem 6.1.

If J={1,2,…,N}, for a given q≥0 a probability distribution that maximizes the Shannon entropy under the constraints

∑S∈ JrS=1,∑S∈ JrS(In(S+q)) := ψ,

is Zipf–Mandelbrot law.

Proof.

If J={1,2,…,N}, we set the Lagrange multipliers λ and t and consider the expression

S˜=−∑S=1NrSln rS−λ(∑S=1NrS−1)−t(∑S=1NrSln(S+q)−ψ)

Just for the sake of convenience, replace λ by ln λ−1, thus the last expression gives

S˜=−∑S=1NrSln rS−(ln λ−1)(∑S=1NrS−1)−t(∑S=1NrSln(S+q)−ψ)

From S˜rS = 0, for S=1,2,…,N, we get

rS=1λ(S+q)t,

and on using the constraint ∑S=1NrS=1, we have

λ=∑S=1N(1(S+1)t)

where t>0, concluding that

rS=1(S+q)tHN,q,t, S=1,2,…,N. □

Remark 6.2.

Observe that the Zipf–Mandelbrot law and Shannon Entropy can be bounded from above (see [23]).

S=−∑S=1Nf(S,N,q,t)ln  f(S,N,q,t)≤−∑S=1Nf(S,N,q,t)ln qS

where (q1,…,qN) is a positive N-tuple such that ∑S=1NqS=1.

Theorem 6.3.

If J={1,…,N}, then probability distribution that maximizes Shannon entropy under constraints

∑S∈JrS:=1, ∑S∈JrS ln(S+q) := Ψ, ∑S∈JSrS := η

is hybrid Zipf–Mandelbrot law given as

rS=wS(S+q)kΦ∗(k,q,w), S∈J,

where

ΦJ(k,q,w)=∑S∈ JwS(S+q)k.

Proof.

First consider J={1,…,N}, we set the Lagrange multiplier and consider the expression

S˜=−∑S=1NrS ln rS+ln w(∑S=1NSrS−η)−(ln λ−1)(∑S=1NrS−1)−k(∑S=1NrS ln(S+q)−Ψ).

On setting S˜rS = 0, for S=1,…,N, we get

−ln rS+S ln w−ln λ−k ln(S+q)=0,

after solving for rS, we get λ=∑S=1NwS(S+q)k, and we recognize this as the partial sum of Lerch’s transcendent that we will denote by

ΦN*(k,q,w)=∑S=1NwS(S+q)k with w≥0,k>0.

□

Remark 6.4.

Observe that for Zipf–Mandelbrot law, Shannon entropy can be bounded from above (see [23]).

S=−∑S=1Nfh(S,N,q,k)ln fh(S,N,q,k)≤−∑S=1Nfh(S,N,q,k)ln qS

where (q1,…,qN) is any positive N-tuple such that ∑S=1NqS=1.

Under the assumption of Theorem 2.1 (i), define the non-negative functionals as follows:

(50)Θ3(f)=Am,r[1]−f(∑S=1nrS∑S=1nqS)∑S=1nqS, r=1,…,m,

(51)Θ4(f)=Am,r[1]−Am,k[1], 1≤r<k≤m.

Under the assumption of Theorem 2.1 (ii), define the non-negative functionals as follows:

(52)Θ5(f)=Am,r[2]−(∑S=1nrS)f(∑S=1nrS∑S=1nqS), r=1,…,m,

(53)Θ6(f)=Am,r[2]−Am,k[2], 1≤r<k≤m.

Under the assumption of Corollary 3.1 (i), define the following non-negative functionals

(54)Θ7(f)=Am,r[3]+∑i=1nqilog(qi), r=1,…,n

(55) Θ8(f)=Am,r[3]−Am,k[3], 1≤r<k≤m.

Under the assumption of Corollary 3.1 (ii), define the following non-negative functionals as

(56)Θ9(f)=Am,r[4]−S, r=1,…,m

(57)Θ10(f)=Am,r[4]−Am,k[4], 1≤r<k≤m.

Under the assumption of Corollary 3.2 (i), let us define the non-negative functionals as follows:

(58) Θ11(f)=Am,r[5]−∑S=1nrSlog(∑S=1nlogrn∑S=1nqS), r=1,…,m

(59)Θ12(f)=Am,r[5]−Am,k[5], 1≤r<k≤m.

Under the assumption of Corollary 3.2 (ii), define the non-negative functionals as follows

(60)Θ13(f)=Am,r[6]−Am,k[6], 1≤r<k≤m.

Under the assumption of Theorem 4.1 (i), consider the following functionals

(61)Θ14(f)=Am,r[7]−Dλ(r,q), r=1,…,m

(62)Θ15(f)=Am,r[7]−Am,k[7], 1≤r<k≤m.

Under the assumption of Theorem 4.1 (ii), consider the following functionals:

(63)Θ16(f)=Am,r[8]−D1(r,q), r=1,…,m

(64)Θ17(f)=Am,r[8]−Am,k[8], 1≤r<k≤m.

Under the assumption of Theorem 4.1 (iii), consider the following functionals:

(65) Θ18(f)=Am,r[9]−Dλ(r,q), r=1,…,m

(66)Θ19(f)=Am,r[9]−Am,k[9], 1≤r<k≤m.

Under the assumption of Theorem 4.2 consider the following non-negative functionals

(67)Θ20(f)=Dλ(r,q)−Am,r[10], r=1,…,m

(68) Θ21(f)=Am,k[10]−Am,r[10], 1≤r<k≤m.

(69) Θ22(f)=Am,r[11]−Dλ(r,q), r=1,…,m

(70) Θ23(f)=Am,r[11]−Am,r[11], 1≤r<k≤m.

(71)Θ24(f)=Am,r[11]−Am,k[10], r=1,…,m, k=1,…,m.

Under the assumption of Corollary 4.3 (i), consider the following non-negative functionals

(72)Θ25(f)=Hλ(r)−Am,r[12], r=1,…,m

(73) Θ26(f)=Am,k[12]−Am,r[12], 1≤r<k≤m.

Under the assumption of Corollary 4.3 (ii), consider the following functionals

(74)Θ27(f)=S−Am,r[13], r=1,…,m

(75) Θ28(f)=Am,k[13]−Am,r[13], 1≤r<k≤m.

Under the assumption of Corollary 4.3 (iii), consider the following functionals

(76)Θ29(f)=Hλ(r)−Am,r[14], r=1,…,m

(77) Θ30(f)=Am,k[14]−Am,r[14], 1≤r<k≤m.

Under the assumption of Corollary 4.4, define the following functionals

(78)Θ31=Am,r[15]−Hλ(r), r=1,…,m

(79) Θ32=Am,r[15]−Am,k[15], 1≤r<k≤m.

(80) Θ33=Hλ(r)−Am,r[16], r=1,…,m

(81) Θ34=Am,k[16]−Am,r[16], 1≤r<k≤m.

(82)Θ35=Am,r[15]−Am,k[16], r=1,…,m, k=1,…,m.

7. Generalization of refinement of Jensen’s, Rényi and Shannon type inequalities Fink’s Identity and Abel–Gontscharoff Green function

In [13], A. M. Fink gave the following result.

Let  f:[α1,α2]→ℝ, where [α1,α2] be an interval, is a function such that f(n−1) is absolutely continuous then the following identity holds

(83)f(z)=nα2−α1∫α1α2f(ζ)dζ+∑λ=1n−1n−λλ!(f(λ−1)(α2)(z−α2)λ−f(λ−1)(α1)(z−α1)λα2−α1)+1(n−1)!(α2−α1)∫α1α2(z−ζ)n−1Fα1α2(ζ,z)f(n)(ζ)dζ,

where

(84)Fα1α2(ζ,z)={ζ−α1,α1≤ζ≤z≤α2;ζ−α2,α1≤z<ζ≤α2.

The complete reference about Abel–Gontscharoff polynomial and theorem for ‘two-point right focal’ problem is given in [1].

The Abel–Gontscharoff polynomial for ‘two-point right focal’ interpolating polynomial for n=2 can be given as

(85)f(z)=f(α1)+(z−α1)f′(α2)+∫α1α2G1(z,w)f″(w)dw,

where

(86)G1(z,w)={α1−w,α1≤w≤z;α1−z,z≤w≤α2.

In [8], S. I. Butt et al. gave some new types of Green functions defined as

(87)G2(z,w)={α2−z,α1≤w≤z;α2−w,z≤w≤α2,

(88)G3(z,w)={z−α1,α1≤w≤z;w−α1,z≤w≤α2,

(89)G4(z,w)={α2−w,α1≤w≤z;α2−z,z≤w≤α2,

Figure 1 shows the graph of Green functions Gi(z,w),i=1,2,3,4 defined in (86)–(89) respectively for fixed value of w. They also introduced some new Abel–Gontscharoff type identities by using these new Green functions in the following lemma.

Lemma A.

Let f:[α1,α2] be a twice differentiable function and Gk (k=2,3,4) be the ‘two-point right focal problem’-type Green functions defined by (87)–(89). Then the following identities hold:

(90)f(z)=f(α2)−(α2−z)f′(α1)−∫α1α2G2(z,w)f″(w)dw,

(91)f(z)=f(α2)−(α2−α1)f′(α2)+(z−α1) f′(α1)+∫α1α2G3(z,w)f″(w)dw,

(92)f(z)=f(α1)+(α2−α1) f′(α1)−(α2−z) f′(α2)+∫α1α2G4(z,w) f″(w)dw.

Theorem 7.1.

Assume (H1), and let f:I=[α1,α2]→R be a function such that for m≥3 (an integer) f(m−1) is absolutely continuous. Also, let x1,…,xn∈I, p1,…,pn, be positive real numbers such that ∑i=1npi=1. Assume that Fα1α2, Gk (k=1,2,3,4) and Θi (i=1,…,35) are the same as defined in (84), (86)–(89), (2), (3), (50)–(82) respectively.

Then:

1. For k=1,3,4 we have the following identities:

   (93)Θi(f)=(m−2)(f′(α2)−f′(α1)α2−α1)∫α1α2Θi(Gk(⋅,w))dw+1α2−α1∫α1α2Θi(Gk(⋅,w))×∑λ=1m−3(m−2−λλ!)(f(λ+1)(α2)(w−α2)λ−f(λ+1)(α1)(w−α1)λ)dw+1(m−3)!(α2−α1)∫α1α2f(m)(ζ)×(∫α1α2Θi(Gk(⋅,w))(w−ζ)m−3Fα1α2α2(ζ,w)dw)dζ, i=1,…,35.

2. For k=2 we have

(94)Θi(f)=(−1)(m−2)(f′(α2)−f′(α1)α2−α1)∫α1α2Θi(G2(⋅,w))dw+(−1)α2−α1∫α1α2Θi(G2(⋅,w))×∑λ=1m−3(m−2−λλ!)(f(λ+1)(α2)(w−α2)λ−f(λ+1)(α1)(w−α1)λ)dw+(−1)(m−3)!(α2−α1)∫α1α2f(m)(ζ)×(∫α1α2Θi(G2(⋅,w))(w−ζ)m−3Fα1α2(ζ,w)dw)dζ, i=1,…,35.

Proof.

• (i) Using Abel–Gontsharoff-typeidentities (85), (91), (92) in Θi(f), i=1,…,35, and using properties of Θi(f), we get

(95)Θi(f)=∫α1α2Θi(Gk(⋅,w))f″(w)dw, i=1,2.

From identity (83), we get

(96) f′(w)=(m−2)(f′(α2)−f′(α1)α2−α1)+∑λ=1m−3(m−2−λλ!)(f(λ)(α2)(w−α2)λ−1−f(λ)(α2)(w−α2)λ−1α2−α1)+1(m−3)!(α2−α1)∫α1α2(w−ζ)m−3Fα1α2(ζ,w)f(m)(ζ)dζ.

Using (95) and (96) and applying Fubini’s theorem we get the result (93) for k=1,3,4.

• (ii) Substituting Abel–Gontscharoff-typeinequality (90) in Θi(f), i=1,…,35, and following similar steps to (i), we get (94). □

Theorem 7.2.

Assume (H1), and let f:I=[α1,α2]→R be a function such that for m≥3 (an integer) f(m−1) is absolutely continuous. Also, let x1,…,xn∈I, p1,…,pn are positive real numbers such that ∑i=1npi=1. Assume that Fα1α2, Gk (k=1,2,3,4) and Θi (i=1,2) are the same as defined in (84), (86)–(89), (2), (3), (50)–(82) respectively. For m≥3 assume that

(97)∫α1α2Θi(Gk(⋅,ζ))(w−ζ)m−3Fα1α2(ζ,w)dw≥0,ζ∈[α1,α2], i=1,…,35,

for k=1,3,4. If f is an m-convex function, then

• (i) For k=1,3,4, the following holds:

(98)Θi(f)≥(m−2)(f′(α2)−f′(α1)α2−α1)∫α1α2Θi(Gk(⋅,w))dw+1α2−α1∫α1α2Θi(Gk(⋅,w))×∑λ=1m−3(m−2−λλ!)(f(λ+1)(α2)(w−α2)λ−f(λ+1)(α1)(w−α1)λ)dw, i=1,…,35.

• (ii) For k=2, we have

(99) Θi(f)≤(−1)(m−2)(f′(α2)−f″(α1)α2−α1)∫α1α2Θi(G2(⋅,w))dw+(−1)α2−α1∫α1α2Θi(G2(⋅,w))×∑λ=1m−3(m−2−λλ!)(f(λ+1)(α2)(w−α2)λ−f(λ+1)(α1)(w−α1)λ)dw, i=1,…,35.

Proof.

(i) Since f(m−1) is absolutely continuous on [α1,α2], f(m) exists almost everywhere. Also, since f is m-convex therefore we have f(m)(ζ)≥0 for a.e. on [α1,α2]. So, applying Theorem 1.1, we obtain (98).

• (ii) Similar to (i). □

Remark A.

We can investigate the bounds for the identities related to the generalization of refinement of Jensen inequality using inequalities for the C˘ebys˘ev functional and some results relating to the Gr¨uss and Ostrowski type inequalities can be constructed as given in Section 3 of [6]. Also we can construct the non-negative functionals from inequalities (98)–(99) and give related mean value theorems and we can construct the new families of m-exponentially convex functions and Cauchy means related to these functionals as given in Section 4 of [6].

Figures

Figure 1

Graph of Green functions for fix w.