Lobachevskii Journal of Mathematics http://ljm.ksu.ru Vol. 17, 2005, 11 – 23

Md. Azizul Baten
ON THE SMOOTHNESS OF SOLUTIONS OF LINEAR-QUADRATIC REGULATOR FOR DEGENERATE DIFFUSIONS
(submitted by A. Lapin)

Abstract. The paper studies the smoothness of solutions of the degenerate Hamilton-Jacobi-Bellman (HJB) equation associated with a linear-quadratic regulator control problem. We establish the existence of a classical solution of the degenerate HJB equation associated with this problem by the technique of viscosity solutions, and hence derive an optimal control from the optimality conditions in the HJB equation.

________________

Key words and phrases. Stochastic differential equation, Hamilton-Jacobi-Bellman equation, Linear-Quadratic problem, Viscosity solutions, Applications to control theory.

2000 Mathematical Subject Classiﬁcation. 60H10, 49N10, 49J15, 49L25, 58E25.

1. Introduction

We are concerned with the quadratic control problem to minimize the expected cost with discount factor $β > 0$ :

J (c) = E [\int_{0}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t]

(1)

over $c \in A$ subject to the degenerate stochastic differential equation

d x_{t} = [A x_{t} + c_{t}] d t + σ x_{t} d w_{t}, x_{0} = x \in R, t \geq 0,

(2)

for non-zero constants $A, σ \neq 0$ , and a continuous function $h$ on $R,$ where $w_{t}$ is a one-dimensional standard Brownian motion on a complete probability space $(Ω, ℱ, P)$ endowed with the natural ﬁltration $ℱ_{t}$ generated by $σ (w_{s}, s \leq t)$ , and $A$ denotes the class of all $ℱ_{t} -$ progressively measurable processes $c = (c_{t})$ with $J (c) < \infty$ .

This kind of stochastic control problem has been studied by many authors [3, 6] for non-degenerate diffusions to (1) and (2). We also assume that $h$ satisﬁes the following properties:

h (x) \geq 0 : convex;

(3)

There exists C > 0 such that h (x) \leq C (1 + ∣ x ∣^{n}), x ε R,

(4)

for some constant $C > 0, n \geq 2 .$ We refer to [5] for the quadratic case of degenerate diffusions related to Riccati equations in case of $h (x) = C x^{2}$ and $n = 2$ with inﬁnite horizon.

The purpose of this paper is to show the existence of a smooth solution $u$ of the associated Hamilton-Jacobi-Bellman (in short, HJB) equation of the form:

- β u + \frac{1}{2} σ^{2} x^{2} u^{''} + A x u^{'} + {min}_{r \in R} (r^{2} + r u^{'}) + h (x) = 0 in R,

(5)

and to give a synthesis of optimal control. Our method consists in ﬁnding the viscosity solution $u$ of (5) [4, 6], by the limit of the solution $v = v_{L}, L > 0,$ to the HJB equation

- β v_{L} + \frac{1}{2} σ^{2} x^{2} v_{L}^{''} + A x v_{L}^{'} + {min}_{∣ r ∣ \leq L} (r^{2} + r v_{L}^{'}) + h (x) = 0 in R,

(6)

as $L \to \infty,$ and then in considering the smoothness of $u$ by it’s convexity. To show the existence of the viscosity solution $v_{L},$ we assume that $h$ has the following property: there exists $C_{ρ} > 0$ , for any $ρ > 0$ , such that

∣ h (x) - h (y) ∣ \leq C_{ρ} ∣ x - y ∣^{n} + ρ (1 + ∣ x ∣^{n} + ∣ y ∣^{n}), \forall x, y \in R,

(7)

for a ﬁxed integer $n \geq 2$ .

This condition acts as the uniform continuity of $h$ with order $n,$ and plays an important role for the existence of viscosity solutions [7, 8]. We notice that (7) holds for $h (x) = ∣ x ∣^{\bar{n}}, \bar{n} \in [2, n]$ ,

In $§$ 2 we show that $u (x) : = {lim}_{L \to \infty} v_{L} (x)$ is a viscosity solution of $(5)$ , as $L \to \infty$ . $§$ 3 is devoted to the study of smoothness of $u$ . Finally in $§$ 4 we present an optimal control to the optimization problem (1) and (2).

2. Convergences of the value function

In this subsection we show that $v_{L} (x)$ is a viscosity solution of the Bellman equation (5) for any ﬁxed $L > 0$ , and then converges to a viscosity solution $u (x)$ of the Bellman equation (5). In order to introduce solutions in the viscosity sense, given a continuous and degenerate elliptic map $H : R \times R \times R \times R \to R$ , we recall by [4] the deﬁnition of viscosity solutions of

H (x, w, w^{'}, w^{''}) = 0 in R .

(8)

Deﬁnition 2.1. $w \in C (R)$ is called a viscosity subsolution (resp., supersolution) of (8) if, whenever for $ϕ \in C^{2} (R), w - ϕ$ attains its local maximum (resp., minimum) at $x \in R$ , then

\begin{array}{rcl} H (x, w (x), ϕ^{'} (x), ϕ^{''} (x)) \leq 0 & (9) \end{array}

\begin{array}{rcl} resp., H (x, w (x), ϕ^{'} (x), ϕ^{''} (x)) \geq 0 . & (10) \end{array}

We also call $w \in C (R)$ a viscosity solution of (8) if it is both viscosity sub- and supersolution of (8).
According to Crandall, Ishii and Lions [4] and Fleming and Soner [6] this deﬁnition is equivalent to the following: for any $x \in R$ ,

\begin{array}{rcl} H (x, w (x), p, q) \leq 0 for (p, q) \in J^{2, +} w (x) \end{array}

\begin{array}{rcl} H (x, w (x), p, q) \geq 0 for (p, q) \in J^{2, -} w (x), \end{array}

where $J^{2, +}$ and $J^{2, -}$ are the second-order superjets and subjets deﬁned by

\begin{matrix} J^{2, +} w (x) \\ = {(p, q) \in R^{2} : {limsup}_{y \to x} \frac{w (y) - w (x) - p (y - x) - \frac{1}{2} q ∣ y - x ∣^{2}}{∣ y - x ∣^{2}} \leq 0}, \\ J^{2, -} w (x) \\ = {(p, q) \in R^{2} : lim {inf}_{y \to x} \frac{w (y) - w (x) - p (y - x) - \frac{1}{2} q ∣ y - x ∣^{2}}{∣ y - x ∣^{2}} \geq 0} . \end{matrix}

Let us deﬁne the value function $v_{L} (x) = {inf}_{c \in A_{L}} J (c),$ where $A_{L} = {c = (c_{t}) \in A : ∣ c_{t} ∣ \leq L for all t \geq 0}$ . We assume that there exists $β_{0} \in (0, β)$ satisfying

- β_{0} + σ^{2} n (2 n - 1) + 2 n ∣ A ∣ < 0,

(11)

and we set $f_{k} (x) = γ + ∣ x ∣^{k}$ for any $2 \leq k \leq 2 n$ and a constant $γ \geq 1$ chosen later.

Lemma 2.2. Assume (11). Then there exist $γ \geq 1$ and $η > 0,$ depending on $L, k,$ such that

- β_{0} f_{k} + \frac{1}{2} σ^{2} x^{2} f_{k}^{''} + A x f_{k}^{'} + {max}_{∣ r ∣ \leq L} (r^{2} + r f_{k}^{'}) + η f_{k} \leq 0

(12)

Further

E [\int_{0}^{τ} e^{- β_{0} s} η f_{k} (x_{s}) d s + e^{- β_{0} τ} f_{k} (x_{τ})] \leq f_{k} (x) for 2 \leq k \leq 2 n,

(13)

E [{sup}_{t} e^{- β_{0} t} f_{k} (x_{t})] < \infty for 2 \leq k \leq n,

(14)

where $τ$ is any stopping time and $x_{t}$ is the response to $(c_{t}) \in A_{L}$ .

Proof. By (11), we choose $η \in (0, β_{0})$ such that

\begin{array}{rcl} - β_{0} + \frac{1}{2} σ^{2} k (k - 1) + k ∣ A ∣ + η < 0, & (15) \end{array}

and then $γ \geq 1$ such that

\begin{array}{rcl} (- β_{0} + \frac{1}{2} σ^{2} k (k - 1) + k ∣ A ∣ + η) ∣ x ∣^{k} + L k ∣ x ∣^{k - 1} + (L^{2} + η γ - β_{0} γ) \leq 0 . \end{array}

Then (12) is immediate. By (12) and Ito’s formula, we deduce (13). Moreover, by moment inequalities for martingales we get

\begin{array}{rcl} E [{sup}_{t} e^{- β_{0} t} f_{k} (x_{t})] & \leq & f_{k} (x) + E [{sup}_{t} ∣ \int_{0}^{t} e^{- β_{0} s} f_{k}^{'} (x_{s}) σ x_{s} d w_{s} ∣] \\ \leq & f_{k} (x) + K E [{(\int_{0}^{\infty} e^{- 2 β_{0} s} σ^{2} ∣ x_{s} ∣^{2 k} d s)}^{1 ∕ 2}], \end{array}

for some constant $K > 0$ . Therefore (14) follows from this relation together with (13).

Theorem 2.3. We assume (3), (4), (7) and (11). Then

\begin{array}{rcl} v_{L} satisﬁes (3), (4), (7), & (16) \end{array}

and the dynamic programming principle holds, i.e.,

\begin{array}{rcl} v_{L} (x) = {inf}_{c \in A_{L}} E [\int_{0}^{τ} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t + e^{- β τ} v_{L} (x_{τ})] & (17) \end{array}

for any stopping time $τ$ .

Proof. We suppress $L$ of $v_{L}$ for simplicity. The convexity of $v$ follows from the same line as [5,Chap. 4, Lemma 10.6]. Let $x_{t}^{0}$ be the unique solution of

\begin{array}{rcl} d x_{t}^{0} = A x_{t}^{0} d t + σ x_{t}^{0} d w_{t}, x_{0}^{0} = x . & (18) \end{array}

Then, by (13) and (4)

v (x) \leq E [\int_{0}^{\infty} e^{- β t} h (x_{t}^{0}) d t] \leq C E [\int_{0}^{\infty} e^{- β_{0} t} f_{n} (x_{t}^{0}) d t] \leq C f_{n} (x) ∕ η .

(19)

For the solution $y_{t}$ of (2) with $y_{0} = y$ , it is clear that $x_{t} - y_{t}$ satisﬁes (18) with initial condition $x - y$ . We note by (15) with $k = n$ and Ito’s formula that

\begin{array}{rcl} E [e^{- β_{0} t} ∣ x_{t}^{0} ∣^{n}] \leq ∣ x ∣^{n} . \end{array}

Thus by (7) and (13)

\begin{matrix} ∣ v (x) - v (y) ∣ \leq {sup}_{c \in A_{L}} E [\int_{0}^{\infty} e^{- β t} ∣ h (x_{t}) - h (y_{t}) ∣ d t] \\ \leq {sup}_{c \in A_{L}} E [\int_{0}^{\infty} e^{- β t} {C_{ρ} ∣ x_{t} - y_{t} ∣^{n} + ρ (1 + ∣ x_{t} ∣^{n} + ∣ y_{t} ∣^{n})} d t] \\ \leq {sup}_{c \in A_{L}} \int_{0}^{\infty} e^{- β t} {C_{ρ} ∣ x - y ∣^{n} e^{β_{0} t} + ρ (h_{n} (x) + h_{n} (y)) e^{β_{0} t}} d t \\ \leq \frac{1}{β - β_{0}} [C_{ρ} ∣ x - y ∣^{n} + 2 ρ γ (1 + ∣ x ∣^{n} + ∣ y ∣^{n})] & (20) \end{matrix}

Therefore we get (16).

To prove (17), we denote by $v^{r} (x)$ the right hand side of (17). By the formal Markov property

\begin{array}{rcl} E [\int_{τ}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t ∣ ℱ_{τ}] & = & E [\int_{0}^{\infty} e^{- β (t + τ)} {h (x_{τ + t}) + ∣ c_{τ + t} ∣^{2}} d t ∣ ℱ_{τ}] \\ = & e^{- β τ} J_{\tilde{c}} (x_{τ}), \end{array}

with $\tilde{c}$ equal to $c$ shifted by $τ$ . Thus

\begin{array}{rcl} J_{c} (x) & = & E [\int_{0}^{τ} + \int_{τ}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t] \\ = & E [\int_{0}^{τ} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t] + E [\int_{τ}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t ∕ ℱ_{τ}] \\ \geq & E [\int_{0}^{τ} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t + e^{- β τ} v_{L} (x_{τ})] . \end{array}

It is known in [6, 9] that this formal argument can be veriﬁed, and we deduce $v_{L} (x) \geq v^{r} (x) .$

To prove the reverse inequality, let $ρ > 0$ be arbitrary. We set

V_{c} (x) = E [\int_{0}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t]

(21)

By the same calculation as (20), there exists $C_{ρ} > 0$ such that

∣ V_{c} (x) - V_{c} (y) ∣ \leq C_{ρ} ∣ x - y ∣^{n} + ρ (1 + ∣ x ∣^{n} + ∣ y ∣^{n}) .

Take $0 < δ < 1$ with $C_{ρ} δ^{n} < ρ$ . Then, we have for $∣ x - y ∣ < δ$ ,

\begin{array}{rcl} ∣ v (x) - v (y) ∣ & \leq & {sup}_{c \in A_{L}} ∣ V_{c} (x) - V_{c} (y) ∣ \\ \leq & ρ (2 + ∣ x ∣^{n} + ∣ y ∣^{n}) \\ \leq & ρ [2 + ∣ x ∣^{n} + 2^{n} (1 + ∣ x ∣^{n})] \\ = & ρ [(2 + 2^{n}) + (1 + 2^{n}) ∣ x ∣^{n}] \\ \leq & Ξ_{ρ} (x) : = ρ (2^{n} + 2) (1 + ∣ x ∣^{n}) . \end{array}

Let ${S_{i}}$ be a sequence of disjoint subsets of $R$ such that

d i a m (S_{i}) < δ a n d \cup_{i} S_{i} = R .

For any $i$ , we take $x^{(i)} \in S_{i}$ and $c^{(i)} \in A_{L}$ such that

V_{c^{(i)}} (x^{(i)}) \leq {inf}_{c \in A_{L}} V_{c} (x^{(i)}) + ρ .

Deﬁne $c^{τ} \in A_{L}$ by

c_{t}^{τ} = c_{t} 1_{{t < τ}} + c_{t - τ}^{(i)} 1_{{x_{τ} \in S_{i}}} 1_{{t \geq τ}}, for x_{τ} \in S_{i} .

Hence,

\begin{array}{rcl} V_{c^{(i)}} (x_{τ}) & = & V_{c^{(i)}} (x_{τ}) - V_{c^{(i)}} (x^{(i)}) + V_{c^{(i)}} (x^{(i)}) \\ \leq & Ξ_{ρ} (x_{τ}) + V_{c^{(i)}} (x^{(i)}) \\ \leq & Ξ_{ρ} (x_{τ}) + {inf}_{c \in A_{L}} V_{c} (x^{(i)}) + ρ \\ = & Ξ_{ρ} (x_{τ}) + v (x^{(i)}) + ρ \\ \leq & 2 Ξ_{ρ} (x_{τ}) + v (x_{τ}) + ρ \end{array}

Now, by the deﬁnition of $v^{r} (x)$ , we can ﬁnd $c \in A_{L}$ such that

v^{r} (x) + ρ \geq E [\int_{0}^{τ} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t] + e^{- β τ} v (x_{τ}) .

Thus, using the formal Markov property [6], we have

\begin{array}{rcl} v^{r} (x) + ρ \\ \geq & \sum_{i} E [\int_{0}^{τ} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t + e^{- β τ} (V_{c^{(i)}} (x_{τ}) - 2 Ξ_{ρ} (x_{τ}) - ρ) : x_{τ} \in S_{i}] \\ = & E [\int_{0}^{τ} e^{- β t} {h (x_{t}^{τ}) + ∣ c_{t}^{τ} ∣^{2}} d t + \int_{τ}^{\infty} e^{- β t} {h (x_{t}^{τ}) + ∣ c_{t}^{τ} ∣^{2}} d t ∣ ℱ_{τ}] \\ - & 2 E [e^{- β τ} Ξ_{ρ} (x_{τ})] - ρ \\ \geq & v (x) - 2 Ξ_{ρ} (x) - ρ, \end{array}

where $x_{t}^{τ}$ is the response to $c_{t}^{τ}$ with $x_{0}^{τ} = x_{τ}$ . Letting $ρ \to 0$ , we deduce $v^{r} (x) \geq v (x)$ , which completes the proof.

Theorem 2.4. We assume (3), (4), (7) and (11). Then $v_{L}$ is a viscosity solution of (5). Furthermore, $v_{L}$ converges locally uniformly to a viscosity solution $u \in C (R)$ of (6) satisfying (4), (7) as $L \to \infty .$

Proof. We note that (13) gives $E [\int_{0}^{h} ∣ x_{t} ∣^{2} d t] \leq e^{β_{0} h} h f_{2} (x) for h > 0,$ and

\begin{array}{rcl} E [{sup}_{0 \leq s \leq h} ∣ x_{s} - x ∣^{2}] \\ \leq & 3^{2} (E [{(\int_{0}^{h} ∣ A x_{t} ∣ d t)}^{2} + {(\int_{0}^{h} ∣ c_{t} ∣ d t)}^{2} + {({sup}_{0 \leq s \leq h} ∣ \int_{0}^{s} σ x_{t} d w_{t} ∣)}^{2}]) \\ \leq & 3^{2} (∣ A ∣^{2} h E [(\int_{0}^{h} ∣ x_{t} ∣^{2} d t)] + h^{2} L^{2} + C E [\int_{0}^{h} ∣ x_{t} ∣^{2} d t]) . \end{array}

for constant $C > 0 .$ Hence we have

\begin{array}{rcl} {lim}_{h \to 0} {sup}_{c \in A_{L}} E [{sup}_{0 \leq s \leq h} ∣ x_{s} - x ∣^{2}] = 0 . \end{array}

Thus we can apply a standard result of viscosity solutions [[4], Thm. 3.1, p. 220] to obtain the viscosity property of $v_{L}$ , taking into account the uniform continuity of $h$ on each compact interval. Since $v_{L} (x)$ is non-increasing, we can deﬁne $u (x)$ by $u (x) = {lim}_{L \to \infty} v_{L} (x) .$ By Theorem 2.3, it is clear that $u$ satisﬁes (4), (7). Thus by Dini’s theorem, we can observe the locally uniform convergence and the viscosity property of $u$ [4]. The proof is complete.

3. Classical solutions

We here study the smoothness of the viscosity solution $u$ of (5).

Proposition 3.1. We assume (3), (4), (7) and (11). Further we assume that $h (x) : convex, then v_{L} (x) and u (x) are convex .$

Proof. For any $ε > 0$ , there exist $c, \hat{c} \in A_{L}$ such that

\begin{array}{rcl} E [\int_{0}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t] & < & v_{L} (x) + ε, \\ E [\int_{0}^{\infty} e^{- β t} {h ({\hat{x}}_{t}) + ∣ {\hat{c}}_{t} ∣^{2}} d t] & < & v_{L} (\hat{x}) + ε, \end{array}

where

\begin{array}{rcl} d x_{t} & = & [A x_{t} + c_{t}] d t + σ x_{t} d w_{t}, x_{0} = x \in R, \\ d {\hat{x}}_{t} & = & [A {\hat{x}}_{t} + {\hat{c}}_{t}] d t + σ {\hat{x}}_{t} d w_{t}, {\hat{x}}_{0} = \hat{x} \in R . \end{array}

We set

\begin{array}{rcl} {\tilde{c}}_{t} & = & ξ c_{t} + (1 - ξ) {\hat{c}}_{t}, \\ {\tilde{x}}_{t} & = & ξ x_{t} + (1 - ξ) {\hat{x}}_{t}, \\ {\tilde{x}}_{0} & = & ξ x + (1 - ξ) \hat{x} \equiv \tilde{x}, \end{array}

for $0 < ξ < 1 .$ Clearly,

\begin{array}{rcl} d {\tilde{x}}_{t} = [A {\tilde{x}}_{t} + {\tilde{c}}_{t}] d t + σ {\tilde{x}}_{t} d w_{t} . \end{array}

Hence, by convexity

\begin{array}{rcl} v_{L} (\tilde{x}) & \leq & E [\int_{0}^{\infty} e^{- β t} {h ({\tilde{x}}_{t}) + ∣ {\tilde{c}}_{t} ∣^{2}} d t] \\ \leq & ξ E [\int_{0}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t] \\ + (1 - ξ) E [\int_{0}^{\infty} e^{- β t} {h ({\hat{x}}_{t}) + ∣ {\hat{c}}_{t} ∣^{2}} d t] \\ \leq & ξ (v_{L} (x) + ε) + (1 - ξ) (v_{L} (\hat{x}) + ε) . \end{array}

Letting $ε \to 0,$ we get

\begin{array}{rcl} v_{L} (\tilde{x}) = v_{L} (ξ x + (1 - ξ) \hat{x}) \leq ξ v_{L} (x) + (1 - ξ) v_{L} (\hat{x}), \end{array}

which completes the convexity of $v_{L} (x) .$ From the deﬁnition of $v_{L} (x),$ for each positive integer $L,$ we have $0 \leq v_{L + 1} (x) \leq v_{L} (x), x \in R .$ Since $v_{L} (x)$ is non-increasing, we can deﬁne $u (x)$ by $u (x) = lim_{L \to \infty} v_{L} (x) .$ Hence we see that $u (x)$ is also convex.

Theorem 3.2. We assume (3), (4), (7) and (11). Then we have

u \in C^{2} (R ∖ {0}) .

(22)

Proof. Step 1: By the convexity of $u$ we recall a classical result of Alexandrov [6] to see that Lebesgue measure of $R ∖ D \cup {0} = 0$ , where
$D = \{x \in R : u is twice differentiable at x \} .$ By the deﬁnition of twice-differentiability, we have $(u^{'} (x), u^{''} (x)) \in J^{+ 2} u (x) \cap J^{- 2} u (x) for all x \in D$ , and hence

\begin{array}{rcl} - β u + \frac{1}{2} σ^{2} x^{2} u^{''} + A x u^{'} - \frac{{(u^{'})}^{2}}{4} + h (x) = 0, \forall x \in D . \end{array}

Let $d^{+} u (x)$ and $d^{-} u (x)$ denote the right- and left-hand derivatives respectively. For all $x \in (R ∖ {0})$ , deﬁne $r^{\pm} (x)$ by

- β u (x) + \frac{1}{2} σ^{2} x^{2} r^{\pm} (x) + A x d^{\pm} u (x) - \frac{{(d^{\pm} u (x))}^{2}}{4} + h (x) = 0 .

(23)

Since $d^{+} u = d^{-} u = u^{'} o n D,$ we have $r^{+} = r^{-} = u^{''}$ a.e. By deﬁnition, $d^{+} u (x)$ is right continuous, and so is $r^{+} (x)$ . Hence it is easy to see that

\begin{array}{rcl} u (y) - u (x) = \int_{x}^{y} d^{+} u (s) d s \end{array}

\begin{array}{rcl} d^{+} u (s) - d^{+} u (x) = \int_{x}^{s} r^{+} (t) d t, s > x . \end{array}

Thus we get

\begin{array}{rcl} R (u; y) : & = & {u (y) - u (x) - d^{+} u (x) (y - x) - \frac{1}{2} r^{+} (x) ∣ y - x ∣^{2}} ∕ ∣ y - x ∣^{2} \\ = & \int_{x}^{y} (d^{+} u (s) - d^{+} u (x) - r^{+} (x) (s - x)) d s ∕ ∣ y - x ∣^{2} & (24) \\ = & \int_{x}^{y} \{\int_{x}^{s} (r^{+} (t) - r^{+} (x)) d t \} d s ∕ ∣ y - x ∣^{2} \to 0 as y ↓ x . \end{array}

Step 2: We claim that $u (x)$ is differentiable at $x \in R ∖ D \cup {0} = 0$ . It is well known in [2] that $δ u (x) = [d^{+} u (x), d^{-} u (x)], for all x \in (R ∖ {0}),$ where $δ u (x)$ is the generalized gradient of $u$ at $x$ . Suppose $d^{+} u (x) > d^{-} u (x)$ . We set

\begin{array}{rcl} \hat{p} & = & ξ d^{+} u (x) + (1 - ξ) d^{-} u (x) \\ \hat{r} & = & ξ r^{+} (x) + (1 - ξ) r^{-} (x), 0 < ξ < 1 . \end{array}

If $lim {inf}_{y \to x} R (u; y) < 0$ , then we can ﬁnd a sequence $y_{m} \to x$ such that
${lim}_{m \to \infty} R (u; y_{m}) < 0$ . By (24), we may consider that $y_{m} \leq y_{m + 1} < x$ for every $m$ , taking a subsequence if necessary. Hence

\begin{array}{rcl} {lim}_{m \to \infty} \frac{u (y_{m}) - u (x) - d^{+} u (x) (y_{m} - x)}{∣ y_{m} - x ∣} \leq 0, \end{array}

this leads to $d^{+} u (x) \leq d^{-} u (x)$ , which is a contradiction. Thus we have $(d^{+} u (x), r^{+} (x)) \in J^{2, -} u (x)$ and similarly, $(d^{-} u (x), r^{-} (x)) \in J^{2, -} u (x)$ . By the convexity of $J^{2, -} u (x)$ , we get $(\hat{p}, \hat{r}) \in J^{2, -} u (x) .$ Now we note that

\begin{array}{rcl} {(\hat{p})}^{2} < ξ {(d^{+} u (x))}^{2} + (1 - ξ) {(d^{-} u (x))}^{2}, \end{array}

and hence by (23)

\begin{array}{rcl} - β u (x) + \frac{1}{2} σ^{2} x^{2} \hat{r} + A x \hat{p} - \frac{{(\hat{p})}^{2}}{4} + h (x) > 0 . \end{array}

On the other hand, by the deﬁnition of viscosity solution

\begin{array}{rcl} - β u (x) + \frac{1}{2} σ^{2} x^{2} q + A x p - \frac{p^{2}}{4} + h (x) \leq 0 \forall (p, q) \in J^{2, -} u (x), \end{array}

which is a contradiction. Therefore we deduce that $δ u (x)$ is a singleton, and so $u$ is differentiable at $x$ [2].
Step 3: We claim that $u^{'}$ is continuous on $(R ∖ {0})$ . Let $x_{m} \to x$ and $p_{m} = u^{'} (x_{m}) \to p$ . Then we have by convexity $u (y) \geq u (x) + p (y - x), for all y .$ Hence we see that $p \in D^{-} u (x)$ , where

\begin{array}{rcl} D^{-} u (x) = {p \in R : {liminf}_{y \to x} {u (y) - u (x) - p (y - x)} ∕ ∣ y - x ∣ \geq 0} . \end{array}

Since $δ u (x) = D^{-} u (x)$ and $δ u (x)$ is a singleton, we deduce $p = u^{'} (x)$ [[2], prop.4.7,p.66]. Step 4: We set $w = u^{'}$ . Since

\begin{array}{rcl} - β w (x_{m}) + \frac{1}{2} σ^{2} {x_{m}}^{2} w^{'} (x_{m}) + A x_{m} w (x_{m}) - \frac{{(w (x_{m}))}^{2}}{4} + h (x_{m}) = 0, \end{array}

where $x_{m} \in D$ , the sequence ${w^{'} (x_{m})}$ converges uniquely as $x_{m} \to x \in R ∖ D \cup {0}$ , and $w$ is Lipschitz near $x$ by monotonicity. Hence, we have a well-known result in nonsmooth analysis that $δ w (x)$ coincides with the convex hull of the set

\begin{array}{rcl} D^{*} w (x) = \{q \in R : q = {lim}_{m \to \infty} w^{'} (x_{m}), x_{m} \in D \to x \} . \end{array}

Then

\begin{array}{rcl} - β u (x) + \frac{1}{2} σ^{2} x^{2} q + A x w (x) - \frac{{(w^{'} (x))}^{2}}{4} + h (x) = 0 \forall q \in δ w (x) . \end{array}

Hence we observe that $δ w (x)$ is a singleton, and then $w (x)$ is differentiable at $x$ . The continuity of $w^{'} (x)$ follows immediately. Thus we conclude that $w \in C^{1} (R ∖ {0})$ and $(R ∖ D \cup {0})$ is empty. The proof is complete.

Theorem 3.3. We assume (3), (4), (7) and (11). Further we assume that

\begin{array}{rcl} h (x) ∕ x^{2} \to \hat{h} \in R_{+} as x \to 0 . & (25) \end{array}

Then we have

\begin{array}{rcl} u \in C^{1} (R) \cap C^{2} (R ∖ {0}) . & (26) \end{array}

In addition, if $\hat{h} = 0$ , then

\begin{array}{rcl} u \in C^{2} (R) . & (27) \end{array}

Proof. We ﬁrst observe that $v_{L}$ is a viscosity solution of the boundary value problem:

\begin{array}{rcl} V^{''} + G (x, V, V^{'}) = 0 in (a, b) & (28) \\ V (a) = v_{L} (a), V (b) = v_{L} (b), \end{array}

for any interval $[a, b] \subset R ∖ {0}$ where

\begin{array}{rcl} G (x, V, V^{'}) = 2 {- α V + A x V^{'} + {min}_{∣ r ∣ \leq L} (∣ r ∣^{2} + r V^{'}) + h (x)} ∕ σ^{2} x^{2} = 0 . \end{array}

Standard elliptic regularity theory Fleming and Soner [[6], Thm. 4.1] and the uniqueness of viscosity solutions Crandall, Ishii and Lions [4] yield that $v_{L}$ is smooth in $(a, b)$ . Thus

\begin{array}{rcl} ∣ {min}_{∣ r ∣ \leq L} (∣ r ∣^{2} + r {v_{L}}^{'}) ∣ & \leq & ∣ {min}_{r \in R} (∣ r ∣^{2} + r {v_{L}}^{'}) ∣ = {(∣ {v_{L}}^{'} ∣ ∕ 2)}^{2} \\ \leq & {{(∣ {v_{L}}^{'} ∣ ∕ 2)}^{2} + 1} . \end{array}

By the Theorem 3.2, we have $u \in C^{2} (R ∖ {0})$ .

To prove (26), it suffices to show that $u$ has the following property:

\begin{array}{rcl} u^{'} (x) = o (1) as x \to 0 . & (29) \end{array}

By (25), there exists $λ > 0$ , for any $ɛ > 0$ such that $h (x) \leq (\hat{h} + ɛ) x^{2} for ∣ x ∣ < λ,$ and hence, by (4)

\begin{array}{rcl} h (x) \leq (\hat{h} + ɛ) x^{2} + C (1 ∕ λ^{n} + 1) ∣ x ∣^{n}, \forall x \in R . & (30) \end{array}

Note that $u (x) \leq E [\int_{0}^{\infty} e^{- β t} h (x_{t}^{0}) d t]$ . Then we have by (13)

\begin{array}{rcl} u^{'} (x) = 0 (x^{2}) as x \to 0 . & (31) \end{array}

Now, by convexity

\begin{array}{rcl} u (y) \geq u (x) + u^{'} (x) (y - x), x \neq 0 . \end{array}

Substituting $y = 2 x$ , and $y = 0$ we get $u (2 x) \geq u (x) + u^{'} (x) x$ and $u (x) - u^{'} (x) x \leq u (0) = 0$ by (31). Hence

\begin{array}{rcl} \frac{u (2 x)}{x^{2}} \geq \frac{u^{'} (x)}{x} \geq \frac{u (x)}{x^{2}}, & (32) \end{array}

which implies (29).

Finally, suppose $\hat{h} = 0$ . Then, by virtue of (30), we have $u (x) = o (x^{2})$ as $x \to 0 .$ Moreover, by (32), $u^{'} (x) = o (x)$ as $x \to 0 .$ Dividing (5) by $x^{2}$ and passing to the limit, we get $u^{''} (0) = 0$ , which implies (27).

4. An application to quadratic control theory

We shall study the quadratic control problem (1) over the class $A_{a d}$ of admissible controls, subject to (2), where

A_{a d} = {c = (c_{t}) \in A : {lim}_{T \to \infty} E [e^{- β T} ∣ x_{T} ∣^{n}] = 0 for the response x_{t} to c_{t}} .

We consider the stochastic differential equation

d x_{t}^{*} = [A x_{t}^{*} - u^{'} (x_{t}^{*}) ∕ 2] d t + σ x_{t}^{*} d w_{t}, x_{0}^{*} = x .

(33)

Theorem 4.1. We assume (3), (4), (7), (11) and (25). Then the optimal control $c_{t}^{*}$ is given by

c_{t}^{*} = - u^{'} (x_{t}^{*}) ∕ 2 .

(34)

Proof. Since $u^{'}$ is continuous, (33) admits a weak solution $x_{t}^{*}$ up to explosion time $σ = inf {t : ∣ x_{t}^{*} ∣ = \infty}$ . Taking into account $x u^{'} (x) \geq 0$ , we can show ${(x_{t}^{*})}^{2} \leq {(x_{t}^{0})}^{2}$ by the comparison theorem. Hence $σ = \infty$ . By the monotonicity of $u^{'} (x)$ , the uniqueness of (33) holds. Thus we conclude that (33) has a unique strong solution $(x_{t}^{*}) .$

It follows from (14) that

\begin{array}{rcl} E [e^{- β T} (1 + ∣ x_{T}^{*} ∣^{n})] \leq e^{- (β - β_{0}) T} E [e^{- β_{0} T} f_{n} (x_{T}^{0})] \to 0 as T \to \infty, \end{array}

where $x_{t}^{0}$ is a unique solution of (18). So $(c_{t}^{*}) \in A_{a d}$ . Since $u$ satisﬁes (4), we see by (32) and (13) that

\begin{array}{rcl} E [\int_{0}^{T} e^{- 2 β t} {(x_{t}^{*} u^{'} (x_{t}^{*}))}^{2} d t] & \leq & E [\int_{0}^{T} e^{- 2 β t} u {(2 x_{t}^{*})}^{2} d t] \\ \leq & C E [\int_{0}^{T} e^{- 2 β t} (1 + ∣ x_{t}^{*} ∣^{2 n}) d t] \\ \leq & C E [\int_{0}^{T} e^{- 2 β t} f_{2 n} (x_{t}^{0}) d t] < \infty, \end{array}

and hence $\int_{0}^{t} e^{- β s} σ x_{s}^{*} u^{'} (x_{s}^{*}) d w_{s}$ is a martingale. Then we apply Ito’s formula for convex functions [7,p.219] to obtain

\begin{array}{rcl} E [e^{- β T} u (x_{T}^{*})] \\ = & u (x) + E [\int_{0}^{T} e^{- β t} (- β u + A x u^{'} + c_{t}^{*} u^{'} + \frac{1}{2} σ^{2} x^{2} u^{''}) ∣_{x = x_{t}^{*}} d t] \\ = & u (x) - E [\int_{0}^{T} e^{- β t} {h (x_{t}^{*}) + ∣ c_{t}^{*} ∣^{2}} d t] . \end{array}

Passing to the limit, we have $J (c^{*}) = u (x) .$ By the same calculation as above , we can see that

\begin{array}{rcl} E [e^{- β T \land τ_{n}} u (x_{T \land τ_{n}})] \geq u (x) - E [\int_{0}^{T \land τ_{n}} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{2}} d t], \end{array}

where ${τ_{n}}$ is a sequence of localizing stopping times for the local martingale. Letting $τ_{n} \to \infty$ and then $T \to \infty$ , we obtain $u (x) \leq J (c)$ for all $c \in A_{a d}$ . The proof is complete.
General stochastic control problem: we can further study a stochastic control problem for linear degenerate systems to minimize the discounted expected cost:

\begin{array}{rcl} J (c) = E [\int_{0}^{\infty} e^{- β t} {h (x_{t}) + ∣ c_{t} ∣^{n}} d t] \end{array}

over $c \in A$ subject to the degenerate stochastic differential equation (2) and a continuous function $f$ on $R$ such that (4) and (7), in addition

\begin{array}{rcl} k_{0} ∣ x ∣^{n} - k_{1} \leq h (x) \end{array}

for some constants $k_{0}, k_{1} > 0$ and for a ﬁxed integer $n \geq 2 .$
Acknowledgements: I would like to express my sincere gratitude to Professor A.B.M. Abdus Sobhan Miah for his advice, support and encouragement.

References

[1] T. M. Apostol. Mathematical Analysis, Addison-Wesley, (1974).

[2] Bardi, M. and Capuzzo-Dolcetta, I. Optimal Control and Viscosity Solutions of Hamilton-Jacobi- Bellman Equations, Birkhäuser, Boston, (1997).

[3] Bensoussan, A. Stochastic Control by Functional Analysis Methods, North-Holland, Amsterdam, (1982).

[4] Crandall, M.G., Ishii, H. and Lions, P.L. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. 27, (1982),1-67.

[5] Da Prato, G. Direct solution of a Riccati equation arising in stochastic control theory. Appl. Math. Optim. 11, (1984),191-208.

[6] Fleming, W.H. and Soner, H.M. Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, (1993).

[7] S. Koike and M. Morimoto. On variational inequalities for leavable bounded-velocity control, Appl. Math. Optim., 18, (2003), pp.1-20.

[8] J.L. Menaldi and M. Robin. On some cheap control problems for diffusions process, Trans. Amer. Math. Soc. vol. 278, (1983), pp. 771-802.

[9] Nisio, M. Stochastic Control Theory, ISI Lecture Notes 9, MacMillan, (1981).

Department of Statistics, ShahJalal University of Science and Technology, Sylhet-3114, Bangladesh.

E-mail address: baten_math@yahoo.com

Received February 18, 2005