Optimal control of compound Poisson processes

. The problem of controlling a compound Poisson process until it leaves an interval is considered. This type of problem is known as a homing problem. To determine the value of the optimal control, we must solve a non-linear integro-di ﬀ erential equation. Exact and explicit solutions are obtained for two possible jumps size distributions.


Introduction
Let {X u (t), t ≥ 0} be the controlled jump-diffusion process defined by where µ ∈ R and σ > 0 are constants, b(•) is a non-zero function, u(•) is the control variable, {B(t), t ≥ 0} is a standard Brownian motion and {N(t), t ≥ 0} is a Poisson process (independent of {B(t), t ≥ 0}) with rate λ.Moreover, the random variables Y 1 , Y 2 , . . .are independent and identically distributed.
In [1], the author considered the following problem: find the control that minimizes the expected value of the cost criterion where θ is a real constant, q(•) is a positive function, K is a general terminal cost function and the final time T (x) is a random variable (called a first-passage time) defined by He was able to find explicit solutions to particular problems when the random variables Y 1 , Y 2 , . . .are exponentially distributed, so that the jumps are positive.This type of problem is known as a homing problem (see [3] and/or [4]).If the parameter θ is positive, then the optimizer must try to minimize the time that the controlled process spends in the continuation region (a, b), whereas the objective is to maximize the time that it spends in (a, b) when θ < 0. In both cases, the optimizer must of course take the quadratic control costs 1 2 q[X u (t)]u 2 [X u (t)] and the termination cost K[X u (T (x))] into account.
In the current paper, we set µ = σ = 0.That is, {X u (t), t ≥ 0} becomes a controlled compound Poisson process; see, for instance, [2].Furthermore, as in the particular problems solved in [1], we assume that the ratio b 2 (x)/q(x) is constant: Finally, we take [a, b] = [0, 1] and we define Thus, there is now a single barrier, at x = 1.To solve the above stochastic optimal control problem, we can use dynamic programming.First, we define the value function The function F(x) is thus the expected cost (or reward, if it is negative) obtained by choosing the optimal value of the control u[X u (t)] for t ∈ [0, T (x)].It satisfies a non-linear integrodifferential equation that must be solved to determine the optimal control u * (x), which is expressed in terms of F(x): In the next section, we will give the integro-differential equation satisfied by F(x) and we will show that it can, in some cases, be transformed into a non-linear ordinary differential equation.We will then solve exactly and explicitly two particular problems.We will give a few concluding remarks in Section 3.

Optimal control
We deduce from Proposition 2.1 in [1] the following result.
Proposition 2.1 The function F(x) defined in Eq. ( 6) satisfies the first-order, non-linear integro-differential equation If Y has an exponential distribution with parameter α, so that the function f Y (y) is given by f Y (y) = αe −αy for y ≥ 0, (10) then, proceeding as in [1], we can find particular solutions of (8), (9).In this paper, we will find new exact and explicit solutions.We assume that the jumps are x-dependent and bounded: Remark 2.1 Notice that if u[X u (s)] ≡ 0, then the process {X u (t), t ≥ 0} will remain inside the interval [0, 1] indefinitely.Therefore, we must assume that the parameter θ in the cost function J(x) defined in Eq. ( 2) is positive; indeed, if θ < 0, we will have F(x) = −∞ by choosing u[X u (s)] ≡ 0.
We assume further that the function g(y) is such that Notice that the integral involving the unknown function F(z) in the above equation does not depend on x.Case I. Suppose first that g(y) = e −αy , where α > 0. This function is indeed such that Eq. ( 13) is satisfied.Moreover, we have which is independent of x.Hence, the integro-differential equation ( 8) reduces to which can be rewritten as the Riccati equation where θ * is a constant that actually depends on the unknown function F(x).Differentiating Eq. ( 18), we obtain This non-linear ordinary differential equation has two solutions: F(x) ≡ c and where c, c 0 and c 1 are arbitrary constants.Because we assumed that the parameter θ is positive, the function F(x) ≡ c does not satisfy Eq. ( 17).Substituting the function defined in Eq. ( 20) into (17), we find that this equation is satisfied if and only if As an illustrative example, let us take α = λ = κ = θ = 1.Equation (21) reduces to 4c 2 1 The two roots of the above equation are c 1 ≃ 1.199 and c 1 ≃ −0.781.Therefore, we have two possible solutions: Now, because θ > 0, we can state that the function F(x) should decrease when x increases in the interval [0, 1).It follows that we must choose the solution The constant c 0 is uniquely determined from the boundary condition F(1) = K(1).Finally, the optimal control is given by Remark 2.2 Notice that the ratio b 2 (x)/q(x) was assumed to be equal to the constant 2 κ.However, the ratio b(x)/q(x) is not necessarily a constant.When it is indeed a constant, the optimal control is an affine function of x.
Case II.Suppose next that g(y) ≡ 1, (26) so that Eq. ( 13) is again satisfied.We calculate The integro-differential equation (8) becomes This equation can be rewritten as Eq. ( 18).It follows that we can state that the function F(x) is of the form given in Eq. (20).The equation that corresponds to Eq. ( 21) is With λ = κ = θ = 1, the above equation is