Mean Field Games with State Constraints

(1)

Mean Field Games with State Constraints

Piermarco Cannarsa

University of Rome “Tor Vergata”

7

TH

T

RILATERAL

M

EETING

(A

USTRALIA

-I

TALY

-T

AIWAN

)

ON

N

ONLINEAR

PDE

S AND

A

PPLICATIONS

National Cheng Kung University, Tainan, Taiwan January 23–28, 2019

Organizers:Yung-fu Fang, Ching-Lung Lin

Scientific Committee:Neil Trudinger, Nicola Fusco, and Tai-Ping Liu joint work with

R. Capuani(NCSU) andP. Cardaliaguet(Paris 9)

(2)

Outline

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(3)

Outline

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(4)

Outline

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(5)

Mean Field Games

A quote from Wikipedia

Mean field game theory is the study of strategic decision making in very large populations of small interacting agents.

This class of problems was considered in the economics literature by B Jovanovic and RW Rosenthal, in the

engineering literature by PE Caines and his co-workers, and

independently and around the same time by mathematicians

J-M Lasry and P-L Lions . . . Under fairly general assumptions

it can be proved that a class of mean field games is the limit

as N → ∞ of an N-player Nash equilibrium.

(6)

Motivation of MFG theory

Goal

To describe equilibria in collective behaviour of large population of rational agents

large population infinite number (a continuum) of players

rational agents each agent is controlling his/her dynamical own

state

(7)

Impact of MFG theory

MFG system allows for ahuge simplification

solution to the macroscopic MFG system providesapproximate Nash equilibria Great potential for applications

finance, market economics(oil producers, carbon markets...) engineering(smart grids...)

crowd dynamics, socio-politics(learning, opinion formation etc...)

(8)

The Lasry-Lions approach

To export the principle of statistical mechanics to interactions within rational particles by introducing amacroscopic descriptionthrough a mean field model

agents are identified with pointsx ∈ Ω ⊂ Rⁿ m(t, dx )is the distribution of agents at time t The generic agent aims to attain

min

γ(0)=x

nZ T 0

L(γ(t), ˙γ(t)) + F (γ(t), m(t)) dt + G(γ(T ), m(T ))o The solution u(t, x ) to the associatedHamilton-Jacobi equation

−ut+H(x , ∇xu) = F (x , m) in [0, T ] × Ω , u(T , x ) = G(x , m(T ))

gives theoptimal feedbackγ⁰(t) = −∇pH(γ(t), ∇xu(t, γ(t))) which in turn leads to thecontinuity equation

mt− div(m ∇pH(x , ∇xu)) = 0 in [0, T ] × Ω , m(0, dx ) = m0(dx )

(9)

The MFG system of PDEs

The MFG system







−ut+H(x , ∇xu) − F (x , m) = 0 mt− div(m ∇pH(x , ∇xu)) = 0

]0, T [×Ω







u(T , x ) = G(x , m(T )) m(0, dx ) = m0(dx ) where

u(t, x ) = infγ(t)=x

nRT

t L(γ(s), ˙γ(s)) + F (γ(s), m(s)) ds + G(γ(T ), m(T ))o H(x , p) := sup_{v ∈R}n − hp, vi − L(x, v)

m0is the agent distribution at time t = 0

has been widely investigated alwayswithout state constraints Ω = Tⁿ, Rⁿ

Achdou, Bardi, Bensoussan, Camilli, Capuzzo Docetta, Cardaliaguet, Carmona, Delarue, Gomes, Gu ´eant, Lachapelle, Porretta, . . .

Ω ⊂ Rⁿbounded with Dirichlet boundary conditions Dweik and Mazanti

(10)

Introducing state constraints into MFG

Solution of MFG system in absence of state constraints

(Notes on Mean Field Games byP. Cardaliaguet, 2013 and 2015) by vanishing viscosity

−ut− ∆u + H(x, ∇xu) = F (x , m) , mt− ∆m − div(m ∇pH(x , ∇xuµ)) =0 by a fixed point argument

µ−→uµ







−ut+H(x , ∇xu) = F (x , µ) u(T , x ) = G(x , µ(T ))

−→mµ

(mt− div(m ∇pH(x , ∇xuµ)) =0 m(0, dx ) =m0(x )dx

Our goalTo study MFGs with state constraints (x ∈ Ω)

Difficulty

Agent distribution may concentrate on small sets

(11)

MFG with state constraints Lagrangian approach

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(12)

Notation

Ω ⊂ Rⁿbounded domain with boundary of class C² P(Ω) Borel probability measures on Ω with Katorovich-Rubinstein distance

d1(m1,m2) =supnZ

Ω

f dm1− Z

Ω

f dm2 :

f (x ) − f (y )| 6 |x − y|o

Recall that, given m ∈ C [0, T ]; P(Ω), agents aim to attain

min

γ(0)=x ,γ(t)∈Ω

nZ T 0

L(γ(t), ˙γ(t)) + F (γ(t), m(t)) dt + G(γ(T ), m(T ))o

Next step

To replace C [0, T ]; P(Ω)

by P

C [0, T ]; Ω

(13)

Lagrangian approach

References

Brenier (1999), Benamou-Brenier (2000), Benamou-Carlier (2015), Benamou-Carlier-Santambrogio (2017)

Cardaliaguet (2015), Cardaliaguet-M ´esz ´aros-Santambrogio (2017) Notation

constrained arcs

Γ =n

γ ∈AC([0, T ]; Rⁿ) : γ(t) ∈ Ω , ∀t ∈ [0, T ]o

with k · k∞

Γ[x ] =γ ∈ Γ : γ(0) = x (x ∈ Ω)

P(Γ)Borel probability measures on Γ: metric space with d1metric evaluation map et: Γ → Ω (t ∈ [0, T ]) defined by et(γ) = γ(t)

Borel measures on Γ which arecompatible with m0∈ P(Ω)are defined as Pm₀(Γ) =η ∈ P(Γ) : e0]η =m0

where e0]η(·) = η e⁻¹₀ (·)

(14)

Assumptions and more notation

F , G : Ω × P(Ω) → R continuous functions L : Ω × Rⁿ→ Rcontinuous such that

• v 7→ L(x , v ) convex ⊕ L > `|v|²− `0 (` >0)

• |∇xL| 6 C(1 + |v|²) ⊕ |∇vL| 6 C(1 + |v|) For any η ∈ P(Γ) we define

the associatedpayoff functional Jη[γ] =

Z T 0

L(γ(t), ˙γ(t)) + F (γ(t), et]η)dt + G(γ(T ), eT]η) ∀γ ∈ Γ

minimizing arcsat x ∈ Ω

Γ^η[x ] =γ ∈ Γ[x] : Jη[γ] =min

Γ[x ]Jη

(15)

MFG with state constraints Relaxed solutions

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(16)

Relaxed equilibria

Let m0∈ P(Ω) Definition

η ∈ Pm₀(Γ)is called arelaxed (CMFG) equilibriumfor m0if spt(η) ⊆ [

x ∈Ω

Γ^η[x ]

Equivalently, for η−a.e. γ ∈ Γ,

Jη[γ] = min

γ∈Γ[γ(0)]Jη[γ]

where

Jη[γ] = Z T

0

L(γ(t), ˙γ(t)) + F (γ(t), et]η)dt + G(γ(T ), eT]η)

(17)

Relaxed solutions

Let m0∈ P(Ω) Definition

(u, m) ∈ C([0, T ] × Ω) × C [0, T ]; P(Ω) is arelaxed solutionto the CMFG problem if m(t)=et]η ∀t ∈ [0, T ]

for some relaxed equilibrium η ∈ Pm₀(Γ)and u(t, x )= min

γ∈Γ,γ(t)=x

nZ T t

L(γ(s), ˙γ(s)) + F (γ(s), m(s))dt + G(γ(T ), m(T ))o

(18)

MFG with state constraints Existence of equilibria

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(19)

MFG with state constraints Existence of equilibria

Existence of relaxed equilibria

Theorem

For any m0∈ P(Ω) there is at least one relaxed equilibrium

Corollary

For any m0∈ P(Ω) there is at least one relaxed solution (u, m) to the CMFG problem Proof of theorem:construction of a fixed point ofE : Pm₀(Γ) ⇒ P^m0(Γ)

E (η)=µ ∈ Pm₀(Γ) |spt(µx) ⊆ Γ^η[x ]for m0−a.e. x ∈ Ω

where{µx}_{x ∈Ω}⊂ P(Γ)is the family of probability measures whichdisintegratesµ µ =

Z

Ω

µxdm0(x ) and spt(µx) ⊆ Γ[x ] m0− a.e. x ∈ Ω Indeed

η ∈ Pm₀(Γ) relaxed equilibrium ⇐⇒ η ∈E (η) The existence of a fixed point of E follows fromKakutani’s Theorem

(20)

MFG with state constraints Uniqueness

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(21)

MFG with state constraints Uniqueness

Uniqueness

Theorem

Assumemonotonicity conditions: for any m1,m2∈ P(Ω)









 Z

Ω

(G(x , m1) −G(x , m2))d (m1− m2)(x ) > 0 Z

Ω

(F (x , m1) −F (x , m2))d (m1− m2)(x ) > 0 if m16= m2

If (u1,m1)and (u2,m2)are relaxed solutions to the CMFG problem, then u1≡ u2 and m1=m2

F satisfies the strict monotonicity condition if F : Ω × P(Ω) → R is of the form F (x , m) =

Z

Ω

f y , (φ ? m)(y )φ(x − y) dy where φ : R^d → R is a smooth even kernel with compact support and

f : Ω × R → R is smooth and f (x, ·) is strictly increasing

(22)

Regularity and CMFG system

More notation and assumptions

Recall Ω ⊂ Rⁿis bounded with ∂Ω ∈ C². Consequently distance dΩ(x ) = min_{y ∈Ω}|x − y |

of class C² Ω⁺_δ for some δ > 0 with Ω⁺_δ =x ∈ Rⁿ\ Ω : dΩ(x ) < δ oriented boundary distance bΩ(x ) = dΩ(x ) − d_Rⁿ\Ω(x )

of class C² Ωδ on Ωδ=x ∈ Rⁿ : |bΩ(x )| < δ

Ω

dΩ

bΩ

(23)

Regularity and CMFG system Necessary conditions

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(24)

References

Dubovitskii – Milyutin (1964) Malanowski (1978)

Hager (1979) Vinter (2000)

Galbraith – Vinter (2003) Frankowska (2006, 2009)

Bettiol – Frankowska (2007, 2008) Bettiol – Khalil – Vinter (2016)

(25)

Necessary conditions for smooth state constraints

Theorem

Given x ∈ Ω letγ^∗minimize over Γ[x ] the functional γ 7→

ZT 0

L(γ(s), ˙γ(s)) + f (s, γ(s))dt + g(γ(T ))

where g ∈ C¹(Ω)and f : [0, T ] × Ω → R satisfies |f^t| + |∇xf | ≤ C Then there exist

p^∗: [0, T ] → RⁿLipschitz

ν ∈ RandΛ ∈ Cb [0, T ] × Ωδ× Rⁿ

(independent of γ^∗,p^∗) such that (I∂Ω=characteristic function of ∂Ω)











˙γ^∗= −∇pH γ^∗,p^∗

˙p^∗= ∇xH γ^∗,p^∗ − ∇xf t, γ^∗ − Λ(t, γ^∗,p^∗) I^∂Ω γ^∗∇bΩ γ^∗ p^∗(T ) = ∇g γ^∗(T ) + ν I∂Ω γ^∗(T )∇bΩ γ^∗(T )

∀t ∈ [0, T ]

Consequently, γ^∗∈ C_Lip¹ [0, T ]; Rⁿ

and k ˙γ^∗kLip6 C(Ω, H, f , g)

(26)

Regularity and CMFG system Lipschitz regularity

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(27)

Regularity and CMFG system Lipschitz regularity

Existence of Lipschitz solutions

Theorem

Let m0∈ P(Ω) and suppose

|F (x1,m1) −F (x2,m2)| + |G(x1,m1) −G(x2,m2)| 6 C

|x1− x2| + d1(m1,m2) Then there exists at least one relaxed solution of CMFG problem (u, m) such that

u ∈ Lip [0, T ] × Ω

and m ∈ Lip [0, T ]; P(Ω)

Such a solution will be called aLipschitz relaxed solutionof the CMFG problem The proof applies necessary conditions to construct a relaxed CMFG equilibrium

η ∈ Pm₀(Γ)such that m(t) := et]ηbelongs to Lip [0, T ]; P(Ω) and uses the Lipschitz continuity of m to deduce that u ∈ Lip [0, T ] × Ω

(28)

Regularity and CMFG system Semiconcavity

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(29)

Adjoint state inclusion / sensitivity relations

Given

a Lipschitz relaxed solution(u, m)of the CMFG problem (t, x ) ∈ [0, T [×Ω and a solutionγ^∗∈ Γto

min

γ∈Γ,γ(t)=x

nZ T t

L(γ(s), ˙γ(s)) + F (γ(s), m(s))dt + G(γ(T ), m(T ))o

the adjoint statep^∗: [t, T ] → Rⁿassociated with γ^∗ we have that

H(γ^∗(s), p^∗(s)) − F (γ^∗(s), m(s)) , p^∗(s)

∈ D⁺u s, γ^∗(s)

∀s ∈ [t, T [ and ∀ρ ∈]0, T [ there exists Cρ> 0 such that ∀ t, t + τ ∈ [0, T − ρ] and all x + h ∈ Ω

u(t + τ, x + h) − u(t, x ) − τ H(x , p^∗(t)) − F (x , m(t)) − hp^∗(t), hi

6 Cρ(|τ | + |h|)^3/2

(30)

Proof of sensitivity relation for τ = 0

We want to show that ∀ t ∈ [0, T − ρ] and all x + h ∈ Ω

u(t, x + h) − u(t, x ) − hp(t), hi 6 Cρ|h|^3/2 Let0 < σ 6 ρto be fixed later and define for all s ∈ [t, T ]

γh(s)= γ^∗(s) +

1 +t − s σ

+

h

Ω γ^∗

x x + h γ^∗+h γh

bγh(s)= γh(s) − d_Ω γh(s)Dd∂Ω γh(s)

(31)

Proof of sensitivity relation (continued)

By dynamic programming

u(t, x + h) − u(x , t) − hp(t), hi 6 Z t+σ

t

L(bγh, ˙γb_h) −L(γ^∗, ˙γ^∗) ds

+ Z t+σ

t

F (γbh,m) − F (γ^∗,m) ds − hp(t), hi (1) We want to express hp(t), hi so we expand

−hp(t), hi= −hp(t + σ),bγh(t + σ) − γ^∗(t + σ)

| {z }

=0

i + Z t+σ

t

d

dshp,bγh− γ^∗i ds

= Z t+σ

t

h ˙p,bγh− γ^∗i ds + Zt+σ

t

hp, ˙bγ_h− ˙γ^∗i ds

By appealing to PMP to represent h ˙p,bγh− γ^∗i and hp, ˙bγ_h− ˙γ^∗i we obtain

u(t, x + h) − u(x , t) − hp(t), hi6 . . . 6 C

Z t+σ t

|bγh− γ^∗|²ds+C Zt+σ

t

| ˙bγ_h− ˙γ^∗|²ds+C Z t+σ

t

|bγh− γ^∗| ds

(32)

Proof of sensitivity relation (completed)

Recalling







γh(s) = γ^∗(s) +

1 +^t−s_σ

+

h

bγh(s) = γh(s) − d_Ω γh(s)Dd∂Ω γh(s) we have that

|γbh(s) − γ^∗(s)| ≤ 2|h| ∀s ∈ [t, t + σ]

Using the regularity of the distance functions one can also prove (technical) Z t+σ

t

| ˙bγ_h(s) − ˙γ^∗(s)|²ds 6 C|h|²

σ +C|h|σ Therefore

u(t, x + h) − u(x , t) − hp(t), hi 6 C|h||h|

σ + σ

6 2C|h|^3/2 by takingσ = |h|^1/2

(33)

Semiconcavity

Theorem

Any Lipschitz relaxed solution (u, m) of CMFG problem islocally semiconcaveon [0, T [×Ω with afractional modulus:

∀ρ ∈]0, T [ there exists Cρ≥ 0 such that

u(t + τ, x + h) + u(t − τ, x − h) − 2u(t, x ) 6 Cρ(|τ | + |h|)^3/2 for all t, t ± τ ∈ [0, T − ρ] and x , x ± h ∈ Ω

(34)

Regularity and CMFG system Point-wise properties

Outline

1

Introduction to Mean Field Games

2

The MFG problem with state constraints The Lagrangian approach

Relaxed solutions to CMFG problem Existence of relaxed equilibria

A uniqueness result for relaxed solutions

3

Regularity and analysis of CMFG system

Necessary conditions and smoothness of minimizers Lipschitz solutions to CMFG problem

Fractional semiconcavity

Point-wise properties of relaxed solutions

(35)

Point-wise solutions of the HJ equation

Given a Lipschitz relaxed solution (u, m) to CMFG problem, we have that (I) u is aconstrained viscosity solutionof

(−ut+H(x , ∇xu) = F (x , m) in ]0, T [×Ω u(x , T ) = G(x , m(T )) ∀x ∈ Ω Moreover, defining

Qm = n

(t, x ) ∈]0, T [×Ω : x ∈ spt m(t)o

∂Qm = n

(t, x ) ∈]0, T [×∂Ω : x ∈ spt m(t)o the following holds true

(II) u isdifferentiableon Qmand −ut+H(x , ∇xu) = F (x , m) on Qm

(III) u has

time derivative, one-sidednormal derivative, andtangential gradienton ∂Qm

(IV) the tangential gradient ∇^τ_xu satisfies

−ut+H^τ(x , ∇^τxu) = F (x , m) on ∂Qm

where H^τ(x , p) = sup − hp, vi − L(x, v) | hv, ν(x)i = 0

(36)

Analysis of the continuity equation

Given a Lipschitz relaxed solution (u, m) to CMFG problem, we have that

(I) there exists abounded continuousvector fieldV :]0, T ] × Ω → Rⁿsuch that m satisfies the continuity equation

mt+div (mV ) = 0 in ]0, T [×Ω in the sense of distributions: ∀ φ ∈ C_c¹ ]0, T [×Ω

Z T 0

Z

Ω

φt+ hV , ∇xφidm(t, dx)dt = 0 (II) V is the expectedoptimal feedbackon Qm, that is,

V (t, x ) =(−∇pH x , ∇xu(t, x )

∀(t, x) ∈ Qm

−∇pH x , ∇^τ_xu(t, x ) + ∂_ν⁺_iu(t, x )νi(x )

∀(t, x) ∈ ∂Qm

(37)

Proof

Consider the continuous mapVm:Qm∪ ∂Qm→ Rⁿ Vm(t, x ) =(−∇pH x , ∇xu(t, x )

∀(t, x) ∈ Qm

−∇pH x , ∇^τ_xu(t, x ) + ∂_ν⁺_iu(t, x )νi(x )

∀(t, x) ∈ ∂Qm

and extend it to a continuous vector field V :]0, T [×Ω → RⁿbyTietze theorem Let η be aconstrained equilibriumassociated with (u, m): then

(t, γ(t)) ∈ Qm∪ ∂Qm and ˙γ(t) = V (t, γ(t)) ∀t ∈]0, T [ thanks to the optimality conditions

So, ∀ φ ∈ C_c¹ ]0, T [×Ω we use the change of variablesm(t) = et]ηto compute d

dt Z

Ω

φ(t, x )m(t, dx ) = d dt

Z

Γ

φ(t, γ(t)))η(d γ)

= Z

Γ

(∂tφ(t, γ(t)) + hDφ(t, γ(t)), ˙γ(t)

|{z}

=V (t,γ(t))

i)η(dγ)

= Z

Ω

(∂tφ(t, x ) + hDφ(t, x ), V (t, x )im(t, dx )

(38)

Concluding remarks

The constrained MFG system enjoys (almost) all the main differential properties that hold true for the unconstrained one.

However, none of these properties come for free:

one has to appeal to a very weak notion of equilibrium in order to give an existence result

(fractional) semiconcavity plays a decisive role in this analysis the structure of the superdifferential of a semiconcave function at a boundary point becomes relevant

the regularity of data is crucial for this approach

(39)