DNA-like Structure of Nonlinear Functions

DNA-like Structure of Nonlinear Functions

Moody T. Chu

(Joint work with Zhenyue Zhang)

North Carolina State University

March 8, 2012 @ National Cheng Kung University

Disclaimer

I This talk is about mathematics, not biology.

I This talk is elementary, involving only fundamental calculus.

I This work is just a beginning. More need be done.

I The importance of DNA is well documented.

• Found in all living organisms.

• Supplies the information for building all cell proteins.

I Basic structure of DNA:

• Two strands coiled around to form a double helix.

• Each rung of the spiral ladder consists of a pair of chemical groups called bases (of which there are four types)

• Base pairing combines A to T and C to G, and the sequence on one strand is complementary to that on the other.

• The specific sequence of bases constitutes the genetic information.

Take Home Message

I There is a considerably similar structure in all nonlinear functions.

• The structure determines the properties of the underlying function?

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Outline

Basics

Gradient Adaption

Singular Value Decomposition Deformation Effect

Singular Curves Dynamical Systems Examples

Critical Curves Local Bearing

Curvilinear Coordinate System Generic Behaviors

Base Pairing

Concavity Property Pairings and Traits Applications

Conclusion

Gradient

I Given a scalar function

η : Rⁿ−→ R, define the gradient ofη by

∇η := ∂η

∂x1

, . . . , ∂η

∂xn

 .

I Significance:

• Points in the direction where the function η(x) ascends most rapidly.

• Attainable maximum rate of change is precisely k∇η(x)k.

Gradient Adaption

I Heat transfer by conduction.

• Opposite to the temperature gradient and is perpendicular to the equal-temperature surfaces.

I Osmosis.

• Passive transport of substances across the cell membrane down a concentration gradient without requiring energy use.

I Image gradients.

• Fundamental building blocks in image processing such as edge detection and computer vision.

Jacobian

I Given a vector function

f : Rⁿ−→ R^m, define the Jacobian of f by

Jf :=







∂f₁

∂x1 . . . _∂x^∂f1 .. n

. . .. ...

∂fm

∂x1 . . . ^∂f_∂x^m

n





.

• A natural generalization of the gradient.

• Both offer linear approximations.

• Does not indicate critical directions or rates of change?

Singular Value Decomposition

I Any given matrix A ∈ R^m×nenjoys a factorization of the form A = V ΣU^>.

• Known as a singular value decomposition (SVD) of A.

I Singular vectors:

• V ∈ R^m×m, U ∈ R^n×nare orthogonal matrices.

I Singular values:

• Σ ∈ R^m×nis diagonal with nonnegative elements σ1≥ σ2≥ . . . ≥ σκ> σκ+1= . . . =0.

• κ =rank(A).

Applications

I A long conceived notion popping up in various disciplines.

I Frequent appearance in a remarkably wide range of important applications.

I A few examples –

• Data analysis.

• Dimension reduction.

• Signal processing.

• Image compression.

• Principal component analysis.

• ...

Variational Formulation

I Many ways to characterize the SVD of a matrix A.

I Cast as an optimization problem over the unit disk:

kxk=1maxkAxk.

• Unit stationary pointsui ∈ Rⁿ= Right singular vectors.

• Singular values = kAuik.

I In the neighborhood of the origin:

• Right singular vectors = Directions where the linear map A changes most critically.

• Singular values = Extent of deformation.

I Similar role by the left singular vectors by the duality theory.

Linear Approximation

I Nearby any given pointex, approximate f (x) by the affine map g(x) := f (ex) + f⁰(ex)(x −ex).

I Under the function g,

• The unit sphere centered atex gets mapped into an ellipsoid centered at f (ex).

• Semi-axes are aligned with the left singular vectors of f⁰(ex).

• Semi-axis lengths are precisely the singular values.

Infinitesimal Deformation

I Reducing the radius of the sphere,

• Downsizes the ellipsoid proportionally.

• Does not alter the directions of the semi-axes.

• g becomes a more accurate approximation of f .

I The gradually reduced ellipsoids silhouette the images of the gradually reduced spheres under f .

I The SVD information of the linear operator f⁰(ex) manifests the infinitesimal deformation property of the nonlinear map f atex.

Directional Derivatives

I Consider the norm of the directional derivative

t→0lim

f (ex + tu) − f (ex) t

= kf⁰(ex)uk.

• u is an arbitrary unit vector.

I Along which direction will the norm of the directional derivative be maximized?

• The right singular vectors of f⁰(ex)!

I This is the generalization of the conventional gradient to vector functions.

Singular Vector Field

I At every pointx ∈ Rⁿ,

• Have a set of orthonormal vectors pointing in particular directions related to the variation of f .

• These orthonormal vectors form a natural frame point by point.

I Tracking down the “motion" of these frames might help to reveal some innate peculiarities of the underlying function f .

Dynamical Systems

I Let (σi, ui, vi)= the ith singular triplet of f⁰(x_i). Interested in the solution flows:

• xi(t) ∈ Rⁿdefined by

˙xi := ±ui(xi), xi(0) =ex.

• yi(t) ∈ R^mdefined by

˙yi:= ±σi(xi)vi(xi), yi(0) = f (ex).

I Minor notes:

• Scaling ensuresyi(t) = f (xi(t)).

• Select the sign ± so as to avoid discontinuity jump.

• Integrate in both forward and backward time.

Critical Points

I The vector field may not be well defined at certain points.

• When singular values coalesce.

• f⁰(x) has multiple singular vector

• Makes ˙xi(or ˙yi) discontinuous.

I Not an issue of the factorization.

• An analytic factorization as a whole for a function analytic inx does exist.

• The continuity of a fixed order singular vectors, say,u1(x), may not be maintained.

First Singular Curve

I Moves in the direction along which f (x) changes most rapidly, when measured in the Euclidean norm.

I Serves as the backbone in the moving frame.

I Can be demonstrated and explained in the case f : R²→ Rⁿ.

• Parametric surfaces.

I More need be done in higher dimensional spaces.

Example 1

 sin (x₁+x₂) +cos (x₂) −1 cos (2 x₁) +sin (x₂) −1



−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Right Singular Curves for Example 1

x1

x 2

Example 2a

 e^x1cos(x₂) 20e^x1sin(x₁)



−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Right Singular Curves for Example 2a

x1

x 2

Example 2b





e^x1cos(x₂) e^x1sin(x₁)

x2





−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Right Singular Curves for Example 2b

x1

x 2

Example 3





4 + x₁cos(x₂/2) x₂ x1sin(x1x2/2)





−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 3

Example 4

 e^x1cos(20x2) 20e^sin(x2)sin(x1)



−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 4

Example 5

 sin(x₁²+x₂²)cos(x₂) 2e^−2x22x12cos(10 sin(x1))



−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 5

Example 6









−270x₁⁴x₂³− 314x1x₂⁴− 689x1x₂³+1428





36x₁⁷+417x₁⁶x2−422x₁⁵x₂²−270x₁⁴x₂³+1428x₁³x₂⁴−1475x₁²x₂⁵+510x1x₂⁶

−200x₁⁶−174x₁⁵x2−966x₁⁴x₂²+529x₁³x₂³+269x₁²x₂⁴+49x1x₂⁵−267x₂⁶+529x₁⁴x2

+1303x₁²x₂³−314x1x₂⁴+262x₂⁵+36x₁⁴−788x₁²x₂²−689x1x₂³+177x₂⁴













−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 6

Example 7







x1−^x₃¹² +x1x₂² x2−^x₆²³ +x2x₁³

x₁²− x₂³







−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 7

Example 8





1

2 2ρ²− φ²− ψ²+2φψ(φ²− ψ²) +ψρ(ρ²− ψ²) +ρφ(φ²− ρ²)

√3

2 φ²− ψ²+ (ψρ(ψ²− ρ²) +ρφ(φ²− ρ²))
(ρ + φ + ψ) (ρ + φ + ψ)³+4(φ − ρ)(ψ − φ)(ρ − ψ)





with







ρ = cos(x1)sin(x2) φ = sin(x1)sin(x2) ψ = cos(x2)

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1

x 2

Right Singular Curves for Example 8

Why?

A Closer Look

I Write

f⁰(x) =

a₁(x), a₂(x)  .

I Define scalar functions

( n(x) := ka₂(x)k²− ka₁(x)k², o(x) := 2a₁(x)^>a₂(x).

• n(x) measures the disparity of lengths.

• o(x) measures nearness of orthogonality.

Critical Curves

I Define

( N := {x ∈ Rⁿ| n(x) = 0} , O := {x ∈ Rⁿ| o(x) = 0} .

I Each forms generically a 1-dimensional manifold in R².

• Possibly composed of multiple curves or loops.

• Will play the role of “polynucleotide" connecting a string of interesting points.

First Right Singular Pair

I The first singular value of f⁰(x):

σ1(x) := 1 2



ka₁(x)k²+ ka2(x)k²+ q

o(x)²+n(x)²

^1/2

I The first right singular vector:

u₁(x) := ±1 p1 + ω(x)²

"

ω(x) 1

# .

with

ω(x) :=







o(x) n(x)+√

o(x)²+n(x)², if n(x)> 0,

−n(x)+√

o(x)²+n(x)²

o(x) , if n(x) < 0.

• Take the limit if ω(x) becomes infinity.

Crossings

I When singular curves coming across critical curves, their tangent vectors point in specific directions.

I Orientations of tangent vectors:

• At N − O, are parallel to either [1, 1]^>or [1, −1]^>, depending on whether o(x) is positive or negative.

• At O − N , are parallel to [0, 1]^>or [1, 0]^>, depending on whether n(x) is positive or negative.

Singular Points

I NT O = singular points.

I At singular points,

• Singular values coalesce.

• The (right) singular vectors become ambiguous.

• Singular curves are“terminated" or “reborn".

I The angles cut by N and O at the singular point affects the intriguing dynamics observed.

• The 1-dimensional manifolds N and O string singular points together along their strands.

x1

x 2

Critical Curves and Singular Curves for Example 1

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

ridges of near singular values

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Right Singular Curves for Example 2a

x₁

x 2

x₁

x 2

Critical Curves for Example 2a

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Right Singular Curves for Example 2b

x₁

x 2

x₁

x 2

Critical Curves for Example 2b

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x₁

x 2

Right Singular Curves for Example 3

x₁

x 2

Critical Curves for Example 3

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1 x2

Right Singular Curves for Example 4

x1 x2

Critical Curves for Example 4

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x1 x2

Right Singular Curves for Example 5

x1 x2

Critical Curves for Example 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x₁

x 2

Right Singular Curves for Example 6

x₁

x 2

Critical Curves for Example 6

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x₁

x 2

Right Singular Curves for Example 7

x₁

x 2

Critical Curves for Example 7

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

x₁

x 2

Right Singular Curves for Example 8

x₁

x 2

Critical Curves for Example 8

−5 −4 −3 −2 −1 0 1 2 3 4 5

−5

−4

−3

−2

−1 0 1 2 3 4 5

Curvilinear Coordinate System

I Denote theα-halves portions of N and O by by nαand oα, where

• The crossing singular vectors are parallel to the unit vectors un_α :=^√1

2[1, 1]^>anduo_α:= [0, 1]^>.

I Flag the sides of nαand oαby arrows .

• Naturally divides the neighborhood ofx0into “quadrants"

distinguished by the signs (sgn(n(x)), sgn(o(x)).

I When the “orientation" is changed, the nearby dynamical behavior might also change its topology.

A Scenario of Propellant

N O x0

(+,+) (+,−) (−,+)

(−,−) N

O x0

(+,+) (+,−) (−,+)

(−,−)

1

I Red segments = tangent vectors crossing the critical curves.

• Take into account the signs of o(x) and n(x).

I Invariant on each half of the critical curves.

I Flows of singular curves nearx₀should move away fromx₀as a repellant.

A Scenario of Roundabout

N

O x0

(+,+)

(+,−) (−,+)

(−,−)

N

O x0

(+,+)

(+,−) (−,+)

(−,−)

1

Generic Behaviors

I Divide the plane into eight sectors with a central angle ^π₄.

I Relative position of nαand oαwith respect to these sectors is critical for deciding the local behavior.

Regular Cases

Aa Aa Ac Ac Bb Bb Bd Bd

Dd Dd Ca Ca Cc Cc Db Db

Db Db Cc Cc Ca Ca Dd Dd

Bb

Bb Bd Bd Aa Aa Ac Ac

Bb

Bb Bd Bd Aa Aa Ac Ac

Ca Ca Dd Dd Db Db Cc Cc

Cc Cc

Db Db

Dd Dd

Ca Ca

Aa

Aa Ac Ac Bb Bb Bd Bd

Mutative Cases

Bb Ca Dd Ac Bb Ca Dd Ac

Aa Db Cc Bd Aa Db Cc Bd

Second Derivative

I Expressω(x) as

ω(x) :=















sgn (o(x))−^n(x)_o(x)+sgn(o(x))n(x)² 2o(x)² +O

n(x)³

, near n(x) = 0,

o(x)

2n(x)−_8n(x)^o(x)33+_16n(x)^o(x)5₅+O o(x)⁷

, near o(x) = 0 and if n(x) > 0,

−1 o(x)

2n(x)−^o(x)3 8n(x)3+^o(x)5

16n(x)5+O(^o(x)7), near o(x) = 0 and if n(x) < 0.

• The first derivative ofx1(t) is related to ω(x1(t)).

• The first term of ω(x) estimates the the second derivative of x1(t).

• Can characterize the concavity property observed.

Variation near N

1 -1

n(x) > 0

n(x) < 0 n(x) < 0

o(x) > 0 o(x) < 0

Increase ω(x), decrease the slope

Slope =±1 when n(x) = 0 A typical point on the critical curveN

ω(x)

1

I In the directionu_nα,ω(x(t)) must be increased if x(t) moves to the side where n(x)< 0.

• The slope ofu1(x(t)) must be less than 1.

I Only four basic ways to cross N .

Four Bases along N

(A)

(C)

n(x) > 0 n(x) > 0 n(x) < 0

n(x) < 0

x0

x0

τn

τn

(B)

(D)

n(x) > 0

n(x) > 0

n(x) < 0

n(x) < 0 x0

x0

τn

τn

1

Variation near O

o(x) n(x)> 0

o(x) n(x)> 0

o(x) n(x)< 0

o(x)

n(x)< 0 Increase ω(x), decrease the slope Slope =∞ or 0 when o(x) = 0

A typical point on the critical curveO ω(x)

1 ω(x)

1

Four Bases along O

(a)

(c)

o(x) n(x)> 0

o(x) n(x)> 0

o(x) n(x)< 0

o(x) n(x)< 0

x0

x0

τn

τn

τo

τo

(b)

(d)

o(x) n(x)> 0

o(x) n(x)> 0

o(x) n(x)< 0

o(x) n(x)< 0

x0

x0

τn

τn

τo

τo

1

Pairing

I Entire dynamics can be classified into 8 categories.

• These base parings are Aa, Ac, Bb, Bd, Ca, Cc, Db, and Dd only, with no other possible combinations.

I Each base pairing results in 8 dynamics in the regular cases and 2 in the mutative cases.

• Distinctive by their characteristic traits.

• Fascinating, but no time in this talk.

I Identify each dynamics by two letters of base paring at the upper left corner.

Trait Characterization

I Base pairings characterize dynamical details.

I Can also characterize the general behavior by a single quantity.

• Define θ(nα,oα)= Angles measured clockwise from τnand to τo.

• Assume the generic condition that τnis not forming an angle^π₄ with the north.

• Singular pointx0is

• A repeller, if 0 < θ(nα,oα) < π.

• A roundabout, if θ(nα,oα) > π.

I Crossovers/hybrids are possible.

I ...

I Too detailed to include here.

Making Mosaics

I Classify of all possible local behaviors.

• A simplistic collection of “tiles" for the delicate and complex

“mosaics".

I Inherent characteristics of the underlying function arrange these local pieces together along the strands of N and O to form the various patterns.

A Comparison

I Consider examples 2a and 2b.

• Easy critical curves.

• O forms horizontal lines with alternating o(x) in between.

• N forms closed loops.

• One additional vertical, continuous, ogee N curve in Example 2b.

• n(x) > 0 inside the loops and to the left of the ogee curve.

I Similar, but different dynamics.

Zoom In

x1

x 2

Curves near Singular Points for Example 2a

R S R S

−1 −0.5 0 0.5 1 1.5 2 2.5 3

0.5 1 1.5 2 2.5

x1

x 2

Curves near Singular Points for Example 2b

T

U

V

−1.50 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4

0.5 1 1.5 2 2.5 3

A Jigsaw Puzzle

x₁

x 2

α −Halves and Base Pairings for Example 1

−2 −1 0 1 2 3 4

−2

−1 0 1 2 3

4 Dd/Db

Ca/Cc

Cc/Ca

Db/Dd Aa/s

Bd

Bd

Ac

Aa

Ac

Bb

Bb

Ac Ac

Bd Bd

Aa/s

Bb

Ac Bb

1

Conclusion

I Gradient adaption is an important mechanism occurring in nature.

• Its generalization to the Jacobian does not “discriminate" directions per se.

I Adaption information is coded in the singular curves.

• Forms a natural moving frame telling intrinsic properties per the given function.

• Results in intricate and complicated patterns.

I Global behavior in general and interpretation in specific are not conclusively understood yet.

• Two stands joined by singular points with one of eight distinct base pairings make up the underlying function.

• Amazingly analogous to the DNA structure essential for all known forms of life.

I Are the patterns discovered “the trace of DNA" within an abstract, “inorganic" function?