2021-22 Ongoing Research

(1)

2021-22 Ongoing Research

‣NEWS Lab

(2)

Operating Systems 作業系統

Op

2

Overview

‣ Autonomous Systems

‣ Distributed Vehicle Decision (ADLink)

‣ Vehicle Lane Change Decision

‣ Distributed Real-Time Messaging for V2V and V2X

‣ Autonomous Driving Middleware (Tier IV, JP)

‣ Smart Sensors

‣ CIM-Friendly Deep Neural Network Inference and Training

‣ Low-Power Always-On 3D Sensor Using CIM and Structure Light (MediaTek)

‣ Structure Light 3D Reconstruction for Endoscope (QuanTa Computers)

(3)

Autonomous Systems

(4)

4

2D SLAM for RoboCup SPL

(5)

Op

5

Autonomous Bus

(6)

Op

6

LiDAR-Based Tracking

Intention Predication on Unsignalized Intersection Sponsored by MediaTek and MOST

Figure 8: data flow

∑^𝑛𝑖=1 min

𝑘∈[1, 𝐾] ∥ (𝑌_𝑖− ^𝑌^(𝑘)𝑖 ) ∥²

(𝑿) (^𝒀^(𝑘))

(𝑀^𝑡^𝑜𝑏𝑠)

(8)

Op

Intention Predication on Unsignalized Intersection

LSTM SGAN(K=20) SCGAN(K=20)

• The LSTM model predicts deviated predictions

• With the social pooling module, SGAN predicts the vehicle to avoid the potential crashes.

However, it does not have location information, and therefore SGAN predicts it to slow down,

or speed up and turn around to avoid the pedestrians

• SCGAN learn to predicts the only feasible vehicle trajectories

to slow down

(9)

Op

9

Distributed Consensus as Virtual Traffic Signals Sponsored by ADLink

‣ Goal: optimize the intersection use subject to safety requirements without traffic lights

‣ Methods:

‣ Allow the vehicles to negotiate with each other and find the optimal decision to cross the intersection

‣ Road Side Unit will serve as the observer, gateway and decision logger.

‣ Challenges:

‣ The number of vehicles dynamically change.

‣ The decision has hard deadline constraint.

(10)

Op

¹⁰

V2X Communication

https://www.yunextraffic.com/global/en/portfolio/traffic-management/connected-mobility- solutions/vehicle2x-communication

(11)

Smart Sensors:

Low-Power r Always-On Real-Time

3D Sensing

(12)

12

現有成果 – 以 VCSEL (15 x 25)為光源

Resolution is 17um at 20cm

(13)

Op

¹³

Traditional endoscope

Stereo endoscope

DaVinci Surgery Robot

From 2D to 3D Endoscope

‣ The visualization of a 3-dimensional surgical field has the theoretical advantage to provide the surgeon with more realistic information about the anatomy of the surgical field which may be beneficial for surgical control and may even reduce complications.

‣ Current Products:

• Da Vinci XI

• Image1 S by KARL STORZ

• Olympus 3D Imaging Solutions

The mean duration for surgery (per sinus) was 7.75 min (SD 4.38) for 2D and 8.06 min. (SD 5.76) for 3D without statistical significance (P

= 0.334).

https://onlinelibrary.wiley.com/doi/10.1111/coa.13494

(14)

Op

¹⁴

Proposed Design

To better fit the light sources and camera into tube of endoscopes, we propose to (1) use fiber to carry the light of different wavelengths and

(2) add space mask to create structure patterns

to create structure patterns on surface. Given our current approaches, we can analyze the images to estimate the depth of the surface or create three-dimension mash of the image.

(15)

15

Architecture of Smart Sensors

Camera (CMOS)

Laser Speckle/

VCSEL

CLK Syn CIM for 3D Point

Cloud Reconstruction

Memory for RGB

3D Image

Reconstruction (on CPU/GPU)

3D Image Output

2D Image Output 3D Point Cloud

2D Image Output RGB Image + Structure Light Pattern

RGB Image

Always-On 3D Sensor (CIM 2022 Platform)

Always-On 3D Sensor (CIM 2023 Platform)

2021-22 Ongoing Research