Work

Cross-Modal Matching Between Neuromorphic Events and Color Images via Adversarial Learning

In this paper, we propose the Event-Based Image Retrieval (EBIR) task. Given an event stream depicting a particular object as query, the aim is to retrieve color images containing the same object that may be captured under different views, backgrounds, or with occlusions. We address the EBIR task by formulating an EBIR framework to jointly model neuromorphic events and color images into a common embedding space, where adversarial learning is used to align the feature distributions of the two modalities of data in the embedding space. We also contribute to the community the first EBIR dataset EC180 with 180 objects, where each object has an event stream and five color images.

Efficient Spatial-Temporal Normalization of SAE Representation for Event Camera

In this work, we proposed a highly efficient normalization method named chain normalization and an improved ordering strategy. By embedding the inherent nature of SAE, our method can not only run fast but also reach the highest performance without manual parameter tuning. In the experiments, our algorithm can run significantly faster than the previous methods. To further validate the normalized results, we conduct object recognition experiments on two large event-based open datasets. Experimental results show that our method achieves the highest classification accuracy among other normalization methods.

[Code] [Preprint]

Toward Autonomous Rotation-Aware Unmanned Aerial Grasping

S. Lin, J. Wang, R. Peng, W. Yang.

In this work, we developed a vision-based autonomous UAM with a 3DoF robotic arm for rotational grasping and with a displacement compensation system for compensating the center of gravity. And we proposed a novel detection approach called Rotation-SqueezeDet to enable rotation-aware grasping, which can give the target position and rotation angle in near real-time on Jetson TX2.

[Open-hardware], [Paper]

Development of an Intelligent Unmanned Aircraft System for Tunnel Inspection (Thesis, In progress)

In this work, I developed a tunnel inspection system based on Pixhawk4 and DJI M100 drone. I Employed point cloud filters and RANSAC line fitting for tunnel perception. And I employed VINS-Mono for localization. With several lab members, I successfully conducted multiple autonomous inspections in the 280-meter long Luojia Shan tunnel.

An Intelligent Unmanned Aircraft System for Wilderness Search and Rescue

H. Yu, S. Lin, J. Wang, K. Fu, W. Yang.

In this work, we developed a wilderness search and rescue (WiSAR) system based on DJI M100 Unmanned Aerial Vehicle (UAV) and a ground station to search and rescue the survivors in wild. We combined infrared and optical target detection to increase the detection speed and accuracy. And we used multiple sensors to make this system can autonomous avoiding obstructions and landing on mobile platform. To increase the detection accuracy of SSD, we adopted ResNet-101 as the base net. And trained it on the UAV-PP dataset. The actual flying test have been conducted in multiple situations to verify the feasibility of our WiSAR system. Paper accepted by IMAV and method has been granted a patent.

[Paper]

ICRA2018 DJI Robomaster AI Challenge

W. He, S. Lin, Q. Cui, R. Peng, J. Wang, H. Yu.

The ICRA 2018 DJI RoboMaster AI Challenge requires robots to drive and launch projectiles using artificial intelligence related technologies. Each team is required to build up to TWO autonomous AI robots to compete in a 5m × 8m arena filled with various obstacles. To win, a team must defeat TWO AI robots presented by the RoboMaster Organizing Committee.

We won the Finalist Prize, Global Rank:6/70 of ICRA2018 DJI Robomaster AI Challenge

Tech: ROS, C++, Python, Tensorflow, Solidworks, Embedded Programming

Smart Braille Reader
My work focused on the development of a universal braille reader, including the design of PCB circuit, communication protocol, touch unit, and an algorithm for touch signal processing. Prototypes have been commercialized by Chengdu Aurora Co. Ltd, helping blind children to "see" the world. And it has been granted two patents in China.

Won the 1^st Prize of Microsoft Image Cup 2016 Global Students Technology Competition (China Round), Apr. 2016

Tech: PCB Design, C, Embedded Programming, Python

Multi-node Intelligent Access Control System

In this work, I implemented a smart access control system with multiple nodes, and each node has a smoke sensor to sense its surrounding. Nodes can automatically exchange messages in the sensor network based on BLE 4.0, and the user can use the Android App to access the system.

Supported by:National Undergraduate Training Programs for Innovation(20000 CNY)

Tech: Android, BLE4.0, Java, C

Air Quality Detector

I didn't believe the official data from weather bureau. So I made my own detector to find it out.

Tech: PCB Design, Embedded Programming, C

A Sliver Ship

Great oaks from little acorns grow.

Tech: My Intelligence + My Hand