CommandPad: Using Finger Identification for Seamless Mode Switching of Command Gesture and Cursor Manipulation on Touchpad

(Personal Research Project, 2019)

Basic Idea

• Use an index finger for triggering command by drawing alphabetic gesture stroke (e.g., "g" shown in above image)
• Use a middle finger for cursor manipulation as in the original touchpad usage.
• Use colored finger strap (e.g., red and blue as shown in above image) to identify the touched finger.

System Component

• C#
  • Cursor Manipulation of main PC
  • Handwriting (Gesture) Recognition: Microsoft.Ink API
• Android
  • Trackpad Simulation
  • Send touch information to main PC (through TCP networking)
• Python
  • Finger Identification (Using Depth Camera: Intel Realsense D415)
  • OpenCV image processing
  • Depth Clipping (To ignore noises from background colors)