Integrated Lensless Camera, Touch and Display

NRI’s Lensless Light Field Imaging technologies that include integrated display and imaging capabilities (for example US 8,754,842, US 8,830,375, and US 15/647,230, as well as related patent assets listed therein), when implemented with LED, micro-LED, or OLED arrays, can be adapted to implement a number of self-illuminating contact-sensing, contact imaging, and simultaneous image display/image sensing functions. These can be used to implement (WITHOUT any need for RF capacitive matrix or front-facing cameras) integrated combinations of finger-operated touch screen, touch gesture sensor, color image display, hand-image gesture sensor, document scanner, secure optical data exchange, and fingerprint processing capabilities for smartphones, tablets, laptops, gaming devices, and other systems and information appliances. With adequate LED, micro-LED, or OLED array resolution, touch sensing capabilities can support implementations of NRI’s 6D TouchTM (roll/pitch/yaw finger-angle) technologies (see Touch, Gesture, and HDTP). (It is also noted that NRI’s Lensless Light Field Imaging technologies can implement software-controllable stereoscopic 3D video capture, providing entirely new capabilities for future “3D phone” concepts.)

As an important opportunity for GPU (“Graphics Processing Unit”) manufacturers provided by NRI’s technology is that (array-based) front-facing imaging formation calculations, touch gesture sensing calculations, and touch gesture recognition calculations can be naturally handled within the GPU idle cycles rather than with external processor chips or the main processor chip. Among many other implications, this permits:

  • Shifting the entire world of gesture recognition and processing from general processors to GPU manufacturers (!)
  • Tight integration of high-speed gestures (including next-generation sophisticated gestures such as NRI’s 6D TouchTM) and high-speed graphics rendering for use in high-performance gaming, 3D, and map applications.
  • Individual LED, micro-LED, or OLED elements in an LED, micro-LED, or OLED array can be used for light sensing as well as light emission functions, either spatially-interleaved or time-multiplexed mode-changes:
      • In one implementation a single OLED array is used as the only
        optoelectronic user interface element in the system.
    • In another implementation two OLED arrays are used, each performing and/or
      optimized
      from different functions. In another implementation, an LCD and an OLED array are
      used in various configurations.

  • A software algorithm can produce touch measurement information responsive to light
    reflected by a finger proximate to the LED, micro-LED, or OLED array.
  • To remove effects of ambient light, the emitted light can be appropriated
    modulated and the system configured to be responsive to received reflected
    modulated light, generating measurement signals responsive to the only reflected
    modulated light.

Issued Patents

TitlePatent NumberApplication NumberPriority DatesPDFText OnlyRelated Patents
Use of LED or OLED array to implement integrated combinations of touch screen tactile, touch gesture sensor, color image display, hand-image gesture sensor, document scanner, secure optical data exchange, and fingerprint processing capabilities 9,632,34413/961,61107/09/2010
07/11/2011
PDFText Integrated Lensless
Camera, Touch, and Display
LED/OLED array approach to integrated display, lensless-camera, and touch-screen user interface devices and associated processors 9,626,02313/180,34507/09/2010PDFText Integrated Lensless
Camera, Touch, and Display

Pending Patents

TitlePublication NumberApplication NumberPriority DatesPDFText OnlyRelated Patents
LED/OLED array approach to integrated display, focusing lensless light-field camera, and touch-screen user interface devices and associated processors2017/022020115/488,29507/09/2010PDFTextIntegrated Lensless Camera, Touch, and Display
Use of Transparent Photosensor Array to Implement Integrated Combinations of Touch Screen Tactile, Touch Gesture Sensor, Color Image Display, Hand-Image Gesture Sensor, Document Scanner, Secure Optical Data Exchange, and Fingerprint Processing Capabilities
2017/0220145 15/492,68107/09/2010
08/07/2013
PDFTextIntegrated Lensless Camera, Touch, and Display
Use of Organic Light Emitting Diode (OLED) Displays as a High-Resolution Optical Tactile Sensor For High Dimensional Touchpad (HDTP) User Interfaces2012/027459613/547,02407/11/2011PDFTextIntegrated Lensless Camera, Touch, and Display

Pending Unpublished Applications

TitleApplication NumberPriority DatesRelated Patents