HOW WE DO IT
Inuitive makes chips. Our core technology is chips optimized for computer vision in mobile devices. The flagship product of this technology is called NU3000. NU3000 is a dedicated signal processor, architected for 3D image processing and computer vision. Its architecture is optimized to support depth sensing, 3D image processing and computer vision. It is a powerful 3D multipurpose image signal processor SoC.
NU3000 General Description
A high-level block diagram of the NU3000 is presented in Figure 3
Several processing units are operating in the NU3000, as described above. NU3000 key functional areas are:
- Command, control and interface all external elements on the module, including:
- Two RGB or IR CMOS sensors, aligned in a stereoscopic setup
- An RGB CMOS sensor, configured to function as a standard web-cam or a fisheye camera for motion tracking
- Two Infra-Red LED current drivers, one for flood illumination and one for pattern illumination
- One or more Analog-to-digital converters for microphone audio pickup
- An external DDR unit
- Several DC elements (DC/DC converters and Linear Regulators)
- Generate a real time depth map, based on the information generated by the two RGB or IR sensors; for this function we employ a dedicated hardware processor.
- Perform all computer vision functions that are specified above in the module features, such as: Extract skeletal information from the depth map, Extract gesture recognition from the skeletal information, voice detection, speech recognition and others.
- Perform all image enhancement functions as specified above in module features on web cam output
Multiple interfaces to support and control a variety of sensors.
- Real time processing capable of synchronizing, time-stamping and process inputs from multiple sensors, to serve as a smart sensor hub. Interface the host processor as a USB3.0 device.
The unique multi-core architecture implemented in NU3000, together with the dedicated hardware accelerated depth-from-stereo core, make NU3000 the most power-efficient solution for depth sensing. It can be shown that apples-for-apples comparison, an NU3000 depth sensor is the lower power solution available today.
NU3000 is a CMOS component. It is sized 10X12mm, in a FPBGA package.
As mentioned above, NU3000 contains powerful vector DSP cores, optimal for image processing, computer vision and real time sensing. Inuitive offers the following software libraries as part of the NU3000 offering:
- Finger tracking / gesture recognition
- Kanade-Lucas-Tomasi (KLT) Feature tracking algorithm
- Gaze tracking
- Real time streaming of depth information to the host / application processor
NU3000 is optimized for consumer mobile applications. It is a low power, small sized unit suitable to fit into mobile devices as a computer-vision co-processor.
NU3000 can off-load all image processing and computer vision tasks from the system application processor. It is a more power-efficient, optimized architecture for such tasks. Furthermore, it can serve as a primary smart sensor hub in the system – i.e. manage, synchronize and even process all sensors in the system to fuse a holistic real-time solution.
NU3000 is the processing core of a stereo based depth sensing camera. It can support many potential stereo topologies, to address many types of use cases and applications. The great flexibility offered NU3000 makes it suitable for various markets – each market with its own unique requirements, needs and constraints.
Virtual Reality products are already available in the market today. They are expected to grow exponentially in the coming years, with players such as Samsung (Gear VR), Facebook (Oculus Rift) and Sony (Morpheus) leading the market. VR products will become more and more present in our lives in the near future. A few examples of VR use cases are:
- Sports and Gaming: Simulating a virtual environment for gaming.
- Social media: creating a personalized environment for communicating with others, while isolating form the physical environment.
- Media consumption: 3D videos, music, “live” shows.
All VR products require sensors, to keep tracking of the unit location and orientation. NU3000 serves as the primary unit to manage all those sensors: it synchronizes all the sensors, reads out the sensors reading and then processes all sensors readings into a single coherent localization solution.
Augmented reality is quickly becoming a thing of the present. More and more companies and promoting and launching their augmented reality glasses. The market is led by Microsoft Hololens, Epson Moverio and Osterhouse Design Group (ODG) R-7. These are all ground-breaking AR technology. Other companies (e.g. Meta View, Magic Leap and others) are quickly following in those footsteps.
Augmented reality can be used in many cases, for example:
- Gaming: superimposing the gaming scene onto the user own living room, table or kitchen.
- Shopping: using 3D sensing, superimposing specific items onto the user environment to help make choices and smart shopping decisions.
- Navigation: add guidance instructions to the user while navigating in a building, or in the street. Also – make comments on specific shopping opportunities by pointing to specific points and shops.
Much like Virtual Reality, Augmented Reality products are consumer products with significant constraints of power and size. NU3000 is a perfect fit for Augmented reality needs, with regards to sensors control, depth sensing and sensor hub.
Robots are becoming a growing part of our everyday life. Vacuum cleaners, drones and other applications are all increasingly present.
For all robotics, the aspiration for autonomous management is primary. NU3000 allows robots to navigate freely is space, thus allow autonomic features.