1. Simultaneous Multiple-Channel Timbre-Based Data Sonification
Data sonification is the use of non-speech audio to convey information or perceptualize data. Powerful specifics of auditory perception (temporal and pitch resolution, timbral parsing, melodic parsing, cocktail-party effect, and many others) permit sonification offers an interesting alternative or complement to data visualization techniques.
NRI’s multichannel data sonification technologies provide parallel perceptual channels for representing complex numerical data to a user seeking to identify correlations within data. NRI’s multichannel data sonification technologies include:
- Representations of several simultaneously-varying data quantities as time-varying sound for presentation to a user for observing variations and correlations among sonic events or trends.
- Each of several simultaneously sounding data-modulated sound timbre classes can each separately rendered within a stereo sound field,
- Interactive capabilities permitting exploration of large complex large-dimension data sets,
- Arrangements and selections of sound timbre classes to convey sonic metaphors,
- Providing a user interface that includes a plurality of metaphors and spatial sound rendering operations resulting in stereo audio output,
- Each audio waveform can be positioned within a sonically-rendered sound field, associating information with positions within the sound field,
- Mappings can be used to associate timbre and other sound parameters with selected values from the multidimensional numerical data,
- Arrangements for explicit use of data sonification with data visualization to provide parallel perceptual channels for representing complex numerical data to a user seeking to identify correlations within the data; for example:
- Several varying data quantities can be represented by time-varying graphics while
several other varying data quantities are represented by time-varying sound, - Visual representations can be used to spatially choose or identify position or
paths in a dataset; for example, a cursor can be moved over visualized data as
associated multidimensional numerical data values are sonified, - Sonification can be used to offload some information-carrying information
capacity from a visualization system, while other information can be rendered via
both sonification and visualization to provide affirming or orienting redundancy, - Joint and coordinated visual-sonic metaphors can used for this or other purposes
(for example, sonification can include multiple data-modulated sound timbre
classes, each rendered within a stereo sound field according to a spatial
metaphor that is shared with the visualization),
- Several varying data quantities can be represented by time-varying graphics while
- These capabilities can be implemented in the context of web browsers and spreadsheets so as to provide web access and a real-time collaboration environment.
NRI’s data sonification can be used to add valuable interactive sonification features mathematical packages such as MathematicaTM (which does not yet have real-time interactive sonficationfeatures) and MatLabTM, used to valuably supplement visual capabilities in business dashboards and spreadsheet programs, employed in plant control consoles, integrated into automotive, medical, and industrial diagnostic systems, and add similar value in many other application areas.
2. Value-Driven Visualization Primitives for Tabular Data of Spreadsheets
NRI’s tabular data visualization technology includes approaches to transcend the capabilities of (and simplify the use of) “conditional formatting” in spreadsheets and related data representation systems. Responsive to associated data value(s), or functions of those values, the color, outlines, and other visual effects of spreadsheet cells, displayed data within the cells, icons, etc. can be simply setup and vary continuously over ranges of associated data values.
NRI’s tabular data visualization technology also includes graphical tools and primitives for visualization of graphical surfaces from tabular data such as in a spreadsheet. Visualizations of a two-dimensional surface can be rendered from provided data according to position, orientation, and viewpoint, some or all of which can be controlled by a user interface. The surface can also represent interactively controlled filtered or statistically-processed tabular data. The surface can be suspended over a region of the displayed tabular data or displayed in other ways. The visually rendered surfaces can comprise symbols, colors, textures, or glyphs representing data, and virtual illumination. Visual height measuring and level sets slicing tools can be provided.
NRI’s tabular data visualization technology can be arranged to rendered graphics in a web browser and responsive to data transmitted over a network. Interactive control can also be transmitted over a network so as to provide web access and a real-time collaboration environment.
NRI’s tabular data visualization technology can be arranged to process data via function blocks, with connectivity among the blocks specified by drawing tool or palette of function blocks, for example clicking on each function block raising dialog windows that can be used for setting parameters.
3. Surface-Curve and Surface-Surface Graphical Intersection Tools and Primitives for Data Visualization
NRI’s graphical intersection-visualization technology includes graphical tools and primitives for interactively intersecting graphical surface with another graphical surface or graphical curve in a visualization setting to produce intersection data and/or intersection graphics. Among other things, such intersections can be interpreted as solutions to systems of equations implied by the data sets. Example features include:
- Visualizations of two-dimensional surfaces are rendered from provided data according
to position, orientation, and viewpoint, some or all of which can be controlled by a
user interface so as to create or control the intersection of parametrized versions
of the surfaces, - The provided data can be provided by an electronic spreadsheet,
- The visualization setting for the intersection can be displayed together with the
display an electronic spreadsheet or other tabular data representation, - Visually rendered surfaces can comprise symbols, colors, textures, or glyphs
representing data, and virtual illumination, - Intersection data corresponding to the mathematical intersection of the surfaces is
computed and provided to further computation. Intersection computation can comprise
interpolation and resampling, - These capabilities can be implemented in the context of web browsers and spreadsheets
so as to provide web access and a real-time collaboration environment.
Potential applications include simple business analysis, minimum cost problems, demographic analysis, economics, logistics, and a wide variety of applications in engineering and science.
4. Interactive WYSIWYG Presentation Control of Mathematical and Statistical 2D and 3D Visual Plots
NRI’s interactive WYSIWYG (“What You See Is What You Get”) plot presentation and formatting technology for mathematical and statistical visual plots provides interactive adjustment of plot and data visualization through clicks, rollovers, menus, and other familiar types of rapid user-machine interaction. Example features and capabilities include:
-
- Interactive adjustments also modify associated software code used to generate the
underlying plot or data visualization. In some implementations this feature may be
always active, - Simple interactive control can be provided by simple mice and their equivalents,
- More intricate and powerful interactive control can be provided by gesture-based
touch interfaces advanced High-Dimensional Touch Pads (HDTP) and associated touch
screens, game controllers, 6D-mice, and extended hyperlink objects (see Touch,
Gesture, and HDTP), - Arrangements to provide related features to more general programming languages not
involved in plots or visualization, allowing users to invoke extended sets of
options via interactive GUIs,
- Interactive adjustments also modify associated software code used to generate the
These capabilities can be implemented in the context of web browsers and
spreadsheets so as to provide web access and a real-time collaboration
environment.
Application areas for NRI’s interactive WYSIWYG plot presentation and formatting technology include business intelligence, simple spreadsheet plots, mathematical packages such as MathematicaTM and MatLabTM, and a wide range of data visualization applications.
Issued Patents
| Title | Patent Number | Application Number | Priority Dates | Text Only | Related Patents | |
|---|---|---|---|---|---|---|
| Sound Synthesis for Data Sonification Employing a Human Auditory Perception Eigenfunction Model in Hilbert Space | 10,832,693 | 15/997,539 | 07/31/2009 | Text | Data Visualization And Sonification | |
| Multi-Channel Data Sonification System with Partitioned Timbre Spaces Including Periodic Modulation Techniques | 10,365,890 | 16/049,573 | 06/16/2009 | Text | Data Visualization And Sonification | |
| Multi-channel data sonification employing data-modulated sound timbre classes | 10,037,186 | 15/585,702 | 06/16/2009 | Text | Data Visualization And Sonification | |
| Value-Driven Visualization Primitives for Tabular Data of Spreadsheets | 9,665,554 | 13/669,431 | 09/02/2009 | Text | Data Visualization And Sonification | |
| Joint and coordinated visual-sonic metaphors for interactive multi-channel data sonification to accompany data visualization | 9,646,589 | 14/176,000 | 06/16/2009 | Text | Data Visualization and Sonification |
|
| Surface-curve graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets | 8,826,114 | 13/673,732 | 09/02/2009 | Text | Data Visualization And Sonification | |
| Surface-surface graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets | 8,826,113 | 13/670,346 | 09/02/2009 | Text | Data Visualization And Sonification | |
| User Interface Metaphor Methods for Multi-Channel Data Sonification | 8,692,100 | 13/846,843 | 06/16/2009 | Text | Data Visualization and Sonification | |
| Interactive Multi-Channel Data Sonification to Accompany Data Visualization with Partitioned Timbre Spaces Using Modulation of Timbre as Sonification Information Carriers | 8,440,902 | 13/450,350 | 06/16/2009 | Text | Data Visualization and Sonification | |
| Multi-Channel Data Sonification in Spatial Sound Fields with Partitioned Timbre Spaces Using Modulation of Timbre and Rendered Spatial Location as Sonification Information Carriers | 8,309,833 | 13/450,339 | 06/16/2009 | Text | Data Visualization and Sonification | |
| Multichannel Data Sonification System with Partitioned Timbre Spaces and Modulation Techniques | 8,247,677 | 12/817,196 | 06/16/2009 | Text | Data Visualization and Sonification |
Pending Published Applications
| Title | Publication Number | Application Number | Priority Dates | Publish Date | Text Only | Related Patents | |
|---|---|---|---|---|---|---|---|
| Interactive graphical surface-rendering data visualization tools for tabular data and spreadsheets | 2017/0262425 | 15/601,892 | 09/02/2009 | Text | Data Visualization And Sonification | ||
| Interactive WYSIWYG Control of Mathematical and Statistical Plots and Representational Graphics for Analysis and Data Visualization | 2012/0317509 | 13/357,595 | 09/02/2009 | 12/13/12 | Text | Data Visualization And Sonification | |
| Interactive Data Visualization Utilizing HDTP Touchpad, HDTP Touchscreens, Advanced Multitouch, or Advanced Mice | 2013/0091437 | 12/875,128 | 09/02/2009 | 04/11/13 | Text | Data Visualization And Sonification |
Pending Unpublished Applications
| Title | Application Number | Priority Dates | Related Patents |
|---|---|---|---|
| Data Flow Management Arrangements for Multi-Channel Data Sonification Systems | 16/525,481 | 07/31/2009 | Data Visualization and Sonification |