Digital Commons Network^™

Directional Audio Technique, Marcel Baker, Alexis Cox, Mathias Paumgarten, Aman Govil

Defensive Publications Series

A directional audio technique is used to manage audio channels of a virtual reality environment. The technique allows a user to hear sounds from features presented in the virtual reality environment in a fashion where sounds are emphasized based on a viewing direction, user navigation, or a virtual position of origin of a feature within the virtual reality environment. When one audio channel associated with a particular viewing direction, user navigation, or virtual position of origin is emphasized, other audio channels are muted, allowing the user to audibly distinguish, for example, different storylines, conversations, or events within the virtual reality …

Three-Dimensional (3d) Audio Laser For Virtual Reality Systems, Xavier Benavides Palos, Adam Glazier

Defensive Publications Series

A three-dimensional (3D) audio laser for VR systems is provided. The 3D audio laser is a system that includes a laser pointer and a dedicated audio location control. The system uses sound to orient a user to a virtual object and to provide an audible description of the virtual object. A location and a name (or other description) of virtual objects in the VR environment are known to the VR system. When the user aims the laser pointer at a particular virtual object and selects the dedicated audio location control, the VR system plays a series of tones with simulated …

Geo-Located Content In Virtual And Augmented Reality, Thomas Anglaret

Defensive Publications Series

A three-dimensional (3D) world representation is used to place virtual objects that represent geolocalized content into a virtual reality (VR) or augmented reality (AR) environment. Browsing navigation may then be performed in 3D by touching, pointing, or focusing on the virtual object to reveal its corresponding media content. The media content may be static and/or interactive content based on video, audio, images, and/or texts. Rather than being presented on flat surfaces, the content surrounds the user in a more-immersive mode, and more-natural VR or AR techniques can be used to select, expand, move, and interact with the content.

Machine Learning For Gesture Recognition In A Virtual Environment, Sandro Feuz, Pedro Gonnet Anders

Defensive Publications Series

A machine-learning technique is provided that allows a VR system to automatically learn user gestures based on a ground-truth data set collected from real users. The technique includes two steps. First, ground-truth data is collected by observing multiple users intentionally performing a specified action in a virtual environment. For example, an action to move an object from one place to another is recorded through input from different sensors in the VR system (e.g., position, orientation, controller actuations, or force/acceleration data). Second, machine-learning techniques (e.g., a recurrent neural network, a feedforward neural network, or a Hidden Markov …

Contextually Organizing Notes In An Assistant Environment

Defensive Publications Series

An assistant is described that stores and organizes user created information (e.g., memos, notes, lists, reminders, etc.) such that the assistant can easily retrieve the information during a future interaction between the user and the assistant. The assistant supplements or tags information with contextual data or other descriptors that are relevant to the real-world or virtual situation the user was in, when the information was originally created or last modified by the assistant. This way, the assistant can later retrieve the information if the user makes references to the specific the information or the previous situation that the user was …

Assistant Recommended Purchases Based On Recurring User Behavior, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that recommends purchases based on recurring user behavior. The assistant monitors shopping lists of its users to determine which items get added to the shopping lists and under what circumstances. The assistant monitors other user behavior, particularly with regards to purchases made (e.g., when, where, and under what circumstances) and develops rules for predicting what items the users will want to purchase and under what conditions. The assistant outputs recommendations based on the rules for adding predicted items to shopping lists. In some implementations, …

Disambiguating Notes Of Different Users In A Multi-User Assistant Environment, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that stores and organizes user created information (e.g., memos, notes, lists, reminders, etc.) on behalf of multiple users of the assistant. The assistant annotates, tags, or otherwise organizes user created information according to which particular user the information was created for. This way, during a future interaction between a particular user and the assistant, the assistant can easily identify and retrieve the information that is associated with that particular user and not some other user of the assistant.

Assistant Autogenerated Planning Lists, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that automatically generates grocery lists for preparing one or more meals or other events. The assistant is provided with a meal plan (e.g., a menu of one or more meals or dishes). The assistant parses the meal plan into individual recipes and from the recipes, generates a list of ingredients and respective quantities. The assistant then creates a shopping list from the ingredients list. The items in the shopping list may be grouped by meal or dish, item type, store location, need for refrigeration, …

Assistant Search Corpus Using Event Metadata, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that includes event metadata in its search corpus to determine a response to queries related to a personal calendar of events. The assistant determines a query related to an event that is recorded on a calendar of a user. To determine a response to the query, the assistant searches through an advanced search corpus that includes past and future event data published on the calendar as well as event metadata associated with each past and future event in the calendar. Reliance on the advanced …

Providing Local Or Other Identified News Information Using An Interactive Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to identify local or other news information from or more content providers and to output such information to a user. For example, an interactive assistant may accept queries from a user, perform searches, and subsequently identify news information that may be of interest to the user based upon these prior searches. In some cases, the interactive assistant may provide news information to a user in direct response to receiving a request for such information from the user. For instance, the …

Context-Based Selection And Presentation Of Content For Presentation To A User, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to select content based on various contextual information and to output such content to a user. For example, a user may engage in various different interactions with a computing system and/or with an interactive assistant over time (e.g., by performing certain actions, providing instructions, requesting select information, performing searches, or the like). Based on these interactions, the interactive assistant may identify and store contextual information associated with these interactions, such as, for instance, one or more of user behavior information, …

Preparing An Assistant-Enabled Vehicle

Defensive Publications Series

An interactive assistant computing system, referred to herein as “an interactive assistant,” “a virtual assistant,” “a computational assistant,” or simply an “assistant,” is described that stores information related to settings of a vehicle (e.g., seat locations, cabin temperature, mirror location, music preferences, directions to a destination, etc.) such that the assistant can automatically adjust one or more parameters of the vehicle in response to receiving an indication of an identity of an operator or passenger of the vehicle. The assistant in the vehicle may store settings for various users and prepare the vehicle once a second computing device sends the …

Provision Of Personalized Date-Based Content, Google Inc.

Defensive Publications Series

An interactive assistant computing system, referred to herein as “an interactive assistant,” “a virtual assistant,” “a computational assistant,” or simply an “assistant,” is described that stores and/or retrieves information related to items associated with particular days for a user such that the assistant can automatically retrieve one or more items associated with the current day (e.g., a local weather forecast, a current weather report, upcoming calendar items for the current day, current news stories, stock prices, indications of the day having personal importance, etc.) and provide a personalized report that includes the items specific to the current day. The assistant …

Assistant-Related Tasks Using Context From Multiple Devices, Google Inc.

Defensive Publications Series

An interactive assistant computing system, referred to herein as “an interactive assistant,” “a virtual assistant,” “a computational assistant,” or simply an “assistant,” is described that, in addition to performing general task requests, will request information from a respective assistant on one or more other computing devices to provide additional information and/or tasks to the user of the interactive assistant. The interactive assistant in the main device may receive an indication of a request from a user to perform a particular task. The interactive assistant, in addition to performing the particular task, may send requests to a respective assistant on one …

Assistant With Historical Location Based Triggers, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that performs actions based on an inferred user location, user direction of movement, and/or historical actions performed for previous locations or directions of movement. In some implementations, a user may explicitly command the assistant to perform a particular action when the user is moving relative to, or at, a particular location. In other implementations, the assistant may learn what actions the user performs or causes the assistant to perform when the user is moving relative to, or at, a particular location. In either case, …

Disambiguating Assistant Tasks Using Supplemental Data, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that relies on supplemental data (e.g., contextual information, user information, etc.) to decipher, and determine actions to perform based on, otherwise ambiguous user commands. With explicit permission from a user, the assistant may access a user’s location history, calendar, e-mail, messages, past assistant interactions, contacts, photos, search history, sensor data, and other contextual or user information to interpret user statements. The supplemental data can be stored locally on a device that is executing the assistant or in a cloud computing environment that is accessible …

Assistance For Calendar Milestone Planning, Google Inc.

Defensive Publications Series

A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described is configured to aid in user planning, celebrating, or otherwise remembering calendar milestones. The assistant determines steps that other users perform, including what sort of gifts to buy or events to plan, when celebrating or otherwise remembering a calendar milestone (e.g., an anniversary, a birthday, a holiday, or other important date). The assistant determines when an important date of a user will occur and automatically recommend ways for the user to plan for, celebrate, or otherwise remember the date. This way, rather than simply …

Device Settings Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant” may execute on counter-top devices, mobile phones, automobiles, and many other type of computing devices. The interactive assistant may be configured to guide the user to access features and settings of the computer device, and may also introduce features and settings of the computer device to the user.

Audible Indicator When A Third Party Enters An Assistant Conversation

Defensive Publications Series

In some cases, a user may primarily interacts with an interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” via voice communications. An interactive assistant may receive input that includes requests for the interactive assistant to perform actions, and the interactive assistant may provide feedback to the user via audible output. An interactive assistant that interacts with third-party services when performing the requested actions may output various audible indications to indicate the state of its interaction with the third-party services.

Compound Task Request And Translation, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to receive a compound task request. A compound task request is a single request to perform multiple (i.e., more than one) tasks. The compound task request may be in the form of a single spoken phrase. The interactive assistance may receive the spoken phrase, determine that the spoken phrase contains a request to perform multiple actions, and perform all of the actions requested by the spoken phrase. For example, the compound task request may be a request to translate a …

Using An Interactive Assistant As A Password Vault, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to securely store passwords, access codes, private keys, digital certificates, and other secret information. When the interactive assistant receives a request from the user to perform an action that requires the use of the secret information stored by the interactive assistant, the interactive assistant can provide such secret information to web sites, services, devices, systems, and the like to authenticate the interactive assistant to gain access to resources as part of performing the requested action. For example, when the interactive …

User Control Of Content Playback By An Interactive Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to dynamically adjust the playback of content (e.g., audible content output by the interactive assistant) in response to receiving a user command. Over the course of time, a user may engage in various different interactions with the assistant by submitting queries, performing searches, requesting select information, providing instructions, and the like. At any point in time, during playback of content provided by the interactive assistant, the user may issue a command to adjust the playback of such content. For example, …

Sleep Mode For An Interactive Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may, after previously receiving explicit user authorization to process user input, be configured to stop processing further user input (e.g., audible user input) for a period of time or until the occurrence of one or more detected events. Over the course of time, a user may engage in various different interactions with the assistant by submitting queries, performing searches, requesting select information, providing instructions, and the like. At any point in time, the user may request that the interactive assistant enter a sleep …

Allocation Of Advertisement Extensions & Formats, Patrick Hummel, Roberto J. Bayardo Jr, Shaohua Sun, Gagan Aggarwal, Kishor Barman, Kshipra Uday Bhawalkar, Jean-Francois Crespo, Ying Wang

Defensive Publications Series

This document describes a process of optimizing the selection of a budget-constrained advertiser's formats when allocating the budget of a budget-constrained advertiser. Such optimization is performed by selectively choosing which formats to allocate to this advertiser based on the advertiser's budget. In particular, this document describes allocating formats to budget-constrained advertisers in advertising auctions to create value for the advertisers and an advertiser manager.

Composite Digital Content Optimization, Wanghuan Chu, Shuchao Bi

Defensive Publications Series

A digital content rotation system that distributes digital content items a sufficient number of times to generate a statistical model for prediction of performance of the digital content items, while also preventing overall system degradation of content provided by content providers. The predicted performance of various digital content items is used as weights in a distribution rotation, thereby providing an optimal balance of exploitation and exploration of content performance.

Reservation Model For Real-Time Bidding Based Advertising System, Renuka Chadha, Bernhard Suter, Reto Strobl, Beverly Wong, Shashank Gupta, Sujata Kosalge, Radu Jurca, Ivan Romanov, Razvan Constantin

Defensive Publications Series

An advertisement management system can enable an advertiser to switch between realtime bidding and reservation delivery modes. A request is received to transition a campaign from a real-time bidding mode to a reservation mode. A reservation engine determines proposed terms for a reservation contract. An acceptance of the proposed terms is received from the advertiser. The campaign is transitioned from the real-time bidding mode to the reservation mode and a reservation contract is established between the advertisement management system and the advertiser, with campaign parameters made immutable for the duration of the contract.

Automatic Context-Based Customization Of Voicemail Response, Alexandru Damian, Victor Carbune

Defensive Publications Series

This disclosure describes a voicemail generator to provide automatic context-based custom voicemail responses, when users provide consent to access contextual data for generation of voicemail responses. The voicemail generator employs a prediction model that is configured to predict multiple factors, e.g., action type, user availability, and caller-specific customizations, that are used to automatically generate a customized voicemail response.

Creation Of Music Playlists Using An Interactive Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to create artist-specific or other criteria-based music playlists based on queries from a user. For example, an interactive assistant may accept queries from a user, perform a search of songs and related content based on certain criteria specified in the queries, and generate or modify music playlists based on the search results. The interactive assistant may also execute one or more applications, such as a music application, to play the songs included in the generated playlists.

Creation Of Theme-Based And/Or Genre-Based Music Playlists Using An Interactive Assistant, Google Inc.

Defensive Publications Series

An interactive assistant, referred to herein as “an interactive assistant,” “a virtual assistant,” or simply “an assistant,” may be configured to create comprehensive playlists based on singer/artist, composer, genre, theme, or other criteria, based on queries from a user. For example, an interactive assistant may accept queries from a user, perform searches for songs and related content based on certain theme-based and/or genre-based criteria specified in the queries (e.g., criteria for one or more love songs, happy songs, scary songs, sad songs, hard-rock songs, and the like), and generate or modify music playlists based on the search results. The interactive …

Using Object Detection To Extract Structured Content From Documents, Natalya Fridman Noy, Jeffrey Dudek

Defensive Publications Series

Structured content such as figures, tables, graphs, captions, and other graphical material often capture the essence of a document. Experienced readers often review the graphical material in a document first to quickly grasp the contents of the document. It is thus evident that identifying and extracting the structured content of a document, e.g., graphical components, is important in building a deeper semantic understanding of the document.

Techniques presented herein automatically extract the structured content of documents. Machine-learning techniques, e.g., object detection, computer vision, etc., are used to recognize and extract the structured content. The techniques work well regardless of the …