Projects such as apple glasses, smart glasses or augmented reality or virtual reality head display, which are often rumored, must rely on providing users with convincing visual effects and enough audio to achieve complete immersion. However, both elements present their own unique problems that companies that produce head mounted displays must overcome. For audio, the audio requirements of AR and smart glasses are quite different from those of VR. The former may hope that the audio is limited and can hear the sound of other environments, while the use of VR usually relies on shielding the external sound, leaving only the audio of the virtual world. Current head mounted displays can do this, but those with built-in audio systems usually do not provide the flexibility users want. < / P > < p > in a patent granted by the U.S. patent and Trademark Office on Tuesday called & quot; a display device with multimodal audio & quot;, apple describes a headwear vision system with audio components that can be switched between different positions on the head display’s supporting structure. Therefore, Apple’s ar glasses or VR head display may include a switchable audio system, which can provide a range of different auditory modes, and a system that can smoothly and rapidly fuse real-world scenes and virtual images. Apple has proposed that the audio system can be switched between the external mode and the in ear mode, the former is to place the loudspeaker of the movable head display system at a short distance from the ear, and the latter is to place the speaker in the ear canal. In the out of ear mode, the system will enable the external sound to be heard more clearly, while the second version will prevent the environment sound from being heard by users. < / P > < p > switching between modes, apple recommends using a driver in each head display to change the way it projects sound through different ports to match two different positions. This includes adjusting the power level of the bass unit depending on whether it is close to the ear, or even a switchable physical element that can change the sound movement. The sound generation module will use a short arm on a pivot, and the farthest point from the pivot contains elements for placing in the ear. Depending on how it pivots, the device either sends audio to the ear in one way or uses another to listen to external sounds and automatically changes it. In addition to the pivot, apple said the audio component may also be able to tilt and away from the ear for ease of placement and may include a telescopic part. < / P > < p > in the second patent, apple tries to solve how the AR head display presents virtual objects on the real-world video source, and combines the real-world camera data and digital production version for users to execute instantly. Because this requires accurate and fast execution, such a process may require a large amount of computing resources to complete to achieve high standards. In a patent called & quot; method and device for combining real and virtual images & quot;, apple proposes a method in which the system can use as few resources as possible to calculate the mixed virtual image. < / P > < p > for this system, it is mentioned in the patent that the injection of the camera can create a series of values for the real pixels, which exist in their virtual counterparts in the form of virtual pixels. Each has its own pixel value, which is calculated to determine what final pixel value should be used and displayed to the user. Two pixel values can be combined into a single value, which can be based on a weighted average between real and virtual pixel values. Apple estimates that the weighting can be determined in part on the alpha or transparency values of real or virtual pixels. To determine the alpha value of the real pixel, we can down sample the whole camera image, create a template according to the camera image, and generate an extinction parameter matrix according to the down sampling image and template. The extinction parameter image can be upsampled to create two pixel values, which can be combined into one value. The alpha value can also be used to create color component pixel values for a specific pixel, so that each color can be mixed instead of the entire pixel value itself. By using this technology, it is believed that the bandwidth required to transmit images to the display can be & quot; less than one tenth of the data generated by the camera alone. < / P > < p > Apple submits a large number of patent applications every week, but although the existence of patents indicates the areas of interest for the company’s R & D work, there is no guarantee that these concepts will appear in future products or services. [image] Google secretly tests 6GHz networks in 17 states of the United States