Key information

Title
Method and system for real time audio in multi-point video conferencing
Application number
20201393
Case type
National
Status
16.06.2025 Patent meddelt (B1)
Filed
18.12.2020
Effective date
18.12.2020
Publicly available
20.06.2022
Next annual fee due
31.12.2027
Applicant
PEXIP AS (NO)
Owner
PEXIP AS (NO)
Inventor
Tulio Beloqui (NO) et al.
Agent
ACAPO ONSAGERS AS (NO)
Granted
16.06.2025
Patent number
348822
Expiry date
18.12.2040

Abstract and drawing


Disclaimer: This text has been machine-scanned and may contain errors – please refer to "Publications" for legally binding content.
A real time audio multi-point videoconferencing system comprising at least two videoconferencing endpoints in communication with a multipoint control unit, and a method of providing the real time audio. The method comprising capturing input lossless digital audio streams on each of the at least two videoconferencing endpoints, transmitting the input lossless digital audio streams from each of the at least two videoconferencing endpoints to the MCU using a frame size in the range 0.25 ms – 3 ms, mixing, using the MCU, the input lossless digital audio streams into a plurality of output lossless digital audio streams, transmitting a different one of the plurality of output lossless digital audio streams to each of the at least two videoconferencing endpoints using a frame size in the range 0.25 ms – 3 ms, and playback the plurality of output lossless digital audio streams on each of the at least two videoconferencing endpoints.

Publications


B1A1
Checking availability of the latest published version...

Documents


Expand Download files (23)
Date
Date Doc. No. Process Case number In/Out Journal description To/from
20.06.2025 10-01 Saksbehandling 20201393(348822) UT PT Registreringsbrev nasjonal patent (15) (PT20201393)
22.04.2025 09-01 Saksbehandling 20201393 UT Intention to grant ACAPO ONSAGERS AS
28.10.2024 08-07 Saksbehandling 20201393 INN P27877NO00 Reply 28 Oct 2024 ACAPO ONSAGERS AS
28.10.2024 08-06 Saksbehandling 20201393 INN P27877NO00 Description Ready for Publication ACAPO ONSAGERS AS
28.10.2024 08-05 Saksbehandling 20201393 INN P27877NO00 Description amended markup ACAPO ONSAGERS AS
28.10.2024 08-04 Saksbehandling 20201393 INN P27877NO00 Claims in Norwegian ACAPO ONSAGERS AS
28.10.2024 08-03 Saksbehandling 20201393 INN P27877NO00 Amended claims markup ACAPO ONSAGERS AS
28.10.2024 08-02 Saksbehandling 20201393 INN P27877NO00 Amended claims Clean ACAPO ONSAGERS AS
28.10.2024 08-01 Saksbehandling 20201393 INN Korrespondanse (Hovedbrev inn) ACAPO ONSAGERS AS
20.08.2024 07-01 Saksbehandling 20201393 UT Substantive examination ACAPO ONSAGERS AS
03.11.2022 06-01 Saksbehandling 20201393 UT PT Varsel om betaling av første årsavgift (3317) (PT20201393)
22.12.2021 05-02 Saksbehandling 20201393 INN Letter to NIPO - in re of office action 20201393 ONSAGERS AS
22.12.2021 05-01 Saksbehandling 20201393 INN Korrespondanse (Hovedbrev inn) ONSAGERS AS
02.12.2021 04-02 Saksbehandling 20201393 INN Copies request Dec 02, 2021 P27877NO00 ONSAGERS AS
02.12.2021 04-01 Saksbehandling 20201393 INN Korrespondanse (Hovedbrev inn) ONSAGERS AS
13.07.2021 03-02 Saksbehandling 20201393 UT PT report 09:15:44
13.07.2021 03-01 Saksbehandling 20201393 UT Substantive examination ONSAGERS AS
22.12.2020 02-01 Saksbehandling 20201393 UT Infobrev til oppfinner ONSAGERS AS
18.12.2020 01-05 Saksbehandling 20201393 INN P27877NO00 English specification ONSAGERS AS
18.12.2020 01-04 Saksbehandling 20201393 INN P27877NO00 Drawings ONSAGERS AS
18.12.2020 01-03 Saksbehandling 20201393 INN P27877NO00 Declaring rights to invention form Dec 18, 2020 ONSAGERS AS
18.12.2020 01-02 Saksbehandling 20201393 INN NO General PoA, Pexip AS ONSAGERS AS
18.12.2020 01-01 Saksbehandling 20201393 INN Søknadsskjema Patent ONSAGERS AS

Description, claims and drawings


Disclaimer: This text has been machine-scanned and may contain errors – please refer to "Publications" for legally binding content. Description
Method and system for real time audio in multi-point video conferencing Technical fieldThe present invention relates to providing real time audio in multi -point videoconferencing.BackgroundTransmission of audio is an integral part of video conferencing, net meetings and videotelephony. Participants in video conferences, particularly when there are many participants in a conference, often experiences that the conversation does not flow as natural as in a physical meeting room and that they unwillingly interrupt each other by speaking simultaneously. The core of this problem is that sound uses more time to travel across the internet than most people would expect.In a normal conversation in a meeting room, those speaking would typically be less than 5 meters apart, which means that the sound delay, or latency, is around 15 milliseconds (ms) as sound travels roughly 1 meter in 3ms. This latency is perceived by the human brain will as instantaneous. In video conferencing it is common to experience latencies up to 500 ms. This is the equivalent of two people yelling at each other 160 meters apart. Even at best-case scenarios with available technologies today, the latency is still around 200ms, which is over 60 meters. While it is difficult to keep a natural conversation going with a latency of 200 ms or more, playing music together is near impossible.In order to be able to play together in sync, to feel a common rhythm or pulse, the latency needs to be very low. In terms of distance, most musicians would agree that being more than 10 meters apart makes it very hard to play together. This means that anything more than 30ms of latency is going to make it hard to play synchronized with a common pulse. Additionally, a comfortable experience for musicians requires clean, high-quality sound with no nose, clicks or pops, while the latency is low.In video conferencing system audio is captured at videoconferencing endpoints using a microphone and an analog-to-digital converter that converts the analog signal into a digital uncompressed audio signal, such as Pulse-code modulation (PCM). The digital uncompressed audio signal is split in frames of 20 ms and sent to an audio codec that compress the data to reduce transmission bandwidth. Video conferencing audio codecs performs lossy compression, that is, the audio codecs reduce the audio quality in order to maximize compression.Transmitting the digital uncompressed audio signal in frames of 20 ms introduces a delay, or latency, in the system of 20 ms. In addition, in order to make sure the codec has sufficient digital uncompressed audio frames to work on to avoid glitches in the sound, the codec keeps a three frame buffer. The three frame buffer adds a latency of 40 – 60 ms, Additionally, even the fastest codecs adds an algorithmic delay of at least 3 - 7 ms. At this stage, we already reached a total latency from one point to another that exceeds the required real time audio latency for playing music together.Fig. 1 schematically illustrates multi-point videoconferencing system 100 with three videoconferencing endpoints 101a, 101b, 101c in communication with a multipoint control unit (MCU) 104. In the multi-point videoconferencing system 100 all media passes through the MCU 104. Input audio 102a, 102b, 102c captured at the videoconferencing endpoints 101a, 101b, 101c is transmitted to the MCU 104, then the input audio 102a, 102b, 102c is mixed with audio from the other videoconferencing endpoints101a, 101b, 101c, and output audio 103a, 103b, 103c is transmitted back out to the videoconferencing endpoints 101a, 101b, 101c. Now, the end to end latency from one videoconferencing endpoint to another videoconferencing endpoint is approximately 200 ms.In addition, in order to avoid echo in the videoconference, the videoconferencing systems comprises an echo cancellation algorithm that adds from 60 ms – several hundred ms of latency depending on the acoustics at the site of a videoconference endpoint.Therefore, there is a need for a method and system that reduces audio latency without compromising audio quality. An example of a prior art attempt to improve user experience among the participants of the video conference is disclosed in patent document US 2012/236111.SummaryIn view of the above, an object of the present invention is to overcome or at least mitigate drawbacks of prior art video conferencing systems.In a first aspect the invention provides a method of providing real time audio in a multi-point videoconferencing system comprising at least two videoconferencing endpoints in communication with a multipoint control unit (MCU). The method comprising capturing one or more input lossless digital audio streams on each of the at least two videoconferencing endpoints; transmitting the one or more input lossless digital audio streams using Real-time Transport Protocol (RTP) from each of the at least two videoconferencing endpoints to the MCU using a frame size in the range 0.25 ms – 3 ms; mixing, using the MCU, the one or more input lossless digital audio streams from each of the at least two videoconferencing endpoints into a plurality of output lossless digital audio streams; transmitting a different one of the plurality of output lossless digital audio streams using Real-time Transport Protocol (RTP) to each of the at least two videoconferencing endpoints using a frame size in the range 0.25 ms – 3 ms; and playback the plurality of output lossless digital audio streams on each of the at least two videoconferencing endpoints.The method provides a very short audio latency from one videoconferencing endpoint to another videoconferencing endpoint.The method may further comprise transmitting the one or more input lossless digital audio streams and the plurality of output lossless digital audio streams further comprises using a single frame buffer.The method may further comprise further comprising using a three-frame buffer for receiving the one or more input lossless digital audio streams from each of the at least two videoconferencing endpoints at the MCU and for receiving one of the plurality of output lossless digital audio streams at each of the at least two videoconferencing endpoints.The method may further comprise transmitting the one or more input lossless digital audio streams from each of the at least two videoconferencing endpoints and the plurality of output lossless digital audio streams further comprises transmitting a bit rate of 200 kbps – 1 Mbps.The method may further comprise transmitting the one or more input lossless digital audio streams and the plurality of output lossless digital audio streams using Pulse -Code Modulation (PCM).The method may further comprise compressing the one or more input lossless digital audio streams and the plurality of output lossless digital audio streams prior to transmission. In one embodiment, compressing the one or more input lossless digital audio streams and the plurality of output lossless digital audio streams prior to transmission, may comprise applying Huffmann encoding on the plurality of input lossless digital audio streams and the plurality of output lossless digital audio streams.In one embodiment, the step of capturing the one or more input lossless digital audio streams on each of the at least two videoconferencing endpoints may further comprise bypassing any echo cancellation algorithm of the at least two videoconferencing endpoints.In a second aspect, the invention provides a real time audio multi-point videoconferencing system comprising at least two videoconferencing endpoints in communication with a multipoint control unit (MCU). Each of the at least two videoconferencing devices comprises an audio capturing unit adapted to capture and encode audio into one or more input lossless digital audio streams; a transmitting unit adapted to transmit the one or more input lossless digital audio streams using Real-time Transport Protocol (RTP) and a frame size in the range 0.25 ms – 3 ms; a receive unit adapted to receive an output lossless digital audio stream transmitted with a frame size in the range 0.25 ms – 3 ms; and a playback unit adapted to playback the received output lossless digital audio stream. The MCU comprises a receive unit adapted to receive one or more input lossless digital audio streams from each of the at least two video conferencing devices, transmitted with a frame size in the range 0.25 ms – 3 ms; a mixing unit adapted to mix the one or more input lossless digital audio streams from each of the at least two video conferencing devices into a plurality of output lossless digital audio streams; and a transmitting unit adapted to transmitting a different one of the plurality of output lossless digital audio streams to each of the at least two videoconferencing endpoints using Realtime Transport Protocol (RTP) and a frame size in the range 0.25 ms – 3 ms.The transmitting unit of each of the at least two videoconferencing endpoints and the transmitting unit of the MCU may be adapted to transmit the respective input and output lossless digital audio streams using a single frame buffer.The receive unit of each of the at least two videoconferencing endpoints and the transmitting unit of the MCU may be adapted to receive the respective output and input lossless digital audio streams using a three-frame buffer.The transmitting unit of each of the at least two videoconferencing endpoints and the transmitting unit of the MCU may be adapted to transmit the respective input and output lossless digital audio stream using a bit rate of 200 kbps – 1 Mbps.The transmitting unit of each of the at least two videoconferencing endpoints and the transmitting unit of the MCU may be adapted to transmit the respective input and output lossless digital audio stream using Pulse-Code Modulation (PCM).The audio capturing unit may be adapted to compress the input lossless digital audio stream. In one embodiment, the audio capturing unit may be adapted to compress the input lossless digital audio stream by applying Huffmann encoding.The audio capturing unit may be adapted to bypass any echo cancellation algorithm of the videoconferencing endpoint.Brief description of the drawingsFigure 1 is a schematic illustration of a multi-point videoconferencing system. Figure 2 is a flowchart illustrating an exemplary method of providing real time audio in a multi-point videoconferencing system. Figure 3 is a schematic illustration of an exemplary videoconferencing endpoint and a multi-point control unit.Detailed descriptionAccording to embodiments of the present invention as disclosed herein, the abovementioned disadvantages of solutions according to prior art are eliminated or at least mitigated.Fig. 2 is a schematic illustration of an exemplary method 200 of providing real time audio in a multi-point videoconferencing system 100, comprising at least two videoconferencing endpoints 101a, 101b, 101c in communication with a multipoint control unit (MCU) 104. An MCU in this context, may refer to all units, both in terms of a device and a software based virtual node, being both distributed or assembled, that are bridging and/or mixing media data from and to endpoints or terminals in a video conference. An endpoint in this context, may refer to both hardware and software video conferencing terminals, including (but not limited to) conventional video conferencing meeting room systems, desktop video conferencing terminals, video conferencing apps for PC and mobile phones, and browser-based video conferencing terminals. The first step 201 comprises capturing one or more input lossless digital audio streams 102a, 102b, 102c on each of the at least two videoconferencing endpoints 101a, 101b, 101c. The input lossless digital audio streams 102a, 102b, 102c may be raw audio from an audio capture device, e.g. a microphone. The input lossless digital audio streams 102a, 102b, 102c may be uncompressed audio, or the videoconferencing endpoints 101a, 101b, 101c may compress the one or more input lossless digital audio streams 102a, 102b, 102c prior to transmission using a lossless compression algorithm. In one exemplary embodiment the compression comprises applying Huffmann encoding on the input lossless digital audio streams 102a, 102b, 102c.The next step 202 comprises transmitting the one or more input lossless digital audio streams 102a, 102b, 102c from each of the at least two videoconferencing endpoints 101a, 101b, 101c to the MCU 104 using a frame size in the range 0.25 ms – 3 ms. In one embodiment, the step of transmitting the one or more input lossless digital audio streams 102a, 102b, 102c further comprises using a single frame buffer. In one exemplary embodiment, where the frame size is 1ms, this limits the latency on the transmit side to 1 ms.The next step 203 comprises mixing, using the MCU 104, the one or more input lossless digital audio streams 102a, 102b, 102c from each of the at least two videoconferencing endpoints 101a, 101b, 101c into a plurality of output lossless digital audio streams 103a, 103b, 103c. The MCU 104 mix the input lossless digital audio streams 102a, 102b, 102c using the frame size of lossless digital audio streams 102a, 102b, 102c. E.g. if the frame size is 1 ms, the MCU mix the plurality of output lossless digital audio streams 103a, 103b, 103c once every 1 ms. In one embodiment, the MCU 104 is using a three-frame buffer for receiving the one or more input lossless digital audio streams 102a, 102b, 102c from each of the at least two videoconferencing endpoints 101a, 101b, 101c. The tree-frame buffer is one embodiment a combined jitter/mixer buffer. When the frame size is 1 ms tree-frame buffer limits the input latency to 3 ms.The input lossless digital audio streams 102a, 102b, 102c are generally mixed such that a videoconferencing endpoint transmitting an input audio stream does not receive that audio in the output stream. In the exemplary multi -point videoconferencing system of Fig. 1, the MCU 104 would typically mix input lossless digital audio streams 102a, 102b in the output lossless digital audio streams 103c, input lossless digital audio streams 102a, 102c in the output lossless digital audio streams 103b and input lossless digital audio streams 102b, 102c in the output lossless digital audio streams 103a.The next step 204 comprises transmitting a different one of the plurality of output lossless digital audio streams 103a, 103b, 103c to each of the at least two videoconferencing endpoints 101a, 101b, 101c using a frame size in the range 0.25 ms – 3 ms. In one embodiment, the step of transmitting the plurality of output lossless digital audio streams 103a, 103b, 103c further comprises using a single frame buffer. In one exemplary embodiment, where the frame size is 1ms, this limits the latency on the transmit side to 1 ms.The output lossless digital audio streams 103a, 103b, 103c may be uncompressed audio, or the MCU 104 may compress the plurality of output lossless digital audio streams 103a, 103b, 103c prior to transmission using a lossless compression algorithm. In one exemplary embodiment the compression comprises applying Huffmann encoding on the output lossless digital audio streams 103a, 103b, 103c . The next step 205 comprises playback the plurality of output lossless digital audio streams 103a, 103b, 103c on each of the at least two videoconferencing endpoints 101a, 101b, 101c. Each of the at least two videoconferencing endpoints 101a, 101b, 101c playback one of the output lossless digital audio streams 103a, 103b, 103c using the frame size of the output lossless digital audio streams 103a, 103b, 103c. E.g. if the frame size is 1 ms, the videoconferencing endpoints 101a, 101b, 101c playback the output lossless digital audio streams 103a, 103b, 103c once every 1 ms. In one embodiment the videoconferencing endpoints 101a, 101b, 101c is using a three-frame buffer for receiving one of the plurality of output lossless digital audio streams 103a, 103b, 103c. When the frame size is 1 ms tree-frame buffer limits the input latency to 3 ms.The step 202 of transmitting the one or more input lossless digital audio streams 102a, 102b, 102c and the step 204 of transmitting plurality of output lossless digital audio streams 103a, 103b, 103c further comprises transmitting at a bit rate of 200 kbps – 1 Mbps.The step 202 of transmitting the one or more input lossless digital audio streams 102a, 102b, 102c and the step 204 of transmitting plurality of output lossless digital audio streams 103a, 103b, 103c may in one embodiment comprise transmitting using Pulse-Code Modulation (PCM).The step 202 of transmitting the one or more input lossless digital audio streams 102a, 102b, 102c and the step 204 of transmitting plurality of output lossless digital audio streams 103a, 103b, 103c may in one embodiment comprise transmitting using Real-time Transport Protocol (RTP).RFC 2586, RFC 4856 and RFC 3190 are exemplary standards describing transmitting PCM over RTP. L16 audio sampled at 48KHz transmitted over RTP requires less than 1Mbps bandwidth. The inventors of the present invention realized that modern networks easily manage such bandwidth requirements. This allows for bypassing the lossy compression codecs used in present multi-point videoconferencing systems. Not only does this allow for higher quality audio, it also allows for very low latency as disclosed in claimed method.In one embodiment, the MCU 104 mix the one or more input lossless digital audio streams using the sampling rate used for capturing the one or more input lossless digital audio streams, e.g. 48KHz, saving resampling time and further reducing latency.In order to avoid the use of echo cancellation algorithms that adds significant latency, musicians playing together in a multi-point videoconferencing system may chose to playback the output lossless digital audio streams 103a, 103b, 103c in headphones. However, due to the very low latency of present method the output lossless digital audio streams 103a, 103b, 103c is perceived as instantaneous with the input lossless digital audio streams 102a, 102b, 102c, and echo cancellation algorithms may be avoided altogether. The musicians may instead use speakers-/monitors for playback and the system may adjust gain and other equalizer settings to avoid “howling”. The inventors have realized that because of the very low latency of the present method, the need for echo cancellation is limited, and by bypassing any echo cancellation algorithms on the at least two videoconferencing endpoints when capturing the one or more input lossless digital audio streams, the latency becomes even lower, and echo cancellation becomes even less needed.The described method may require considerable processing resources at both the transmitting endpoint, the receiving endpoint and the MCU. It also may require more bandwidth than the receiver and transmitting side are able to provide. It may therefore be necessary to manually set the system in a low-latency mode.Fig. 3 is a schematic illustration of one of at least two videoconferencing endpoints 101a, 101b, 101c in communication with the MCU 104. Each of the videoconferencing endpoints 101a, 101b, 101c comprises an audio capturing unit 301 adapted to capture and encode audio into the one or more input lossless digital audio streams 102a, 102b, 102c, a transmitting unit 302 adapted to transmit the one or more input lossless digital audio streams 102a, 102b, 102c using a frame size in the range 0.25 ms – 3 ms, a receive unit 303 adapted to receive the output lossless digital audio stream 103a, 103b, 103c transmitted with a frame size in the range 0.25 ms – 3 ms, and a playback unit 304 adapted to playback the received output lossless digital audio stream 103a, 103b, 103c.The MCU 104 comprises a receive unit 305 adapted to receive the one or more input lossless digital audio streams 102a, 102b, 102c from each of the at least two video conferencing devices 101a, 101b, 101c, transmitted with a frame size in the range 0.25 ms – 3 ms, a mixing unit 306 adapted to mix the one or more input lossless digital audio streams 102a, 102b, 102c from each of the at least two video conferencing devices 101a, 101b, 101c into a plurality of output lossless digital audio streams 103a, 103b, 103c, and a transmitting unit 307 adapted to transmitting a different one of the plurality of output lossless digital audio streams 103a, 103b, 103c to each of the at least two videoconferencing endpoints using 101a, 101b, 101c a frame size in the range 0.25 ms – 3 ms.In one embodiment, the transmitting unit 302 of each of the at least two videoconferencing endpoints 101a, 101b, 101c and the transmitting unit 307 of the MCU 104 is adapted to transmit the respective input and output lossless digital audio streams using a single frame buffer.In one embodiment the receive unit 303 of each of the at least two videoconferencing endpoints 101a, 101b, 101c and the receiving unit 305 of the MCU is adapted to receive the respective output and input lossless digital audio stream using a three-frame buffer. In one embodiment the transmitting unit 302 of each of the at least two videoconferencing endpoints 101a, 101b, 101c and the transmitting unit 307 of the MCU is adapted to transmit the respective input and output lossless digital audio stream using a bit rate of 200 kbps – 1 Mbps.In one embodiment the transmitting unit 302 of each of the at least two videoconferencing endpoints 101a, 101b, 101c and the transmitting unit 307 of the MCU is adapted to transmit the respective input and output lossless digital audio stream using Pulse-Code Modulation (PCM).In one embodiment the transmitting unit 302 of each of the at least two videoconferencing endpoints 101a, 101b, 101c and the transmitting unit 307 of the MCU is adapted to transmit the respective input and output lossless digital audio stream using Real-time Transport Protocol (RTP).In one embodiment the audio capturing unit 301 is adapted to compress the input lossless digital audio stream.In one embodiment the audio capturing unit 301 is adapted to compress the input lossless digital audio stream by applying Huffmann encoding.In one embodiment the audio capturing unit 301 is adapted to bypass any echo cancellation algorithm of the at videoconferencing endpoint 101a, 101b, 101c. In the preceding description, various aspects of the method and imaging processing device according to the invention have been described with reference to the illustrative embodiment. For purposes of explanation, specific numbers, systems and configurations were set forth in order to provide a thorough understanding of the system and its workings. However, this description is not intended to be construed in a limiting sense. Various modifications and variations of the illustrative embodiment, as well as other embodiments of the method and image processing device, are possible as long as they fall within the scope of the appended claims.
Claims
CLAIMS1. A method of providing real time audio in a multi-point videoconferencing system (100) comprising at least two videoconferencing endpoints (101a, 101b, 101c) in communication with a multipoint control unit, MCU, (104), wherein the method comprising-capturing one or more input lossless digital audio streams (102a, 102b, 102c) by each of the at least two videoconferencing endpoints (101a, 101b, 101c);- transmitting the one or more input lossless digital audio streams (102a, 102b, 102c) using Real-time Transport Protocol, RTP, from each of the at least two videoconferencing endpoints (101a, 101b, 101c) to the MCU (104) using a frame size in the range 0.25 ms – 3 ms;-mixing, by the MCU (104), the one or more input lossless digital audio streams (102a, 102b, 102c) from each of the at least two videoconferencing endpoints (101a, 101b, 101c) into a plurality of output lossless digital audio streams (103a, 103b, 103c);- transmitting, by the MCU (104), a different one of the plurality of output lossless digital audio streams (103a, 103b, 103c) using Real-time Transport Protocol, RTP, to each of the at least two videoconferencing endpoints (101a, 101b, 101c) using a frame size in the range 0.25 ms – 3 ms; and- playback the plurality of output lossless digital audio streams (103a, 103b, 103c) by each of the at least two videoconferencing endpoints (101a, 101b, 101c).2. The method of claim 1, wherein the step of transmitting the one or more input lossless digital audio streams (102a, 102b, 102c) and the plurality of output lossless digital audio streams (103a, 103b, 103c) further comprises using a single frame buffer.3. The method of claim 1 – 2, further comprising using a three-frame buffer for receiving the one or more input lossless digital audio streams (102a, 102b, 102c) from each of the at least two videoconferencing endpoints (101a, 101b, 101c) at the MCU (104) and for receiving one of the plurality of output lossless digital audio streams (103a, 103b, 103c) at each of the at least two videoconferencing endpoints (101a, 101b, 101c).4. The method of any of the preceding claims, wherein the step of transmitting the one or more input lossless digital audio streams (102a, 102b, 102c) from each of the at least two videoconferencing endpoints (101a, 101b, 101c) and the plurality of output lossless digital audio streams (103a, 103b, 103c) further comprises transmitting a bit rate of 200 kbps – 1 Mbps.5. The method of any of the preceding claims, further comprising transmitting the one or more input lossless digital audio streams (102a, 102b, 102c) and the plurality of output lossless digital audio streams (103a, 103b, 103c) using Pulse-Code Modulation, PCM.6. The method of any of the preceding claims, further comprising compressing the one or more input lossless digital audio streams (102a, 102b, 102c) and the plurality of output lossless digital audio streams (103a, 103b, 103c) prior to transmission.7. The method according to claim 6, wherein the step of compressing the one or more input lossless digital audio streams (102a, 102b, 102c) and the plurality of output lossless digital audio streams (103a, 103b, 103c) prior to transmission, comprises applying Huffmann encoding on the plurality of input lossless digital audio streams (102a, 102b, 102c) and the plurality of output lossless digital audio streams (103a, 103b, 103c).8. The method according to any the preceding claims, wherein the step of capturing the one or more input lossless digital audio streams (102a, 102b, 102c) on each of the at least two videoconferencing endpoints (101a, 101b, 101c) further comprises bypassing any echo cancellation algorithm of the at least two videoconferencing endpoints (101a, 101b, 101c).9. A real time audio multi-point videoconferencing system (100) comprising at least two videoconferencing endpoints (101a, 101b, 101c) in communication with a multipoint control unit, MCU, (104), whereineach of the at least two videoconferencing devices (101a, 101b, 101c) comprises -an audio capturing unit (301) adapted to capture and encode audio into one or more input lossless digital audio streams (102a, 102b, 102c);-a transmitting unit (302) adapted to transmit the one or more input lossless digital audio streams (102a, 102b, 102c) using Real-time Transport Protocol (RTP) and a frame size in the range 0.25 ms – 3 ms;- a receive unit (303) adapted to receive an output lossless digital audio stream (103a, 103b, 103c) transmitted with a frame size in the range 0.25 ms – 3 ms; and - a playback unit (304) adapted to playback the received output lossless digital audio stream (103a, 103b, 103c); andthe MCU (104) comprises- a receive unit (305) adapted to receive one or more input lossless digital audio streams (102a, 102b, 102c) from each of the at least two video conferencing devices (101a, 101b, 101c), transmitted with a frame size in the range 0.25 ms – 3 ms;- a mixing unit (306) adapted to mix the one or more input lossless digital audio streams (102a, 102b, 102c) from each of the at least two video conferencing devices (101a, 101b, 101c) into a plurality of output lossless digital audio streams (103a, 103b, 103c); and- a transmitting unit (307) adapted to transmitting a different one of the plurality of output lossless digital audio streams (103a, 103b, 103c) to each of the at least two videoconferencing endpoints (101a, 101b, 101c) using Real-time Transport Protocol (RTP) and a frame size in the range 0.25 ms – 3 ms.10. The system according to claim 9, wherein the transmitting unit (302) of each of the at least two videoconferencing endpoints (101a, 101b, 101c) and the transmitting unit (307) of the MCU (104) is adapted to transmit the respective input (102a, 102b, 102c) and output (103a, 103b, 103c) lossless digital audio streams using a single frame buffer.11. The system according to claim 9 or 10, wherein the receive unit (303) of each of the at least two videoconferencing endpoints (101a, 101b, 101c) and the receive unit (305) of the MCU (104) is adapted to receive the respective output (103a, 103b, 103c) and input (102a, 102b, 102c) lossless digital audio streams using a three-frame buffer.12. The system according to any of claim 9 – 11, wherein the transmitting unit (302) of each of the at least two videoconferencing endpoints (101a, 101b, 101c) and the transmitting unit (307) of the MCU (104) is adapted to transmit the respective input (102a, 102b, 102c) and output (103a, 103b, 103c) lossless digital audio stream using a bit rate of 200 kbps – 1 Mbps.13. The system according to any of claim 9 – 12, wherein the transmitting unit (302) of each of the at least two videoconferencing endpoints (101a, 101b, 101c) and the transmitting unit (307) of the MCU (104) is adapted to transmit the respective input (102a, 102b, 102c) and output (103a, 103b, 103c) lossless digital audio stream using Pulse-Code Modulation, ,PCM).14. The system according to any of claim 9 – 13, wherein the audio capturing unit (301) is adapted to compress the input lossless digital audio stream (102a, 102b, 102c).15. The system according to claim 14, wherein the audio capturing unit (301) is adapted to compress the input lossless digital audio stream (102a, 102b, 102c) by applying Huffmann encoding.16. The system according to any of any of claim 9 – 15, wherein the audio capturing unit (301) is adapted to bypass any echo cancellation algorithm of the videoconferencing endpoint (101a, 101b, 101c).PATENTKRAV1. Fremgangsmåte for å tilveiebringe sanntidsaudio i et multipunktvideokonferansesystem (100) som omfatter minst to videokonferanse-endepunkter (101a, 101b, 101c) i kommunikasjon med en multipunkt-styringsenhet, MCU, (104), der fremgangsmåten omfatter- fange én eller flere input-tapsfrie digitale audiostrømmer (102a, 102b, 102c) med hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c),- sende den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) å benytte sanntid-transportprotokoll, RTP, fra hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) til MCU’en (104) ved å benytte en rammestørrelse i området av 0,25 ms – 3 ms,- mikse, med MCU’en (104), den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) fra hvert av de minst to videokonferanseendepunktene (101a, 101b, 101c) til et flertall av output-tapsfrie digitale audiostrømmer (103a, 103b, 103c),- sende, med MCU’en (104), en forskjellig én i flertallet av output -tapsfrie digitale audiostrømmene (103a, 103b, 103c) å benytte sanntid-transportprotokoll, RTP, til hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) ved å benytte en rammestørrelse i området av 0,25 ms – 3 ms, og- spille tilbake flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) med hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c).2. Fremgangsmåte ifølge krav 1,der trinnet med å sende den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) videre omfatter å benytte en enkelt rammebuffer.3. Fremgangsmåte ifølge krav 1-2,videre omfattende å benytte en trerammebuffer for å motta den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) fra hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) på MCU’en (104) og for å motta én i flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) på hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c).4. Fremgangsmåte ifølge ethvert av de foregående krav,der trinnet med å sende den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) fra hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) videre omfatter å sende en bit-rate på 200 kbps - 1 Mbps.5. Fremgangsmåte ifølge ethvert av de foregående krav,videre omfattende å sende den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) ved å benytte pulskodemodulering, PCM.6. Fremgangsmåte ifølge ethvert av de foregående krav,videre omfattende å komprimere den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) før sending.7. Fremgangsmåte ifølge krav 6,der trinnet med å komprimere den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) før sending omfatter å benytte Huffman-koding på flertallet av input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) og flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c).8. Fremgangsmåte ifølge ethvert av de foregående krav,der trinnet med å fange den ene eller de flere input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) på hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) videre omfatter å forbigå enhver ekkokanselleringsalgoritme for de minst to videokonferanse-endepunktene (101a, 101b, 101c).9. Sanntidsaudio-multipunktvideokonferansesystem (100) som omfatter minst to videokonferanse-endepunkter (101a, 101b, 101c) i kommunikasjon med en multipunkt-styringsenhet, MCU, (104), derhver av de minst to videokonferanseinnretningene (101a, 101b, 101c) omfatter- en audiofangende enhet (301) som er tilpasset å fange og kode audio inn i én eller flere input-tapsfrie digitale audiostrømmer (102a, 102b, 102c),en sende-enhet (302) som er tilpasset å sende den ene eller de flere inputtapsfrie digitale audiostrømmene (102a, 102b, 102c) ved å benytte sanntidtransportprotokoll, RTP, og å benytte en rammestørrelse i området av 0,25 ms - 3 ms,en mottaksenhet (303) som er tilpasset å motta en output -tap sfri digital audiostrøm (103a, 103b, 103c) som er sendt med en rammestørrelse i området av 0,25 ms - 3 ms, ogen tilbakespillenhet (304) som er tilpasset å spille tilbake den mottatt output -tap sfri e digitale audio strømmen, ogMCU’ en (104) omfatteren mottaksenhet (305) som er tilpasset å motta én eller flere input-tapsfrie digitale audiostrømmer (102a, 102b, 102c) fra hver av de minst to videokonferanseinnretningene (101a, 101b, 101c), sendt med en rammestørrelse i området av 0,25 ms - 3 ms,en mikse-enhet (306) som er tilpasset å mikse den ene eller de input-tapsfrie digitale audiostrømmene (102a, 102b, 102c) fra hver av de minst to videokonferanseinnretningene (101a, 101b, 101c) til et flertall av output -tap sfri e digitale audiostrømmer (103a, 103b, 103c), og- en sende-enhet (307) som er tilpasset å sende en forskjellig én i flertallet av output-tapsfrie digitale audiostrømmene (103a, 103b, 103c) til hvert av de minst to videokonferanse-endepunktene (101a, 101b, 101c) ved å benytte sanntid transportprotokoll, RTP, og å benytte en rammestørrelse i området av 0,25 ms - 3 ms.10. System ifølge krav 9,der sende-enheten (302) for hvert av de minst to videokonferanseinnretningene (101a, 101b, 101c) og sende-enheten (307) for MCU’en (104) er tilpasset å sende de respektive input- (102a, 102b, 102c) og output- (103a, 103b, 103c) tapsfrie digitale audiostrømmene ved å benytte en enkelt rammebuffer.11. System ifølge krav 9 eller 10,der mottaks-enheten (303) for hvert av de minst to videokonferanseendepunktene (101a, 101b, 101c) og mottaksenheten (305) til MCU’en (104) er tilpasset å motta de respektive input- (102a, 102b, 102c) og output- (103a, 103b, 103c) tapsfrie digitale audiostrømmene ved å benytte en trerammebuffer.12. System ifølge ethvert av kravene 9-11,der sende-enheten (302) til hvert av de minst to videokonferanseendepunktene (101a, 101b, 101c) og sende-enheten (307) til MCU’en (104) er tilpasset å sende den respektive input- (102a, 102b, 102c) og output- (103a, 103b, 103c) tapsfrie digitale audiostrømmene ved å benytte en bit-rate på 200 kbps - 1 Mbps.13. System ifølge ethvert av kravene 9-12,der sende-enheten (302) til hvert av de minst to videokonferanseendepunktene (101a, 101b, 101c) og sende-enheten (307) til MCU’en (104) er tilpasset å sende den respektive input- (102a, 102b, 102c) og output- (103a, 103b, 103c) tapsfrie digitale audiostrømmene ved å benytte pulskodemodulering, PCM.14. System ifølge ethvert av kravene 9-13,der audiofangeenheten (301) er tilpasset å komprimere den input-tapsfrie digitale audiostrømmen (102a, 102b, 102c).15. System ifølge krav 14,der den audiofangende enheten (301) er tilpasset å komprimere den input -tapsfrie digitale audiostrømmen (102a, 102b, 102c) ved å benytte Huffman -koding.16. System ifølge ethvert av kravene 9-15,der den audiofangende enheten (301) er tilpasset å forbigå enhver ekkokanselleringsalgoritme for videokonferanse -endepunktet (101a, 101b, 101c).
Drawings

Priority


No priority

Patent classes


IPC classesH04N 7/15H04N 21/6437H04L 29/06G10L 19/00H04M 3/56CPC classesH04N 7/15H04N 21/6437H04L 29/06176G10L 19/0017H04L 65/00H04M 3/567

Citations


WO 0215556 A2 (A2)WARDROBE K, Complete Guide to Zoom Audio Settings For Music Teachers, dated September 2020, downloaded 11.07.2021 from https://midnightmusic.com.au/2020/09/complete-guide-to-zoom-audio-settings-for-music-teachers/ ()ZHU Q. ET AL, Low-delay, low-complexity rate reduction and continuous presence for multipoint videoconferencing, IEEE Transactions on Circuits and Systems for Video Technology, volume 9, issue 4, June 1999, pages 666-676, DOI: 10.1109/76.767130 ()WO 2019038019 A1 (A1)WO 2008080426 A1 (A1)WO 2007111842 A2 (A2)WO 2020231577 A1 (A1)

Applicants, owners and inventors


Owners
PEXIP AS
Lilleakerveien 2 A 0283 OSLO NO (OSLO Municipality, Oslo county)Org. number: 819850232
Inventors
Tulio Beloqui
Konows gate 35 0196 OSLO NO (OSLO Municipality, Oslo county)
Håvard Graff
Finnhaugveien 16 0760 OSLO NO (OSLO Municipality, Oslo county)

Agent


ACAPO ONSAGERS AS
ACAPO ONSAGERS AS

Org. number: 932214636

Name and address are automatically synchronized with the Brønnøysund Register Centre.

Open in The Brønnøysund Register Centre
Edvard Griegs vei 1 5059 BERGEN NO (BERGEN Municipality, Vestland county)
Org. number: 932214636
Reference: P27877NO00

Status history


Payment history


Oppositions and appeals


No oppositions or appeals

Administrative limitation


No administrative limitation

Liens and licences


No liens or licenses registered

Gazette entries


No Gazette entries foundOnly events from 2024.03.04 and onwards are shown.