多媒體通信資料

1、多媒體通信清華大學(xué)電子工程系 崔慧娟 汪志兵 多媒體通信終端技術(shù)基本概念歷史發(fā)展系統(tǒng)介紹關(guān)鍵技術(shù)系統(tǒng)應(yīng)用未來發(fā)展基本概念:多媒體：多種混合信息（文本、數(shù)據(jù)、圖像、語音、視頻、音頻等數(shù)字信號）的表示形式理想的多媒體通信：實(shí)現(xiàn)任何人在任何時(shí)間、任何地點(diǎn)，通過通信網(wǎng)獲得任何需要的多媒體信息多媒體通信：通過網(wǎng)絡(luò)通信信道操作、傳輸和控制這些視聽多媒體信號的技術(shù)。 多媒體通信基本概念感覺媒體 (Preception Medium)表示媒體 (Representation Medium)顯示媒體 (Presentation Medium)存儲(chǔ)媒體 (Storage Medium)傳輸媒體 (Transmis

2、sion Medium) 媒體分類(ITU-T I.374)基本概念 多媒體通信中的“媒體”多媒體通信中的媒體特指表示媒體,也就是多媒體通信系統(tǒng)中要有存儲(chǔ)、傳輸、處理、顯現(xiàn)多種表示媒體信息（即多種編碼的信息）的功能基本概念集成性交互性同步性互通性 多媒體通信系統(tǒng)的基本特征基本概念集成性交互性同步性互通性 多媒體通信系統(tǒng)的基本特征多媒體通信能處理、存儲(chǔ)、傳輸和顯示多媒體信息。因此，它是集多種編碼器、多種顯示方式于一體，能與多種傳輸媒體進(jìn)行接口，與多種傳輸媒體進(jìn)行通信的系統(tǒng)。基本概念集成性交互性同步性互通性 多媒體通信系統(tǒng)的基本特征是指終端與系統(tǒng)的交互通信能力。多媒體通信終端的用戶對通信的全過程有

3、完備的交互控制能力。它是通過人機(jī)接口協(xié)議和應(yīng)用層協(xié)議來實(shí)現(xiàn)的?；靖拍罴苫换バ酝叫曰ネㄐ?多媒體通信系統(tǒng)的基本特征多媒體通信終端上顯現(xiàn)的圖像、聲音和文字是以同步方式工作的。這可以在個(gè)層面上實(shí)現(xiàn)，這個(gè)層面是：鏈路層同步；表示層同步；應(yīng)用層同步即，盡管圖像、聲音和文字等信息來自不同的信息源，并通過不同的傳輸途徑，但終端用戶得到的是一個(gè)完全同步了的多媒體信息。這種同步性，是區(qū)別多媒體通信和多種媒體通信的一個(gè)重要標(biāo)志?；靖拍罴苫换バ酝叫曰ネㄐ?多媒體通信系統(tǒng)的基本特征工作在不同網(wǎng)絡(luò)環(huán)境下的多媒體通信終端通過網(wǎng)關(guān)實(shí)現(xiàn)互操作基本概念會(huì)議型業(yè)務(wù) (MM Conference Service)會(huì)

4、話型業(yè)務(wù) (MM Conversation Service)回放型業(yè)務(wù) (MM Retrieval Service)消息型業(yè)務(wù) (MM Message Service)分配型業(yè)務(wù) (MM Distribution Service)采集型業(yè)務(wù) (MM Collection Service) 業(yè)務(wù)類型(ITU-T F.700)歷史發(fā)展年代初，美國、日本和歐洲,首先建立了基于局域網(wǎng)的多媒體通信系統(tǒng)，如美國Xerox公司的以太電話（Ehterphone）年月，RCA公司推出了交互式數(shù)字視頻系統(tǒng)(DVI) ，用標(biāo)準(zhǔn)光盤存儲(chǔ)和檢索靜止圖像、動(dòng)態(tài)圖像、聲音和其它數(shù)據(jù) 多媒體通信系統(tǒng)的雛形歷史發(fā)展美國的AT&

5、T、PacificBell、Chico和加利福尼亞洲公立大學(xué)（）于1991年5月開始利用ISDN Bri進(jìn)行了多媒體遠(yuǎn)程學(xué)習(xí)網(wǎng)絡(luò)試驗(yàn)德國電信公司、德國科學(xué)研究中心于1994年10月共同開發(fā)公用多媒體業(yè)務(wù)，主要面向中小型商業(yè)機(jī)構(gòu)，利用ISDN路由器把各地的以太網(wǎng)和FDDI（光纖分布式數(shù)據(jù)接口）鏈接起來，支持遠(yuǎn)程辦公、遠(yuǎn)程醫(yī)療1994年7月日本開始進(jìn)行了國家級的試驗(yàn)項(xiàng)目B-ISDN，對點(diǎn)播電視、電子游戲傳送點(diǎn)播業(yè)務(wù)、家庭購物、遠(yuǎn)程教育、遠(yuǎn)程醫(yī)療、電子圖書館、多媒體會(huì)議電視、市民畫廊等多項(xiàng)多媒體通信的應(yīng)用逐項(xiàng)展開試驗(yàn) 多媒體通信早期發(fā)展歷史發(fā)展80年代日本TTC標(biāo)準(zhǔn)的靜態(tài)可視電話、1992年美國AT

6、&T公司的2500型可視電話產(chǎn)品均以不實(shí)用而告敗90年代以來ITU專家組制定了一系列的ITU-T 多媒體通信終端及系統(tǒng)的標(biāo)準(zhǔn)美國、英國、日本、新加坡、法國的相關(guān)廠商紛紛于1997年先后開發(fā)出該類最新產(chǎn)品國內(nèi)可視電話技術(shù)發(fā)展動(dòng)態(tài)基本上逼近國外的發(fā)展趨勢基于IP的可視電話,會(huì)議電視異軍突起 多媒體通信的發(fā)展歷史發(fā)展ITU-T是國際電信聯(lián)盟標(biāo)準(zhǔn)化組織。主要職能是完成國際電信聯(lián)盟有關(guān)電信標(biāo)準(zhǔn)化的目標(biāo)，使全世界的電信標(biāo)準(zhǔn)化IETF：國際工程師任務(wù)組，是一個(gè)專門負(fù)責(zé)研究因特網(wǎng)標(biāo)準(zhǔn)和協(xié)議并使之標(biāo)準(zhǔn)化的組織，由許多工作組組成，受IESG、IAB和ISOC管理。IMTC：國際多媒體電信會(huì)議聯(lián)合體，由全球150多

7、個(gè)多媒體設(shè)備生產(chǎn)廠商組成的非盈利性組織，該組織的宗旨是為多媒體會(huì)議編制接口、主辦互操作性測試以及把現(xiàn)有標(biāo)準(zhǔn)擴(kuò)展到新的網(wǎng)絡(luò)中,推動(dòng)不同廠商間產(chǎn)品的兼容性。 多媒體通信的標(biāo)準(zhǔn)化系統(tǒng)介紹ITU-T H.32X: 多媒體通信系統(tǒng)協(xié)議MPEGX 存儲(chǔ)轉(zhuǎn)發(fā)式多媒體系統(tǒng)通信協(xié)議系統(tǒng)介紹 ITU-T H.32XH.320 N-ISDN上多媒體通信系統(tǒng)協(xié)議H.324 PSTN上的可視電話系統(tǒng)協(xié)議H.322/H.323 LAN上的會(huì)議電視系統(tǒng)協(xié)議H.321 B-ISDN上多媒體通信系統(tǒng)協(xié)議H.310 ATM上多媒體通信系統(tǒng)協(xié)議( T.120 多方數(shù)據(jù)會(huì)議協(xié)議系列 )ITU 多媒體通信終端終端 H.320H.321

8、H.322H.323H.324通信網(wǎng)絡(luò)窄帶綜合業(yè)務(wù)數(shù)字網(wǎng)N-ISDN寬帶綜合業(yè)務(wù)數(shù)字網(wǎng)B-ISDN局域網(wǎng)(QoS)局域網(wǎng)(non-QoS)電話網(wǎng)GSTN批準(zhǔn)時(shí)間19901995199519961996視頻編碼H.261H.261H.261H.263 H.261H.261 H.263音頻編碼G.711 G.722G.728G.711G.722G.728G.711G.722G.728G.723.1G.723.1數(shù)據(jù)復(fù)接H.221H.221H.221H.225.0H.223系統(tǒng)控制H.230H.242H.242H.242H.230H.245H.225.0H.245多點(diǎn)會(huì)議H.231H.243H.23

9、1H.243H.231H.243T.120SeriesT.120Series傳輸接口I.400I.400I.400 TCP/IPTCP/IPV.34/ V.80 ITUT 多媒體通信終端系統(tǒng)介紹 ITU-T H.32XITU-T H.320概況H.320：窄帶ISDN可視電話系統(tǒng)和終端設(shè)備(多媒體會(huì)議電視)Narrow-Band ISDN Visual Telephone Systems and Terminal EquipmentITU-T H.320概況基于H.320標(biāo)準(zhǔn)的可視電話和會(huì)議電視系統(tǒng)運(yùn)行在窄帶綜合業(yè)務(wù)數(shù)字網(wǎng)(ISDN)上，提供語音，圖象和其他數(shù)據(jù)的實(shí)時(shí)通信。它是ITU最早制定的

10、多媒體通信標(biāo)準(zhǔn)。系統(tǒng)介紹 ITU-T H.320ITU-T H.320系統(tǒng)框圖 H.320 Scope系統(tǒng)介紹 ITU-T H.320H.324：低比特率多媒體通信終端Terminal for Low Bitrate Multimedia CommunicationITU-T H.324概況ITU-T H.324是運(yùn)行在普通電話網(wǎng)(GSTN)上的可視電話，提供語音，圖象和其他數(shù)據(jù)的實(shí)時(shí)通信。由于電話網(wǎng)帶寬的限制，決定了語音和圖象都必須采用高壓縮比的算法，以低數(shù)碼率傳輸。這樣語音雖然仍然能夠和普通電話的質(zhì)量相當(dāng)，但是圖象卻一定不會(huì)太好。 ITU-T H.324概況系統(tǒng)介紹 ITU-T H.324

11、 ITU-T H.324系統(tǒng)框圖 H.324 Scope系統(tǒng)介紹 ITU-T H.324H.323建議描敘了一個(gè)基于包交換網(wǎng)絡(luò)的多媒體通訊系統(tǒng)。這樣的網(wǎng)絡(luò)有局域網(wǎng)（LAN），城域網(wǎng)（MAN），廣域網(wǎng)（WAN），企業(yè)網(wǎng)（EAN）和英特網(wǎng)（Internet）等等。H.323系統(tǒng)組成有終端（Terminal），網(wǎng)關(guān)（Gateway），關(guān)守（Gatekeeper），多點(diǎn)控制器（MC），多點(diǎn)處理器（MP）和多點(diǎn)控制單元（MCU）。H.323終端提供語音通訊能力，活動(dòng)圖像和數(shù)據(jù)通訊為可選項(xiàng)。通訊方式包括：點(diǎn)對點(diǎn)會(huì)議和點(diǎn)對多點(diǎn)的會(huì)議。通過H.323網(wǎng)關(guān)，可以和其它H.32X系列的終端通訊，如基于GSTN的H

12、.324終端，基于ISDN的H.320終端等等。關(guān)守提供進(jìn)入許可，地址解析等服務(wù)。多點(diǎn)控制器，多點(diǎn)處理器和多點(diǎn)控制單元支持多點(diǎn)會(huì)議。ITU-T H.323概況系統(tǒng)介紹 ITU-T H.323ITU-T H.323系統(tǒng)框圖 H.323 Scope系統(tǒng)介紹 ITU-T H.323What H.323 DOES do提供了連接建立協(xié)議提供了互操作性框架利用了現(xiàn)存的相關(guān)協(xié)議的優(yōu)勢What H.323 does NOT do管理地址映射限制媒體數(shù)據(jù)格式和內(nèi)容限制與會(huì)者的能力集合 H.323 Overview系統(tǒng)介紹 ITU-T H.323終端 - Terminal網(wǎng)關(guān)(GW) - gateway網(wǎng)守(G

13、K) - gatekeeper多點(diǎn)控制器(MC) - multipoint controller多點(diǎn)處理器(MP) - multipoint processor多點(diǎn)控制單元(MCU) - multipoint control unitH.323系統(tǒng)組件系統(tǒng)介紹 ITU-T H.323 H.323通信系統(tǒng)電路交換網(wǎng)(POTs and ISDN)GatekeeperH.323H.320(over ISDN)H.324(over POTs)普通電話(telephones)局域網(wǎng)GatewayH.323因特網(wǎng)系統(tǒng)介紹 ITU-T H.32X系統(tǒng)介紹 ITU-T H.323不同網(wǎng)絡(luò)之間的“橋接”eg.L

14、AN to ATM, LAN to GSTN, LAN to ISDN)協(xié)議轉(zhuǎn)換H.245 to H.242/H.230H.225.0(RTP) to H.221/H.223媒體信息流的編碼轉(zhuǎn)換與速率匹配呼叫控制轉(zhuǎn)換可以包含 MCU/GK 網(wǎng)關(guān)(Gateway)系統(tǒng)介紹 ITU-T H.323一個(gè)可選的設(shè)備必要特征地址翻譯 (alias/E.164 to transport within zone)身份認(rèn)證 (ARQ/ACF/ARJ)帶寬控制 (BRQ/BCF/BRJ)可選特征呼叫控制和轉(zhuǎn)移 (under GK control)呼叫管理 (call status, tracking etc.)

15、進(jìn)一步研究 網(wǎng)守(Gatekeeper)系統(tǒng)介紹 ITU-T H.323物理上可位于其它終端設(shè)備中 (endpoint, GK, MCU, or GW )基于 H.245 的會(huì)議管理建立一般的會(huì)議模式發(fā)布多播地址兩種媒體通信方式集中式 (centralizedAudio, centralizedVideo)非集中式 (decentralizedAudio, decentralizedVideo 多點(diǎn)控制器(MC)系統(tǒng)介紹 ITU-T H.323系統(tǒng)介紹 ITU-T H.323H.323相關(guān)協(xié)議集系統(tǒng)介紹 ITU-T H.323 H.323協(xié)議棧系統(tǒng)介紹 ITU-T H.323實(shí)時(shí)傳輸協(xié)議（RT

16、P）包的結(jié)構(gòu) 0 12 34 56 78 910 1112 1314 1516 1718 1920 2122 2324 2526 2728 2930 31V P XCC M (1bit) PT(7bit)sequence number: 16 bitTimestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers。 Payload 系統(tǒng)介紹 ITU-T H.323 UDP 報(bào)文格式UDP源端口UDP信宿端口UDP長度UDP校驗(yàn)和數(shù)據(jù)區(qū)由頭標(biāo)和數(shù)據(jù)區(qū)組成系統(tǒng)介紹 ITU-T H.32

17、3UDP 報(bào)文封裝UDP頭標(biāo) UDP數(shù)據(jù)區(qū) IP頭標(biāo) IP數(shù)據(jù)區(qū) 系統(tǒng)介紹 ITU-T H.323TCP段格式 序號Sequence Number：段中數(shù)據(jù)在發(fā)送數(shù)據(jù)流中的位置確認(rèn)號Acknowledge Number：本機(jī)希望下一個(gè)接收字節(jié)的序號碼位Code Bits：指出段的目的與內(nèi)容，各位意義如下： URG：緊急指針有效 ACK：確認(rèn)域有效 PSH：push操作 RST：連接復(fù)償位 SYN：同步序號 FIN： 發(fā)送方已到達(dá)字節(jié)末尾窗口Window：通告接收端接收緩沖區(qū)的大小0 1631源端口信宿端口序號 確認(rèn)號 頭標(biāo)長度保留碼位窗口校驗(yàn)和緊急指針選項(xiàng)填充字節(jié)數(shù)據(jù)系統(tǒng)介紹 ITU-T H.

18、323IP數(shù)據(jù)報(bào)格式 IP協(xié)議：無連接數(shù)據(jù)報(bào)傳送由報(bào)頭和數(shù)據(jù)區(qū)組成0 16 31版本頭標(biāo)長服務(wù)類型總長標(biāo)識標(biāo)志片偏移生存時(shí)間協(xié)議頭標(biāo)校驗(yàn)和源 IP 地址信宿 IP 地址選項(xiàng)填充域數(shù)據(jù)區(qū)系統(tǒng)介紹 ITU-T H.323 典型的H.323通信過程Pre-call setup operationsRAS signaling to GK for Registration/admissionPhase A: Connection (Q.931 signaling)Phase B: Capability Exchage (H.245)Phase C: Establishment of AV communi

19、cations (H.245)Phase D: Steady State Phase E: Call Teardown系統(tǒng)介紹 ITU-T H.323TCP connectionSETUPALERTING(optional)CONNECT (H245 Address)Q.931(over TCP)TCP connectionH.245 MessagesOpen Logical Channels(RTCP address)(RTCP & RTP addresses)(RTCP address)(RTCP & RTP addresses)H.245(over TCP)RTP streamRTP s

20、treamRTCP streamMedia(over UDP)H.323H.323呼叫建立和通信過程圖例系統(tǒng)介紹 ITU-T H.323TCP Disconnect(Releae Complete)Q.931(over TCP)TCP DisconnectH.245 MessagesClose Logical ChannelsEnd SessionEnd SessionH.245(over TCP)H.323H.323通信終止過程圖例系統(tǒng)介紹 ITU-T H.323MPEG - The mandateMPEGs original mandate was the development of s

21、tandards for coding moving pictures. The mandate was extended with the addition of and associated audio”and their combinationLater on the limitation for digital storage media was removedMPEG is now “Coding of moving pictures and audio”系統(tǒng)介紹 MPEGXThe MPEG Working ProcessSelect target applicationsIdent

22、ify the functionalities needed by each applicationBreak down the functionalities into requirements Identify the common requirements across the systems of interest and other relevant requirements Specify tools that support the requirements aboveVerify that these tools can be used to assemble the targ

23、et systems and provide the desired functionalities 系統(tǒng)介紹 MPEGXThe development of an MPEG standard (1)Identify the need for a standard the time it will take to develop itRequest ISO to approve a new standard projectDefine precise requirements technologies needed to satisfy the requirementsIssue a call

24、 for proposals of technologiesAssess the technologies proposed系統(tǒng)介紹 MPEGXThe development of an MPEG standard (2)Define a common model as a platform to assess the relative merits and integrate the different technologies (no rubberstamp!)Implement the common model in software and perform ”core experime

25、nts”Produce a series of working draftsExchange bitstreams to remove all bugsVerify the performance of the standardSubmit the draft standard to NB ballotsCD, FCD, FDIS, IS系統(tǒng)介紹 MPEGXStructure of MPEG standardsPart 1SystemsPart 2VideoPart 3AudioPart 4Conformance testingPart 5Reference softwarePart 6Del

26、ivery系統(tǒng)介紹 MPEGXMPEG-1 (ISO/IEC 11172) Title: Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/sProject historyStarted in May 88IS status reached in November 92Main goal:storage of interactive movies on CDs (1.4 Mbit/s) withvideo quality equivalent to V

27、HSs CD-quality stereo audio系統(tǒng)介紹 MPEG-1MPEG-1: a sequence of “first”First integrated audio-visual standard : ISO/IEC 11172-1 (Systems), -2 (Video), -3 (Audio)First audio-visual standard defining the “receiver” and not the “transmitter”First video coding standard independent of video format (NTSC/PAL/

28、SECAM)First standard jointly developed by all industries interested in audio and videoFirst standard developed entirely in softwareFirst standard to include a software implementation系統(tǒng)介紹 MPEG-1MPEG-1 - an assessment Video CD (60 million players are used in China) “The” format of audio and video for

29、PCWindows 95/98/NT contain an MPEG-1 software decoder MPEG-1 Audio (recently, layer 3) is widely used for Web music Digital Audio Broadcasting (DAB) utilises MPEG-1 Audio (adopted in Europe and Canada) MPEG-1 video cameras from several anufacturers are on sale系統(tǒng)介紹 MPEG-1MPEG-2 (ISO/IEC 13818)Title:

30、Generic coding of moving pictures and audioProject history Started in July 90 IS status achieved in November 94Main goalmigration of television from analogue to digital withcomposite quality at 6 Mbit/s, component quality at 9 Mbit/smultichannel audio coding系統(tǒng)介紹 MPEG-2MPEG-2 - an assessment56 millio

31、n digital TV set top boxes will be sold at the end of 2000Digital television VHF/UHF broadcastingUK and other countries for conventional definitionUS for ATV 4 million DVD players sold in 1999 in USA aloneThe MPEG-2 4:2:2 profile is being adopted in the television production industryMPEG-2 has creat

32、ed the entirely new digital television industry worth 20 billion USD 系統(tǒng)介紹 MPEG-4MPEG-4 (ISO/IEC 14496)Title: Coding of audio-visual objectsProject historyStarted in July 93 IS status to be achieved in May 1999Main goalcompression and manipulation of audio and visual objects, the web page paradigm ap

33、plied to audio and video!系統(tǒng)介紹 MPEG-4Features of MPEG-4An audio-visual scene is made up of objects with each of which the user can interact. Like what a user can do with a hot spot in a web page. The difference is that the interaction becomes possible with audio and video objects. When these are indi

34、vidually represented, it becomes possible to create scenes that contain not just natural objects (as was the case of MPEG-1 and -2), but also synthetic objects;系統(tǒng)介紹 MPEG-4Features of MPEG-4MPEG-1 and MPEG-2 were implicitly designed having hardware implementation in mindMPEG-4 has been designed havin

35、g both hardware and software in mind. The reference software (part 5 of the standard), developed by an ad-hoc Open Source Software process by MPEG experts, has even the same normative value as the textual part.系統(tǒng)介紹 MPEG-4Features of MPEG-4In the past standards were largely static” but today standard

36、s need to be able to evolve. The response of MPEG-4 is the version approach. The version is a compatible extension of the standard. Version 1 (baseline) has been approved in October 1998.Version 2 in December 1999, Versions 3 and 4 will be approved in January 2001. More versions are in the pipeline.

37、系統(tǒng)介紹 MPEG4MPEG-4: Reference ModelSourceDemultiplexCompos.VideoAudioAnimationText/Grap.InteractionCompositionPresentationDelivery系統(tǒng)介紹 MPEG-4MPEG-4, the model系統(tǒng)介紹 MPEG-4Visual objects系統(tǒng)介紹 MPEG-4The basic audio-visual objects系統(tǒng)介紹 MPEG-4A composite audio-visual object系統(tǒng)介紹 MPEG-4An Object-Based SceneSpor

38、ts results: Portugal - Brazil Sports results: Portugal - Brazil系統(tǒng)介紹 MPEG-4Some technical features of MPEG-4 (1)Video covers the 5 kbit/s - 5 Mbit/s rangescalability, error resilience, 12 bits pixelsAudio covers the 2 kbit/s - 64 kbit/s rangespeech: good quality starting from 2 kbit/smusic: CD qualit

39、y 128 kbit/s, good quality 24 kbit/sSystems Streaming Format and File Format definedSupports Object Content Information for annotationsSupports individual media encryption and authenticationDeliveryAbstraction from the underlying transport layer系統(tǒng)介紹 MPEG-4Some technical features of MPEG-4 (2)Texture

40、 coding with high scalability levelsHuman face animationStructured audioText-to-speech interface系統(tǒng)介紹 MPEG-4A practical solution: Profiles系統(tǒng)介紹 MPEG-4Some MPEG-4 applicationsAudio on demand on the WebDigital radio broadcasting digital AM radioStreaming video on the WebInteractive mobile multimediaDigi

41、tal Multimedia Broadcasting Electronic Program Guides (EPG)Virtual sites on the WebInteractive local multimediaSurveillance系統(tǒng)介紹 MPEG-4Metering of content usageTodays applications are largely monolithic (a movie, a CD etc.)Fixed price for content usageBut in a multimedia environment a user can naviga

42、te in a large application and consume only 10% of the content in itShould the user pay for all content in the application or only for the content actually consumed?The MPEG-4 IPMP (Intellectual Property Management and Protection) framework provides a complete answer系統(tǒng)介紹 MPEG-4MPEG-4 IPMP系統(tǒng)介紹 MPEG-4M

43、etering of technology usageToday the right to use the technology is paid once for as long as the device holdsabout 4 USD to use the technology of an MPEG-2 decoderBut there are new licensing models. AAC (Advanced Audio Coding) can be licensed according to two different modelsFor a hardware playback

44、device it is the same as MPEG-2For downloadable software the service provider pays a licence fee for each piece of music consumed. The MPEG-4 IPMP framework provides an answer for the general multimedia framework系統(tǒng)介紹 MPEG-4Patent IP management系統(tǒng)介紹 MPEG-4MPEG-7 (ISO/IEC 15938) Title: Multimedia Conte

45、nt Description InterfaceProject historyStarted in Jul 97IS status to be reached in Sep. 01Main goal:Goal: allowing efficient search for multimedia content using standardized descriptions系統(tǒng)介紹 MPEG-7Context: Stating the Question More information is in digital form, .More digital information is stored,

46、 broadcast, or available on-line, .More of that is audio/visual, .Efficiently searching what you want is increasingly important!However - There is no easy - and generally agreed upon - way to locate audio/visual information!系統(tǒng)介紹 MPEG-7Goal of MPEG-7Included: multimedia informationstill pictures, aud

47、io, graphics, moving video, 3D models, etc.no text-only documents and: information on how these elements are combined in multimedia presentatione.g. scenarios, composition information. and: information that cannot be extracted (e.g. context)系統(tǒng)介紹 MPEG-7Goal of MPEG-7Information will be associated wit

48、h the content content could be MPEG-4 objectscould also be MPEG-2 or MPEG-1 material could even be an analogue movieInformation could be together or separate (pointers to and from the data en the description are then needed)系統(tǒng)介紹 MPEG-7Scope of the MPEG-7 Standard: some dimensionsPush & Pull service

49、modelsso: searching as well as filtering supportedso: standardisation of descriptors is neededAlso applications that do not fall under either categoryOn-line and off-line; Local and remoteReal-time and non Real-time annotation(NB: These dimensions are not independent)系統(tǒng)介紹 MPEG-7What is exactly stand

50、ardised?Descriptors A value associated with one or more FeaturesDescription Schemes structure and semantics for Descriptors and their relationsDescription Definition LanguageLanguage to define Description Schemes Description = Description Scheme + instantiated Description系統(tǒng)介紹 MPEG-7The elements of t

51、he MPEG-7 StandardMPEG-7 Description Schemes (DSs)MPEG-7 Descriptors Multimedia Content(streamed or stored) Description Generation(analysis tools, annotation, etc. )Search / QueryEngine UserFilterAgent UserMPEG-7 Description Definition Language (DDL)Push applicationsPull applications系統(tǒng)介紹 MPEG-7New O

52、bjective for MPEGMPEG-1, -2 and -4: representation of content itself (the bits)MPEG-7: representation of information about the content (the bits about the bits) but the boundary is not so clear-cut!shapes, colour, motion vector fields, 系統(tǒng)介紹 MPEG-7What Form of Descriptions?Description depends on appl

53、ication! Extraction automatic or human-assistedthe higher the abstraction level, the more difficult automatic extraction will be whats both possible and useful; still an open question!系統(tǒng)介紹 MPEG-7What is Exactly standardised?featureextractiondescriptionsearchenginethis is the scope of MPEG-7!Not the

54、analysis, Not the search engine,Just the description.系統(tǒng)介紹 MPEG-7A Few Example Application AreasDigital libraries image catalog, musical dictionary,Multimedia directory services e.g. yellow pagesBroadcast media selection radio channel, TV channel,Multimedia editing personalized news service, media au

55、thoring系統(tǒng)介紹 MPEG-7MPEG-21 Title: Multimedia FrameworkProject historyStarted in May 00IS status to be reached in Dec. 01Main goal:provide the integration of different technologies needed for Multimedia系統(tǒng)介紹 MPEG-21MPEG-21 - Multimedia FrameworkContent Identification & DescriptionContentRepresentationC

56、ontentManagement& Usage Networks & TerminalsEvent ReportingIPMPMPEG-21User系統(tǒng)介紹 MPEG-21MPEG-21 IPMP (1)Languages to codify Norms and Rules 1Adopt/extend existing rights expression languages for describing contractual usage rules2Expand these languages to allow the expression of public policies and ru

57、les stemming from sources other than Rights Holders, such as governments and other relevant rule-making bodies. (Not for 1st Phase of MPEG-21)系統(tǒng)介紹 MPEG-21MPEG-21 IPMP (2)Trusted Framework of IPMP Systems1Define attributes of a trusted environment for persistent management and protection of MPEG-21 D

58、igital Items in accordance with codified norms and rules2Start with MPEG-4 and MPEG-7 and widen scope for other types of MPEG-21 Digital Items系統(tǒng)介紹 MPEG-21多媒體通信關(guān)鍵技術(shù)之一音頻編碼 音頻、視頻信號數(shù)字化的優(yōu)點(diǎn) 壓縮的必要性 音頻編碼分類及發(fā)展簡史 音頻壓縮依據(jù) 語音編碼性能評價(jià) 語音信號的特點(diǎn) 目前水平及現(xiàn)有標(biāo)準(zhǔn) 語音生成模型及參數(shù)編碼音頻、視頻信號數(shù)字化的優(yōu)點(diǎn)u 提高存儲(chǔ)、傳輸?shù)馁|(zhì)量 數(shù)字存儲(chǔ)與模擬存儲(chǔ)相比具有如下優(yōu)點(diǎn)： F 存儲(chǔ)過程無

59、失真，所以存儲(chǔ)質(zhì)量好 F 便于讀寫及快速搜索 F 便于利用固態(tài)存儲(chǔ)器，不需要機(jī)械傳動(dòng)機(jī)構(gòu) 數(shù)字傳輸與模擬傳輸相比具有如下優(yōu)點(diǎn)： F 采用多電平調(diào)制可以節(jié)省頻帶 F 在相同接收質(zhì)量下，數(shù)字調(diào)制節(jié)省發(fā)射功率 F 便于采用誤碼校驗(yàn)技術(shù)，提高傳輸質(zhì)量 F 抗干擾能力強(qiáng) F 便于交換，提高網(wǎng)絡(luò)效率u 便于處理u 使用靈活，便于多種媒體（視頻、音頻、文字、數(shù)據(jù)）相結(jié)合應(yīng)用u 易于加密u 適合大規(guī)模集成u 可靠性高、體積功耗小u 價(jià)格便宜壓縮的必要性AUDIO類型帶寬KHZ采樣率KHZ比特/樣點(diǎn)比特率kb/s電話語音0.23.481296寬帶語音0.0571614224調(diào)頻廣播0.02153216512CD

60、光盤0.012044.116705.6DAB/DAT0.01204816768音頻編碼發(fā)展簡史1)語音 (Telephone Speech)：主要應(yīng)用于數(shù)字電話 波形編碼 PCM原理（37年，法Alec Reeres）電子管PCM（46年，Bell實(shí)驗(yàn)室）晶體管PCM（62年，市話擴(kuò)容，64kb/s）單片IC PCM（70年代，微波、衛(wèi)星、光纖） 增量編碼原理（46年，法De Loraine）自適應(yīng)增量 CVSD（60年代末，軍用，32、16kb/s）Continuously Variable Slope Delta Modulator 連續(xù)變化斜率增量調(diào)制器 其他編碼（70年代，ADPCM、