http://dx.doi.org/10.5909/jeb.2012.17.2.207 MPEG DASH service-compatible 3D HTTP adaptive streaming a), a), a), a) HTTP Adaptive Streaming Method for Service-compatible 3D Contents Based on MPEG DASH Gijun Park a), Gilbok Lee a), Jangwon Lee a), and Kyuheon Kim a) 3D 3D TV, 3D Monitor, 3D. 3D QoS(Quality of Service) IP 3D., IP MPEG(Moving Picture experts Group) DASH(Dynamic Adaptive Streaming over HTTP). DASH, segment 2 Elementary Stream(ES) service- compatible scheme. DASH service-compatible 3D. DASH 3D IP 3D 3D, 3D. Abstract Recently, many consumer electronics manufacturers have produced 3D devices such as 3DTVs and 3D monitors as interests in a stereoscopic video service are getting increased. However, most of 3D services are focused on local storage or bandwidth guaranteed service since 3D stereoscopic video service require bandwidth more stable and larger. This property causes difficulties in seamless stereoscopic video streaming services under IP based open network environment that cannot guarantee quality of services. In order to achieve a seamless video streaming service the international standard organization MPEG (Moving Pictures Experts Group) has developed the adaptive HTTP streaming technology called as DASH (Dynamic Adaptive Streaming over HTTP). However, the DASH doesn t have obvious scheme which can express the two elementary video streams based service-compatible stereoscopic contents in one segment. Therefore, this paper proposes a scheme of efficient 3D adaptive streaming service based on the DASH, which covers not only frame-packing stereoscopic contents but also service-compatible ones. The 3D adaptive HTTP streaming scheme introduced in this paper is able to provide 3D contents with various qualities to user and also has benefit that single 3D content can be applied to a variety of devices. Keyword : DASH, stereoscopic, service-compatible
. 3D 3D, 3D. 3D.,.,,, 3D. 3D. 3D 1. 3D, (HTTP)., 2. 1 1. Fig. 1. Concept of various device service using single content a) Kyung Hee University : (kyuheonkim@khu.ac.kr) ( Edutainment )( : 10035486) IT (NIPA-2011-(C1090-1111-0001)). (2011 12 30 ), (2012 3 9 ), (2012 3 9 )
2. Fig. 2. Figure of various quality service of single content, 2., 3D, Qos IP. MPEG DASH(ISO-IEC_ 23009-1 Dynamic Adaptive Streaming over HTTP) [1]. DASH framepacking service-compatible., service-compatible,, elementary stream(es) segment scheme. DASH segment, ES service-compatible 3D DASH MPD.. 2 DASH. 3 segment, ES service-compatible 3D DASH MPD(Media Presentation Description), 4 3 3D DASH. 5.. MPEG DASH MPEG DASH. 1 DASH, 2 MPD. 3 MPD. 1. DASH DASH MPD segment. MPD XML, segment MPD. MPD segment HTTP Client DASH., (segment ) MPD. Client HTTP, Client. Client MPD, MPD segment. segment Client DASH. 3 DASH Client Model, Client MPD, MPD Client segment data. segment data DASH access en-
3. DASH Fig. 3. DASH Client Model gine MPEG format, Media engine. Media engine. 2. Media Presentation Description MPD., MPD. MPD 4,. MPD Period, Period AdaptationSet. AdaptationSet Representation, Representation segment. MPD,, 4. DASH Fig. 4. Hierarchical DASH Data Model
,, URL. MPD Period Media Presentation, Period Period, segment. Period AdaptationSet, /,,,,, ContentComponent.,, stereo pair Role. Role Content Component, 2 ContentComponent segment,. segment service-compatible MPD ContentComponent, 2.1. AdaptationSet Representation. Representation,, segment URL., Representation. AdaptationSet, Representation.,, mime Type,, Type. FramePacking, framepacking. Representation MPD segment SegmentList, SegmentTemplate, SegmentBaseInformation, MultipleSegmentBaseInformation [1]. segment URL. MPD. 3. 3.1 Frame-packing,, frame-packing. frame-packing side-by-side, top and bottom, column/row based interleaving. frame- packing, MPD Adap- tationset FramePacking. FramePacking 1 schemeiduri, value. 1. FramePacking Table 1. Structure of FramePacking element Element or Attribute Name FramePacking @schemeiduri @value Use FramePacking schemeiduri, AVC(Advanced Video Coding) schemeiduri urn:mpeg:dash:14496:10:frame_packing_arrangement_type:2011 ISO/IEC 14496-10, value ISO/IEC 14496-10 Part 10 Table D-8 value, AdaptationSet frame-packing [1][3]. MPEG 2 Video scheme- M O
Iduri urn:mpeg:dash: 13818:1:stereo_video_format_ type:2011 ISO/IEC 13818-2, value ISO/IEC 13818-2 AMD4 Table L-1 value, AdaptationSet frame-packing [1][4]. frame-packing MPD. 3.2 Two segments service-compatible,, 3D service-compatible., AdaptationSet segment, MPD 5. 5 3D segment, segment service-compatible, AdaptationSet segment, AdaptationSet segment. MPD AdaptatinSet Role. Role FramePacking schemeiduri, value schemeiduri urn:mpeg: dash:stereoid:2011 stereo pair id, value segment, stereo pair. segment,.. One segment service-compatible DASH 3 segment, service-compatible MPD. DASH one segment service-compatible 5. Two segments service-compatible MPD Fig. 5. MPD structure for service-compatible stereoscopic content based on two segments
MPD,. 1 segment,, 2 1 3D DASH MPD. 1. One segment service-compatible,, 3D service-compatible. service-compatible service compatible MPEG-2 TS [5], SVAF left/right view sequence type [6]. Service compatible MPEG-2 TS PMT(Program Map Table) 2. MPEG-2 TS PMT Table 2. PMT syntax of MPEG-2 TS Format TS_program_map_section(){ table_id 8... program_info_length 12 for(i=0 ; i<n ; I++){ Stereoscopic_program_info_descriptor() for(i=0 ; i<n1 ; I++){ stream_type 8 0x02 reserved 3 elementary_pid 13 0x68 reserved 4 ES_info_length for(i=0 ; i<n1 ; I++){ Stereoscopic_video_info_descriptor() } for(i=0 ; i<n2 ; I++){ stream_type 8 0x23 reserved 3 elementary_pid 13 0x69 reserved 4 ES_info_length for(i=0 ; i<n3 ; I++){ Stereoscopic_video_info_descriptor() CRC_32 32 } 2. PMT PID. 2, PMT Stereoscopic_ program_info_descriptor 'stereoscopic_ service_type' 3 Service compatible service service-compatible, PMT ES_Info 2, 2 ES_Info elementary_pid [5][7]. 3. MPEG-2 TS Table 3. Stereoscopic service type of MPEG-2 TS Values Description 010 Frame compatible service 011 Service compatible service 100~111 Reserved SVAF(Stereoscopic Video Application Format) 4. SVAF ISO base media file format, 4 Left/ Right view sequence type segment, [6]. 4. SVAF Table 4. Stereoscopic composition type of SVAF Value 0x00 0x01 0x02 0x03 0x04 0x05-0xFF Stereoscopic_composition_type Side-by-side type Vertical line interleaved type Frame sequential type Left/Right view sequence type Top-Bottom type Reserved Left/Right view sequence type 6. 6 ISO base media file format. mdat moov
6. Left/Right view sequence Fig. 6. File structure of Left/Right view sequence type, trak track_id 2,. segment,, 'trak' svmi,. 2. One segment service-compatible MPD 1 service compatible MPEG-2 TS [5], SVAF left/right view sequence type [6], service-compatible, segment ContentComponent MPD. 2.1. ContentComponent ContentComponent AdaptationSet 5 @id, @lang, @contenttype, 5. ContentComponent Table 5. Structure of ContentComponent element Element or Attribute Name ContentComponent @id @lang @contenttype @par Accessibility Role Rating Viewpoint Use _ O O O O 0... N 0... N 0... N 0... N
@par, Accessibility, Role, Rating, Viewpoint. ContentComponent AdaptationSet. ContentComponent MPD 7. Content Component id, contenttype AdaptationSet segment, AdaptationSet segment. 7 contenttype, ContentComponent Adaptation Set. 3.2 Role,, AdaptationSet segment, service-compatible MPD. 2.2. ContentComponent MPD Service-compatible, segment ContentComponent MPD. 5 ContentComponent Role. Role 1 FramePacking schemeiduri, value., 3.2 schemeiduri, value. segment, Content Component 2 AdaptationSet segment. AdaptationSet segment MPD 8. Adaptation Set Representation segment, segment,. 7. ContentComponent MPD Fig. 7. Example of MPD structure using ContentComponent element
8. service compatible 3D MPD Fig. 8. MPD structure for 3D service based on service compatible 7 AdaptationSet 2 ContentComponent AdaptationSet segment, segment, AdaptationSet 2 ContentComponent., AdaptationSet ContentComponent Role value "l0", "r0", AdaptationSet Representation segment,. MPD segment 3D DASH.. 3D DASH MPD 3D DASH. 1 3D DASH, 3D DASH 2. 1. 3D DASH 3D DASH 9. Microsoft Expression Encoder
9. 3D DASH Fig. 9. Structure of the 3D DASH service system, *.ism, *.ismc, *.ismv. *ism, *.ismc MPD, *.ismv. DASH MPD segment 3D DASH File Generator segment MPD. Client MPD MPD segment URL, segment segment. segment segment,. Windows 7 (64bits), Microsoft visual studio 2008 C/C++. 2., 9 3D DASH 3D DASH File Generator MPD/segment. MPD frame-packing MPD MPD Generator MPD. frame-packing MPD 10. 10 frame-packing 3D MPD, 4 MPD, Period, AdaptationSet, Representation, Segment. BaseURL segment, BaseURL SegmentURL segment. frame-packing segment AdaptationSet FramePacking schemeiduri urn:mpeg:dash:14496:10:frame_packing_arrangement_type:2011 value "3" AVC side-by-side frame-packing
10. Frame packing 3D DASH MPD Fig. 10. MPD file for 3D DASH service based on frame packing. segment, ES MPD MPD Generator MPD. service-compatible, segment MPD 11. 8 MPD AdaptationSet 2 ContentComponent Role segment,. MPD MPD, MPD, MPD segment URL. URL segment URL segment segment., segment 11. Service compatible 3D DASH MPD Fig. 11. MPD file for 3D DASH service based on service compatible
박기준 외 : MPEG DASH 기반 service-compatible 3D 구현하였다. 다시 말해, MPD에서 Representation 요소 단 위로 세 가지 품질을 분류하여 기술하여 줌으로써, 각 품질 에 해당하는 segment URL정보를 클라이언트 측에서 알 수 있고, 이에 따라 가변적인 품질로 segment 파일을 요청할 수 있는 것이다. 이러한 과정을 통해 디스플레이 된 3D 영상을 아래의 그 림 12, 13에서 나타내고 있다. 우선, 그림 12는 frame-packing 기반의 3D 영상과 service-compatible기반의 3D 영상을 각각 나타낸다. 재생 화면의 윗부분에 mode, service type, 콘텐츠 대상 HTTP adaptive streaming 적용방안 219 등의 정보를 알 수 있도록 표시하여 주었 다. 그림 12의 위쪽 영상은 type이 frame-packing형식의 side-by-side 타입을 나타내고 있음을 볼 수 있고, 아래쪽 영상에서의 type은 service-compatible 형식을 나타내고 있 다. 또한 512Kbps, 1024Kbps, 2500Kbps 세 가지의 품질 중, 가장 높은 품질로 재생되고 있음을 알 수 있다. 여기서 아래쪽 영상의 품질이 5000Kbps로 나타나는 이유는 좌, 우 영상이 하나의 프레임으로 표현되는 frame-packing 방식과 는 달리 service-compatible 방식은 좌 영상의 한 프레임과 resolution, bitrate 그림 12. Frame packing과 service compatible 기반의 3D 재생 화면 Filg. 12. 3D playback scene based on frame packing and service compatible
220 방송공학회논문지 년제 권제호 2012 17 2 그림 13. 품질 별 3D 재생 화면 Fig. 13. 3D playback scene with different quality 우영상의 한 프레임이 3D로 디스플레이 되므로 bitrate가 frame-packing 영상에 비해 두 배로 나타나는 것이다. 다음으로 service-compatible 기반의 3D 영상이 품질별 로 재생되는 화면을 그림 13에서 보여주고 있다. 우선 그림 의 위쪽 영상은 1024Kbps, 2048Kbps, 5000Kbps 세 가 지의 품질 중, 가장 낮은 품질의 1024Kbps 640 360 사이즈 영상을 나타내고 있고, 아래쪽 영상은 가장 높은 품질의 5000Kbps 1920 1080 사이즈 영상을 나타내고 있다. 이처 13
3D, 3D... MPEG DASH 3D MPD, segment MPD. framepacking, service-compatible 3D., [1] ISO/IEC_23009-1 Dynamic adaptive streaming over HTTP (DASH)-Part 1:Media presentation description and segment format [2],,,, HTTP Streaming User Profile Seamless Framework,, 16 1, pp.155-173 [3] ISO/IEC_14496-10 Information technology -Coding of audio-visual objects-part 10: Advanced Video Coding [4] ISO/IEC_13818-2 AMD4: Frame Packing Arrangement Signalling for 3D Content [5] ISO/IEC_13818-1 AMD7: Signaling of stereoscopic video in MPEG-2 system [6] ISO/IEC_23000-11 Information technology Multimedia application format (MPEG-A) Part 11: Stereoscopic video application format [7],,,,, MPEG-2 MPEG-C part 3,, 2011 16 5, pp.693-893 - 2011 2 : - 2011 3 ~ : - :, MPEG-2/4 systems, MPEG-DASH - 2010 2 : - 2010 3 ~ : - :, MPEG-2/4 systems
- 2007 2 : - 2007 3 ~ : - :, - 1989 2 : - 1992 9 : University of Newcastle upon Tyne - 1996 7 : University of Newcastle upon Tyne - 1996 ~ 1997 : University of Sheffield, Research Fellow - 1997 ~ 2006 : - 2006 ~ : - :,,