QoE Enhancement for Stereoscopic 3DVideo Quality Based on Depth and Color Transmission over IP Networks: A Review
Keywords:
3D video, Stereoscopic, QoE, Color-plus-Depth, error concealment.Abstract
In this review paper we focus on the enhancement of Quality of Experience (QoE) for stereoscopic 3D video based on depth information. We focus on stereoscopic video format because it takes less bandwidth than other format when 3D video is transmitted over an error channel but it is easily affected by the network parameters such as packets loss, delay and jitter. The packet loss on 3D video has more impact in the depth information than other 3D video factors such as comfort, motion, disparity and discomfort. The packet loss on depth information causes undesired effect on color and depth maps. Therefore, in order to minimize quality degradation, the application of frame loss concealment technique is preferred. This technique is expected to improve the QoE for end users. In this paper we will also review 3D video factors and their challenges, methods of measuring the QOE, algorithms used for packets loss recovery.
References
[1] Chaminda Hewage. “The transmission aspect of 3D video” In 3D processing and transmission fundamentals, 1st edition ,2014, PP. 63- 68 .
[2] Rec BT.2021, “Subjective methods for the assessment of stereoscopic 3DTV systems”, 2008.
[3] Recommendation REC. ITU-R BT.1438, “subjective assessment of stereoscopic television pictures”, 2000.
[4] Smolic, A., Mueller, K. , Merkle, P. and Vetro, A. , “Development of a new MPEG Standard for Advanced 3D Video Applications,” , Image and Signal Processing and Analysis, ISPA Proceedings of 6th International Symposium on 2009 , PP. 400 – 407.
[5] Takashi Matsuyama, Shohei Nobahara,Takeshi takai, Tony Tung . 3D video and Its Application. , Springer-Verlag : London ,2012,PP.2-6.
[6] Bruno Zatt, Muhammad Shafique,Sergio Bampi,Jorg Herkee,Energy. 3D video coding for Embedded Devices , Efficient Algorithms & Architectures. Springer Science+Business Media : New York, 2013,PP.2-3.
[7] Kazuhisa Yamagishi, Taichi Kawano, and Kimiko Kawashima, “QoE Assessment Methodologies for 3D Video Services.” NTT Technical Review, Vol. 11 No. 5, PP.1-5, May 2013
[8]. Philipp MERKLE, Karsten MÜLLER, and Thomas WIEGAND, “3D video: Acquisition, Coding and Display.” IEEE Transactions on Consumer Electronics, vol.56, no.2, pp.946-950, 2010.
[9] Anil Fernando, Stewart T, Worall and Erhan Ekmekcio?lu. (2013,AUG,16). 3DTV Processing and transmission of 3D video signals. (First Edition). [on-line]:Available: www.onlinelibrary.wiley.com/book/10.1002/9781118705711[Feb, 23, 2016].
[10] Guan-Ming Su, Yu-chi Lai, Andres Kwasinski and Haohong Wang. (2012,Dec,12).3D visual Communications.(first edition) .[on-line]:Available:www.onlinelibrary.wiley.com/book/10.1002/9781118358023 [JUL, 3, 2016].
[11] E.Kohler, M.Handley and S.Floyd,, “ Datagram Congestion Control Protocol (DCCP) ” ,Internet Engineering Task Force RFC 4340”, Mar .2006.
[12] S.Floyd, E.Kohler and Padhye, “Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 3: TCP-Friendly Rate Control (TFRC)”, Internet Engineering Task Force RFC 4342, Mar .2006
[13] Frederic dufuax Beatrice Pesquet-Popescu, & Marco cagnazzo. (2013, APR, 22). Emerging technologies for 3D video creation, coding, transmission and Rendering.(first edition).[on-line].Available: http://onlinelibrary.wiley.com/book/10.1002/9781118583593 [Feb, 24, 2016].
[14] Ekmekcioglu, Guildford, Fernando, Worrall, S.T, Kondoz, A.M., “ Utilization of motion similarity in Color-plus-Depth 3D video for improved error resiliency, ”, published Information and Automation for Sustainability (ICIAFs), 5th International Conference on 17-19 Dec. 2010 , PP:475 – 478.
[15] Renaud, ManLi,DavidSoldani , “ QoE management in UMTS Cellular Systems.”, Vehicular Technology Conference, 2006. VTC 2006-Spring. IEEE 63rd , Vol:2 ,PP:981-985.
[16] Amendment 1 to ITU-T Recommendation P.10/G.100 (2006). “Vocabulary for performance and quality of service”, was agreed on 25 January 2007 by ITU-T Study Group 12 (2005-2008)
[17] Chaminda T. E. R. Hewage and Maria G. Martini, “Quality of Experience for 3D video streaming.” , IEEE Journals & Magazines of Communications Magazine, Vol.51, No.5,PP: 101 - 107 ,2013.
[18] ETSI TR 102 643 V1.0.2 (2010-01) , “ Human Factors (HF); Quality of Experience (QoE) requirements for real-time communication services” , 2010.
[19] Kjell Brunnström, David Hands, Filippo Speranza and Arthur Webster, “VQEG Validation and ITU Standardization of Objective Perceptual Video Quality Metrics.”, IEEE Signal processing magazine, Vol. 26, PP .96-10 , 2009.
[20] Recommendation ITU-T P.910, “Subjective video quality assessment methods for multimedia applications”, 2008.
[21] Recommendation ITU-T P.913 (01/2014) ,“ Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment”, 2014.
[22] Recommendation ITU-T J.247 , “objective perceptual multimedia video quality measurement in the presence of a full reference” , 2008.
[23] Recommendation ITU-T J.246, “perceptual visual quality measurement techniques for multimedia services over digital cable television Networks in the presence of a reduced bandwidth reference”, 2008.
[24] D. Hands, O. V. Barriac, and F. Telecom, “Standardization activities in the ITU for a QoE assessment of IPTV.” IEEE Communications Magazine, vol. 46, pp. 78–84, 2008.
[25] NNT group. “Communication services QoE Assessment Research portal site @NNT.” Internet: http://www.ntt.co.jp/qos/qoe/eng/technology/3d/01_1.html , Feb.5.2016 [Jule.3.2016].
[26] Pierre Lebreton, Alexander Raake, Member, Marcus Barkowsky, Member, and Patrick Le Callet, Member, “Evaluating Depth Perception of 3D Stereoscopic Videos.”, Vol.6, No.6,PP.710- 720, 2012
[27] A.S Amar, R.M.Swash and A.H.Sadka “Subjective Quality Assessment of 3D Videos,”, The Falls Resort and Conference Centre, Livingstone, Zambia ,September 2011,PP.13 – 15.
[28] Mysirlidis, C, Dagiuklas, T, Politis, I, Ekmekcioglu, E, Dogan, S, Kotsopoulos, S, “Quality evaluation of 3D video using color-plus-depth & MDC over IP networks,” , 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2014, PP.1- 4.
[29] Taewan Kim, Jiwoo Kang, Sanghoon Lee and Alan C. Bovik, “ Multimodal Interactive Continuous Scoring of Subjective 3D Video Quality of Experience ,”, Vol.16, PP.387- 402 , 2014
[30] Hossein Malek mohamadi, W. A. C. Fernando 5and A. M. Kondoz, “ A New Reduced Reference Objective Quality Metric for Stereoscopic Video,”, GC12 Workshop: Quality of Experience for Multimedia Communications , 2012. PP.1325-1328.
[31] Hewage and Maria G. Martini , “Reduced-Reference Quality Evaluation For Compressed Depth Maps Associated With Color Plus Depth 3d Video,” Image Processing (ICIP), IEEE International Conference on, 2010,PP.4017- 4020
[32] Chaminda T. E. R. Hewage and Maria G. Martini, “Reduced-Reference Quality Assessment for 3D Video Compression and Transmission.”, consumer electronics IEEE Transaction, Vol.57, PP.1185-1193, 2011.
[33] P. Joveluro, H. Malek mohamadi, W.A.C Fernando, and A.M. Kondoz ,“Perceptual Video Quality Metric for 3D Video Quality Assessment,”, 3DTV- conference: the true vision- caption, transmission and display of 3D video (3DTV-CON), 2010, PP.1-4.
[34] Politis, V. De Silva, S. Dogan, C. De Alwis , T. Dagiuklas, and A. M. Kondoz , “ Objective Measurement of Transmission Losses and Compression Artefacts in Stereoscopic 3D Video,”, Communications Workshops (ICC), IEEE International Conference on , 2013,PP.667-671.
[35] Yi Han, Zhenhui Yuan, Gabriel-Miro Muntean, “ No Reference Objective Quality Metric for Stereoscopic 3D Video,”, Broadband Multimedia System and broadcasting (BMSB) ,2014 , PP.1-6.
[36] Yi Han, Zhenhui Yuan, Gabriel-Miro Muntean,“Extended No Reference Objective Quality Metric for Stereoscopic 3D Video,”, Communication Workshop (ICCW),IEEE International Conference on , 2015,PP.1729-1734.
[37] Sawsen Abdulhadi Mahmood , Rana Fareed Ghani , “Objective Quality Assessment of 3D Stereoscopic Video Based on Motion Vectors and Depth Map Features,”, Computer Science and Electronic Engineering Conference (CEEC) , 2015, PP.179-183
[38] Tselios, Athanasios Kordelas, Tasos Dagiuklas, and Andreas Papadaki , “A model of network related QoE for 3D video, ”, Globecom Workshops (GC Wkshps)EEE, 2012, PP.1335-1340.
[39] J. Carreira, L. Pinto, N. Rodrigues, S. Faria, P. Assuncao “Subjective Assessment Of Frame Loss Concealment Methods In 3D Video,”, Picture Coding Symposium (PCS) , 3,2010 ,PP. 182–185.
[40] L.Pinto,J.Carreira , S.Faria, N.Rodrigues,P.Assuncao “Subjective Quality Factors In Packet 3D video,”, in Quality of Multimedia Experience (QoMEX) Third International Workshop,Sept.2011,PP. 149–154 .
[41] Tae-Young Chung, SanghoonSull, and Chang-Su Kim, “Frame Loss Concealment for Stereoscopic Video Plus Depth Sequences.”, IEEE Trans. on Consumer Electronics, vol. 57, PP,. 336-344 , 2011.
[42] Meng Yang, Xuguang Lan, Nanning Zheng, Cosman.P, “ Depth-Assisted Temporal Error Concealment for Intra Frame Slices in 3-D Video.” IEEE Trans. on Broadcasting , vol. 60, PP. 385- 393,2014.
[43] E.Ekmekcioglu, “ Utilization of Motion Similarity in Color-plus-Depth 3D video for Improve Error Resiliency,” , in Information and Automation for Sustainability (ICIAFs), on. 17–19.Dec. 2010, PP. 475– 47.
[44] Khalid Mohamed Alajel, Wei Xiang, and Yafeng Wang,“ Unequal Error Protection Scheme Based Hierarchical 16-QAM for 3-D Video Transmission.” , IEEE Trans. on Consumer Electronics , vol. 58,PP.731–738., 2012.
[45] Wen-Nung Lie , Guan-Hua Lin “Error Concealment for 3D Video Transmission,” , IEEE International Symposium on Circuits and Systems (ISCAS), May. 2013,PP. 2856– 2559.
[46] Bo Yan, Senior Member, IEEE, and Jie Zhou “Efficient Frame Concealment for Depth Image-Based 3-D Video Transmission.”, IEEE/Journal Trans. on Multimedia, vol. 4 ,PP. 936-941, 2014.
[47] Chaminda T.E.R. Hewage, Stewart T. Worrall, Safak Dogan, and Ahmet M. Kondoz “A novel frame concealment method for depth maps using corresponding color motion vectors, ”, vectors,on 3DTV,2008, PP. 149–152.
[48] C.T.E.R. Hewage, S. Worrall, S. Dogan, and A.M. Kondoz “Frame concealment algorithm for stereoscopic video using motion vector sharing,” , IEEE on Multimedia and Expo, 2008 , PP. 485–488.
[2] Rec BT.2021, “Subjective methods for the assessment of stereoscopic 3DTV systems”, 2008.
[3] Recommendation REC. ITU-R BT.1438, “subjective assessment of stereoscopic television pictures”, 2000.
[4] Smolic, A., Mueller, K. , Merkle, P. and Vetro, A. , “Development of a new MPEG Standard for Advanced 3D Video Applications,” , Image and Signal Processing and Analysis, ISPA Proceedings of 6th International Symposium on 2009 , PP. 400 – 407.
[5] Takashi Matsuyama, Shohei Nobahara,Takeshi takai, Tony Tung . 3D video and Its Application. , Springer-Verlag : London ,2012,PP.2-6.
[6] Bruno Zatt, Muhammad Shafique,Sergio Bampi,Jorg Herkee,Energy. 3D video coding for Embedded Devices , Efficient Algorithms & Architectures. Springer Science+Business Media : New York, 2013,PP.2-3.
[7] Kazuhisa Yamagishi, Taichi Kawano, and Kimiko Kawashima, “QoE Assessment Methodologies for 3D Video Services.” NTT Technical Review, Vol. 11 No. 5, PP.1-5, May 2013
[8]. Philipp MERKLE, Karsten MÜLLER, and Thomas WIEGAND, “3D video: Acquisition, Coding and Display.” IEEE Transactions on Consumer Electronics, vol.56, no.2, pp.946-950, 2010.
[9] Anil Fernando, Stewart T, Worall and Erhan Ekmekcio?lu. (2013,AUG,16). 3DTV Processing and transmission of 3D video signals. (First Edition). [on-line]:Available: www.onlinelibrary.wiley.com/book/10.1002/9781118705711[Feb, 23, 2016].
[10] Guan-Ming Su, Yu-chi Lai, Andres Kwasinski and Haohong Wang. (2012,Dec,12).3D visual Communications.(first edition) .[on-line]:Available:www.onlinelibrary.wiley.com/book/10.1002/9781118358023 [JUL, 3, 2016].
[11] E.Kohler, M.Handley and S.Floyd,, “ Datagram Congestion Control Protocol (DCCP) ” ,Internet Engineering Task Force RFC 4340”, Mar .2006.
[12] S.Floyd, E.Kohler and Padhye, “Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 3: TCP-Friendly Rate Control (TFRC)”, Internet Engineering Task Force RFC 4342, Mar .2006
[13] Frederic dufuax Beatrice Pesquet-Popescu, & Marco cagnazzo. (2013, APR, 22). Emerging technologies for 3D video creation, coding, transmission and Rendering.(first edition).[on-line].Available: http://onlinelibrary.wiley.com/book/10.1002/9781118583593 [Feb, 24, 2016].
[14] Ekmekcioglu, Guildford, Fernando, Worrall, S.T, Kondoz, A.M., “ Utilization of motion similarity in Color-plus-Depth 3D video for improved error resiliency, ”, published Information and Automation for Sustainability (ICIAFs), 5th International Conference on 17-19 Dec. 2010 , PP:475 – 478.
[15] Renaud, ManLi,DavidSoldani , “ QoE management in UMTS Cellular Systems.”, Vehicular Technology Conference, 2006. VTC 2006-Spring. IEEE 63rd , Vol:2 ,PP:981-985.
[16] Amendment 1 to ITU-T Recommendation P.10/G.100 (2006). “Vocabulary for performance and quality of service”, was agreed on 25 January 2007 by ITU-T Study Group 12 (2005-2008)
[17] Chaminda T. E. R. Hewage and Maria G. Martini, “Quality of Experience for 3D video streaming.” , IEEE Journals & Magazines of Communications Magazine, Vol.51, No.5,PP: 101 - 107 ,2013.
[18] ETSI TR 102 643 V1.0.2 (2010-01) , “ Human Factors (HF); Quality of Experience (QoE) requirements for real-time communication services” , 2010.
[19] Kjell Brunnström, David Hands, Filippo Speranza and Arthur Webster, “VQEG Validation and ITU Standardization of Objective Perceptual Video Quality Metrics.”, IEEE Signal processing magazine, Vol. 26, PP .96-10 , 2009.
[20] Recommendation ITU-T P.910, “Subjective video quality assessment methods for multimedia applications”, 2008.
[21] Recommendation ITU-T P.913 (01/2014) ,“ Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment”, 2014.
[22] Recommendation ITU-T J.247 , “objective perceptual multimedia video quality measurement in the presence of a full reference” , 2008.
[23] Recommendation ITU-T J.246, “perceptual visual quality measurement techniques for multimedia services over digital cable television Networks in the presence of a reduced bandwidth reference”, 2008.
[24] D. Hands, O. V. Barriac, and F. Telecom, “Standardization activities in the ITU for a QoE assessment of IPTV.” IEEE Communications Magazine, vol. 46, pp. 78–84, 2008.
[25] NNT group. “Communication services QoE Assessment Research portal site @NNT.” Internet: http://www.ntt.co.jp/qos/qoe/eng/technology/3d/01_1.html , Feb.5.2016 [Jule.3.2016].
[26] Pierre Lebreton, Alexander Raake, Member, Marcus Barkowsky, Member, and Patrick Le Callet, Member, “Evaluating Depth Perception of 3D Stereoscopic Videos.”, Vol.6, No.6,PP.710- 720, 2012
[27] A.S Amar, R.M.Swash and A.H.Sadka “Subjective Quality Assessment of 3D Videos,”, The Falls Resort and Conference Centre, Livingstone, Zambia ,September 2011,PP.13 – 15.
[28] Mysirlidis, C, Dagiuklas, T, Politis, I, Ekmekcioglu, E, Dogan, S, Kotsopoulos, S, “Quality evaluation of 3D video using color-plus-depth & MDC over IP networks,” , 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2014, PP.1- 4.
[29] Taewan Kim, Jiwoo Kang, Sanghoon Lee and Alan C. Bovik, “ Multimodal Interactive Continuous Scoring of Subjective 3D Video Quality of Experience ,”, Vol.16, PP.387- 402 , 2014
[30] Hossein Malek mohamadi, W. A. C. Fernando 5and A. M. Kondoz, “ A New Reduced Reference Objective Quality Metric for Stereoscopic Video,”, GC12 Workshop: Quality of Experience for Multimedia Communications , 2012. PP.1325-1328.
[31] Hewage and Maria G. Martini , “Reduced-Reference Quality Evaluation For Compressed Depth Maps Associated With Color Plus Depth 3d Video,” Image Processing (ICIP), IEEE International Conference on, 2010,PP.4017- 4020
[32] Chaminda T. E. R. Hewage and Maria G. Martini, “Reduced-Reference Quality Assessment for 3D Video Compression and Transmission.”, consumer electronics IEEE Transaction, Vol.57, PP.1185-1193, 2011.
[33] P. Joveluro, H. Malek mohamadi, W.A.C Fernando, and A.M. Kondoz ,“Perceptual Video Quality Metric for 3D Video Quality Assessment,”, 3DTV- conference: the true vision- caption, transmission and display of 3D video (3DTV-CON), 2010, PP.1-4.
[34] Politis, V. De Silva, S. Dogan, C. De Alwis , T. Dagiuklas, and A. M. Kondoz , “ Objective Measurement of Transmission Losses and Compression Artefacts in Stereoscopic 3D Video,”, Communications Workshops (ICC), IEEE International Conference on , 2013,PP.667-671.
[35] Yi Han, Zhenhui Yuan, Gabriel-Miro Muntean, “ No Reference Objective Quality Metric for Stereoscopic 3D Video,”, Broadband Multimedia System and broadcasting (BMSB) ,2014 , PP.1-6.
[36] Yi Han, Zhenhui Yuan, Gabriel-Miro Muntean,“Extended No Reference Objective Quality Metric for Stereoscopic 3D Video,”, Communication Workshop (ICCW),IEEE International Conference on , 2015,PP.1729-1734.
[37] Sawsen Abdulhadi Mahmood , Rana Fareed Ghani , “Objective Quality Assessment of 3D Stereoscopic Video Based on Motion Vectors and Depth Map Features,”, Computer Science and Electronic Engineering Conference (CEEC) , 2015, PP.179-183
[38] Tselios, Athanasios Kordelas, Tasos Dagiuklas, and Andreas Papadaki , “A model of network related QoE for 3D video, ”, Globecom Workshops (GC Wkshps)EEE, 2012, PP.1335-1340.
[39] J. Carreira, L. Pinto, N. Rodrigues, S. Faria, P. Assuncao “Subjective Assessment Of Frame Loss Concealment Methods In 3D Video,”, Picture Coding Symposium (PCS) , 3,2010 ,PP. 182–185.
[40] L.Pinto,J.Carreira , S.Faria, N.Rodrigues,P.Assuncao “Subjective Quality Factors In Packet 3D video,”, in Quality of Multimedia Experience (QoMEX) Third International Workshop,Sept.2011,PP. 149–154 .
[41] Tae-Young Chung, SanghoonSull, and Chang-Su Kim, “Frame Loss Concealment for Stereoscopic Video Plus Depth Sequences.”, IEEE Trans. on Consumer Electronics, vol. 57, PP,. 336-344 , 2011.
[42] Meng Yang, Xuguang Lan, Nanning Zheng, Cosman.P, “ Depth-Assisted Temporal Error Concealment for Intra Frame Slices in 3-D Video.” IEEE Trans. on Broadcasting , vol. 60, PP. 385- 393,2014.
[43] E.Ekmekcioglu, “ Utilization of Motion Similarity in Color-plus-Depth 3D video for Improve Error Resiliency,” , in Information and Automation for Sustainability (ICIAFs), on. 17–19.Dec. 2010, PP. 475– 47.
[44] Khalid Mohamed Alajel, Wei Xiang, and Yafeng Wang,“ Unequal Error Protection Scheme Based Hierarchical 16-QAM for 3-D Video Transmission.” , IEEE Trans. on Consumer Electronics , vol. 58,PP.731–738., 2012.
[45] Wen-Nung Lie , Guan-Hua Lin “Error Concealment for 3D Video Transmission,” , IEEE International Symposium on Circuits and Systems (ISCAS), May. 2013,PP. 2856– 2559.
[46] Bo Yan, Senior Member, IEEE, and Jie Zhou “Efficient Frame Concealment for Depth Image-Based 3-D Video Transmission.”, IEEE/Journal Trans. on Multimedia, vol. 4 ,PP. 936-941, 2014.
[47] Chaminda T.E.R. Hewage, Stewart T. Worrall, Safak Dogan, and Ahmet M. Kondoz “A novel frame concealment method for depth maps using corresponding color motion vectors, ”, vectors,on 3DTV,2008, PP. 149–152.
[48] C.T.E.R. Hewage, S. Worrall, S. Dogan, and A.M. Kondoz “Frame concealment algorithm for stereoscopic video using motion vector sharing,” , IEEE on Multimedia and Expo, 2008 , PP. 485–488.
Downloads
Published
2016-08-07
How to Cite
Ismaiel, W., Osman, N., & Mkwawa, I.-H. (2016). QoE Enhancement for Stereoscopic 3DVideo Quality Based on Depth and Color Transmission over IP Networks: A Review. American Scientific Research Journal for Engineering, Technology, and Sciences, 23(1), 146–171. Retrieved from https://asrjetsjournal.org/index.php/American_Scientific_Journal/article/view/1968
Issue
Section
Articles
License
Authors who submit papers with this journal agree to the following terms.
