Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira. ISSN 0122-1701 y ISSN: 2344-7214

434

Abstract— Video services are becoming more and more popular

for mobile network users and require greater and greater

resources and provisions from telecommunications service

HDS (HTTP dynamic streaming), HLS (HTTP2 live streaming)

and HSS (HTTP smooth streaming) - used in a live video playback

by a user in different test scenarios on an emulated long-term

evolution (LTE) network. Comparison of performance was

carried out using the mean opinion score (MOS) metric calculated

based on ITU-T Recommendation P.1203. This recommendation

focuses on bitstream-based parametric quality assessment of

progressive downloads and adaptive audiovisual streaming

services through a reliable transport protocol. From the results

obtained in the evaluation of the different adaptive transmission

Resumen— El servicio de video es cada vez más popular por parte

de los usuarios de redes móviles, además exige mayores recursos y

prestaciones por parte de los proveedores de servicios de

telecomunicaciones. Para satisfacer la calidad de la experiencia del

servicio suministrado a los usuarios - QoE y mejorar el

rendimiento de las redes, los operadores utilizan diferentes

técnicas de transmisión adaptativa, las cuales presentan

inconvenientes de interoperabilidad entre ellas. En este artículo se

presenta una comparación de las técnicas de streaming DASH

T P.1203. Esta recomendación se centra en la evaluación de la

calidad paramétrica basada en flujos de bits de descargas

progresivas y servicios de transmisión audiovisual adaptativa a

This manuscript was sent on June 11, 2021 and accepted on November 25,

2021.

H-F.Bermudez. Author, He is now with the Faculty of Engineering, Quindio

University, Armenia - Colombia (e-mail: hfbermudez@uniquindio.edu.co).

través de un protocolo de transporte fiable. A partir de los

resultados obtenidos en la evaluación de las diferentes técnicas de

Video.

I. INTRODUCTION

ecent years have seen an increase in the number of users

using video streaming services to play content that is either

live (LVS - live video streaming) or on demand (VoD -

video on demand). The number of users and mobile devices

(e.g. laptop, smartphone, tablet) that access these types of

service through wireless mobile networks is also increasing [1].

Moreover, the wide variety of devices offered by the market

increases the variability of characteristics between devices:

certain network management tasks - the monitoring and control

of bandwidth, delay, jitter, throughput and packet loss - that

ensure an adequate level of quality for users. These tasks are

much more complex in a wireless environment where

difficulties include wireless signal coverage, a high rate of

packet loss, and instability of the wireless channel, these being

the result of phenomena pertaining to the channel itself, such as

multipaths, fading, interference, and noise [2], [3], [4]. These

can considerably affect the performance of wireless mobile

networks and bring down the quality of experience evaluation

University, Armenia - Colombia (e-mail: wycampo@uniquindio.edu.co).

University, Armenia - Colombia (e-mail: eastaiza@uniquindio.edu.co).

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

most popular video services on the Internet, for example, VoD

or LVS use adaptive transport techniques in preference to

Hypertext Transfer Protocol (HTTP) [6]. These techniques

adapt the speed of video data transmission to the network

available bandwidth, so that even while in progress they adopt

the most suitable video encoding settings given the conditions

of end user and data path [7]. The following may be considered

among the most popular video adaptation techniques: (i) HTTP

smooth streaming (HSS) from Microsoft [8], (ii) HTTP2 live

streaming (HLS) from Apple [9], (iii) HTTP dynamic streaming

mobile networks, LTE technology since its inception has

progressively incorporated improvements in response to the

increase in mobile network traffic, seeking to enhance the video

experience for its end users. These enhancements support

features such as fast initial startup, bandwidth efficiency,

adaptive bit rate switching, adaptation to content delivery

network (CDN) properties, reuse of HTTP servers and caches,

reuse of existing media playback engines, support for delivery

of services on demand, live and displaced in time and simplicity

for wide adoption [12].

several recommendations, among which are ITU-T P.910 and

P.911, [14], [15], which look to obtain the average quality

rating of all users for a particular video transmission. This rating

is known as the mean opinion score (MOS). Objective quality

metrics, on the other hand, are algorithms designed to

characterize video quality and predict the MOS of users. These

metrics are not based on service user surveys but on parameters

measured within the network or video stream itself. Thus,

Recommendations P.1201 (ITU-T, 2012), P.1202 [16] and

P.1203 [17] provide an overview of models for non-intrusive

The paper is organized as follows. In Section II, studies

related to techniques of sending video through adaptive

streaming are analyzed. Section III presents the methodology.

In section IV, the results and their discussion are found and

finally, Section V presents the conclusions and identifies future

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

among end users, through QoE estimates, using for the first

PC1 hosts the 4G-LTE network emulated with the LENA

library on NS3. This LTE network is composed of a remote

Network Gateway), an eNB base station (evolved NodeB) and

an end user equipment (UE) that acquires the role of mobile

into the system by the LVS Wowza streaming server (PC2)

PC3 is used for the reproduction and monitoring of the

(Hardware in the Loop) platform is used to enable

communication between PC2-Host-Remote and UE-PC3,

components that receives and delivers data to real end systems,

PC2 and PC3.

Three test scenarios were defined in order to carry out

different network delay times {0, 25, 50, 75, 100, 125 and 150

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

CODING ADJUSTMENTS

Description

IV. RESULTS AND DISCUSSION

Below, a comparison is found of the four HTTP adaptive

streaming techniques for the different test scenarios.

the P.1203 standard. All measurements were made for the

provision of the LVS service in the emulated 4G network

described above.

Figs. 2-5 present the QoE estimate from MOS calculations

for the different test scenarios. Fig. 2 shows for Test Scenario 1

how DASH and HLS adaptive streaming techniques performed

best when controlled delays of less than or equal to 50 ms are

introduced to the emulated system, with HSS and HDS doing

less well. It can be seen that, after 75 ms of delay, the QoE

estimate for the different techniques studied is considered poor,

other streaming techniques when different packet loss values

it is clearly seen that for values less than 0.5% packet loss, the

QoE estimate through MOS is greater than 4, considered a good

evaluation, i.e. in the assessment range: 4-4.9. On the contrary,

for values greater than 0.5% packet loss, QoE evaluation was

considered as bad, i.e. in the assessment range: 2-2.9, rendering

the service unviable from the user point of view.

Fig. 2. Estimated QoE in scenario 1 (user located 30m from the eNB to which

different network delay times were varied in a controlled manner).

seen that the performance measured by estimating QoE was fair

(evaluation range: 3-3.9) for speeds of 1 to 2.5 m/s. Only HSS

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

Fig. 4. Estimated QoE in scenario 2 (UE moving away from eNB with uniform

speed and direction).

Fig. 5 shows the results obtained for test scenario 3. It was

observed how the DASH technique again returned the best

performance (MOS on the scale 4-4.9) in terms of QoE when

Fig. 5. Estimated QoE in scenario 3 (UE moving away from the eNB with

random speed and directions).

Figs.6-10 show results obtained for stalling events for

different test scenarios, as defined in the P.1203 standard as

having a high impact on QoE estimation [24]; Fig. 6 and 7 show

duration and quantity of stalling events for test scenario 1. It

was observed, in Fig. 6, how as controlled delays greater than

100 ms are introduced in the emulation system, a longer

duration of stalling events is encountered in DASH and HDS

techniques. On the other hand, the number of stalling events

DASH performance improves from 100 ms to 125 or 150 ms.

can be seen that, although there were fewer stalling events, their

QoE estimate is very low for DASH in delays above 100 ms, as

evidenced in Fig. 2, where a QoE estimate of bad was observed

(evaluation range: 1-2). In conclusion, it can be stated that for

DASH and HDS streaming techniques, the higher the delay, the

fewer stalling events but the longer the duration. While for HLS

and HSS, the more delay the more stalling events but shorter.

For test scenario 2, where the UE moves away from the eNB

with uniform speed and a single direction, it was observed in

Figures 8 and 9 that the duration and number of stalling events

Fig. 6. Duration of stalling events for emulation scenario 1.

Fig. 7. Number of stalling events for emulation scenario 1.

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

Fig. 8. Duration of stalling events for emulation scenario 2.

Fig. 10. Stalling events for emulation scenario 3.

Fig. 11. Initial buffer for emulation scenario 2.

Meanwhile, with respect to the initial buffer parameter, Fig.

11 shows its duration for scenario 2, when different values of

percent packet loss were entered in a controlled manner. It was

observed that, as the percentage of packet loss increases, the

associating these results with the QoE estimates, it could be

A comparative study of the performance of different video

streaming techniques based on QoE was carried out through a

controlled variation of such QoS parameters as delay and packet

loss in different static and dynamic experimentation scenarios

defined within the research. Regarding the delay parameter, the

following observations were made: (i) the value of 50 ms was

considered to be the maximum limit to ensure quality of LVS

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

operators present due to the use of different adaptive streaming

techniques. Currently, the authors are conducting research to

establish relationships between different QoS metrics and the

objective estimate of QoE for the streaming service;

additionally, this research will allow extending studies and

emulation scenarios to other access technologies, such as WiFi

networks, LTE-A Pro and 5G.

Cambridge University Press, 2005.

[9] Apple, “HTTP Live Streaming (HLS) - Apple Developer,” 2014.

[24] M. Taha, J. Lloret, A. Ali, and L. Garcia, “Adaptive video streaming

testbed design for performance study and assessment of QoE,”

2018, doi: 10.1002/dac.3551.

[25] F. Abdurrahman et al., “Real Time Video Streaming over NS3 based

Emulated LTE Networks,” 2014, vol. 4, pp. 659–663.

[26] H.-F. Bermudez, R. Sanchez-Iborra, J. L. Arciniegas, W. Y. Campo,

and M.- Cano, “Performance validation of NS3-LTE emulation for live

video streaming under QoS parameters,” in 2017 IEEE 13th

[28] Blender, “Big Buck Bunny » Download,” 2008.

https://peach.blender.org/download/ (accessed Sep. 09, 2020).

[29] ITU-T, “Y.1541 : Objetivos de calidad de funcionamiento de red para

servicios basados en el protocolo Internet,” Dec. 2011.

https://www.itu.int/rec/T-REC-Y.1541-201112-I/es (accessed Nov. 08,

2020).

Scientia et Technica Año XXVI, Vol. 26, No. 04, diciembre de 2021. Universidad Tecnológica de Pereira.

Telecommunications in 2010 and PhD in Telematic

Engineering in 2020, where he received a laureate mention for

his doctoral research thesis. He made a doctoral research stay at

the Polytechnic University of Cartagena UPCT in Cartagena

ORCID: https://orcid.org/0000-0002-8101-3764

His fields of interest are: Software Defined Networking SDN,