documentsn.com

3. Wireless Video Streaming TestsAfter the definition of the system architecture and the practical realisation of the hardware prototypeswe set up a testing campaign to verify the quality of service achievable by our wireless videostreaming testbed. The tests started with the selection of a set of video sequences at HDTVresolutions. The video content source we used was formed by uncompressed (raw) YUV sequences(1920x1080 pixels), part of the Philips portfolio of videos inside the Innovation Lab of Philips. Theyare also used for HDTV plasma and LCD experimental panels quality checks.Each video frame is saved in uncompressed YUV format and putting a lot of them in one containerfile (DPX for example) results in a video sequence of uncompressed HDTV material. The sequencesused come from movies and sport events, as the PULSERS wireless DVI setup will mostly be usedin a home environment.We used 2 different video sources:Video Source n.1:Philips HDTV Promotion video showing recent movie scenes and sport event (47 seconds) Shows a mix of very fast changing action scenes as well as slower moving scenes Detailed explosions, fast moving objects and panoramic overviews with lots ofwarms colours and flashing lights.Figure 3 Sample screenshots from video sequence n.1Video Source n.2:HDTV Air show (32 seconds) Shows a mix of very fast changing action scenes as well as slower moving scenes Detailed zoom-in of aircraft and people faces, detailed clouds

Figure 4 Sample screenshots from video sequence n.2In order to test the different possible options and resolution we pre-processed the video sequencesand pre-recorded different files representing the video sequences at different resolutions and withdifferent compression ratios.4. Streaming Quality Tests ResultsWe were able to perform both a physical layer test related to the wireless part of the high qualityvideo streaming platform (effective throughput) and a full quality of service analysis if the end-toend gigabit video link. The main results are presented in the following paragraphs.Initial reference tests – effective PHY throughputIn the initial (reference setting) wireless video testing set-up we connected the transmitting streamingPC to different potential physical layer wireless transceivers interfaces, among which commercial802.11g equipment, 802.11 Pre-N equipment, Ultrawaves UWB transceivers, free space opticaltransceivers. We also set a reference point with a wired Gigabit Ethernet IP connection.The first metric we selected for our video tests was the effective throughput, which combinesthroughput with coverage area of the wireless network. All of our platforms were tested on theirthroughput in Mbps over distances from 1, 2, 3, 4, 6, 8, 11 and 15 meters. We could measure theexact data rates of the platforms that we used for the compressed scenario using the software packageIPerf. IPerf was originally developed as a modern alternative for measuring TCP and UDPbandwidth performance over an IP network. It is therefore a good tool to measure the effectivethroughput of a wireless network, as in the PULSERS test configuration. IPerf is also a tool tomeasure maximum TCP bandwidth, and allows the fine tuning of parameters connected to the UDPprotocol. Iperf reports in an easy format bandwidth, delay jitter, datagram loss.

Figure 5: Initial wireless video testing set-upThroughput rates at measured ring distancesWired LAN Gigabit Ethernet10000Throughput in MbpsWLAN 802.11g1000WLAN 802.11 Pre-N100ULTRAWAVES UWB (1stgen.) platformFree space optical link(uncompressed)10PULSERS UWB (2nd ters11meters15meters1DistanceFigure 6: Effective (uncompressed) throughput from different physical layers for the PULSERS VHDR PlatformFinal tests – video sequences QoSFor the final tests we placed the receiving platform on a stationary table below a flat screen highquality Philips TV. The transmitter was moved on a trolley along a straight line at constant spacingaway from the receiver, as shown in Figure 7.

Figure 7 PULSERS VHDR: TX (bottom in the picture) and RX (top) components 6 meters apart.As explained in the Chapter 2, different metrics exist and can help in measuring the quality of avideo signal. What has been found however is that the measured subjective quality of a videosequence is not always perfectly correlated to the objective inverse mean square error or to the peaksignal to noise ratio. Therefore in order to evaluate the quality of our wireless video streamingsolution we called in a panel of experts that scored each particular sequence seen on screen.Having registered good and stable connectivity results in the throughput of the PULSERS UWBplatforms up to 5 meters, we were expecting good QoS in the video sequences transmissions as well.But the subjective QoS tests at this distance were very disappointing. We therefore decided to reducethe test distance between transmitter antenna and receiver antenna to 1,5 meter, which was themaximum distance at which we could have error-free MPEG-2 transmissions. The MotionJPEG2000 content was stable up to about 3 meter, before noticeable blurring artefacts came on thescreen. But for the sake of comparison we had to keep the transmitter-receiver distance fixed to 1.5meter for all different physical layers. This 1,5-meter distance between transmitter and receiver alsorepresents a typical cable replacement distance.The first set of tests were performed in ideal conditions, without interference of other wireless links.The second set of tests were performed in conditions where we interfered the wireless links byplacing obstructive objects (person, flowers and cabinet) between transmitter-receiver. To addemulated errors in the Wired LAN (reference) setup we used a utility that overloaded the traffic overthe Wired LAN link called ‘Network Traffic Emulator’ from Nsasoft.

We compressed our Philips promotion video from raw video into H.264, MPEG-2 and MotionJPEG2000 formats. We made compressed content that would be able to run on the WLAN MIMOpre-N, Wired LAN and PULSERS UWB platforms.In the table below you see the description of bit rates used for each type of HDTV 1080i compressedvideo on the different platforms, which were evaluated. Note that we were not able to reproducevisual lossless video (PSNR 40 dB) for the WLAN MIMO 802.11 pre-N platform. In that case wewere bound to the hardware platform’s maximum throughput of 42 Mbit/s, which was short ofachieving a full PSNR of 40 dB.Final subjectiveQoS test - 1080i content descriptionPSNR 40dB (except for MIMO)MPEG-2H.264M-JP2000Mbit/sMbit/sMbit/sWired Gigabit LAN8050150WLAN MIMO pre-N4040408050150PlatformsPULSERS UWBTable 3 Final subjective QoS test - content descriptionThe marks from 1-10 were assigned exclusively for the subjective visual quality of the video on thereceiving display. Effects like skipping of frames (all), MPEG artefacts (MPEG-2 and H.264),motion blurring (Motion-JPEG2000) and complete black screens (FS optical link) in the sequencesaffected the subjective impressions of the testers. The tested video resolutions were the ones allowedby the related platforms. Only Gigabit Ethernet, PULSERS UWB platform and Free-Space opticallinks were able to transmit sequences at high quality resolutions (HDTV). The other (compressed)links were limited by the underlying radio technology.QoS marksQoS subjective tests without interference / ressedWired GigabitLANWLAN MIMOpre-NPULSERSUWBFS optical linkPlatformsFigure 8 Final PULSERS MOS in LOS at 1,5m distance – comparison of different PY and compressions –LOSconditionsWhen our subjective group took place in front of the display for the analysis of perceived quality forthe HDTV 1080i content, they were asked to give marks for the visual quality of the different videocompression types used and the different platforms used. The results were then averages and formed

the so callled Mean Opinion Score (MOS), the basis for comparison of the quality of service offeredby our cascaded compression and wireless transmission HDTV link.As we can see from Figure 8, our evaluators perceived the uncompressed Free-space optical link asthe absolute best, as expected. Close to this platform mark we see that the M-JPEG2000 and H.264compressed video on the PULSERS UWB and Wired LAN platforms are perceived almost with thesame quality as the uncompressed free space optical link. M-JPEG2000, H.264 and MPEG-2 wereperceived lower on the WLAN MIMO system, confirming the fact that this platform was not capableof supporting high quality/low compressed video data, compared to the Wired LAN and PULSERSUWB platform.When the QoS tests in LOS conditions were completed, the test group sit in front of the display againand they were asked to give new marks for the visual quality of the different HDTV 1080i videocompression types. This time the radio link was not in full line of sight, but was disturbed byobstructions and interference in the radio path as introduced with hand waving and walking troughthe link as it might happen in normal domestic situations. The results for these tests in Non Line ofSight (NLOS) obstructed conditions are mentioned in the Figure 9.QoS marksQoS subjective tests with interference / ssedWiredWLAN MIMOGigabit LANpre-NPULSERSUWBFS opticallinkPlatformsFigure 9 Final PULSERS MOS in LOS at 1,5m distance distance – comparison of different PY and compressions– with obstructionsIn the case of severe interference we can notice how the free space optical link was not judged asideal anymore. This was because it showed black frames as soon as the line of sight path betweentransmitter and receiver was blocked and this is a noticeable and clearly unacceptable behaviour.When we take a look at the M-JPEG2000 ratings we see that people prefer this compression methodinstead of MPEG-2 and H.264 on all platforms. This can be explained because of the different visualerrors appearing on the screen and noticed by our test group. People found much more annoying tosee the MPEG block artefacts coming on the screen and blocking a lot of visual information, ratherthan the blurred errors coming of the M-JPEG2000 final visual compressed videos (see Figure 10).

Figure 10: Different types of errors: blocky (left) for the MPEG-2 and blurred (right) for the M-JPEG2000 codecEven in very badly obstructed conditions, our test people were still able to see the content of theMJPEG 2000 video and only in some places in the screen the image was a bit blurrier, which isperceived as acceptable given the conditions.We also registered a noticeable difference between the ratings for M-JPEG2000 and those for H.264in the UWB PULSERS platform. This could be due to the fact that high bandwidth M-JPEG2000compared to MPEG-2 and H.264 is more resilient to block errors, with an efficient use of entropyencoding and frame by frame compression with no memory that could cause in the H.264propagation of errors.5. ConclusionsThe IST Sixth Framework Integrated project PULSERS has specified and developed an innovativeexperimental hardware architecture to allow high quality wireless video transmission and to helpverify the feasibility of a cable-less replacement of an advanced wired video transmission protocol,known as Digital Video Interface (DVI).The cascaded combination of fast compression (based on M-JPEG 2000, MPEG-2 and H.264techniques) and fast UWB radio links has been analysed and compared with reference fast wiredlinks (Gigabit Ethernet) and wireless Optical Links. The quality of the resulting link has beenmeasured both in terms of throughput and subjective perceived quality (Mean Opinion Scoreanalysis).From our experiments we can conclude that: For low quality (VGA-type) video, all the analysed radio platforms (802.11g, 802.11N,UWB) in conjunction with a hardware compression engine can deliver reasonably long radiolinks (typically over 10m), even if the subjective quality of the received video is notexcellent. For high-resolution videos (HDTV or above) only free-space optical links and the UWBPULSERS platform have been able to match the quality of wired solutions. The subjectivetesting confirmed that the quality of wireless videos is not yet exceptional in all conditions,but is already perceived as good for short (1.5m) distances.

The wireless DVI system implemented and tested in PULSERS during 2005 was top of the class andwas a good testbed to demonstrate feasibility of what could become a killer application of the future.Future work should now start also addressing possibilities for lower cost and lower complexitycompression techniques which could be coupled with the faster UWB platforms expected on themarket in the next few years.References[1] Sixth Framework Integrated Project PULSERS, IST FP6 506897, www.pulsers.net[2] Digital Display Working Group, “Digital Visual Interface DVI”, 02 April 1999[3] Domenico Porcino (Philips Research, UK), Bram van der Wal (Philips Consumer Electronics,NL), Ying Zhao (Department of Electronics Engineering, Tsinghua University, China), "Asimple architecture for a Wireless DVI system", IEEE International Conference on UltraWideband 2005 (ICU 2005), Zurich, Switzerland, September 5-8, 2005[4] Zhou Wang, Alan Bovik, Hamid Sheikh, Eero Simoncelli, “Image Quality Assessment: FromError Visibility to Structural Similarity”, IEEE Transactions on Image Processing, Vol 13, N4,April 2004

HD-SDI Host CPU PCI bus PCI bus RAM disk with stored M-JPEG2000 content Memory bus RAM disk with received M-JPEG2000 content M-JPEG 2000 decoder M-JPEG 2000 decoder Video signal merger Y bus CbCr bus Clock data bus Host CPU PCI bus PCI bus Memory bus HDTV video out of HD-SDI HD-SDI to DVI convert