MASTER THESIS TITLE: Implementation of a Low Cost Video Wall using Raspberry Pi devices MASTER DEGREE: Master of Science in Telecommunication Engineering & Management AUTHORS: Ramón García García Carlos Viñas Rodríguez DIRECTOR: Marc Fernández Vanaclocha SUPERVISOR: Toni Oller Arcas DATE: 24th April 2014
Thanks to Marc Fernández for guiding us during this project. Thanks to i2CAT for providing us the necessary material. Thanks to Toni Oller for supervising the project.
Overview Keywords: Video Wall, Raspberry Pi, H.264, Streaming Description: This project is a part of a more extensive project developed by i2CAT Foundation. The aim of this master thesis is to build a Low Cost Video Wall using Raspberry Pi devices. A Video Wall can be defined as an array of monitors or screens synchronized to display content, simulating a bigger screen.
Resum Paraules clau: Video Wall, Raspberry Pi, H.264, Streaming Descripció: Aquest projecte forma part d’un projecte més gran desenvolupat per la fundació i2CAT. L’objectiu d’aquesta tesi és construir un Video Wall de baix cost emprant els dispositius Raspberry Pi. Podem definir un Video Wall com un conjunt de monitors o pantalles sincronitzades per mostrar contingut de vídeos o imatges, simulant una pantalla de major dimensió.
INDEX CHAPTER 1. INTRODUCTION........................................................................ 11 1.1. Work Structure ................................................................................................................... 11 1.2. What is a Video Wall? ........................................................................................................ 12 1.3. Project Objectives ............................................................................................................
4.2.1. Milestone 2 - Sending a HD/Full HD video to one Raspberry Pi ............................. 46 4.2.2. Milestone 3 - Sending a real time web cam video flow to one Raspberry Pi .......... 50 4.2.3. Milestone 4 - Receiving a Full HD video / WebCam and playing back ................... 51 4.2.4. Milestone 5 - Sending a Full HD video using MPEG-2 TS ...................................... 52 4.2.5. Milestone 6 - Receiving a Full HD video using MPEG2-TS .................................... 53 4.3.
CHAPTER 10. COMPARISON WITH COMERCIAL PRODUCTS ................... 85 10.1. SAGE Overview ................................................................................................................ 85 10.2. SAGE Features and Tiled Display Wall Configurations ............................................... 86 10.3. SAGE Minimum Systems Requirements and Minimun Budget................................... 86 10.4. Minimum Systems Requirements and Budget of this project .....................................
Figure list Figure 1. 3x3 Video Wall ............................................................................................. 12 Figure 2. System components .................................................................................... 14 Figure 3. HP Pavilion DV6 Notebook PC .................................................................... 16 Figure 4. Debian OS Logotype .................................................................................... 16 Figure 5.
Figure 54. Backplane capacity Milestone 11 ............................................................... 80 Figure 55. Commercial use scenario........................................................................... 81 Figure 56. Start up screen Raspberry Pi devices ........................................................ 82 Figure 57. Server VideoWall GUI ................................................................................ 82 Figure 58. SAGE working demonstration ...........................
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 1. INTRODUCTION This chapter focuses on explaining in detail the work structure of this project, what a video wall is, the objectives of the project and the main features of the components employed in this thesis. 1.1. Work Structure The work structure of this project has been organized in 12 chapters. For each one, a brief description of the work done is provided.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Chapter 10 provides a comparison of the system implemented with current commercial products (SAGE) in terms of cost, power consumption and backplane to demostrate that the system can compete in these three aspects. Chapter 11 shows the timeline of the project and how tasks and time have been organized.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Video walls are sometimes found in control rooms, stadiums, and other large public venues. However,it is also possible to find simple video walls that can be driven from just using a set of personal computers and a set of monitors screen at home. Video Wall Latest Advancements Researchers around the U.S.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The system displays high-resolution still images, HD videos, live HD video streams and PC applications. Multiple feeds can be displayed on the wall simultaneously and users can reposition and resize each feed in much the same way they move and resize windows on a PC desktop. Each feed can be scaled up for viewing on several monitors or the entire wall instantly depending upon the user’s discretion. 1.3.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Component Description and Functions Source The source is a notebook that acts as a server and its function is to stream encoded audio and video flows to the receivers using some scripts. Switch / Router This element is in charge of distributing the flow sent by the server to the different receivers. Receivers The receivers are the five Raspberry Pi devices.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 1.3. Server Features The server in this project is a HP Pavilion DV6 Notebook PC. Figure 3. HP Pavilion DV6 Notebook PC The server has the following features: - CPU: Intel Core i3 M330 2.13 GHz (4 CPUs) - RAM: 4 GB - GPU Card: NVIDIA GeForce G105M - Operating System (OS): Debian GNU/Linux 7.1 (wheezy) - Hard Disk Drive Size: 78 GB Logic Partition 1.3.1. Server OS The server uses Debian GNU/ Linux 7.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The Debian Project is an association of individuals who have made common cause to create a free operating system. 1.4. Client Features The Raspberry Pi is a credit-card sized computer that plugs into the TV and a keyboard. It is a capable little PC which can be used for many of the things that a desktop PC does, like spreadsheets, word-processing and games.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Item Minimum recommended specifications 1 - SD card Minimum size 4Gb; class 4 (the class indicates how fast the card is). It is recommended using branded SD cards as they are more reliable. 2a - HDMI to HDMI / DVI lead HDMI to HDMI lead (for HD TVs and monitors with HDMI input) or HDMI to DVI lead (for monitors with DVI input).
Implementation of a Low Cost Video Wall using Raspberry Pi devices - 2 x USB 2.0 , 10/100 MBIT - HDMI output. Some small differences that will affect the performance have been encountered between both Raspberry Pi devices.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 2. PROTOCOL ARQUITECTURE This chapter is centered on the different standards of real time video transmission. The different protocols used in the project to implement the video wall are explained. The following two figures describe the set of protocols and video codecs used in the project. In the following sections the most relevant aspects of H.264, Real Time Protocol (RTP) and Real Time Control Protocol (RTCP) are explained.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.1. Video Compression techniques Video applications require some form of data compression to achieve reasonable precondition for storage and transmission. The digital video compression is one of the main issues in digital video coding, enabling efficient distribution and interchange of visual information. Video codecs are devices that are used to compress and decompress as well as to encode and decode video streams.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Figure 10. Block diagram of video compression In addition to the previous compression technique, the temporal redundancy between frames can be utilized for further compression. The basic method is to calculate the prediction error between corresponding blocks in the current and previous frames. The error values are then sent to the compression process. Compressed frames generated using predictions are usually called P-frames.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.2. H.264 video codec H.264/AVC8,, also known as MPEG-4, is the latest international video coding standard. It was jointly developed by the ITU-T (International Telecommunication Union) “Video Coding Experts Group” (VCEG) and the ISO/IEC (International Organization for Standardization/ International Electrotechnical Commission) “Moving Picture Experts Group” (MPEG).
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.2.2. Structure The H.264/AVC standard defines two conceptual layers: the “Video Coding Layer” (VCL) and the “Network Adaptation Layer” (NAL). The first one defines the efficient encoding representation of the video whereas the second one is designed to provide “network friendliness” by facilitating the ability to map H.264/AVC VCL data to different transport layers.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.3. Real Time Protocol Operation 2.3.1 What is RTP RTP18 provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services; such as videoconferences. RTP does not address resource reservation and does not guarantee quality of service for real-time services.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.3.2. RTP functions The RTP protocol carries out a control task on media data flow adding, for example, temporal marks and sequence numbers in each data packet to identify them in reception. Furthermore, the protocol is capable of identifying transported information according to the type (audio or video) and separate it in diverse flows.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Version Number (V) (2 bits) Indicates the version of the protocol. The current version is 2. Padding (P) (1 bit) It is used to indicate if there are extra padding bytes at the end of the RTP packet. Extension(X) (1 bit) It indicates presence of an extension header between standard header and payload data. CSRC Count (CC) (4 bits) Contains the number of Contributing Source identifiers that follow the fixed header.
Implementation of a Low Cost Video Wall using Raspberry Pi devices SSRC (32 bits) Synchronization source identifier uniquely identifies the source of a stream. The optional header extensions are described below: Extended Sequence Number (16 bits) The high order bits of the extended 32-bit sequence number, in network byte order. Line No (15 bits) Scan line number of encapsulated data, in network byte order.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.4. Real Time Control Protocol operation This section aims to provide a global vision of RTCP operational (RFC 3550), their functions and the packet types contemplated. RTCP18 is introduced as an adjacent control protocol to the RTP. The RTCP functionality is to report the network state to different nodes of communication.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Session Control Information Finally, RTCP can carry session control information. For example, it could carry information regarding the participant identification and displayed to the user. The first two functions must be met in all possible RTCP scenarios, but are especially important in multicast sessions.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 2.5. MPEG-2 Transmission The MPEG-215 (Moving Picture Expert Group-2) standards define how to format the various component parts of a multimedia program (which may consist of: MPEG-2 compressed video, compressed audio, control data and/or user data). It also defines how these components are combined into a single synchronous transmission bit stream. The process of combining the streams is known as multiplexing.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Figure 13. Combining elementary streams from encoders into a TS or a PS The Program Stream is widely used in digital video storage devices, and also where the video is reliably transmitted over a network (e.g. video-clip download). Digital Video Broadcast (DVB) uses the MPEG-2 Transport Stream over a wide variety of underlying networks.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Transmission of the MPEG-TS Although the MPEG TS may be directly used over a wide variety of media, it may also be used over a communication network. It is designed to be robust with short frames, each one being protected by a strong error correction mechanism. It is constructed to match the characteristics of the generic radio or cable channel and expects an uncorrected Bit Error Rate (BER) of better than 1010.
Implementation of a Low Cost Video Wall using Raspberry Pi devices particular PID value of 0x1FFF is reserved to indicate that the packet is a null packet (and has to be ignored by the receiver). The two scrambling control bits are used by conditional access procedures to encrypt the payload of some TS packets. - Two adaption field control bits which may take four values: - 01: No adaptation field, payload only. - 10: Adaptation field only, no payload. - 11: Adaptation field followed by payload.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 3. FRAMEWORKS During this project, there has been an investigation on different frameworks available to form the video wall. The following are the four frameworks to focus on: GStreamer, FFmpeg, VLC and OMXPlayer.
Implementation of a Low Cost Video Wall using Raspberry Pi devices GStreamer can bridge to other multimedia frameworks in order to reuse existing components (e.g. codecs) and use platform input/output mechanisms: - Linux/Unix: OpenMAX-IL (via gst-omx) - Windows: DirectShow - Mac OS X: QuickTime Comprehensive Core Library - Graph-based structure allows arbitrary pipeline construction. - Based on GLib 2.0 object model for object-oriented design and inheritance.
- Implementation of a Low Cost Video Wall using Raspberry Pi devices Audio: integer and float audio in various bit depths and multichannel configurations. Extensive Development Tools - gst-launch command-line tool for quick prototyping and testing. - A lot of documentation, including partially completed manual and plugin writer's guide. - Large selection of test programs and example code in each module. - Access to GStreamer API with various programming languages. 3.2.
Implementation of a Low Cost Video Wall using Raspberry Pi devices - ffserver is a multimedia streaming server for live broadcasts. - ffplay is a simple media player based on SDL and the FFmpeg libraries. - ffprobe is a simple multimedia stream analyzer.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 3.3. VLC VLC media player54 is a portable, free and open-source, cross-platform media player and streaming media server written by the VideoLAN project. VLC media player supports many audio and video compression methods and file formats, including DVDVideo, video CD and streaming protocols. It is able to stream over computer network and to transcode multimedia files.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 3.4. OMXPlayer OMXPlayer is a video player specifically made for the Raspberry Pi devices. It takes advantage of the GPU to decode H.264 videos and it comes with the standard Raspbian Wheeze distribution. OMXPlayer has been used in the Raspberry Pi devices to play back the streamed videos. OMXPlayer is installed by default on the Raspberry Pi.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The following key commands do the following actions when pressed while playing a video in OMXPlayer.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 4. MILESTONES This chapter will describe the different milestones defined in this project and how they are achieved by using the different frameworks available. The work methodology employed is the following: 1. Defining the deployment scenarios. 2. For each scenario, defining different milestones. 3. For each milestone, trying to implement a solution using the available frameworks. 4.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.1.1. Milestone 1 - Playing back a H.264 Full HD Video GStreamer After some investigation and testing, how to create a pipeline able to play back a H.264 1080p video was discovered: gst-launch-1.0 filesrc location= /home/pi/Monsters.mp4 ! qtdemux name=dmux dmux.video_0 ! queue ! decodebin ! autovideosink dmux.audio_0 ! queue ! aacparse ! faad ! alsasink The description of this pipeline is simple. The command gst-launch-1.
Implementation of a Low Cost Video Wall using Raspberry Pi devices FFmpeg Due to there is no HW Support of FFmpeg in the Raspberry Pi, it was decided to use the GStreamer and OMXPlayer reception to play back the videos on the Raspberry Pi. The same reason applies to the rest of reception milestones in this chapter. VLC There is HW Support of VLC in the Raspberry Pi60 but due to the complexity and the constant request of multiple third party libraries, this option was impossible to implement.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.2. Deployment scenario 2 The deployment scenario 2 is the following: Figure 20. Deployment scenario 2 The scenario is composed by a computer that will act as a server, an Ethernet Cat 5e cable to connect the computer with the Raspberry Pi, a HDMI cable and a monitor screen. In this scenario, the following milestones will be performed: - Sending a HD/Full HD video to one Raspberry Pi.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.2.1. Milestone 2 - Sending a HD/Full HD video to one Raspberry Pi GStreamer The aim is to find out how to stream a H.264 Full HD video over the network using the RTP protocol. gst-launch-1.0 filesrc location=Videos/Monsters.mp4 ! qtdemux name=dmux dmux.video_0 ! queue ! rtph264pay config-interval=1 ! udpsink host=192.168.137.3 port=5002 dmux.
Implementation of a Low Cost Video Wall using Raspberry Pi devices To begin writing a pipeline the command ffmpeg has to be used. The parameter –re allows ffmpeg to read input at native frame rate. It is mainly used to simulate a grab device or live input stream (e.g. when reading from a file). Should not be used with actual grab devices or live input streams (where it can cause packet loss). By default ffmpeg attempts to read the input(s) as fast as possible.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The range of the quantizer scale is 0-51: where 0 is lossless, 23 is default, and 51 is worst possible. A lower value means a higher quality and a subjectively sane range is 18-28. Considered 18 to be visually lossless or nearly, it should look the same or nearly the same as the input but it is not technically lossless.
Implementation of a Low Cost Video Wall using Raspberry Pi devices value. The value increases up to 28 (the limit value inside the sane range) and the results are the following: Figure 23. FFmpeg statistics (CRF applied) As expected, bitrate decreases and the fps increases but it is still not enough to fluently play back the video in reception. Increasing CRF up to insane value, correct fps rate is achieved but quality becomes a disaster.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.2.2. Milestone 3 - Sending a real time web cam video flow to one Raspberry Pi This milestone has been implemented only with GStreamer framework. The aim is to find out how to stream a WebCam video flow from the server to the client. The resolution of the server web cam is 640x480. gst-launch-1.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.2.3. Milestone 4 - Receiving a Full HD video / WebCam and playing back This milestone has been implemented only with GStreamer framework. This pipeline describes the inverse process done in Milestones 2 and 3. The aim is to decode the H.264 encoded video in the server and play it back in the receiver without latency and without reducing quality. gst-launch-1.0 udpsrc uri=udp://192.168.137.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.2.4. Milestone 5 - Sending a Full HD video using MPEG-2 TS GStreamer The aim of this pipeline is to be able to send the video using a RTP MPEG-2 TS container. When done, it will be possible to use this project for other types of receivers that use this type of container. gst-launch-1.0 filesrc location= Descargas/FullHD/Monsters.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The results obtained with VLC were not satisfactory, even using the SDP file. 4.2.5. Milestone 6 - Receiving a Full HD video using MPEG2-TS GStreamer This pipeline describes the inverse process done in Milestone 5. The aim is to decode the H.264 video over RTP MPEG2-TS container encoded in the server and play it back in the receiver without reducing quality. gst-launch-1.0 udpsrc uri=udp://192.168.137.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.3. Deployment scenario 3 The deployment scenario 3 is the following: Figure 25. Deployment scenario 3 This is the first deployment scenario in which the audio will be streamed and processed. In the figure above the speaker is connected to the Raspberry Pi device using a 3.5 mm jack to provide the stereo audio.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.3.1. Milestone 7 - Extracting and sending the audio stream from a Full HD video This milestone has been implemented only with GStreamer framework. As commented in Milestone 2, some problems with audio stream appeared. After investigating, it was found how to stream the audio contained in a Full HD video correctly. The pipeline able to do it is the following: gst-launch-1.0 filesrc location=Videos/DesMe2.mp4 ! qtdemux name=dmux dmux.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.4. Deployment scenario 4 The deployment scenario 4 is the following: Figure 26. Deployment Scenario 4 This is the last deployment scenario that combines all the previous ones. It is composed by the server, a router to forward the audio and video flows to the corresponding Raspberry Pi device and monitor screens and speakers to visualize the video and listen to the audio.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.4.1. Milestone 9 - Splitting and sending Full HD video to Raspberry Pi devices GStreamer In this case there are 2 Raspberry Pi, each one connected to a different monitor screen. In addition a router (or switch) is needed in between to send two video flows (one to each Raspberry). It is necessary because our server only has one Ethernet port.
Implementation of a Low Cost Video Wall using Raspberry Pi devices original video for each Raspberry. Obviously, the server must have better features than the ones used in this project. Figure 27. Video cropping example Now decoding is done using the H.264 encoder and the first video stream is encapsulated and sent through the network. Exactly the same is done in parallel for the other part of the video. The element t is used to separate the two video flows contained in the same pipeline.
Implementation of a Low Cost Video Wall using Raspberry Pi devices FFmpeg The aim of the pipeline defined below is to split the video into two different video flows and send them to the respective Raspberry Pi devices. It is important to say that when using a Full HD video, instead of HD video, the same problem that in Milestone 2 happened. The fps ratio was not enough so this pipeline had to be used with 720p videos. ffmpeg -re -i /home/carlos/Descargas/HD/Serenity.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 4.4.2. Milestone 10 - Receiving a Full HD/Cam and playing back / cropping the video This milestone has been implemented only with GStreamer framework. The pipeline is very similar with to one described in Milestone 4. gst-launch-1.0 udpsrc uri=udp://192.168.137.
Implementation of a Low Cost Video Wall using Raspberry Pi devices is similar to the one defined in Milestone 2. For this reason it is not taken into account in the CPU comparison in chapter 6. 4.5. Translation of GStreamer Pipelines to C Code GStreamer pipelines created in different milestones have been translated to C Code. The advantages of writting the code in C are that the programs run quickly because they can access low level facilities (both, in the Raspberry and in the server).
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 5. SYNCHRONIZATION In a Video Wall the idea of synchronizing a set of screens is that you can use individual content on each screen but still get the monitors to work together in a synchronized way. In the figure below there are three animations (video clips) that are synchronized to start simultaneously and maintain this synchronization during all the streaming.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Third of all, focusing on the capture, there are some packets called NAL unit (Network Abstraction Layer). As explained in chapter 2, the encoded video data is organized in NAL units and each NAL contains a header that indicates the type of data and the payload of the data defined in the header. In the wireshark captures showed below different types of NAL units are seen.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The P frames and B frames are captured and are inside the non-IDR frames. Just to remember a P-frame ('Predicted picture') holds only the changes in the image from the previous frame. For example, in a scene where a car moves across a stationary background, only the car's movements need to be encoded. The encoder does not need to store the unchanging background pixels in the P-frame, thus saving space.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Figure 36. FU-A Marked packet The difference between 2 Marked FU-A packets will provide us the time difference between two equal video frames (one frame for Raspberry A and one frame for Raspberry B). For instance, the difference between the packets 711 and 712 is 0.046 ms, which is really small. Fluctiations in this time difference appears.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 5.2. FFmpeg This section explains the basis of the packets captured with Wireshark during the FFmpeg transmission and why we have synchronization in our Video Wall when using it. The aim now is to explain the differences between both frameworks. As it can be observed in the figure below, for each video flow a SPS, PPS13 and SEI NAL units are sent. The functionality is the same as the explained in the previous section. Figure 38.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 6. CPU USAGE COMPARISON CPU usage is a term used to describe how much the processor is working. A computer's CPU usage can vary depending on the types of tasks that are being performed by the processor. CPU usage can be monitored to see how much of the processor's capacity is in use.
Implementation of a Low Cost Video Wall using Raspberry Pi devices After measuring average CPU usage in Client 1 and 2, the following results are obtained: Client 1 CPU Usage The CPU usage for the WebCam Streaming is the lowest because only 640x480 pixels resolution is being processed. The second lowest CPU usage is for 2 Video Flow Streaming because the Raspberry only receives a quarter part of the original 1920x1080p (that is 960x540p).
Implementation of a Low Cost Video Wall using Raspberry Pi devices 6.2. FFmpeg Server CPU Usage After measuring average CPU usage in the server, the following results are obtained: A single Video Streaming at 720p has the lowest CPU usage (around 24 %). The second lowest CPU usage is for the Single Video Streaming at 1080p with a CPU usage around 25 %. There is a very low difference in CPU usage but this can be due to Streaming 1080p flow does not work perfectly with FFmpeg.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Figure 43. CPU Usage FFmpeg Raspberry 1 Client 2 CPU Usage Something similar happens with the second Raspberry. Just note that Raspberry 2 CPU usage is always higher than Raspberry 1. Figure 44.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 6.3. VLC Server CPU Usage Using VLC the video flows can be directly streamed even if the video flow uses MPEG2-TS container. In this case, the CPU usage for the Single Video Streaming is the lowest. The video stream is simply payloaded into RTP packets and the RTP flow is sent via UDP. The CPU usage for the Single Video Streaming with MPEG2-TS encapsulation increases a bit. The increase in CPU usage is due to the muxer.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Client 2 CPU Usage Figure 47. VLC CPU Usage Client 2 Quite similar CPU usage for Raspberry 2 compared to the first Raspberry Pi. The difference in both graphs, if we compare them to GStreamer, is that videos have a resultion of 1280x720 meanwhile in GStreamer videos had a resolution of 1920x1080. That means that CPU usage is higher when VLC is used as emmiter than when GStreamer becomes the emmiter.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 6.4. OMXPlayer Some measures using top command have been done and the results obtained are the following: Figure 48. OMXPlayer CPU usage Both Raspberry Pi devices have similar CPU usage when OMXPlayer is executed. It is for that reason that there is just one figure for CPU usage. Surprisingly, CPU consumption is 0.7% when we play back a Full HD video.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 7. NETWORK TESTING This chapter will explain the tests performed in the system network used. To perform the testing a program called DummyNet45 has been used. DummyNet is a live network emulation tool, originally designed for testing networking protocols, and since then, used for a variety of applications including bandwidth management.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 7.2. DummyNet Scenario Our scenario is very similar to the scenario in the Figure 26, though there is a new element between the Server and the Router ADSL. This new element is a PC running Frenzy. This is a portable system administrator toolkit, LiveCD based on FreeBSD. It generally contains software for hardware tests, file system check, security check, network setup and analysis. DummyNet is a standard component in FreeBSD.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Now the command ifconfig-up is executed to the bridge and the two interfaces. ifconfig em0 up ifconfig bge0 up ifconfig bridge0 up The gateway list has to be verified too, and the gateways, added, to achieve the endpoints (Server and 2 Raspberry Pi devices). netstat -rn route add 192.168.137.5 192.168.137.3 route add 192.168.137.4 192.168.137.3 route add 192.168.137.1 192.168.137.2 7.4.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The results of these tests are very satisfactory. Our Video Wall can recover perfectly the 2 video flows after applying some delays or losses. When losses are applied, the image quality decreases a lot because it is a digital communication. If no more losses are added to the network, the Video Wall recovers the streaming perfectly after refreshing the image.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 8. THROUGHPUT AND BACKPLANE ANALYSIS This chapter will analyse the network throughput and the backplane that will handle the switch/router in the scenario of Figure 26. Here are the definitions of the two concepts that will be analysed later: Throughput or network throughput is the average rate of successful message delivery over a communication channel.
Implementation of a Low Cost Video Wall using Raspberry Pi devices In this case, the backplane capacity of the switch should be: Backplane capacity = 4 * 2.5 Mbps = 10 Mbps Figure 52. Backplane capacity Milestone 9 8.2. Milestone 11 - Throughput and Backplane Throughput In milestone 11, the pipeline sends in broadcast the original video flow.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Backplane The Backplane capacity in this case is the following: Backplane capacity = 10 Mbps Figure 54. Backplane capacity Milestone 11 A backplane of 10 Mbps is not very high. A wide range of switchs provides this backplane or more for a low price. In case of having a server with good features it is possible to split the video and then send it through the network.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 9. VIDEO WALL CONTROL SYSTEM Pipelines have been used all along this project: pipelines for emmiting and pipelines for receiving video flows. A lot of testing has been done. Changes from one pipeline to another, as well as verifying different scenarios. Obviously, the future user of the implemented system wants the system to be very simple and easy to use.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Web Cams should be Full HD cameras since the cropping script is designed to split a Full HD video in 4 differents parts. 2. In the Raspberry that will handle the audio stream the following pipeline will start: Pipeline H.264 audio receiver Port 60000 Description Milestone 8 Pipeline (GStreamer) Table 5. Initial Script for Audio The following figure shows a capture of the initial screen of Raspberry Pi devices when the H.
Implementation of a Low Cost Video Wall using Raspberry Pi devices The destination IP in the emitter pipelines will be the broadcast address. The port will change depending on the type of source (video flow, video flow cropped, web cam flow or webcam cropped). The user will indicate the preferred option in the shell script in the server as in the following table: Pipeline H.264 emmiter crop H.264 emmiter Web cam emmiter crop Web cam emmiter Option 1 2 3 4 IP Address 192.168.137.255 192.168.137.
Implementation of a Low Cost Video Wall using Raspberry Pi devices sudo chmod 755 /etc/init.d/z01.sh Finally, to register the script to be run at start-up and shutdown, the following command is run: sudo update-rc.d z01.sh defaults If ever the script wants to be removed from start-up, the following command will be run: sudo update-rc.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 10. COMPARISON WITH COMERCIAL PRODUCTS As explained in chapter 1, there are several Video Wall commercial products. One of the most popular is SAGE. This chapter will compare the requirements of SAGE with the requirements of the Video Wall in this project to demonstrate that it is a Low Cost Video Wall, in terms of cost, power consumption, backplane and throughput needed. 10.1.
Implementation of a Low Cost Video Wall using Raspberry Pi devices configuration supports highly interactive, multi-site remote, collaborative work, and enables multi-user interaction using a variety of input devices: 3D mouse, touch-screen and motion sensing input devices such as the Microsoft Kinect. 10.2.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Basic SAGE Minimun Requirements 1 control computer (SAGE manager) 2 displays/computer (# display tiles to computer ratio) Specifications/Computer Dual Xeon E5-2690 Supermicro GPU server NVIDIA GTX680 graphics card (per machine) Dual 10Gbps network card (per machine) 160GB SSD 2TB hard drive Networking 10Gbps network switch Asus Xonar DX PCI Express Sound Card 2 speakers (Dell AY410 Multimedia Speaker System) c
Implementation of a Low Cost Video Wall using Raspberry Pi devices 10.4. Minimum Systems Requirements and Budget of this project Chapter 1 described the features of the Server and of the Raspberry Pi devices. We already had some of the elements of the system and the others elements were provided by the i2CAT Foundation. The following table is an estimated summary of the cost of the components that needed in the Video Wall in case of buying all the elements.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Conclusion The Basic SAGE costs about 5989 € and the amount of money needed to build the system of this project's Video Wall costs only 670 €. That means, 5319 € less than the Basic SAGE. 10.5. Comparison of the power consumption results In this section the approximate Power Consumption of the Basic SAGE and this project are calculated.
Implementation of a Low Cost Video Wall using Raspberry Pi devices For the same price, 8 hours more on uniterrumped work functioning is obtained by the system created along the project. 10.6. Throughput and Backplane analysis In this section the network throughput and the backplane that will handle the Basic SAGE system are estimated in order to compare them to the results obtained with Low Cost Video Wall.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 11. PROJECT PLANNING This chapter provides a summary of all the diferent tasks performed in this project and the time investment per task. Also, a time line is drawn to show the evolution of the project planning. 11.1. Task table The following table shows a summary of all tasks performed in the project and the time investment to achieve them.
Implementation of a Low Cost Video Wall using Raspberry Pi devices 7. Throughput and Backplane 8. Design of VideoWall control system 9. Comparison with commercial products 10. Written Report 11.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 12. CONCLUSIONS The main milestone of this project was to build a cheap Video Wall using Raspberry Pi and we are proud to announce that this milestone is accomplished after working tirelessly during seven months. It is important to remark that all the project milestones have been acomplished in the following way: 1. Low cost video wall is a reality.
Implementation of a Low Cost Video Wall using Raspberry Pi devices FFmpeg is another insteresting framework able to decode, encode, transcode, mux, demux, stream, filter and play a lot of format. However, it does not provide hardware support of FFmpeg in the Raspberry Pi devices. VLC, a portable free and open source media player and streaming media server, has compiled version with hardware acceleration. Although, it was impossible to install in our Raspberry Pi devices as commented in Chapter 4.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Environmental impact Finally, about the environmental impact of this project, it is important to take into account: - Power consumption: A comparison with the commercial product SAGE has been done in Chapter 10, and the power consumption (Watts) of the Video Wall implemented is much lower. This fact reduces the environmental impact because it is possible to obtain more operating time for the same power.
Implementation of a Low Cost Video Wall using Raspberry Pi devices CHAPTER 13. BIBLIOGRAPHY Raspberry Pi [1] BroadCom VideoCore [online]. [Reference date: 25/07/2013] Available on: http://www.broadcom.com/products/technology/mobmm_videocore.php [2] BroadCom VideoCore Specifications [online]. [Reference date: 26/07/2013] Available on: http://www.broadcom.com/press/release.php?id=s430181 [3] Raspberry Pi VideoCore APIs [online]. [Reference date: 26/07/2013] Available on: http://elinux.
Implementation of a Low Cost Video Wall using Raspberry Pi devices [12] FU-A description [online]. [Reference date: 12/08/2013] Available on: http://tools.ietf.org/html/rfc3984#section-5.8 http://tools.ietf.org/html/rfc6184 [13] NAL Unit - SPS & PPS description [online]. [Reference date: 14/08/2013] Available on: http://tools.ietf.org/html/rfc6184 [14] H.264 ITU Recommendation [online]. [Reference date: 14/08/2013] Available on: http://www.itu.int/rec/T-REC-H.
Implementation of a Low Cost Video Wall using Raspberry Pi devices [23] OpenMAX Development Guide [online]. [Reference date: 10/09/2013] http://processors.wiki.ti.com/index.php/OpenMax_Development_Guide [24] GStreamer Pipelines examples [online]. Reference date: 11/09/2013] Available on: http://processors.wiki.ti.com/index.php/Example_GStreamer_Pipelines#Loopback:_Aud io_.2B_Video_2 [25] GStreamer Multimedia and Internet [online]. Reference date: 20/09/2013] Available on: http://www.polytech2go.
Implementation of a Low Cost Video Wall using Raspberry Pi devices [35] Useful GStreamer commands [online]. [Reference date: 28/09/2013] Available on: https://community.freescale.com/docs/DOC-93487 [36] GStreamer Streaming project [online]. [Reference date: 28/09/2013] Available on: https://github.com/peonic/streaming [37] Description of Gstreamer Core plugins elements [online]. [Reference date: 29/09/2013] Available on: http://gstreamer.freedesktop.
Implementation of a Low Cost Video Wall using Raspberry Pi devices DummyNet Testing [45] The dummynet project [online]. [Reference date: 05/11/2013] http://info.iet.unipi.it/~luigi/dummynet/ [46] Study of implementation of a real time online football game for mobile devices [online]. [Reference date: 06/11/2013] http://upcommons.upc.edu/pfc/bitstream/2099.1/14976/1/memoria.pdf FFmpeg [47] Information about FFmpeg [online]. [Reference date: 07/11/2013] Available on: http://www.ffmpeg.
Implementation of a Low Cost Video Wall using Raspberry Pi devices [57] VLC Download page [online]. [Reference date: 11/12/2013] Available on: http://www.videolan.org/vlc/download-debian.html [58] Backports Source List VLC. [online]. [Reference date: 12/12/2013] Available on: http://backports.debian.org/Instructions/#index2h2 [59] Compiled VLC with hardware acceleration support on Raspberry Pi [online]. [Reference date: 13/12/2013] Available on: http://intensecode.blogspot.com.
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEXES
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEX A.
Implementation of a Low Cost Video Wall using Raspberry Pi devices In this annex we will explain how to prepare a SD card for the Raspbian OS installation in our Raspberry Pi, perform the Raspbian OS installation and configure the Raspberry Pi network to allow internet connection. A1. Preparing our SD card for the Raspberry Pi In order to use our Raspberry Pi, we will need to install an Operating System (OS) onto a SD card.
Implementation of a Low Cost Video Wall using Raspberry Pi devices A2. Installation Raspbian “wheezy” We have decided to install the Raspbian “wheezy” OS. This OS is based on the Raspbian optimized version of Debian, and containing LXDE, Midori, development tools and example source code for multimedia functions. To perform the installation in Windows 7 we will need a program which is called Win32DiskImager and we will need to follow the next steps: 1. Download the distribution from the raspberrypi.
Implementation of a Low Cost Video Wall using Raspberry Pi devices A3. Configuration of Raspberry Pi Network Initially we need to know the IP of our Raspberry. To archive that we will use a program which is called Advanced IP Scanner. Figure A3. Advanced IP Scanner Program To establish a connection between our PC and the Raspberry Pi we will need a program called PuTTY. PuTTY is a free and open-source terminal emulator; serial console and network file transfer application.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Figure A4. PuTTY Configuration (1) Figure A5.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Once we press Open button, the connection is established as we can see in Figure A6 and Figure A7. Figure A6. Connection established Figure A7.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Next step is to share Internet to our Raspberry Pi. To make that possible we need to share the Wireless Connection with the Local Area Connection in Windows 7. Figure A8.Wireless Connection shared - Windows 7 After that, we need to open the file /etc/network/interfaces with a text editor (i.e. GNU Nano) to add a static address of our Raspberry Pi, a net mask and a gateway. As it is shown in the Figure A9, the IP selected is 192.168.
Implementation of a Low Cost Video Wall using Raspberry Pi devices In addition it is necessary to configure the DNS. We need to open the file /etc/resolv.conf and add a nameserver (in this case, the name server is 147.83.2.3 which is the DNS of UPC). The other two name servers are the ones that we use in our home, so Raspberry Pi can work in the different wireless networks that we will use during this project. Figure A10.
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEX B.
Implementation of a Low Cost Video Wall using Raspberry Pi devices By default GStreamer 0.10 is installed in the Raspbian OS. But, due to project requirement a GStreamer 1.0 installation will be performed in both Server and Clients. B1. Installing GStreamer 1.0 in the Server Due there is no an official package of GStreamer 1.0 available, we need to compile and install GStreamer manually.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Install libvisual-0.4-dev for xvimagesink (if running X11) $ sudo apt-get install libxv-dev libvisual-0.4-dev Install base plugins Gstreamer base plugins is a well-groomed and well-maintained collection of GStreamer plugins and elements. Also contains helper libraries and base clases useful for writing elements. A wide range of video and audio decoders, encoders, and filters are included. $ wget http://gstreamer.freedesktop.
Implementation of a Low Cost Video Wall using Raspberry Pi devices $ wget http://gstreamer.freedesktop.org/src/gst-plugins-bad/gstplugins-bad-1.0.2.tar.xz $ tar xvf gst-plugins-bad-1.0.2.tar.xz $ cd gst-plugins-bad-1.0.2 $ ./configure --prefix=/usr $ make $ sudo make install Install the ugly plugins This is a set of plugins that have good quality and correct functionality, but distributing them might pose problems. The code might be widely known to present patent problems.
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEX C.
Implementation of a Low Cost Video Wall using Raspberry Pi devices C1. Installing VLC in the Server It is really simple to install VLC in the server. It is highly recommended to get a recent version of VLC on wheezy by installing the version from wheezy-backports. We need to add the wheezy-backports repository to our sources.list by writing: deb http://mirror.debian.org/debian wheezy-backports main (or add a new file with the ".list" extension to /etc/apt/sources.list.
Implementation of a Low Cost Video Wall using Raspberry Pi devices An alternative method that it is expensive in time (around 20 hours, and that time investment does not guarantee a success) is to compile ourselves the VLC. The instructions to compile it by ourselves are the following: Step 0: environment preparation - build tools installation # apt-get install git libtool build-essential pkg-config autoconf Step 1: get VLC sources from git # git clone git://git.videolan.org/vlc.git # cd vlc # .
Implementation of a Low Cost Video Wall using Raspberry Pi devices Step 5 (optional) To play back HD content it is strongly recommended to overclock the Raspberry. The file to change CPU frequency is /boot/config.txt. We followed the 6 steps commented above, but due to the complexity and the constant request of multiple third party libraries, this option has a very low probability of success. We devoted 2 days (7 hours per day) but the compilation failed.
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEX D.
Implementation of a Low Cost Video Wall using Raspberry Pi devices D1. H.261 video codec H.261 is a video coding standard published by ITU in 1990. It is designed for ISDN network data rates which are multiples of 64 kbit/s. A functional block diagram of the video codec is shown in Figure B1. Figure B1. A block diagram of H.261 video codec The main element in the encoding process is a source coder which compresses the incoming video signal.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Each macroblock is further divided into six blocks. Four of them represent the luminance and two the chrominance components of the signal. Each block has 8x8 pixels, so the color resolution is half of the luminance resolution in both dimensions. Luminance (Y) 1 2 3 4 Chrominance / blue (Cb) Chrominance / red (Cr) 5 6 Table 18.
Implementation of a Low Cost Video Wall using Raspberry Pi devices Multiplexing The video multiplex coder combines the compressed data with various side information, which indicates various operation modes, into hierarchical bitstream that can be interpreted by decoders. Transmission buffer The transmission buffer is employed to smooth the varying bit rate from the source coder to adapt it to the communication channel.
Implementation of a Low Cost Video Wall using Raspberry Pi devices ANNEX E.
Implementation of a Low Cost Video Wall using Raspberry Pi devices To create a useful diagram of our pipelines we have followed the next steps: 1. Install dot sudo apt-get install graphviz The "dot" command is available in the "graphviz" package. 2. Run in the prompt: # export GST_DEBUG_DUMP_DOT_DIR=/tmp/ This will place the .dot generated files in the /tmp/ directory. We can change that directory if we need to. 3. Run the pipeline gst-launch-1.0 filesrc location= /home/carlos/Descargas/ Monsters.
E1. Milestone 1 E2. Milestone 2 E3. Milestone 3 E4.
E5. Milestone 5 E6. Milestone 6 E7.
Implementation of a Low Cost Video Wall using Raspberry Pi devices
