Plex Setup for Many Simultaneous Streams

Over the past couple of days I have been compiling a bunch of already available information and personalizing it for my current and future need.

I love Plex. It does everything (for the most part) that I need it to.

I work for an educaitonal institution. We recently deployed a bunch of iPads and AppleTV 2s (with TVs) to 8 of our k-12 classrooms. Using Airplay works pretty well for media that the teachers can find online (youtube, vimeo, netflix, etc.).  Still, I needed a way to provide access to the old media that this school has collected over the years; whatever format it may be in.

Enter: Ubuntu, Plex, and CPUlimit.

I have pieced together what I believe is a viable solution that can provide many multiple simultaneous transcoding sessions to any client device from the same server.

Basically, it boils down to a headless Ubuntu 12.04 server (in this case, it was built in VMware, but physical HW would work just fine) with at least 4 high-end desktop or server grade CPU cores. I installed Plex, setup some NFS mounts, and installed CPUlimit and configured it with some associated scripts.

We are now able to successfully stream as many transcoding sessions as we have physical cores in the system (minus 2 cores for OS and other Plex processes).

In my particular current setup, the server has 8 cores and 4GB of ram. Media is stored on a network share.

If you are interested in the actual fairly straight forward instructions to setup your own server this way, click the “more” link below these two images:

Running processes before the script is enabled:

Running processes after the script is enabled:

Click here for a .txt of the instructions

#### Plex Media Server 9.6.9 Installation and Configuration on Ubuntu 12.04 With CPUlimit for Many Simultaneous Streams ####
#### Installation and configuration done on Ubuntu Server 12.04 LTS (64-bit) but not specific to this version ####
#### Instructions assume firewall turned off or exception made for port 32400 for Plex ####
# Firewall instructions found here: https://help.ubuntu.com/community/UFW #
#### Instructions assume properly configured Static or DHCP IP address WITH correct DNS server entries ####
#! MyPlex login will fail if DNS not properly setup #

==============================================
==============================================
#PLEX INSTALLATION (ASSUMES UBUNTU INSTALLED, WITH ACCESS TO INTERNET, AND UPDATED)
#INSTALLATION INSTRUCTIONS FOUND HERE:
# http://forums.plexapp.com/index.php/topic/26727-how-to-plex-media-server-on-ubuntu/

#install Avahi
$sudo apt-get install avahi-daemon

##install Plex

#add plexapp to repo sources
$sudo nano /etc/apt/sources.list

#append this to end of sources.list
# deb http://www.plexapp.com/repo lucid main
#previous line is for stable versions not betas; see website above for beta repo.
#ctrl-x then y to save and close

#update apt-get sources
$sudo apt-get update

#install Plex Media Server
$sudo apt-get install plexmediaserver

#verify Plex is running with web browser.

# locate and document ip address
$ifconfig

# open web browser on another computer and visit documented ip address
# http://x.x.x.x:32400/manage/index.html

#updating plex
$sudo aptitude update
$sudo aptitude safe-upgrade

=========

==============================================
==============================================
#NFS SETUP
#This portion is intended for those who want to host the actual media files on another physical server.
#Install NFS and automount share(s) from other server(s) (NAS, server, etc.)
#Instructions found here:
# http://www.cyberciti.biz/tips/ubuntu-linux-nfs-client-configuration-to-mount-nfs-share.html

#update sources
$sudo apt-get update

#install nfs
$sudo apt-get install nfs-common

#get and display available NFS shares from another server (need NFS sharing server’s IP).
$showmount -e x.x.x.x

#example: $showmount -e 10.3.254.249
#output: Export list for 10.3.254.249
# /c/media *
# /c/backup *

#make new directory for nfs mounts to be stored.
$sudo mkdir /nfs

#make subdirectories if mounting multiple sub-shares or multiple servers.
#this makes managing media easier in the Plex gui.
#examples:
# $sudo mkdir /nfs/TV
# $sudo mkdir /nfs/Movies

#mount NFS (sub-)shares inside the /NFS folder just created.
$sudo mount -o soft,intr,rsize=8192,wsize=8192 x.x.x.x:/media/videos/movies /nfs/Movies
$sudo mount -o soft,intr,rsize=8192,wsize=8192 x.x.x.x:/media/videos/tv /nfs/TV

#verify access to newly mounted director(ies) contents.
$cd /nfs/Movies
$ls
$cd ./directoryName

#check freespace in human readable form; shows free space of nfs mounts, also.
$cd /
$df -h

#automount NFS shares
#open /etc/fstab
$sudo nano /etc/fstab

#append contents of next line to end of file.
# x.x.x.x:/media/videos/movies /nfs/Movies nfs soft,intr,rsize=8192,wsize=8192

#### UNMOUNT NFS SHARE(S) ####
$cd /
$sudo umount /nfs/Movies
$df -h

#notice mounted share is now gone.
$sudo umount /nfs/TV
$df -h

==============================================
==============================================
#Add Media directories from NFS subdirectories (/c/nfs/…) to Plex Media Manager

==============================================
==============================================
#CPULIMIT INSTALLATION AND CONFIG
#Use CPUlimit to restrict each Plex Transcoder instance to approx. (1) core on the CPU.
#The idea for this is to provide the ability to stream multiple transcoding/transmuxing sessions
#simultaneously without maxing-out/choking the CPUs. Normal usage of an 8-core cpu is between
#300%-600% (roughly 3-6 cores working at full speed to finish the transcode as quickly as possible). Only
#99% (roughly 1-core) is usually required for faster-than-real-time transcoding.
#Use an N-2 formula (# of cores minus 2 for system processes/services) for caluating the number of
#potential transcoding/transmuxing streams on a particular system.
#! This is only useful on a system with high(er)-end desktop CPU(s) or Server class CPUs such as
#Xeon or Core i7. This may work well on Core i5 CPUs, but I have not tested this.
==========
#preliminary code sample to test cpulimit

#/bin$ sudo cpulimit –exe “/usr/lib/plexmediaserver/Resources/Plex Transcoder” –limit 50

#Start transcoding instance by playing video on iOS device, or PlexMediaCenter with Force Transcoding enabled, and bitrate lower than original file.

#stdout:
#Process XXXX detected

#stdout:
#Process XXXX dead!

==========

==========
#Most content in this section found and copied from: http://ubuntuforums.org/showthread.php?t=992706 – Thanks to abcuser for writing it and thanks to Trudge for sending me there.
#Black/White Processes Lists: processes lists like: “Plex Transcoder|firefox|mysql” etc.
#LINE 8: CPU_LIMIT=x — this is the percentage of (1) core processing ability that you want to
#assign to an instance of Plex Transcoder. !Note: If you set this value to >/=100% will not limit
#any process instance.

#install cpulimit program
$sudo apt-get install cpulimit

#install gawk program
$sudo apt-get install gawk

#create cpulimit daemon script
$sudo nano /usr/bin/cpulimit_daemon.sh

==========
#contents of cpulimit_daemon.sh below. paste into nano

#######################################
#!/bin/bash
# ==============================================================
# CPU limit daemon – set PID’s max. percentage CPU consumptions
# ==============================================================

# Variables
CPU_LIMIT=99 # Maximum percentage CPU consumption by each PID
DAEMON_INTERVAL=60 # Daemon check interval in seconds
BLACK_PROCESSES_LIST=”`Plex\ Transcoder`” # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
WHITE_PROCESSES_LIST= # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.

# Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
if [[ -n “$BLACK_PROCESSES_LIST” && -n “$WHITE_PROCESSES_LIST” ]] ; then # If both variables are defined then error is produced.
echo “At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty.”
exit 1
elif [[ -n “$BLACK_PROCESSES_LIST” ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND=”top -b -n1 -c | grep -E ‘$BLACK_PROCESSES_LIST’ | gawk ‘\$9>CPU_LIMIT {print \$1}’ CPU_LIMIT=$CPU_LIMIT”
elif [[ -n “$WHITE_PROCESSES_LIST” ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND=”top -b -n1 -c | gawk ‘NR>6’ | grep -E -v ‘$WHITE_PROCESSES_LIST’ | gawk ‘\$9>CPU_LIMIT {print \$1}’ CPU_LIMIT=$CPU_LIMIT”
else
NEW_PIDS_COMMAND=”top -b -n1 -c | gawk ‘NR>6 && \$9>CPU_LIMIT {print \$1}’ CPU_LIMIT=$CPU_LIMIT”
fi

# Search and limit violating PIDs
while sleep $DAEMON_INTERVAL
do
NEW_PIDS=$(eval “$NEW_PIDS_COMMAND”) # Violating PIDs
LIMITED_PIDS=$(ps -eo args | gawk ‘$1==”cpulimit” {print $3}’) # Already limited PIDs
QUEUE_PIDS=$(comm -23 <(echo “$NEW_PIDS” | sort -u) <(echo “$LIMITED_PIDS” | sort -u) | grep -v ‘^$’) # PIDs in queue for i in $QUEUE_PIDS do cpulimit -p $i -l $CPU_LIMIT -z & # Limit new violating processes done done #################################### ========= #Set cpulimit_daemon.sh to auto-start #change permissions $sudo chmod 755 /usr/bin/cpulimit_daemon.sh #add to startup $sudo nano /etc/init.d/cpulimit ========= #contents of /etc/init.d/cpulimit below. paste into nano ######################################## #!/bin/sh ### BEGIN INIT INFO # Provides: cpulimit # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start daemon at boot time # Description: Enable service provided by daemon. ### END INIT INFO # #located @ /etc/init.d/cpulimit # # Script to start CPU limit daemon # set -e case “$1″ in start) if [ $(ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print $1}’ | wc -l) -eq 0 ]; then nohup /usr/bin/cpulimit_daemon.sh >/dev/null 2>&1 &
ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print}’ | wc -l | gawk ‘{ if ($1 == 1) print ” * cpulimit daemon started successfully”; else print ” * cpulimit daemon can not be started” }’
else
echo ” * cpulimit daemon can’t be started, because it is already running”
fi
;;
stop)
CPULIMIT_DAEMON=$(ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print $1}’ | wc -l)
CPULIMIT_INSTANCE=$(ps -eo pid,args | gawk ‘$2==”cpulimit” {print $1}’ | wc -l)
CPULIMIT_ALL=$((CPULIMIT_DAEMON + CPULIMIT_INSTANCE))
if [ $CPULIMIT_ALL -gt 0 ]; then
if [ $CPULIMIT_DAEMON -gt 0 ]; then
ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print $1}’ | xargs kill -9 # kill cpulimit daemon
fi

if [ $CPULIMIT_INSTANCE -gt 0 ]; then
ps -eo pid,args | gawk ‘$2==”cpulimit” {print $1}’ | xargs kill -9 # release cpulimited process to normal priority
fi
ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print}’ | wc -l | gawk ‘{ if ($1 == 1) print ” * cpulimit daemon can not be stopped”; else print ” * cpulimit daemon stopped successfully” }’
else
echo ” * cpulimit daemon can’t be stopped, because it is not running”
fi
;;
restart)
$0 stop
sleep 3
$0 start
;;
status)
ps -eo pid,args | gawk ‘$3==”/usr/bin/cpulimit_daemon.sh” {print}’ | wc -l | gawk ‘{ if ($1 == 1) print ” * cpulimit daemon is running”; else print ” * cpulimit daemon is not running” }’
;;
esac
exit 0
###########################################

=========

#change cpulimit owner
$sudo chown root:root /etc/init.d/cpulimit

#change cpulimit permissions
$sudo chmod 755 /etc/init.d/cpulimit

#add cpulimit script to boot defaults
$sudo update-rc.d cpulimit defaults

#reboot and check for running script
$sudo shutdown now -r

#### script created and setup. use below steps to verify it is in working order.

=========
#### status, start, stop, restart cpulimit

#check if cpulimit is running
$sudo service cpulimit status

#manually start cpulimit
$sudo service cpulimit start

#manually stop cpulimit
$sudo service cpulimit stop

#manually restart cpulimit
$sudo service cpulimit restart

=========

#### check cpu usage of Plex Transcoder process instances

#check for running processes and cpu utilization
$top

======================
======================

#### REMOVAL ONLY ####
#UNINSTALL CPULIMIT DAEMON AND CPULIMIT PROGRAM

#stop cpulimit daemon
$sudo service cpulimit stop

#remove da2emon from startup
$sudo update-rc.d -f cpulimit remove

#delete boot script
$sudo rm /etc/init.d/cpulimit

#delete daemon
$sudo rm /usr/bin/cpulimit_daemon.sh

#uninstall gawk if not needed
$sudo apt-get remove gawk

=================

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s