Hi Mike,
I thought I would check in with you on the plex optimization features. It's been several months since we chatted about this and I'm curious if you're still using your own conversion scripts prior to dropping them into Plex? I'm tempted to try and just let Plex to the conversion when I drop the files in my media library folder to see if conversion scripts are still necessary.
Ironically, right now I have it set to optimize all of my "Newly Added" movies automatically, but a part of me feels this is overkill because it's optimizing an already converted movie file.
Any feedback would be appreciated including what settings you found to be optimal for TV viewing.
Thanks
Optimizing media for less to no transcoding
-
MikeG.6.5
- Posts: 917
- Joined: Fri May 15, 2015 1:56 am
Re: Optimizing media for less to no transcoding
No, I don't use the Plex Optimized media feature, except for some testing right now. When we get bit rate limits on a per user basis and adaptive bitrates that might very likely change, though. As then I can set someone up for say 3Mbps and also make a 2Mbps or 1.5Mbps version and as the server needs to adjust for lagging, it has a way to do so without the need for transcoding.
Ideally, with bitrate limits, adaptive bitrates and OM versions, there should always be a Direct Playable version that will fit the limits on the user's account, and the bitrates the server decides is needed for streaming the media. Bitrate limits are coming, and I have it on good authority that adaptive and tying this to OM is the plan Plex is going towards. This puts a truly fully capable streaming service into the hands of any Plex Media Server operator, with very little configuration. And small servers are going to be just as capable as some of the very large servers, if media is Direct Playable from the start!
BTW, I'm going to outline one method of bitrate limits here that many people don't already know:
In your ADM File Manager, make a folder in your media folder for OM versions. (/volume1/media/OM-Versions) Go to PMS Web App, and edit the settings for your main library. to include this new folder as part of the library's search path. Next, make a NEW library pointing ONLY to this new folder for it's search path. Then, back in the main library, start an OM job, for all files, for a given bitrate/setting with a custom path set to the new folder. (test this on a few client devices with just a few files to start...) As the OM feature makes the new versions, it adds the media to both the old library as a second version and to the new library.
Now, this is the KEY to making this work. For all of the users you have on a remote connection, only share out to the NEW LIBRARY! They will never ever see the other higher bitrate files, so they can never stream that media. Only the media that is already made that fits your bitrates in the OM job. this is going to take a bit of work, as some movies might already be smaller bitrates than the OM job is set for, and you have to do a bit of creative juggling to get those to show in the new library, but I think you can figure that out from this description.... A second OM for an even smaller bitrate might be needed to catch those odd one-offs, and that's not that big of a deal to do either, but I advise waiting to build this until you get the first one done....
This, in a nutshell, is fully working, tested and verified bitrate limit for remote users. And as media is added to the main library, a new version is automatically generated for that media item in both libraries, one for the full bitrate and one for each library at the new bitrate as the OM is generated.
Ideally, with bitrate limits, adaptive bitrates and OM versions, there should always be a Direct Playable version that will fit the limits on the user's account, and the bitrates the server decides is needed for streaming the media. Bitrate limits are coming, and I have it on good authority that adaptive and tying this to OM is the plan Plex is going towards. This puts a truly fully capable streaming service into the hands of any Plex Media Server operator, with very little configuration. And small servers are going to be just as capable as some of the very large servers, if media is Direct Playable from the start!
BTW, I'm going to outline one method of bitrate limits here that many people don't already know:
In your ADM File Manager, make a folder in your media folder for OM versions. (/volume1/media/OM-Versions) Go to PMS Web App, and edit the settings for your main library. to include this new folder as part of the library's search path. Next, make a NEW library pointing ONLY to this new folder for it's search path. Then, back in the main library, start an OM job, for all files, for a given bitrate/setting with a custom path set to the new folder. (test this on a few client devices with just a few files to start...) As the OM feature makes the new versions, it adds the media to both the old library as a second version and to the new library.
Now, this is the KEY to making this work. For all of the users you have on a remote connection, only share out to the NEW LIBRARY! They will never ever see the other higher bitrate files, so they can never stream that media. Only the media that is already made that fits your bitrates in the OM job. this is going to take a bit of work, as some movies might already be smaller bitrates than the OM job is set for, and you have to do a bit of creative juggling to get those to show in the new library, but I think you can figure that out from this description.... A second OM for an even smaller bitrate might be needed to catch those odd one-offs, and that's not that big of a deal to do either, but I advise waiting to build this until you get the first one done....
This, in a nutshell, is fully working, tested and verified bitrate limit for remote users. And as media is added to the main library, a new version is automatically generated for that media item in both libraries, one for the full bitrate and one for each library at the new bitrate as the OM is generated.
-
Jagstyles
Re: Optimizing media for less to no transcoding
Great feedback as always Mike. Ideally, I'd like to get to a point where I no longer need to run conversion scripts. Simply put the MKV files into the media folder and let Plex do it's thing to optimize the files and away we go. I'm also wondering if they will ever provide a feature to delete the original MKV file and use just the optimized version - seems pointless to have both media files to be honest.
-
MikeG.6.5
- Posts: 917
- Joined: Fri May 15, 2015 1:56 am
Re: Optimizing media for less to no transcoding
Well, it does and it doesn't. Let me explain.
Using the conversion scripts ensures that you should never need to transcode for anything local, as everything should be in the right container, codec and ready to Direct Play out of the box, on any local device. Remember, the costs of streaming something locally is already paid for, as you bought the hardware up front, and electricity costs are the only other expenses you are going to have. And any time you have to transcode media On Demand, the quality of the conversion isn't going to be nearly as good as something that is pre-converted. Transcoding, IMO, is almost an evil thing. And MKV's almost always have something in the files, codecs, or layouts that require transcoding on my own clients. So MKV is another evil thing, IMO....
Remote streaming is where maintaining more than one copy of the media starts to shine through. You have the 2 libraries I outline above, and any users you share with that are going to use you media remotely only see the second library, at a much lower bitrate than the stuff reserved for local streaming. As this media is much smaller, maintaining this second copy is going to be over-all much smaller than the original file would. For instance, if you have a 20Mbps copy of a movie that might take up 20GB or more, a 4Mbps copy is likely to be at or near 1GB. So you can store 10-20 different 4Mbps media items in the same space you store just one 20Mbps movie. Having it duplicated is much cheaper, long term, then it is to rely on transcoding for remote, especially on a server that might not be able to handle On Demand transcoding in Real Time.
BTW, this is how Netflix and a lot of other streaming services handle streaming to different connection speeds. They have many different bitrates of a file, and the client and server negotiate the speeds that can work. Then the appropriate file is selected and streamed. If buffering starts to happen due to network congestion, the client initiates a renegotiation, and a different files is now used at the same rough starting place as the first was. Right now this auto-bitrate renegotiation isn't live in Plex. But I know first hand it is coming some time.
When we get bitrate limits for users, renegotiation of the bit rates automatically, and can have the PMS select and use the right OM version on the fly, we're going to be set up pretty close to how Netflix and the other services are doing their streaming. And then the fun starts to really get ramped up. Folks with lower bitrate connections are going to be able to stream to a few friends at a time, as well as locally, on even a small NAS.
Using the conversion scripts ensures that you should never need to transcode for anything local, as everything should be in the right container, codec and ready to Direct Play out of the box, on any local device. Remember, the costs of streaming something locally is already paid for, as you bought the hardware up front, and electricity costs are the only other expenses you are going to have. And any time you have to transcode media On Demand, the quality of the conversion isn't going to be nearly as good as something that is pre-converted. Transcoding, IMO, is almost an evil thing. And MKV's almost always have something in the files, codecs, or layouts that require transcoding on my own clients. So MKV is another evil thing, IMO....
Remote streaming is where maintaining more than one copy of the media starts to shine through. You have the 2 libraries I outline above, and any users you share with that are going to use you media remotely only see the second library, at a much lower bitrate than the stuff reserved for local streaming. As this media is much smaller, maintaining this second copy is going to be over-all much smaller than the original file would. For instance, if you have a 20Mbps copy of a movie that might take up 20GB or more, a 4Mbps copy is likely to be at or near 1GB. So you can store 10-20 different 4Mbps media items in the same space you store just one 20Mbps movie. Having it duplicated is much cheaper, long term, then it is to rely on transcoding for remote, especially on a server that might not be able to handle On Demand transcoding in Real Time.
BTW, this is how Netflix and a lot of other streaming services handle streaming to different connection speeds. They have many different bitrates of a file, and the client and server negotiate the speeds that can work. Then the appropriate file is selected and streamed. If buffering starts to happen due to network congestion, the client initiates a renegotiation, and a different files is now used at the same rough starting place as the first was. Right now this auto-bitrate renegotiation isn't live in Plex. But I know first hand it is coming some time.
When we get bitrate limits for users, renegotiation of the bit rates automatically, and can have the PMS select and use the right OM version on the fly, we're going to be set up pretty close to how Netflix and the other services are doing their streaming. And then the fun starts to really get ramped up. Folks with lower bitrate connections are going to be able to stream to a few friends at a time, as well as locally, on even a small NAS.
-
Jagstyles
Re: Optimizing media for less to no transcoding
Great feedback.... The thing for us is we consume all of our Plex Media content locally e.g: NAS to various TV's in the house via wired ethernet. When we take things on the road, I either use the SYNC option to only copy movies I want for a particular flight I'm taking but even that's a rarity.
So, in the end, if we consume our content mostly locally, is it good enough to just plop the MKV files right into my media library folders and let Plex optimize them accordingly? The optimization settings seem to translate into a file very similar to what I would normally get via my conversion scripts.
So, in the end, if we consume our content mostly locally, is it good enough to just plop the MKV files right into my media library folders and let Plex optimize them accordingly? The optimization settings seem to translate into a file very similar to what I would normally get via my conversion scripts.
-
MikeG.6.5
- Posts: 917
- Joined: Fri May 15, 2015 1:56 am
Re: Optimizing media for less to no transcoding
Again, yes and no....
The MKV's you know right out of the gate are likely to require transcoding to watch for anything locally, and that's where the conversion scripts come in. Even with my NAS I do not ever want to rely on something to transcode on-demand, because this is a potential bottleneck for ANY CPU, and the smaller NASes especially.
There are so many things that need to be right for Direct Play to work correctly, that the scripts you are running do right at the start. You always know that those files should be the best possible candidates for Direct Play. It's what's inside of the MKV's that always going to be questionable. Yeah, they might be in H264, but unless the MOV atom is moved tot eh proper place in the file (also called Web Optimized.) then it transcodes. If timing internal to the file is off, or 20 different subs, or 15 different audio tracks, or a slew of other issues that are way beyond your control unless you specifically made that MKV yourself....
How the Optimized Media works now is to make a file as Direct Playable as possible from existing media. And at the bitrates you set up for the OM task. Something reduced to fit remote or small displays or an original for something local that dan't digest the original file, for what ever reason.
I can't tell you what to do here. All I can do is advise you based on my own experiences. I still run the scripts you are using, and they handle any conversions I need for 90% of my streaming. And if I had the disk space for it, I would also have an OM version made for everything in the libraries, to facilitate any remotes or synced requests the server has to deal with. At this point, I don't have the space to duplicate everything... I literally have too much stuff!
Yes, running the scripts does create just a touch more hands-on than other methods. But that's also part of being an administrator on a full blown Client/Server suite. You have to do a bit more to make sure everything runs smoothly, even if it is just for your own consumption. And having said that, it's up to you. Let me know how you decide to proceed, as others might find some value in their usage. But keep in mind, too... Not everyone running Plex on an Asustor has the CPU I or even you have in their devices... And what works for either of us might very well NOT work well for someone else...
The MKV's you know right out of the gate are likely to require transcoding to watch for anything locally, and that's where the conversion scripts come in. Even with my NAS I do not ever want to rely on something to transcode on-demand, because this is a potential bottleneck for ANY CPU, and the smaller NASes especially.
There are so many things that need to be right for Direct Play to work correctly, that the scripts you are running do right at the start. You always know that those files should be the best possible candidates for Direct Play. It's what's inside of the MKV's that always going to be questionable. Yeah, they might be in H264, but unless the MOV atom is moved tot eh proper place in the file (also called Web Optimized.) then it transcodes. If timing internal to the file is off, or 20 different subs, or 15 different audio tracks, or a slew of other issues that are way beyond your control unless you specifically made that MKV yourself....
How the Optimized Media works now is to make a file as Direct Playable as possible from existing media. And at the bitrates you set up for the OM task. Something reduced to fit remote or small displays or an original for something local that dan't digest the original file, for what ever reason.
I can't tell you what to do here. All I can do is advise you based on my own experiences. I still run the scripts you are using, and they handle any conversions I need for 90% of my streaming. And if I had the disk space for it, I would also have an OM version made for everything in the libraries, to facilitate any remotes or synced requests the server has to deal with. At this point, I don't have the space to duplicate everything... I literally have too much stuff!
Yes, running the scripts does create just a touch more hands-on than other methods. But that's also part of being an administrator on a full blown Client/Server suite. You have to do a bit more to make sure everything runs smoothly, even if it is just for your own consumption. And having said that, it's up to you.
Let me know how you decide to proceed, as others might find some value in their usage. But keep in mind, too... Not everyone running Plex on an Asustor has the CPU I or even you have in their devices... And what works for either of us might very well NOT work well for someone else...