I spent last weekend thinking through and implementing some ideas that have been rattling around my head for months. I wanted a way to automatically download videos from every YouTube channel I followed and to download any ad hoc video I came across in my day-to-day life that I wanted to watch later. What I didn’t want was to manage any of this myself. I had been using my RSS reader to get new videos from the channels I subscribed to, but I had to then use my RSS reader, filter through the items, add each item to my YouTube list, then initiate the download command in the terminal myself. I explained how I did this in this post from November, but again, I wanted something more automated.

So I figured out how to make it more automated… by automating most of it!

Here’s how it works.

What you need

I won’t go through what yt-dlp is or why I set it up the way I did because I explained that in my previous post, so please read that if you need a quick explainer. What this post will go through is how to setup yt-dlp to “subscribe” to your own list of YouTube channels and to download your own list of ad hoc (or watch later) videos automatically.

I’m using a 2018 Intel-based Mac mini with yt-dlp installed per their installation instructions. That means this setup is intended for people with a device running macOS. Optionally, to make this setup just that tad bit more awesome, you will also need:

• Plex
• YouTube-Agent.bundle (a Plex plug-in to add YouTube metadata to each video)
• Hazel
• Apple Shortcuts

First things first

Before we can automate anything, we have to create a few files first. In my previous post, I created an archive.md file to log all the videos I’ve downloaded. This is important. The archive file is the most important piece to this process, so create it now. Good? Okay. I then created a file called subscriptions.md and placed that in my main YouTube folder where all my files live. In this subscriptions.md file, I added each channel I wanted to subscribe to, one channel per line. For example, a snippet of my file looks like:

https://www.youtube.com/@PeopleMakeGames
https://www.youtube.com/@PeterMcKinnon
https://www.youtube.com/@PickUpLimes
https://www.youtube.com/@primitivetechnology9550

Essentially, that’s all you need. There’s nothing fancy to this. Keep in mind that this will download all the videos these channels produce, including shorts and other things. I don’t mind that, so this is good enough for me.

The next thing, and this is where the magic happens, is to simulate your yt-dlp command and add each file to your archive.md file. To do this, add both the --simulate and --force-write-archive commands. --simulate tells yt-dlp not to download anything and --force-write-archive adds all the videos from all your subscriptions to your archive.md file. Do you see why this is important? When you’re ready, you can now run your command. Mine looks like this:

yt-dlp -P "/path/to/YouTube/" -P "temp:tmp" -P "subtitle:subs" --simulate --force-write-archive -o '%(uploader)s-%(upload_date)s-%(title)s [%(id)s].%(ext)s' --download-archive '/path/to/archive.md' -f 'bestvideo+bestaudio/best' --sub-langs all,-live_chat --embed-subs --yes-playlist --batch-file '/path/to/subscriptions.md'

Depending on how many subscriptions you’ve added, this process could take a long time, so go outside, hang out with friends, read a book, do something else while this process runs. By the end of this, your archive.md file will most likely be huge. Mine has over 15,000 lines on it but barely cracks 300 KB. So really, you can leave this file alone forever and it’ll never cause you any issues (knock on wood).

This takes care of my subscriptions, but what about any ad hoc videos I may want to download and watch later? Well, I mostly answered that in my previous post (you should really read that post, it’s pretty good). Specifically, the important part is creating a youtube.md file and including the --batch-file command that points to it. Then, as I go along with my regular internet surfing life, I add links to any videos I want to watch later to this file using a simple Shortcut and my system automatically downloads it. How?

Well, I’m glad you asked!

Let’s automate the shit out of this

Time to create a few more files. I created four and called them: subscriptions.sh, subscriptions.plist, youtube.sh, and youtube.plist. I placed these files in my main YouTube folder, but the plist files can be moved or copied to /Library/LaunchAgents, which we’ll do later. I’ll focus on my subscriptions first.

In my subscriptions.sh file, I added:

#!/bin/bash
/usr/local/bin/yt-dlp -P "/path/to/YouTube/" -P "temp:tmp" -P "subtitle:subs" -o '%(uploader)s-%(upload_date)s-%(title)s [%(id)s].%(ext)s' --download-archive '/path/to/archive.md' -f 'bestvideo+bestaudio/best' --sub-langs all,-live_chat --embed-subs --yes-playlist --batch-file '/path/to/subscriptions.md'

This is similar to the code above but without the --simulate and --force-write-archive commands. This is the real deal command, so if you haven’t run the --simulate command, then you will be downloading everything in your subscriptions.md file. Maybe that’s what you want, so you do you. I’m not your mom.

The subscriptions.plist file is where the magic happens (lots of magic happening today). This file contains this bit of code:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>subscriptions</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/subscriptions.sh</string>
    </array>
    <key>StartInterval</key>
    <integer>21600</integer>
    <key>disabled</key>
    <false/>
</dict>
</plist>

I’ve simplified my plist file compared to say, Jason, which I took a lot of inspiration from, but I customized his setup to fit my needs. He includes a few log files that are good practice to include, but I like living on the edge, so I didn’t include them. If you’d like to add them, include something like this in your plist file:

<key>StandardErrorPath</key>
<string>/path/to/ytdl_error.log</string>
<key>StandardOutPath</key>
<string>/path/to/ytdl_st_out.log</string>

A few things to note:

The Label key is essential if you plan on adding more than one file to the LaunchAgents folder. I gave this one a label of subscriptions to differentiate it from the others.

I set my StartInterval to 21600, which is 21,600 seconds, or 6 hours. I set it to this because this is not something I want running all the time, and it’s not something I want to be checking my downloads folder for to see if there’s something new. That’s the behavior I wanted to eliminate, so putting it to 6 hours has helped me pull away from my devices while still ensuring I have something new to watch a few times throughout the day. Honestly, I could set this to 24 hours and still be happy, which might be something I do in the future. Stay tuned.

Once this is all done and setup to your liking, navigate to /Library/LaunchAgents and copy your plist file into it. macOS might notify you that subscriptions.sh is an item that can run in the background. That’s exactly what you want. You may also have to add bash to your Files and Folders section in the Privacy & Security pane in your System Preferences. This ensures bash has permission to run on your system. Boy did that one have me scratching my head for a while.

And that’s essentially it. Rinse and repeat with the youtube.sh file:

#!/bin/bash
/usr/local/bin/yt-dlp -P "/path/to/YouTube/" -P "temp:tmp" -P "subtitle:subs" -o '%(uploader)s-%(upload_date)s-%(title)s [%(id)s].%(ext)s' --download-archive '/path/to/archive.md' -f 'bestvideo+bestaudio/best' --sub-langs all,-live_chat --embed-subs --yes-playlist --batch-file '/path/to/youtube.md'

And the youtube.plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>youtube</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/youtube.sh</string>
    </array>
    <key>StartInterval</key>
    <integer>60</integer>
    <key>disabled</key>
    <false/>
</dict>
</plist>

And you have a fully automated YouTube setup. Note, I set the StartInterval to 60 because I don’t mind this running all the time because I don’t add too many ad hoc videos anymore, so when I do, I’d like to watch it sooner rather than later.

Because you took the time to add all your channel videos to your archive.md file, from this point forward, this setup will ensure you only download new videos as they’re posted to your subscriptions. Pretty nice, right?

But there’s one more piece to this that’s like icing on the cake. It’s not required, but I feel like this really completes the entire setup.

Tidying things up

The last thing I did was use a few apps to make this entire experience a tad nicer. First, I launched Hazel and added my YouTube folder. I then created… many rules:

• Go into subfolders. I set the Kind to Folder and selected Run rules on folder contents. This rule ensures every rule hereafter is run within all the folders contained in my main YouTube folder. This is important because:
• ‌Delete empty folders. Each rule hereafter creates lots of folders, and this rule tidies things up by deleting any empty folder. This rule is also simple. First, I set the Kind to Folder, and the Sub-file/folder Count to 0. I then Move any matches to the Trash. This is very important because:
• Sorting subscriptions by folder. I created a rule for each and every subscription I have. The rule here is simple. I set the Name to contains the name of my subscription, e.g. MKBHD. I then used the Sort into subfolder option to sort my videos into its own folder. The pattern I used was Subscriptions ▸ <name of subscription>. Why do this? Because I don’t have time to watch TV all the time, and sometimes I want to binge through one channel’s output, and this makes that easier. After adding a rule for each of my subscriptions, I created my final rule:
• Recently Added. This rule moves any file that hadn’t already been matched to a folder I called Recently Added. Again, this rule is very simple. All it has is Kind set to Movie and it Sort[s] into subfolder any matches to my folder called Recently Added.

Having this run 24/7 removes yet another cognitive weight from my life and just keeps things tidy. Work smarter not harder, right? And finally, on to the final step.

In Plex (you did download and install it, right?), I downloaded the YouTube-Agent.bundle plug-in and installed it per their installation instructions. This plug-in requires a bit of setup before you can take advantage of it. You have to create your own YouTube API key, which isn’t too tough, and each one of your movies needs to have the YouTube ID in its filename. Review my yt-dlp command and you may notice I’ve added [%(id)s] to my filename. This option is there to ensure YouTube-Agent.bundle adds the correct metadata to my movies.

Finally, I created two libraries, one called Subscriptions and the other called Recently Added. I pointed each to its respective folder, and in the Advanced section, I chose the YouTubeMovie plug-in in the Agent option. This is also where I added my API key.

If everything went well, then you truly have a (mostly) automated YouTube setup. I say mostly because I still have to add any ad hoc videos to my youtube.md file, but it literally takes 2 seconds to do so and that’s the only actual work I have to do. For the most part, I haven’t had to troubleshoot this setup yet, but knowing the pace of technology, I will have to sometime in the future.

Until then, I’ve been really enjoying this setup. Dare I say, it feels a tad magical, and I love it. Maybe you will, too.

I grew up in the ‘90s, and one of the most popular channels in my household growing up was MTV. Our family loved tuning into MTV and watching music video after music video (yes, MTV used to play music videos!). Every day before school, my mom would turn on the TV, and we would all get ready for school listening to the music from these videos. When I finished getting ready for school, I would sit on the couch waiting for my siblings to finish getting ready and watch music videos by Aerosmith and Jamiroquai and Mariah Carey and Nirvana, and on and on it went, music video after music video. This was my childhood, and I didn’t know I missed it until I inadvertently stumbled into my own music video channel.

A few years ago, I made an attempt to get rid of my reliance on Google and their services. At the time, ever since Google Reader’s demise (RIP), I had began transitioning away from Google, but I never made the full transition. I had switched from Google Search to DuckDuckGo, from Google’s productivity suite to Markdown files and Apple’s capable alternatives, from Gmail to iCloud Mail, but the one service I could not replace was YouTube. For years I kept my Google account active because of YouTube. I wanted to keep track of my subscriptions, to like videos, to create playlists—to use YouTube, essentially. But there was a way I could delete my Google account and still use YouTube without having to visit their website and watch their ads and train their algorithm.

That way was by using youtube-dl.

Using a command-line and a bit of configuration, I could use youtube-dl to download any video from YouTube at whatever quality I wanted, with whatever settings I wanted, and watch it later on whatever device I wanted. The original youtube-dl has, sadly, gone dormant, but because the project is open-source, some awesome people have forked it and made their own version, yt-dlp.

yt-dlp picks up where youtube-dl left off, and they have been awesome at keeping this project active and up-to-date.

Once I had yt-dlp setup, the next challenge was to “subscribe” to all my favorite channels so I wouldn’t miss any videos. Fortunately, YouTube has made it so each channel has its own RSS feed, and many RSS readers support YouTube right out of the box. My feed reader of choice, Reeder, supports YouTube, so adding all my favorite channels was a breeze.

Finally, I needed a place to watch my videos, and for me, the best app for this is Plex. Plex has been around forever, unlike others (remember Boxee???), and they have apps for most devices out there. I use them on my Apple TV, and the app has been nothing but great.

With yt-dlp setup, with a way to get all the videos I want, and with a way to watch them, my dependence on my Google account vanished, and I could finally delete my Google account. So a few years ago, I did.

But wait, I might hear you saying, wasn’t this supposed to be about creating my own music video channel?

Why yes! Yes it was. I wanted to get all that out of the way to get to how I do things. First things first, here is the command I use to download my videos:

yt-dlp -o '/path/to/YouTube/%(uploader)s-%(upload_date)s-%(title)s.%(ext)s'
--download-archive '/path/to/archive.md'
-f 'bestvideo+bestaudio/best'
--sub-langs all,-live_chat
--embed-subs
--yes-playlist
--batch-file '/path/to/youtube.md'

I have a dedicated YouTube folder on an external SSD (CALYPSO) that saves each video with the channel name first, the upload date, then the title of the video. For example, downloading this video by Jomboy Media will output as: Jomboy Media-20221117-Tom Brady falls and trips player during botched trick play, a breakdown.webm. I prefer this format because sometimes I can go days or weeks without watching videos, and when I find the time, I like watching a certain channel’s output by the order they were released and catch up that way. It’s how I like to watch my videos.

The --download-archive setting helps ensure I don’t download the same video twice.

The -f 'bestvideo+bestaudio/best' ensures I get the highest quality version available.

I follow lots of foreign-language channels, so --sub-langs all,-live_chat helps download subs, and --embed-subs simply embeds the file in the video itself, and when I go to watch it on Plex, I can select the file and view the video with subtitles.

--yes-playlist downloads playlists. Simple enough.

Finally, --batch-file and the file itself is where some of the magic happens. I can go through my day, and I can simply add the URLs for all the YouTube videos I come across in my various feeds and append them to this file, and when I’m ready to download them, I run my yt-dlp command once, and all my videos start downloading. It’s really nice.

I know there are ways to run this automatically or on a schedule, but I download my videos to an SSD I take with me everywhere, and I don’t want my desktop at home to be my only media server. So I run this command manually when I need to, and it has worked fine for me.

As part of my RSS subscriptions, I subscribe to a lot of music websites and YouTube channels. Whenever there’s a new video out, either from someone I know or, especially, someone I don’t, I add the video to my YouTube.md file, and sometime later, after adding more and more videos to this file, I download all the videos.

Within my main YouTube folder, I have another folder called music, and within this folder, I add every music video and song I have downloaded. I do this for days, weeks, sometimes months, and I don’t watch them. I let them pile up for a while, and when I’m feeling the urge to sit on my couch and jam out to some music videos, I navigate to this folder in Plex—and here’s the fun part—I click on the “shuffle” button.

Music video bliss.

Doing this has brought back all the nostalgia from my childhood, back when I could sit on my couch before having to go to school and simply watch and listen to some amazing music. Those really were the days…