Fix YouTube downloads failing with 403 by updating Python and adding Deno

yt-dlp deprecated Python 3.10, which is what the Ubuntu 22.04 base image
ships, and now requires a JavaScript runtime to solve YouTube's player
challenges. Without one, extraction falls back to the android_vr client
and every YouTube download fails with "HTTP Error 403: Forbidden".

- Move the base image to Ubuntu 24.04 (Python 3.12)
- Fetch Deno in the utils stage and ship it in the final image
- Replace the pip install of pycryptodomex with python3-pycryptodome,
  which drops the build-essential install/purge dance entirely
- Remove the default "ubuntu" user of 24.04, which occupies UID/GID 1000
- Bump libicu70 to libicu74 for TwitchDownloaderCLI
- Document the Python and JavaScript runtime requirements

Deno does not publish armv7 binaries, so the fetch script skips that
architecture and the image is built without a JS runtime there, same as
before.

Fixes #1168

Dockerfile

@@ -6,11 +6,12 @@ COPY docker-utils/*.sh .
 RUN chmod +x *.sh
 RUN sh ./ffmpeg-fetch.sh
 RUN sh ./fetch-twitchdownloader.sh
+RUN sh ./fetch-deno.sh
 
 
-# Create our Ubuntu 22.04 with node 16.14.2 (that specific version is required as per: https://stackoverflow.com/a/72855258/8088021)
-# Go to 20.04
-FROM ubuntu:22.04 AS base
+# Create our Ubuntu 24.04 with node 16.14.2 (that specific version is required as per: https://stackoverflow.com/a/72855258/8088021)
+# 24.04 is required because yt-dlp deprecated Python 3.10, which is what 22.04 ships
+FROM ubuntu:24.04 AS base
 ARG TARGETPLATFORM
 ARG DEBIAN_FRONTEND=noninteractive
 ENV UID=1000
@@ -23,9 +24,11 @@ ENV npm_config_cache=/app/.npm
 
 # Use NVM to get specific node version
 ENV NODE_VERSION=16.14.2
-RUN groupadd -g $GID $USER && useradd --system -m -g $USER --uid $UID $USER && \
+# Ubuntu 24.04 ships a default "ubuntu" user occupying UID/GID 1000, remove it first
+RUN { userdel -r ubuntu || true; groupdel ubuntu || true; } 2>/dev/null; \
+    groupadd -g $GID $USER && useradd --system -m -g $USER --uid $UID $USER && \
     apt update && \
-    apt install -y --no-install-recommends curl ca-certificates tzdata libicu70 libatomic1 && \
+    apt install -y --no-install-recommends curl ca-certificates tzdata libicu74 libatomic1 && \
     apt clean && \
     rm -rf /var/lib/apt/lists/*
 
@@ -74,17 +77,14 @@ RUN npm config set strict-ssl false && \
 FROM base
 RUN npm install -g pm2 && \
     apt update && \
-    apt install -y --no-install-recommends gosu python3-minimal python-is-python3 python3-pip atomicparsley build-essential && \
-    pip install pycryptodomex && \
-    apt remove -y --purge build-essential && \
-    apt autoremove -y --purge && \
+    apt install -y --no-install-recommends gosu python3-minimal python-is-python3 python3-pycryptodome atomicparsley && \
     apt clean && \
     rm -rf /var/lib/apt/lists/*
 WORKDIR /app
 # User 1000 already exist from base image
-COPY --chown=$UID:$GID --from=utils [ "/usr/local/bin/ffmpeg", "/usr/local/bin/ffmpeg" ]
-COPY --chown=$UID:$GID --from=utils [ "/usr/local/bin/ffprobe", "/usr/local/bin/ffprobe" ]
-COPY --chown=$UID:$GID --from=utils [ "/usr/local/bin/TwitchDownloaderCLI", "/usr/local/bin/TwitchDownloaderCLI"]
+# Copies ffmpeg, ffprobe, TwitchDownloaderCLI and (where available) deno in one go.
+# deno is optional as it does not ship binaries for every architecture we build for.
+COPY --chown=$UID:$GID --from=utils [ "/usr/local/bin/", "/usr/local/bin/" ]
 COPY --chown=$UID:$GID --from=backend ["/app/","/app/"]
 COPY --chown=$UID:$GID --from=frontend [ "/build/backend/public/", "/app/public/" ]
 #COPY --chown=$UID:$GID --from=python ["/app/TwitchDownloaderCLI","/usr/local/bin/TwitchDownloaderCLI"]

README.md

@@ -31,7 +31,8 @@ NOTE: If you would like to use Docker, you can skip down to the [Docker](#Docker
 Required dependencies:
 
 * Node.js 16
-* Python
+* Python 3.11 or above (yt-dlp deprecated older versions)
+* A JavaScript runtime, [Deno](https://deno.com/) by default (required by yt-dlp to solve YouTube's player challenges, see the [yt-dlp wiki](https://github.com/yt-dlp/yt-dlp/wiki/EJS)). Without it YouTube downloads fail with `HTTP Error 403: Forbidden`
 
 Optional dependencies:
 

docker-utils/fetch-deno.sh

@@ -0,0 +1,37 @@
+#!/bin/sh
+
+# yt-dlp requires a JavaScript runtime to solve YouTube's player challenges.
+# Without one, YouTube downloads fail with "HTTP Error 403: Forbidden".
+# See: https://github.com/yt-dlp/yt-dlp/wiki/EJS
+
+case $(uname -m) in
+  x86_64)
+    ARCH=x86_64-unknown-linux-gnu;;
+  aarch64)
+    ARCH=aarch64-unknown-linux-gnu;;
+  *)
+    echo "(INFO) Deno is not available for $(uname -m), skipping JavaScript runtime installation."
+    echo "(INFO) YouTube downloads may be limited on this architecture."
+    exit 0
+esac
+
+echo "(INFO) Architecture detected: $ARCH"
+echo "(1/5) READY - Install temp dependencies in deno obtain layer"
+apt-get update && apt-get -y install curl unzip ca-certificates
+echo "(2/5) DOWNLOAD - Acquire latest deno release"
+curl -o deno.zip \
+    --connect-timeout 5 \
+    --max-time 120 \
+    --retry 5 \
+    --retry-delay 0 \
+    --retry-max-time 40 \
+    -L "https://github.com/denoland/deno/releases/latest/download/deno-${ARCH}.zip"
+echo "(3/5) PROVISION - Provide deno from deno obtain layer"
+unzip -o deno.zip -d /usr/local/bin
+chmod +x /usr/local/bin/deno
+echo "(4/5) Smoke test"
+/usr/local/bin/deno --version
+echo "(5/5) CLEANUP - Remove temporary downloads from deno obtain layer"
+rm -f deno.zip
+apt-get -y remove curl unzip
+apt-get -y autoremove