(port) > Software > YouTube >

https://github.com/philbot9/youtube-comment-scraper/

Downloads the comments on YouTube videos.

Used to be available at ytcomments.klostermann.ca but that IP got banned because of overuse.

• WARNING - Do not overuse this program, or maybe you will get IP-banned. The authors' GUI website did.

• See my youtube-comment-scraper.sh script for how I use it.

• 2020-06-09 - 1.0.2 cli on Devuan_beowulf_3.0.0_amd64
• 2020-05-09 - 1.0.1 cli on Debian 10.1.0-amd64-xfce-CD-1
• 2019-11-01 - 1.0.1 (cli) on Debian 10.1.0-amd64-xfce-CD-1
• 2019-11-01 - git (gui) on Debian 10.1.0-amd64-xfce-CD-1
• 2019-10-13 - git (cli) on Debian 9.9.0-i386-xfce-CD-1

• I think I had this working on Windows momentarily, but I changed nothing and it just stopped working after I forgot about it for a while.

2020-05-28 - 1.0.2 (cli) ∞

Horrifically-large comment downloads were failing. See:

• https://github.com/philbot9/youtube-comment-scraper-cli/issues/46
• https://github.com/philbot9/youtube-comment-scraper/issues/35

• https://github.com/philbot9/youtube-comments-task/issues/26

Per the author's instructions, I did:

\sudo  \npm  install  -g youtube-comment-scraper-cli@1.0.2
youtube-comment-scraper --format csv --stream -- koPmuEyP3a0 | tee output.csv

✕ API response does not contain a "content_html" field

.. testing.

2019-11-01 - 1.0.1 (cli) ∞

Installation ∞

# Optionally keep a copy..
# \git  clone https://github.com/philbot9/youtube-comment-scraper-cli/
# \cd                                   ./youtube-comment-scraper-cli/

\sudo  \apt-get  -y  install  \
  node.js  \
  npm  \
` # `
# WARNING - as of 2019-11-01 I get:
#   npm WARN npm npm does not support Node.js v10.15.2
\sudo  \npm  install  -g  youtube-comment-scraper-cli

Running ∞

Do note that downloading can take a whole lot of time..

source_video_id='jNQXAC9IVRw'
\youtube-comment-scraper  --format csv  --outputFile "$source_video_id".csv  "$source_video_id"

It seemed too slow, so I switched to the GUI for a while, but the dependencies for the GUI, and running it, is just abysmal!

2019-11-01 - git (gui) ∞

not recommended

Installation (GUI/web browser) ∞

Install various dependencies, see https://github.com/spiralofhope/shell-random/tree/master/zombie/youtube-comment-scraper--install-dependencies.sh

\sudo  \apt-get  -y  install  \
  apt-transport-https  \
  ca-certificates  \
  curl  \
  gnupg2  \
  software-properties-common  \
` # `
\curl  -fsSL https://download.docker.com/linux/debian/gpg  |  \sudo  \apt-key  add -

# optionally verify the docker key is correct:
\sudo  \apt-key  fingerprint 0EBFCD88  |  \grep  '9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88'

\sudo  \add-apt-repository  \
   "deb [arch=amd64] https://download.docker.com/linux/debian/ \
   $(lsb_release -cs) \
   stable"
\sudo  \apt-get  update
\sudo  \apt-get  -y  install docker-ce docker-ce-cli containerd.io

# optionally test docker:
\sudo  \docker  run hello-world

if  [ ! -f '/var/run/docker.pid' ]; then
  \sudo  \dockerd  &
fi
\sudo  \docker  pull mongo

Running ∞

NOTE: You should probably examine youtube-comment-scraper.sh

if  [ ! -f '/var/run/docker.pid' ]; then
  \sudo  \dockerd  2> /dev/null &
fi
\sudo  \docker  run  --name="ytcomments-mongo" --restart="always"  -v  "$PWD":/data/db  -d mongo
\sudo  ./deploy

• Visit http://localhost:49161/

  • http://0.0.0.0:49161/ also worked for me.

Stopping:

\docker  stop  $( \sudo  \docker  ps -aq )
\docker  rm    $( \sudo  \docker  ps -aq )
# This worked once but stopped working..
\systemctl  stop  containerd
\systemctl  stop  dockerd
# I have no idea how to actually fucking stop docker.
\killall  dockerd  containerd
\rm  /var/run/docker.pid
\umount  /var/lib/docker/overlay2/*

# For some fucked up reason the permissions get messed up..
#\find  /home/user/  -exec \chmod  g+rw       {} \;
#\find  /home/user/  -exec \chown  user:user  {} \;

# See what is running:
\sudo  \docker  ps
\sudo  \docker  restart $container_id
\sudo  \docker  stop    $container_id

2019-10-13 - git (cli) ∞

some random notes

https://nodejs.org/dist/latest-v0.12.x/

node-ansi-regex
node-marked

not sure:

npm WARN package.json [email protected] Non-dependency in bundleDependencies: imurmurhash

linuxbrew - https://brew.sh/

sh -c "$( curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh )"

.. fails sigh

test -d ~/.linuxbrew && eval $(~/.linuxbrew/bin/brew shellenv)
test -d /home/linuxbrew/.linuxbrew && eval $(/home/linuxbrew/.linuxbrew/bin/brew shellenv)
test -r ~/.bash_profile && echo "eval \$($(brew --prefix)/bin/brew shellenv)" >>~/.bash_profile
echo "eval \$($(brew --prefix)/bin/brew shellenv)" >>~/.profile

to .zshrc

export PATH="$HOME/.linuxbrew/bin:$PATH"
export MANPATH="$HOME/.linuxbrew/share/man:$MANPATH"
export INFOPATH="$HOME/.linuxbrew/share/info:$INFOPATH"

installation notes ∞

#!/usr/bin/env  sh


# FIXME - I still don't have "npm"


# As root:
if ! [ "$USER" = 'root' ]; then
 \echo  "enter root password"
 /bin/su  --command  "$0"
else


#:<<'}'  # NodeJS
         # https://github.com/nodesource/distributions/blob/master/README.md
{
  \apt-get  install  --yes  curl

  # It seems to just be available:
  #\apt-get  install -y nodejs
  # Also possibly:
  #\apt-get  install  --yes  build-essential nodejs
  # The simple manual way was:
  #\curl -sL https://deb.nodesource.com/setup_12.x | bash -
  #\apt-get  install -y  nodejs

  # The more thorough way:

  \curl  --silent  --location  https://dl.yarnpkg.com/debian/pubkey.gpg | \apt-key  add -
  \echo  'deb https://dl.yarnpkg.com/debian/ stable main' | \tee  '/etc/apt/sources.list.d/yarn.list'
  # The following does an apt-get update:
  \curl  --silent  --location  https://deb.nodesource.com/setup_12.x | bash -
  \apt-get  install  --yes  \
    nodejs  \
    yarn  \
  ` # `

  # You may also need development tools to build native addons:
  #\apt-get  install  --yes  gcc  g++  make

  # I don't know why the fuck it doesn't do this already..
  \ln --symbolic  --force  /usr/bin/nodejs  /usr/bin/node


# Other people say these are dependencies, but some are not pulled in from installing nodejs from the repo.
sudo apt-get install build-essential curl git m4 ruby texinfo libbz2-dev libcurl4-openssl-dev libexpat-dev libncurses-dev zlib1g-dev

}


fi   # The above is run as root

another install script I had..

\sudo  \tar  -xf node-v0.12.18.tar.xz   --directory /usr/local --strip-components 1

Alternatives ∞

• /tag/youtube+archivism+writing

• https://github.com/egbertbouman/youtube-comment-downloader

Last updated 2021-02-20 at 08:59:47