Search youtube without a browser

ohnonot · 2020-10-04 08:27:56

Search youtube without a browser, choose from results & copy page link to clipboard & a wrapper to use tor.
I moved 3 scripts to their own repository now, which I should've called "media-streaming-site-tools", but I just called it "youtube-tools". In light of recent events, maybe that will turn out to be a mistake?
wink

Last edited by ohnonot (2021-07-04 09:11:01)

Martin · 2020-10-04 12:44:09

martin@he2:~$ youtube-search.py GC32
  File "/home/martin/bin/youtube-search.py", line 32
    def __init__(self, search_terms: str, max_results=None):
                                   ^
SyntaxError: invalid syntax

/Martin

Sector11 · 2020-10-04 12:56:08

same error here.

vasa1 · 2020-10-04 14:53:27

Works for me (on Kubuntu 20.04) with Python 3.8.2.

Martin · 2020-10-04 15:48:34

Changing to

#!/usr/bin/python3

made it work for me.

/Martin

sleekmason · 2020-10-04 16:16:56

Martin wrote:

Changing to
#!/usr/bin/python3
made it work for me.
/Martin

Here as well:) Thank you! What a cool script!

ohnonot · 2020-10-06 06:08:31

Martin wrote:

Changing to
#!/usr/bin/python3
made it work for me.

Wow, I didn't know Debian still defaults to python2!
Anyhow, Code fixed, updated.

vasa1 · 2020-10-07 04:29:55

@ohnonot, when I enter "python" or "python scripts" or "coding in python" with or without quotes as the search term, I don't get any results in youtube-search.

Changing "python" to "haskell" works fine

Last edited by vasa1 (2020-10-07 13:05:02)

ohnonot · 2020-10-07 20:35:54

vasa1 wrote:

@ohnonot, when I enter "python" or "python scripts" or "coding in python" with or without quotes as the search term, I don't get any results in youtube-search.
Changing "python" to "haskell" works fine

Not sure why you crossed that out again, but I have noticed something similar too.
I suspect YT has its own way of determining whether it likes your query or not - esp. after you've been playing with the script for a while, making dozens of searches. Usually it works again when you simply enter the same search term again.
But I cannot enter the search term "search youtube" - it always returns 0 results. Weird.

vasa1 · 2020-10-08 02:27:39

ohnonot wrote:

Not sure why you crossed that out again, but I have noticed something similar too.
I suspect YT has its own way of determining whether it likes your query or not - esp. after you've been playing with the script for a while, making dozens of searches. Usually it works again when you simply enter the same search term again.
But I cannot enter the search term "search youtube" - it always returns 0 results. Weird.

Thanks for confirming!

I crossed out my post because I noticed the problem wasn't 100% reproducible and had a feeling it was possibly something wrong at my end.

All the same it's very useful. Search for something and pop the link into mpv!

(BTW, for my personal use, I've removed the input_with_prefill aspect and changed |xclip to |xclip -selection clipboard.)

ohnonot · 2020-10-10 07:13:24

vasa1 wrote:

for my personal use, I've removed the input_with_prefill aspect

Yes, that can be annoying when the clipboard is filled with useless and sometimes long text, but again very useful for highlight & search situations (like e.g. FF's context menu has).

The script would need command line options, I guess that will be my next foray into the wrold of python.

ohnonot · 2020-10-15 19:14:28

I added command line options to youtube-search:

$> youtube-search -h

Search Youtube.

Usage: youtube-search [OPTIONS] [SEARCH STRING]

You can provide a search string on the command line or enter it interactively.

It helps if xclip and/or tput are available, but the script works without those.

OPTIONS
    -c      Use Clipboard instead of primary selection
    -p      Prefill the search with whatever is found in the selected clipboard
    -l      List more detail for each result
    -f      Don't cut long strings to terminal width
    -m int  Maximum number of results shown
    -a      Append to /tmp/youtube-search.log instead of overwriting
    -h      This help

Empty search results can sometimes happen; AFAICS that is entirely YT's decision and not a problem with the script. Just try a few times.

Last edited by ohnonot (2021-07-04 09:11:52)

damo · 2020-10-15 21:24:04

Very cool script smile

Now in my main menu as

Youtube Search,x-terminal-emulator -e yt-search.py

ohnonot · 2020-10-16 12:47:04

Thanks.
I have now added tor functionality: the script can connect to an existing tor session. It actually works better that way for me (I guess YT does not like my IP very much).
Then I've added a wrapper shell script that starts a dedicated tor session, then performs the YT search.
If both youtube-search and youtube-search-tor are in PATH it should work.
Tor functionality also adds a dependency for python 'requests', which seems to be provided by the 'python3-requests' package. I would appreciate feedback about how this works on Debian stable.

Last edited by ohnonot (2021-05-27 17:31:19)

Martin · 2020-10-16 20:35:15

My first try was not a smashing success:

martin@he2:~$ youtube-search-tor
tor is /usr/sbin/tor
youtube-search is /home/martin/bin/youtube-search
Pidof tor: 15002
------------------------------------------------------------------------------------
Enter search terms for youtube (minimum length 3 chars): 
eastory
Timeout reached, no response.
Traceback (most recent call last):
  File "/home/martin/bin/youtube-search", line 267, in <module>
    results = YoutubeSearch(uts, max_results).to_dict()
  File "/home/martin/bin/youtube-search", line 54, in __init__
    self.videos = self.search()
  File "/home/martin/bin/youtube-search", line 81, in search
    results = self.parse_html(response)
UnboundLocalError: local variable 'response' referenced before assignment

Doing the same search just using youtube-search works fine.

I have never used tor before...

/Martin

ohnonot · 2020-10-17 07:48:48

Apparently tor is installed.
Try using the tor browser - does that work normally? In other words, does tor work on your system?
This seems to be a timeout problem. You might have hit a very slow node, tor chooses them randomly. Have you just tried again?
There's also /tmp/youtube-search-tor-<username>.log

Naik · 2020-10-17 14:33:53

Hey! Thanks for that!

I had to slightly change both scripts according to naming and version problems, I guess.
I just took the script from notabug and saved it as ~/bin/yt-search.py, so I had to update the calls in youtube-search-tor accordingly and than the script from notabug doesn't take the -P and -T options, but after coping with this everything works fine:

┌─[naik@GerryTSSD ~]
└──$ ~/bin/youtube-search-tor
tor ist /usr/sbin/tor
/home/naik/bin/youtube-search-tor: Zeile 4: type: youtube-search: Nicht gefunden.
┌─[naik@GerryTSSD ~]
└──$ nano ~/bin/youtube-search-tor
┌─[naik@GerryTSSD ~]
└──$ ~/bin/youtube-search-tor
tor ist /usr/sbin/tor
yt-search.py ist /home/naik/bin/yt-search.py
Pidof tor: 23197
/home/naik/bin/youtube-search-tor: Zeile 32: youtube-search: Kommando nicht gefunden.
┌─[naik@GerryTSSD ~]
└──$  ~/bin/youtube-search-tor
tor ist /usr/sbin/tor
yt-search.py ist /home/naik/bin/yt-search.py
Pidof tor: 23293
/home/naik/bin/youtube-search-tor: Zeile 32: youtube-search: Kommando nicht gefunden.
┌─[naik@GerryTSSD ~]
└──$ geany ~/bin/youtube-search-tor
┌─[naik@GerryTSSD ~]
└──$ ~/bin/youtube-search-tor
tor ist /usr/sbin/tor
yt-search.py ist /home/naik/bin/yt-search.py
Pidof tor: 23644
option -T not recognized

Search Youtube.

Usage: youtube-search [OPTIONS] [SEARCH STRING]

You can provide a search string on the command line or enter it interactively.

It helps if xclip and/or tput are available, but the script works without those.

OPTIONS
    -c      Use Clipboard instead of primary selection
    -p      Prefill the search with whatever is found in the selected clipboard
    -l      List more detail for each result
    -f      Don't cut long strings to terminal width
    -m int  Maximum number of results shown
    -a      Append to /tmp/yt-search.py.log instead of overwriting
    -h      This help

┌─[naik@GerryTSSD ~]
└──$ ~/bin/youtube-search-tor "Käpt'n Peng"
tor ist /usr/sbin/tor
yt-search.py ist /home/naik/bin/yt-search.py
Pidof tor: 23812
option -T not recognized

Search Youtube.

Usage: youtube-search [OPTIONS] [SEARCH STRING]

You can provide a search string on the command line or enter it interactively.

It helps if xclip and/or tput are available, but the script works without those.

OPTIONS
    -c      Use Clipboard instead of primary selection
    -p      Prefill the search with whatever is found in the selected clipboard
    -l      List more detail for each result
    -f      Don't cut long strings to terminal width
    -m int  Maximum number of results shown
    -a      Append to /tmp/yt-search.py.log instead of overwriting
    -h      This help

┌─[naik@GerryTSSD ~]
└──$ geany ~/bin/youtube-search-tor
┌─[naik@GerryTSSD ~]
└──$ ~/bin/youtube-search-tor "Käpt'n Peng"
tor ist /usr/sbin/tor
yt-search.py ist /home/naik/bin/yt-search.py
Pidof tor: 24534
Search string: 9222 Käpt'n Peng
1 ----------------------------------------------------------------------------------
  Title:       Shaban & Käptn Peng - Sie mögen sich
  Description: Ein Lied vom Album "Die Zähmung der Hydra" Zeichnungen: Lane Schäfer 
  Duration:    6:56
  Url:         https://www.youtube.com/watch?v=apCal7ihvy0
2 ----------------------------------------------------------------------------------
  Title:       Käptn Peng im Wortgewand
  Description: Der Käptn hat seinen Verstand noch immer nicht wieder gefunden. Dafür
  Duration:    2:30
  Url:         https://www.youtube.com/watch?v=RvUeTzcBdsI
3 ----------------------------------------------------------------------------------
  Title:       shaban & Käptn Peng - werbistich
  Description: Ein Waldspaziergang, eigentlich nix verfängliches.. Aber wenn einen d
  Duration:    6:47
  Url:         https://www.youtube.com/watch?v=MpOVJWi6ul8
4 ----------------------------------------------------------------------------------
  Title:       Käptn Peng & Die Tentakel von Delphi - Der Anfang ist nah
  Description: Das erste Lied vom Album "Expedition ins O" von den Tentakeln von Del
  Duration:    5:45
  Url:         https://www.youtube.com/watch?v=vqRWMDv0r78
5 ----------------------------------------------------------------------------------
  Title:       KÄPTN PENG & DIE TENTAKEL VON DELPHI - Gelernt
  Description: Ein Lied aus dem Album "Das Nullte Kapitel" konsumquatsch: Kreismusik
  Duration:    3:21
  Url:         https://www.youtube.com/watch?v=JtULNuoS3jg
6 ----------------------------------------------------------------------------------
  Title:       shaban & Käptn Peng - parantatatam
  Description: Nachdem der Käptn den Verstand verlor und das Rätsel seines eigenen G
  Duration:    4:23
  Url:         https://www.youtube.com/watch?v=QEXVqYgHrsY
7 ----------------------------------------------------------------------------------
  Title:       Shaban & Käptn Peng -  Kündigung 2.0
  Description: Ein Lied vom Album "Die Zähmung der Hydra" http://www.kreismusik.de K
  Duration:    3:15
  Url:         https://www.youtube.com/watch?v=ojrwKSWrNCs
8 ----------------------------------------------------------------------------------
  Title:       KÄPTN PENG & DIE TENTAKEL VON DELPHI - Neue Freunde
  Description: Album "Das nullte Kapitel" ab dem 19.05.2017 bei www.kreismusik.de un
  Duration:    4:56
  Url:         https://www.youtube.com/watch?v=g3oQxIxaJIU
9 ----------------------------------------------------------------------------------
  Title:       Käptn Peng - Liebes Leben
  Description: http://www.kreismusik.de.
  Duration:    2:37
  Url:         https://www.youtube.com/watch?v=c1cIn4Ni7rw
------------------------------------------------------------------------------------
Your choice (1-9, q to quit, s to search again): q
Results can be found in /tmp/yt-search.py.log

Naik --greetz

Martin · 2020-10-18 08:19:36

ohnonot wrote:

Apparently tor is installed.
Try using the tor browser - does that work normally? In other words, does tor work on your system?
This seems to be a timeout problem. You might have hit a very slow node, tor chooses them randomly. Have you just tried again?
There's also /tmp/youtube-search-tor-<username>.log

Sorry for the late reply. I spent yesterday covering my boat for winter.

I have downloaded and tried the Tor Browser as well as trying Brave's Tor mode. Both work but does it tell us anything about the tor called by your script? While trying Brave's Tor mode I noticed youtube's response times, but nothing else, became really long and eventually I was blocked from watching videos (including my own). I saw none of this while using the Tor Browser.

I don't have a /tmp/youtube-search-tor-<username>.log

/Martin

ohnonot · 2020-10-18 10:28:50

@Naik: wouldn't it be easier to just symlink /path/to/repo/youtube-search to ~/bin, instead of copying it over under a different name then rewriting the script?
If you need to edit to the script, tell me what it is, I'll try to make that a command line option instead (so that the repo remains untouched).

@all:
During your last posts development was in heavy flux, there might have been a temporary mismatch between the 2 scripts.
I changed the -T (=Tor) option to -S (=Socks) because all it does is try to connect to a socks5 proxy. It does not provide actual tor functionality, but tor provides a socks5 proxy to connect to by default.
So -T is now -S, in both scripts.

So please make sure you use the newest version, and tell me if you felt the need to edit the scripts, I'll try to accomodate that.

youtube-search-tor uses only European nodes by default - if you want to remove that restriction, pass -c '', like so:

youtube-search-tor -c ''

All further arguments are passed on to youtube-search.

Martin wrote:

I have downloaded and tried the Tor Browser as well as trying Brave's Tor mode. Both work but does it tell us anything about the tor called by your script?

It just tells us that tor is working in general.
But my script creates a new connection on a dedicated port (9222, currently hardcoded).
BTW, tor-browser uses port 9150 by default, you can try youtube-search -S (it defaults to port 9150) while the tor-browser is running.

I don't have a /tmp/youtube-search-tor-<username>.log

That's weird; is your /tmp not user-writeable?
BTW, it should be just /tmp/youtube-search-tor.log now.

johnraff · 2020-10-19 07:05:17

@ohnonot maybe this project deserves its own thread, rather than sitting in the middle of a bunch of CLI tips?

Would you like me to split it off?

ohnonot · 2020-10-19 12:58:01

johnraff wrote:

Would you like me to split it off?

Yes please!
Thanks.

johnraff · 2020-10-20 02:57:17

Done.
"Search Youtube without a browser" is OK as a title?

ohnonot · 2020-10-20 05:56:38

^ Yes, it is.

And I would love to get some more feedback from those brave beta testers.
For me it's working quite well now, with and without tor.

damo · 2020-10-20 09:23:51

I'm using it via gmrun, then Ctrl click on the link:

x-terminal-emulator -e yt-search.py <search term>

The next thing is to make a wrapper to reduce the typing wink

Martin · 2020-10-20 17:17:18

ohnonot wrote:

^ Yes, it is.
And I would love to get some more feedback from those brave beta testers.
For me it's working quite well now, with and without tor.

Too much work but testing again with newly downloaded code. youtube-search works but not youtube-search-tor. This time I do get log files. Here is the one for youtube-search-tor:

tor is /usr/sbin/tor
youtube-search is /home/martin/bin/youtube-search
#########################
Tor: nodes only from these countries:
{fi},{ax},{se},{no},{dk},{ee},{lv},{lt},{pl},{cz},{sk},{at},{si},{it},{ch},{fr},{es},{pt},{be},{nl},{de},{gb},{ie},{is},{gr}
#########################
Oct 20 19:16:44.607 [notice] Tor 0.3.5.10 running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1d, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd 1.3.8.
Oct 20 19:16:44.607 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Oct 20 19:16:44.607 [notice] Read configuration file "/home/martin/.config/tor/yt-search-tor/torrc".
Oct 20 19:16:44.612 [notice] Opening Socks listener on 127.0.0.1:9222
Oct 20 19:16:44.612 [notice] Opened Socks listener on 127.0.0.1:9222
Oct 20 19:16:44.612 [warn] Fixing permissions on directory /home/martin/.config/tor/yt-search-tor
Oct 20 19:16:44.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Oct 20 19:16:44.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Oct 20 19:16:44.000 [notice] Bootstrapped 0%: Starting
Oct 20 19:16:44.000 [notice] Starting with guard context "default"
#########################
Pidof tor: 26489
#########################Oct 20 19:16:45.000 [notice] Bootstrapped 5%: Connecting to directory server
Oct 20 19:16:45.000 [notice] Bootstrapped 10%: Finishing handshake with directory server
Oct 20 19:16:45.000 [notice] Bootstrapped 15%: Establishing an encrypted directory connection
Oct 20 19:16:45.000 [notice] Bootstrapped 20%: Asking for networkstatus consensus
Oct 20 19:16:45.000 [notice] Bootstrapped 25%: Loading networkstatus consensus
Oct 20 19:16:46.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
Oct 20 19:16:46.000 [notice] Bootstrapped 40%: Loading authority key certs
Oct 20 19:16:46.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services.
Oct 20 19:16:46.000 [notice] Bootstrapped 45%: Asking for relay descriptors for internal paths
Oct 20 19:16:46.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6779, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
Oct 20 19:16:46.000 [notice] Bootstrapped 50%: Loading relay descriptors for internal paths
Oct 20 19:16:47.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
Oct 20 19:16:47.000 [notice] Bootstrapped 56%: Loading relay descriptors
Oct 20 19:16:47.000 [notice] Bootstrapped 62%: Loading relay descriptors
Oct 20 19:16:48.000 [notice] Bootstrapped 67%: Loading relay descriptors
Oct 20 19:16:48.000 [notice] Bootstrapped 73%: Loading relay descriptors
Oct 20 19:16:48.000 [notice] Bootstrapped 80%: Connecting to the Tor network
Oct 20 19:16:48.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
Oct 20 19:16:48.000 [notice] Bootstrapped 90%: Establishing a Tor circuit
Oct 20 19:16:49.000 [notice] Bootstrapped 100%: Done
Oct 20 19:16:56.000 [notice] Catching signal TERM, exiting cleanly.

/Martin

#1 2020-10-04 08:27:56

Search youtube without a browser

#2 2020-10-04 12:44:09

Re: Search youtube without a browser

#3 2020-10-04 12:56:08

Re: Search youtube without a browser

#4 2020-10-04 14:53:27

Re: Search youtube without a browser

#5 2020-10-04 15:48:34

Re: Search youtube without a browser

#6 2020-10-04 16:16:56

Re: Search youtube without a browser

#7 2020-10-06 06:08:31

Re: Search youtube without a browser

#8 2020-10-07 04:29:55

Re: Search youtube without a browser

#9 2020-10-07 20:35:54

Re: Search youtube without a browser

#10 2020-10-08 02:27:39

Re: Search youtube without a browser

#11 2020-10-10 07:13:24

Re: Search youtube without a browser

#12 2020-10-15 19:14:28

Re: Search youtube without a browser

#13 2020-10-15 21:24:04

Re: Search youtube without a browser

#14 2020-10-16 12:47:04

Re: Search youtube without a browser

#15 2020-10-16 20:35:15

Re: Search youtube without a browser

#16 2020-10-17 07:48:48

Re: Search youtube without a browser

#17 2020-10-17 14:33:53

Re: Search youtube without a browser

#18 2020-10-18 08:19:36

Re: Search youtube without a browser

#19 2020-10-18 10:28:50

Re: Search youtube without a browser

#20 2020-10-19 07:05:17

Re: Search youtube without a browser

#21 2020-10-19 12:58:01

Re: Search youtube without a browser

#22 2020-10-20 02:57:17

Re: Search youtube without a browser

#23 2020-10-20 05:56:38

Re: Search youtube without a browser

#24 2020-10-20 09:23:51

Re: Search youtube without a browser

#25 2020-10-20 17:17:18

Re: Search youtube without a browser

Board footer