1
0
Fork 0
mirror of https://github.com/yt-dlp/yt-dlp synced 2025-01-15 03:41:33 +01:00
Commit graph

1270 commits

Author SHA1 Message Date
sepro
1732eccc0a
[core] Parse release_year from release_date ()
Closes 
Authored by: seproDev
2023-11-26 02:12:05 +00:00
coletdjnz
ccfd70f4c2
[rh:websockets] Migrate websockets to networking framework ()
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
2023-11-20 08:04:04 +00:00
sepro
45d82be65f
[ie/nebula] Overhaul extractors ()
Closes , Closes , Closes , Closes , Closes 
Authored by: elyse0, pukkandan, seproDev

Co-authored-by: Elyse <26639800+elyse0@users.noreply.github.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-11-20 01:03:33 +00:00
bashonly
f4b95acafc
Remove Python 3.7 support ()
Closes 
Authored by: bashonly
2023-11-16 18:39:00 +00:00
bashonly
f04b5bedad
[ie] Do not smuggle http_headers
See: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-3ch3-jhc6-5r8x

Authored by: coletdjnz
2023-11-14 22:04:25 +01:00
bashonly
87264d4fda
[test:update] Implement simple updater unit tests
Authored by: bashonly
2023-11-12 18:30:55 -06:00
Simon Sawicki
4e38e2ae9d
[rh:requests] Handle both bytes and int for IncompleteRead.partial (Fix 8a8b54523a) ()
Authored by: bashonly, coletdjnz, Grub4K
2023-10-15 10:54:38 +02:00
coletdjnz
8a8b54523a
[rh:requests] Add handler for requests HTTP library ()
Adds support for HTTPS proxies and persistent connections (keep-alive)

Closes https://github.com/yt-dlp/yt-dlp/issues/1890
Resolves https://github.com/yt-dlp/yt-dlp/issues/4070
Resolves https://github.com/ytdl-org/youtube-dl/issues/32549
Resolves https://github.com/ytdl-org/youtube-dl/issues/14523
Resolves https://github.com/ytdl-org/youtube-dl/issues/13734

Authored by: coletdjnz, Grub4K, bashonly
2023-10-13 23:33:00 +00:00
Awal Garg
9d7ded6419
[utils] js_to_json: Fix Date constructor parsing ()
Authored by: awalgarg, Grub4K
2023-10-08 01:57:23 +02:00
Simon Sawicki
088add9567
[cleanup] Misc
Authored by: Grub4K
2023-09-24 02:35:23 +02:00
Simon Sawicki
de015e9307
[core] Prevent RCE when using --exec with %q (CVE-2023-40581)
The shell escape function is now using `""` instead of `\"`. `utils.Popen` has been patched to properly quote commands.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-42h4-v29r-42qg for reference.

Authored by: Grub4K
2023-09-24 02:29:01 +02:00
Simon Sawicki
61bdf15fc7
[core] Raise minimum recommended Python version to 3.8 ()
Authored by: Grub4K
2023-09-24 02:24:47 +02:00
bashonly
5ca095cbcd
[cleanup] Misc ()
Closes , Closes 
Authored by: barsnick, sqrtNOT, gamer191, coletdjnz, Grub4K, bashonly
2023-09-23 20:00:31 +00:00
bashonly
661c9a1d02
[test:download] Test for expected_exception
Authored by: at-wat

Co-authored-by: Atsushi Watanabe <atsushi.w@ieee.org>
2023-09-21 17:48:57 -05:00
bashonly
52414d64ca
[utils] js_to_json: Handle Array objects
Authored by: Grub4K, std-move

Co-authored-by: std-move <26625259+std-move@users.noreply.github.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-09-21 16:51:57 -05:00
coletdjnz
20fbbd9249
[networking] Fix various socks proxy bugs ()
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes https://github.com/yt-dlp/yt-dlp/issues/7959
Fixes https://github.com/ytdl-org/youtube-dl/issues/15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
2023-09-18 07:33:26 +00:00
coletdjnz
fcd6a76adc
[tests] Add tests for socks proxies ()
Authored by: coletdjnz
2023-08-25 07:10:44 +00:00
coletdjnz
db7b054a61
[networking] Add request handler preference framework ()
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-08-04 22:17:48 +00:00
pukkandan
6148833f5c
[cleanup] Misc 2023-07-30 04:06:18 +05:30
pukkandan
3f7965105d
[utils] HTTPHeaderDict: Handle byte values 2023-07-30 03:18:10 +05:30
pukkandan
de20687ee6
[test] Fix test_load_certifi
Closes , 
2023-07-29 21:53:00 +05:30
coletdjnz
4bf912282a
[networking] Remove dot segments during URL normalization ()
This implements RFC3986 5.2.4 remove_dot_segments during the URL normalization process.

Closes , 

Authored by: coletdjnz
2023-07-28 22:40:20 +00:00
coletdjnz
bbeacff7fc
[networking] Ignore invalid proxies in env ()
Authored by: coletdjnz
2023-07-27 20:26:02 +05:30
coletdjnz
95abea9a03
[test] Fix httplib_validation_errors test for old Python versions ()
Fixes https://github.com/yt-dlp/yt-dlp/issues/7674

Authored by: coletdjnz
2023-07-24 19:18:52 +00:00
coletdjnz
86aea0d3a2
[networking] Add strict Request extension checking ()
Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-07-23 05:17:15 +00:00
pukkandan
a250b24733
[compat] Ensure submodules are imported correctly
Closes 
2023-07-22 18:10:35 +05:30
pukkandan
62b5c94cad
[cleanup] Misc fixes
Closes 
2023-07-22 09:09:52 +05:30
pukkandan
a264433c9f
[outtmpl] Fix replacement for playlist_index 2023-07-22 08:57:43 +05:30
bashonly
e57eb98222
[fd/external] Fix ffmpeg input from stdin ()
Bugfix for 1ceb657bdd

Authored by: bashonly
2023-07-22 02:32:49 +00:00
bashonly
af86873218
[utils] Improve parse_duration
Authored by: bashonly
2023-07-20 08:40:31 -05:00
bashonly
75dc8e673b
[networking] Fix --legacy-server-connect ()
Bugfix for 227bf1a33b

Authored by: bashonly
2023-07-20 13:31:17 +00:00
bashonly
71baa490eb
[networking] Fix POST requests with zero-length payloads ()
Bugfix for 227bf1a33b

Authored by: bashonly
2023-07-20 13:23:30 +00:00
bashonly
42ded0a429
[fd/external] Fixes to cookie handling
- Fix bug in `axel` Cookie header arg
- Pass cookies to `curl` as strings
- Write session cookies for `aria2c` and `wget`

Closes 
Authored by: bashonly
2023-07-15 15:25:51 -05:00
bashonly
6c5211cebe
[core] Fix HTTP headers and cookie handling
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor

Closes 
Authored by: bashonly, pukkandan
2023-07-15 15:25:45 -05:00
coletdjnz
3d2623a898
[compat, networking] Deprecate old functions ()
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
coletdjnz
227bf1a33b
[networking] Rewrite architecture ()
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
2023-07-15 16:18:35 +05:30
pukkandan
c365dba843
[networking] Add module ()
No actual changes - code is only moved around
2023-07-15 16:18:34 +05:30
pukkandan
1b392f905d
[utils] Add temporary shim for logging
Related: , 
2023-07-15 16:18:34 +05:30
Mahmoud Abdel-Fattah
2af4eeb772
[utils] clean_podcast_url: Handle more trackers ()
Authored by: mabdelfattah, bashonly
Closes 
2023-07-11 06:30:38 +05:30
Simon Sawicki
3121512228
[core] Change how Cookie headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
2023-07-06 23:14:39 +05:30
coletdjnz
f8b4bcc0a7
[core] Prevent Cookie leaks on HTTP redirect
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: coletdjnz
2023-07-06 23:14:39 +05:30
bashonly
1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
2023-07-06 23:14:38 +05:30
pukkandan
47bcd43724
[outtmpl] Pad playlist_index etc even when with internal formatting
Closes 
2023-07-06 20:22:03 +05:30
pukkandan
cd810afe2a
[extractor/youtube] Improve nsig function name extraction 2023-06-22 13:27:18 +05:30
pukkandan
42f2d40b47
Update to ytdl-commit-07af47
[YouTube] Improve fix for ae8ba2c
07af47960f
2023-06-21 09:21:23 +05:30
pukkandan
1619ab3e67
Bugfix for ebe1b4e34f 2023-06-21 09:21:22 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes , Closes , Closes , Closes , Closes 
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
pukkandan
ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes 
2023-06-21 06:10:39 +05:30
pukkandan
eedda5252c
[utils] FormatSorter: Improve size and br
Closes 

Previously, when some formats have accurate size and some approximate,
the ones with accurate size was always prioritized

For formats with known tbr and unknown vbr/abr, we were setting
(vbr=tbr, abr=0) for sorting to work. This is no longer needed.

Authored by pukkandan, u-spec-png
2023-06-21 06:10:38 +05:30
pukkandan
01aba2519a
[jsinterp] Fix global object extraction
Closes 
2023-06-18 04:11:15 +05:30