Commit graph

45 commits

Author SHA1 Message Date
Simon Sawicki
ffbd4f2a02
[utils] traverse_obj: Support xml.etree.ElementTree.Element ()
Authored by: Grub4K
2024-01-05 21:26:17 +01:00
Simon Sawicki
f9fb3ce86e
[cleanup] Misc ()
Authored by: bashonly, pukkandan, seproDev, Grub4K

Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2023-12-30 22:27:36 +01:00
Simon Sawicki
65de7d204c
Update to ytdl-commit-be008e6 ()
- [utils] Make restricted filenames ignore some Unicode categories (by dirkf)
- [ie/telewebion] Fix extraction (by Grub4K)
- [ie/imgur] Overhaul extractor (by bashonly, Grub4K)
- [ie/EpidemicSound] Add extractor (by Grub4K)

Authored by: bashonly, dirkf, Grub4K

Co-authored-by: bashonly <bashonly@protonmail.com>
2023-12-26 01:40:24 +01:00
coletdjnz
196eb0fe77
[networking] Strip whitespace around header values ()
Fixes https://github.com/yt-dlp/yt-dlp/issues/8729
Authored by: coletdjnz
2023-12-20 19:15:38 +13:00
Simon Sawicki
0b6f829b1d
[utils] traverse_obj: Move is_user_input into output template ()
Authored by: Grub4K
2023-12-06 21:46:45 +01:00
pukkandan
a174c453ee
Let read_stdin obey --quiet
Closes 
2023-11-29 05:48:40 +05:30
coletdjnz
ccfd70f4c2
[rh:websockets] Migrate websockets to networking framework ()
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
2023-11-20 08:04:04 +00:00
bashonly
f04b5bedad
[ie] Do not smuggle http_headers
See: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-3ch3-jhc6-5r8x

Authored by: coletdjnz
2023-11-14 22:04:25 +01:00
bashonly
84e26038d4
[utils] write_xattr: Use os.setxattr if available ()
Closes 
Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-10-09 18:30:36 +00:00
Simon Sawicki
1c51c520f7
[fd/fragment] Improve progress calculation ()
This uses the download speed from all threads and also adds smoothing to speed and eta

Authored by: Grub4K
2023-10-08 02:01:01 +02:00
Awal Garg
9d7ded6419
[utils] js_to_json: Fix Date constructor parsing ()
Authored by: awalgarg, Grub4K
2023-10-08 01:57:23 +02:00
Simon Sawicki
de015e9307
[core] Prevent RCE when using --exec with %q (CVE-2023-40581)
The shell escape function is now using `""` instead of `\"`. `utils.Popen` has been patched to properly quote commands.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-42h4-v29r-42qg for reference.

Authored by: Grub4K
2023-09-24 02:29:01 +02:00
bashonly
52414d64ca
[utils] js_to_json: Handle Array objects
Authored by: Grub4K, std-move

Co-authored-by: std-move <26625259+std-move@users.noreply.github.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-09-21 16:51:57 -05:00
Simon Sawicki
836e06d246
[core] Fix support for upcoming Python 3.12 ()
This also adds the following test runners:
- `3.12-dev` on `ubuntu-latest`
- `3.12-dev` on `windows-latest`
- `pypy-3.10` on `ubuntu-latest`

Authored by: Grub4K
2023-09-17 12:56:50 +02:00
Simon Sawicki
f659e64394
[ie/bpb] Overhaul extractor ()
Authored by: Grub4K
2023-09-16 17:50:06 +02:00
pukkandan
3f7965105d
[utils] HTTPHeaderDict: Handle byte values 2023-07-30 03:18:10 +05:30
coletdjnz
4bf912282a
[networking] Remove dot segments during URL normalization ()
This implements RFC3986 5.2.4 remove_dot_segments during the URL normalization process.

Closes , 

Authored by: coletdjnz
2023-07-28 22:40:20 +00:00
coletdjnz
bbeacff7fc
[networking] Ignore invalid proxies in env ()
Authored by: coletdjnz
2023-07-27 20:26:02 +05:30
pukkandan
a250b24733
[compat] Ensure submodules are imported correctly
Closes 
2023-07-22 18:10:35 +05:30
bashonly
af86873218
[utils] Improve parse_duration
Authored by: bashonly
2023-07-20 08:40:31 -05:00
coletdjnz
3d2623a898
[compat, networking] Deprecate old functions ()
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
coletdjnz
227bf1a33b
[networking] Rewrite architecture ()
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
2023-07-15 16:18:35 +05:30
pukkandan
c365dba843
[networking] Add module ()
No actual changes - code is only moved around
2023-07-15 16:18:34 +05:30
pukkandan
1b392f905d
[utils] Add temporary shim for logging
Related: , 
2023-07-15 16:18:34 +05:30
Mahmoud Abdel-Fattah
2af4eeb772
[utils] clean_podcast_url: Handle more trackers ()
Authored by: mabdelfattah, bashonly
Closes 
2023-07-11 06:30:38 +05:30
coletdjnz
f8b4bcc0a7
[core] Prevent Cookie leaks on HTTP redirect
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: coletdjnz
2023-07-06 23:14:39 +05:30
pukkandan
337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
pukkandan
91302ed349
[utils] clean_podcast_url: Handle protocol in redirect URL
Closes 
2023-06-26 16:19:49 +05:30
pukkandan
e59e20744e
Bugfix for b4e0d75848 2023-06-22 23:45:53 +05:30
pukkandan
812cdfa06c
[cleanup] Misc 2023-06-22 13:31:07 +05:30
pukkandan
b4e0d75848
Improve --download-sections
* Support negative time-ranges
* Add `*from-url` to obey time-ranges in URL

Closes 
2023-06-22 13:03:07 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes , Closes , Closes , Closes , Closes 
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
Nicolai Dagestad
db3ad8a676
Add option --netrc-cmd ()
Authored by: NDagestad, pukkandan
Closes 
2023-06-21 08:37:42 +05:30
pukkandan
ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes 
2023-06-21 06:10:39 +05:30
pukkandan
a35af4306d
[utils] strftime_or_none: Handle negative timestamps
Closes 
Authored by pukkandan, dirkf
2023-06-21 06:10:39 +05:30
pukkandan
eedda5252c
[utils] FormatSorter: Improve size and br
Closes 

Previously, when some formats have accurate size and some approximate,
the ones with accurate size was always prioritized

For formats with known tbr and unknown vbr/abr, we were setting
(vbr=tbr, abr=0) for sorting to work. This is no longer needed.

Authored by pukkandan, u-spec-png
2023-06-21 06:10:38 +05:30
hoaluvn
7bcd481321
[extractor/urplay] Extract all subtitles ()
Authored by: hoaluvn
2023-06-14 21:22:17 +05:30
garret
4cbfa570a1
[extractor/camfm] Add extractors ()
Authored by: garret1317
2023-05-29 20:14:26 +05:30
coletdjnz
daafbf49b3
[core] Support decoding multiple content encodings ()
Authored by: coletdjnz
2023-05-27 10:40:05 +00:00
coletdjnz
b87e01c123
[cookies] Move YoutubeDLCookieJar to cookies module ()
Authored by: coletdjnz
2023-05-27 07:08:19 +00:00
coletdjnz
08916a49c7
[core] Improve HTTP redirect handling ()
Aligns HTTP redirect handling with what browsers commonly do and RFC standards. 

Fixes issues afac4caa7d missed.

Authored by: coletdjnz
2023-05-27 07:06:13 +00:00
pukkandan
4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
2023-05-24 23:30:43 +05:30
pukkandan
46f1370e9a
[devscripts/cli_to_api] Add script 2023-05-24 23:29:30 +05:30
coletdjnz
955c89584b
[core] Deprecate internal Youtubedl-no-compression header ()
Authored by: coletdjnz
2023-05-20 22:55:09 +00:00
coletdjnz
69bec6730e
[cleanup, utils] Split into submodules ()
Closes https://github.com/yt-dlp/yt-dlp/pull/2173

Authored by: pukkandan, coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-05-20 21:56:23 +00:00