Commit graph

239 commits

Author SHA1 Message Date
Kenshin
bcec568ea7 Add ffmpeg progress tracking to FFmpegFD
Add ffmpeg progress tracking to FFmpegPostProcessor

Apply changes from the code review

Fix a bug where the subprocess didn't capture any output thus an empty stdout and stderr were sent back

Add missing hooks

Revert "Add missing hooks"

This reverts commit a359c5ea10bb35b965e80801e736f43cdbcf3294.

Add support of -ss=132 timestamp format

Infer filename from ffmpeg args instead of info_dic

Remove redundant parenthesis and switch from to_stodout to to_screen

Add info kwarg with multiple files and ffmpeg to track progress

Moved format progress function to util

Moved format progress function to util

Add progress tracking to postprocessing operations

Fix typing error

Handle self._downloader is None at __init__

Move format progress functions to utils

Move format progress functions to utils

Handle case where ydl passed is None

Handle case where ydl passed is None

Handle case where _multiline isn't initialized

Handle case where _multiline isn't initialized

Fix streams incorrectly returned

Fix case where ydl is nested in the downloader

Add progress_hook attribute

Fix bug after merge

Fix import bugs after merge

Catch up with upstream

Fix merge errors #1

Adapt tests and implementatation for ffmpeg progress tracking args
2024-01-17 15:14:41 +01:00
Simon Sawicki
1c51c520f7
[fd/fragment] Improve progress calculation (#8241)
This uses the download speed from all threads and also adds smoothing to speed and eta

Authored by: Grub4K
2023-10-08 02:01:01 +02:00
pukkandan
62b5c94cad
[cleanup] Misc fixes
Closes #7528
2023-07-22 09:09:52 +05:30
bashonly
e57eb98222
[fd/external] Fix ffmpeg input from stdin (#7655)
Bugfix for 1ceb657bdd

Authored by: bashonly
2023-07-22 02:32:49 +00:00
bashonly
42ded0a429
[fd/external] Fixes to cookie handling
- Fix bug in `axel` Cookie header arg
- Pass cookies to `curl` as strings
- Write session cookies for `aria2c` and `wget`

Closes #7539
Authored by: bashonly
2023-07-15 15:25:51 -05:00
bashonly
6c5211cebe
[core] Fix HTTP headers and cookie handling
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor

Closes #7558
Authored by: bashonly, pukkandan
2023-07-15 15:25:45 -05:00
coletdjnz
3d2623a898
[compat, networking] Deprecate old functions (#2861)
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
coletdjnz
227bf1a33b
[networking] Rewrite architecture (#2861)
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
2023-07-15 16:18:35 +05:30
Simon Sawicki
3121512228
[core] Change how Cookie headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
2023-07-06 23:14:39 +05:30
bashonly
1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
2023-07-06 23:14:38 +05:30
pukkandan
bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
2023-07-06 21:40:01 +05:30
pukkandan
337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
pukkandan
662ef1e910
[downloader/http] Avoid infinite loop when no data is received
Closes #7504
2023-07-06 20:22:00 +05:30
pukkandan
8a8af356e3
[downloader/aria2c] Add --no-conf
Closes #7404
2023-06-26 16:13:31 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
pukkandan
424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes #6599
2023-06-21 06:10:39 +05:30
Lesmiscore
f8f9250fe2
[extractor/niconico:live] Add extractor (#5764)
Authored by: Lesmiscore
2023-05-29 18:35:10 +09:00
Simon Sawicki
edbe5b589d
Bugfixes for 4823ec9f46
Hotfix for fragmented downloads

Authored by: bashonly
2023-05-25 22:52:44 +02:00
Simon Sawicki
8417f26b8a
[core] Implement --color flag (#6904)
Authored by: Grub4K
2023-05-24 20:35:07 +02:00
pukkandan
4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
2023-05-24 23:30:43 +05:30
pukkandan
46f1370e9a
[devscripts/cli_to_api] Add script 2023-05-24 23:29:30 +05:30
coletdjnz
955c89584b
[core] Deprecate internal Youtubedl-no-compression header (#6876)
Authored by: coletdjnz
2023-05-20 22:55:09 +00:00
pukkandan
66aeaac9aa
[downloader/curl] Fix progress reporting
Bug in 8c53322cda
Closes #6490
2023-03-09 21:58:07 +05:30
pukkandan
5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan
f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2023-02-28 23:34:43 +05:30
pukkandan
65f6e80780
[dependencies] Simplify Cryptodome
Closes #6292, closes #6272, closes #6338
2023-02-28 23:15:13 +05:30
Felix Yan
65e5c021e7
[utils] Don't use Content-length with encoding (#6176)
Authored by: felixonmars
Closes #3772, #6178
2023-02-17 08:38:45 +05:30
Simon Sawicki
6839ae1f6d
[utils] traverse_obj: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
2023-02-10 19:36:55 +05:30
bashonly
7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
pukkandan
f6a765ceb5
[dependencies] Standardize Cryptodome imports 2023-02-08 07:28:46 +05:30
pukkandan
ad68b16a1e
[downloader/aria2c] Disable native progress
Closes #5931, closes #5928, Re-opens #2038
2023-01-03 17:25:56 +05:30
Lesmiscore
8c53322cda
[downloader/aria2c] Native progress for aria2c via RPC (#3724)
Authored by: Lesmiscore, pukkandan

Closes #2038
2023-01-02 02:16:25 +09:00
bashonly
3cf50fa8e9
[downloader/ffmpeg] Fix headers for video+audio formats (#5659)
Authored by: bashonly, Grub4K
2022-12-09 23:36:38 +00:00
pukkandan
71df9b7fd5
[cleanup] Misc 2022-12-03 19:52:31 +05:30
bashonly
3b021eacef
[extractor/generic] Add fragment_query extractor arg for DASH and HLS (#5528)
* `fragment_query`: passthrough any query in generic mpd/m3u8 manifest URLs to their fragments
* Add support for `extra_param_to_segment_url` to DASH downloader
Authored by: bashonly, pukkandan
2022-11-21 00:51:45 +00:00
pukkandan
64c464a144
[utils] Move FileDownloader.parse_bytes into utils 2022-11-17 08:40:34 +05:30
pukkandan
d5d1df8afd
[cleanup Misc
Closes #5162
2022-10-18 23:52:44 +05:30
pukkandan
814bba3933
[downloader/fragment] HLS download can continue without first fragment
Closes #5274
2022-10-18 19:20:51 +05:30
pukkandan
a057779d5e
[cleanup] Minor fixes
Closes #5129, Closes #4982
2022-10-04 01:48:14 +05:30
std-move
af7a5eef2f
[downloader/aria2c] Fix filename containing leading whitespace (#5099)
Similar to eb55bad5a0, but for fragmented downloads
Authored by: std-move
2022-10-01 21:00:14 +05:30
nixxo
81b6102d20
[downloader/ism] Support ec-3 codec (#5004)
Closes #296
Authored by: nixxo
2022-09-30 23:03:29 +05:30
pukkandan
deae7c1711
[cleanup] Misc 2022-09-10 03:46:54 +05:30
pukkandan
da4db748fa
[utils] Add deprecation_warning
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
2022-08-30 21:03:07 +05:30
pukkandan
e1eabd7beb
[downloader/external] Smarter detection of executable
Closes #4778
2022-08-30 18:13:38 +05:30
Lesmiscore
62b58c0936
[docs] Consistent use of e.g. (#4643)
Authored by: Lesmiscore
2022-08-14 17:34:13 +05:30
pukkandan
be5c1ae862
Standardize retry mechanism (#1649)
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
2022-08-02 01:43:18 +05:30
pukkandan
3df4f81dfe
[downloader] Add average speed to final progress line
Fixes: https://github.com/ytdl-org/youtube-dl/issues/31122
2022-07-31 04:20:02 +05:30
pukkandan
b4daacb4ec
[mhtml, cleanup] Use imghdr 2022-07-31 02:20:12 +05:30
pukkandan
6929b41a21
Remove Python 3.6 support
Closes #3764
2022-07-18 06:31:14 +05:30
pukkandan
f5ea47488a
[cleanup] Minor fixes 2022-07-11 02:24:36 +05:30