Kemal's subclass of the stdlib `HTTP::StaticFileHandler` is not as
maintained as its parent, and so misses out on many enhancements and bug
fixes from upstream, which unfortunately also includes the patches for
security vulnerabilities...
Though this isn't necessarily Kemal's fault since the bulk of the stdlib
handler's logic was done in a single big method, making any changes hard
to maintain. This was fixed in Crystal 1.17.0 where the handler
was refactored into many private methods, making it easier for an
inheriting type to implement custom behaviors while still leveraging
much of the pre-existing code.
Since we don't actually use any of the Kemal specific features added by
`Kemal::StaticFileHandler`, there really isn't a reason to not just
create a new handler based upon the stdlib implementation instead which
will address the problems mentioned above.
This PR implements a new handler which inherits from the stdlib variant
and overrides the helper methods added in Crystal 1.17.0 to add the
caching behavior with minimal code changes. Since this new handler
depends on the code in Crystal 1.17.0, it will only be applied on
versions greater than or equal to 1.17.0. On older versions we'll
fallback to the current monkey patched `Kemal::StaticFileHandler`
* feat: Add configurable max_request_line_size to handle long URLs
This commit adds a new configuration option `max_request_line_size` that allows
users to increase the HTTP request line size limit. This is particularly useful
for handling very long continuation tokens that can cause 414 (URI Too Long) errors.
Changes:
- Add `max_request_line_size` property to Config class
- Configure Kemal server to use the custom limit if specified
- Document the option in config.example.yml with recommendations
- Add examples in docker-compose.yml for both YAML and env var configuration
The default behavior remains unchanged (8KB limit) unless explicitly configured.
This provides a solution for users experiencing 414 errors without affecting
existing installations.
* Hardcode max_request_line_size to 16384
---------
Co-authored-by: Sunghyun Kim <hello@sunghyun.me>
* Add link to GitHub release/tag/commit in footer
* Only show tag if there is one
Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
---------
Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
* Remove signature helper completely from Invidious
The official way to reproduce video with Invidious now is by using
Invidious Companion which uses Youtube.JS with a Javascript Interpreter
that can successfully decrypt youtube video URLs.
Sig helper has not been used for a long time, is beyond broken and no
one has plans to fix it and maintain it.
* Remove DECRYPT_FUNCTION and shrink player function
* remove `sp = cfr[sp]`
* Improve message
The exception handling for database connections results in an
`ex` variable which Ameba sees as overshadowing the `ex` used by the
`ex` block arg used to define the HTTP status code 500 handler below.
Although this is a non-issue since the db connection exception handling
will cause Invidious to exit, Ameba's nature as a static checker means
that it isn't aware of this.
The simplest fix without a dirty ameba ignore comment is to rename `ex`
within the Kemal handler block below, since `ex` within a begin rescue
block is a Crystal convention that will also cause Ameba to raise when
not adhered to.
* add support for invidious companion
* redirect latest_version and dash manifest to invidious companion
* fix Shadowing outer local variable `response`
* fixing condition for Content-Security-Policy
* throw error if inv_sig_helper and invidious_companion used same time
* Use sample instead of Random.rand
Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
* Remove debug puts functions
Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
* modify the description for config.example.yaml about invidious companion
* move config checks for invidious companion
* separate invidious_companion logic + better config.yaml config
* fixing "end" misplacement
* fix linting + use .empty?
* crystal handle decompression already by itself
* fix download function when invidious companion used
* fix linting
* invidious companion always used so always add CSP and redirect latest_version
* apply all the suggestions + rework invidious_companion parameter
* format watch.cr
* fix ameba Redundant use of `Object#to_s` in interpolation
* add ability for invidious companion to check request from invidious
* Better document private_url and public_url
* Better doc for invidious_companion_key
* !empty? to present?
* skip proxy for invidious companion
* fixing format
* missing ,
* add companion pooling http
* fix: don't use http proxy when sending requests to companion
* fix: logic where we want to have the invidious logic if companion is not used
* chore: remove baseurl usage from invidious companion
* chore: change from inv-sig-helper to companion for required playback
* fix: use puts + add warning for inv-sig-helper deprecated
---------
Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
The crystal compiler seems to evaluate `require` in an alphabetical way,
so if anyone in the future, wants to add another job and that job is
above `base_job.cr` in alphabetical order, the compiler is going to fail
with `Error: undefined constant: Invidious::Jobs::BaseJob`.
This doesn't fix anything, but it will prevent a future headache.
Kilt is unmaintained and the ECR templating logic has been
natively integrated into Kemal with the issues previously seen
having been resolved.
This commit is mostly a precursor to support the next Kemal
release which will add the ability to create error handlers for
raised exceptions.
See https://github.com/kemalcr/kemal/pull/688
Theoretically this should improve memory usage and performance by quite a bit
as we aren't creating a new HTTP::Client and in a turn a new connection for
every image we request from YouTube.
Closes issue 4009
The automatic instance redirection implemented in #1940 fetches a new list of
instances each time someone queries the /redirect endpoint. This is extremely
inefficient...
This PR optimizes all that into a background job that only fetches a single
list every 30 minutes. This should performance quite a bit.
No related issue was opened.
This handler should no have been removed in 4276, as it adds the required CORS
header (Access-Control-Allow-Origin) for public acces to the API.
Thanks to iBicha for noticing this!
