-
-
Notifications
You must be signed in to change notification settings - Fork 8
Version 4.0-rc docs #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gzigzigzeo
wants to merge
7
commits into
master
Choose a base branch
from
version/4
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
4f4c6c5
Version 4.0-rc docs
gzigzigzeo 7116b34
Cache docs
gzigzigzeo 839b176
Grammarly, fixed classification, addressed PR comments
gzigzigzeo 3d65af0
Various fixes, addressed PR comments
gzigzigzeo 69d0666
Minor updates by the comments
gzigzigzeo 2c66198
Version 4-preview
gzigzigzeo ad7bfb1
Correct banner in latest version
gzigzigzeo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 20 additions & 0 deletions
20
versioned_docs/version-4-preview/about_processing_pipeline.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| --- | ||
| description: Learn about imgproxy's processing pipeline | ||
| --- | ||
|
|
||
| # About the processing pipeline | ||
|
|
||
| imgproxy has a specific processing pipeline tuned for maximum performance. When you process an image with imgproxy, it does the following: | ||
|
|
||
| * If the source image format allows shrink-on-load, imgproxy uses it to quickly resize the image to the size that is closest to desired. | ||
| * If it is needed to resize an image with an alpha-channel, imgproxy premultiplies one to handle alpha correctly. | ||
| * imgproxy resizes the image to the desired size. | ||
| * If the image colorspace need to be fixed, imgproxy fixes it. | ||
| * imgproxy rotates/flip the image according to EXIF metadata. | ||
| * imgproxy crops the image using the specified gravity. | ||
| * imgproxy fills the image background if the background color was specified. | ||
| * imgproxy applies filters. | ||
| * imgproxy adds a watermark if one was specified. | ||
| * And finally, imgproxy saves the image to the desired format. | ||
|
|
||
| This pipeline, using sequential access to source image data, allows for significantly reduced memory and CPU usage — one of the reasons imgproxy is so performant. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| --- | ||
| description: External cache options for imgproxy, including CDNs, nginx, and Varnish | ||
| --- | ||
|
|
||
| # External cache | ||
|
|
||
| External cache refers to caching layers in front of imgproxy, such as CDNs, reverse proxies, or caching appliances. These solutions provide edge caching, geo-distribution, and high-performance image delivery. | ||
|
|
||
| ## CDN options | ||
|
|
||
| Most major CDNs (Cloudflare, Fastly, AWS CloudFront, Akamai, etc.) can cache imgproxy responses: | ||
|
|
||
| ### Recommended CDN settings | ||
|
|
||
| * **Image optimization** - Disable CDN image optimization to avoid double-processing | ||
| * **Compression** - In case your CDN supports conditional compression of SVG images, enable gzip/Brotli compression for bandwidth savings | ||
| * **Include essential headers*** - Include `Accept` header in the cache key if you're using [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection). If you have [client hints](../configuration/options.mdx#client-hints-support) enabled, include also `DPR`, `Sec-CH-Dpr`, and `Sec-CH-Width` headers. | ||
| * **Origin shield** - Add a secondary cache layer between your CDN and imgproxy origin if your CDN supports it. | ||
|
|
||
| ## nginx caching | ||
|
|
||
| nginx can act as a reverse proxy cache in front of imgproxy: | ||
|
|
||
| ```nginx | ||
| proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=imgproxy_cache:10m max_size=50g inactive=30d; | ||
|
|
||
| upstream imgproxy { | ||
| server localhost:8080; | ||
| } | ||
|
|
||
| server { | ||
| listen 80; | ||
| server_name images.example.com; | ||
|
|
||
| # Uncomment if you do not use format negotiation and client hints | ||
| # proxy_cache_key "$scheme$proxy_host$request_uri"; | ||
|
|
||
| # Uncomment if you use format negotiation | ||
| # proxy_cache_key "$scheme$proxy_host$request_uri$http_accept"; | ||
|
|
||
| # Uncomment if you use client hints | ||
| # proxy_cache_key "$scheme$request_method$host$request_uri:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width"; | ||
|
|
||
| # Uncomment if you use format negotiation and client hints | ||
| # proxy_cache_key "$scheme$request_method$host$request_uri:accept=$http_accept:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width"; | ||
|
|
||
| location / { | ||
| proxy_pass http://imgproxy; | ||
|
|
||
| # Cache configuration | ||
| proxy_cache imgproxy_cache; | ||
| proxy_cache_valid 200 30d; | ||
| proxy_cache_valid 404 1m; | ||
| proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; | ||
| proxy_cache_background_update on; | ||
| proxy_cache_lock on; | ||
| proxy_cache_bypass $http_pragma $http_authorization; | ||
| proxy_no_cache $http_pragma $http_authorization; | ||
|
|
||
| # Add cache status header for debugging | ||
| add_header X-Cache-Status $upstream_cache_status; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Possible cache key setup | ||
|
|
||
| The exact key depends on which imgproxy features can change output for the same URL. | ||
|
|
||
| 1. Basic key when output depends only on URL/path options: | ||
|
|
||
| ```nginx | ||
| proxy_cache_key "$scheme$proxy_host$request_uri"; | ||
| ``` | ||
|
|
||
| 2. Include [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection) signal when WebP/AVIF/JXL preference can vary by client: | ||
|
|
||
| ```nginx | ||
| proxy_cache_key "$scheme$proxy_host$request_uri$http_accept"; | ||
| ``` | ||
|
|
||
| 3. Include [Client Hints](../configuration/options.mdx#client-hints-support) dimensions when width/DPR affects output: | ||
|
|
||
| ```nginx | ||
| proxy_cache_key "$scheme$request_method$host$request_uri:accept=$http_accept:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width"; | ||
| ``` | ||
|
|
||
| Use the smallest key that still prevents collisions for your traffic profile. | ||
|
|
||
| ### Key recommendations | ||
|
|
||
| * Include `Accept` in the cache key if [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection) is used. | ||
| * Include DPR/width-related hints in the cache key if [Client Hints](../configuration/options.mdx#client-hints-support) are used to vary output. | ||
| * Set long `inactive` timeout (30d) to keep popular images in cache. | ||
| * Configure `proxy_cache_use_stale` for graceful degradation during origin issues. | ||
| * Monitor `/proc/sys/fs/file-max` and increase if needed for large caches. | ||
| * Use separate cache zones for different image types if needed. | ||
|
|
||
| ## Varnish caching | ||
|
|
||
| Varnish is a powerful reverse proxy cache optimized for HTTP performance: | ||
|
|
||
| ```vcl | ||
| vcl 4.1; | ||
|
|
||
| backend imgproxy { | ||
| .host = "localhost"; | ||
| .port = "8080"; | ||
| .connect_timeout = 600ms; | ||
| .first_byte_timeout = 600s; | ||
| .between_bytes_timeout = 60s; | ||
| } | ||
|
|
||
| sub vcl_recv { | ||
| set req.backend_hint = imgproxy; | ||
|
|
||
| # Cache only GET and HEAD requests | ||
| if (req.method != "GET" && req.method != "HEAD") { | ||
| return (pass); | ||
| } | ||
|
|
||
| # Include important headers in cache key | ||
| if (req.http.Accept) { | ||
| set req.http.X-Accept = req.http.Accept; | ||
| } | ||
|
|
||
| # Keep DPR if Client Hints are not enabled | ||
| if (req.http.DPR) { | ||
| set req.http.X-DPR = req.http.DPR; | ||
| } | ||
|
|
||
| if (req.http.Width) { | ||
| set req.http.X-Width = req.http.Width; | ||
| } | ||
|
|
||
| if (req.http.Sec-CH-DPR) { | ||
| set req.http.X-CH-DPR = req.http.Sec-CH-DPR; | ||
| } | ||
|
|
||
| if (req.http.Sec-CH-Width) { | ||
| set req.http.X-CH-Width = req.http.Sec-CH-Width; | ||
| } | ||
| } | ||
|
|
||
| sub vcl_hash { | ||
| # Base hash | ||
| hash_data(req.url); | ||
| hash_data(req.http.Host); | ||
|
|
||
| # Include Accept header for format negotiation | ||
| if (req.http.X-Accept) { | ||
| hash_data(req.http.X-Accept); | ||
| } | ||
|
|
||
| # Include DPR if present | ||
| if (req.http.X-DPR) { | ||
| hash_data(req.http.X-DPR); | ||
| } | ||
|
|
||
| if (req.http.X-Width) { | ||
| hash_data(req.http.X-Width); | ||
| } | ||
|
|
||
| if (req.http.X-CH-DPR) { | ||
| hash_data(req.http.X-CH-DPR); | ||
| } | ||
|
|
||
| if (req.http.X-CH-Width) { | ||
| hash_data(req.http.X-CH-Width); | ||
| } | ||
| } | ||
|
|
||
| sub vcl_backend_response { | ||
| # Cache successful responses for 30 days | ||
| if (beresp.status == 200) { | ||
| set beresp.ttl = 30d; | ||
| set beresp.keep = 30d; | ||
| } | ||
|
|
||
| # Cache 404s for a short period | ||
| if (beresp.status == 404) { | ||
| set beresp.ttl = 1m; | ||
| } | ||
|
|
||
| # Enable cache using stale object if backend is down | ||
| set beresp.grace = 24h; | ||
| } | ||
|
|
||
| sub vcl_deliver { | ||
| # Add cache hit/miss header | ||
| if (obj.hits > 0) { | ||
| set resp.http.X-Cache = "HIT"; | ||
| set resp.http.X-Cache-Hits = obj.hits; | ||
| } else { | ||
| set resp.http.X-Cache = "MISS"; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Key recommendations | ||
|
|
||
| * Use `sub vcl_hash` to customize cache key behavior. | ||
| * Include `Accept` for [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection). | ||
| * Include DPR/width-related headers when [Client Hints](../configuration/options.mdx#client-hints-support) support is enabled (`DPR`, `Width`, `Sec-CH-DPR`, `Sec-CH-Width`). | ||
| * Set grace period (`beresp.grace`) for graceful error handling. | ||
| * Monitor Varnish statistics with `varnishstat`. | ||
| * Use tags for efficient cache purging: add `set beresp.http.Surrogate-Key = "images:product:123"` in `vcl_backend_response`. | ||
| * Configure workspace limits if caching many large images. | ||
|
|
||
| ## Cache headers from imgproxy | ||
|
|
||
| imgproxy sends the following headers useful for external caching: | ||
|
|
||
| * `Cache-Control: public, max-age=31536000` - Long-term caching (1 year default, configurable with `IMGPROXY_TTL`) | ||
| * `ETag` - If `IMGPROXY_USE_ETAG=true` (by default), enables conditional requests | ||
| * `Last-Modified` - If `IMGPROXY_USE_LAST_MODIFIED=true` (by default), enables conditional requests | ||
| * `Content-Type` - Important for cache key if not in `Accept` header |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| --- | ||
| title: Internal cache | ||
| description: Learn about how to configure and use imgproxy's internal cache | ||
| --- | ||
|
|
||
| # Internal cache ((pro)) | ||
|
|
||
| imgproxy Pro provides an internal cache that stores processed images on disk or in cloud storage. This cache can act as both a primary and a secondary cache, serving as a fallback when your CDN cache misses. | ||
|
|
||
| :::tip | ||
| While putting a CDN in front of imgproxy is and will always be a best practice, the internal cache provides long-term storage for cases that require an additional caching layer. | ||
| ::: | ||
|
|
||
| ## Why use internal cache? | ||
|
|
||
| The internal cache provides long-term persistent storage for processed images, unlike CDNs, which typically delete rarely accessed content. It stores images in a single location rather than across multiple edge stores, eliminating cache misses when requests hit different edges. The cache is designed specifically for imgproxy, working seamlessly with features like modern image format detection and client hints support that generic external caches don't understand by default. | ||
|
|
||
| The cache is protected by the same security measures as imgproxy itself, including URL signatures and processing restrictions. Importantly, URL signatures are not part of the cache key, so you can rotate keys or use multiple key/salt pairs without invalidating cached images. You maintain full control over where the cache is stored and how it integrates with your infrastructure. | ||
|
|
||
| ## Configuration | ||
|
|
||
| You need to define the following config variables to enable the internal cache: | ||
|
|
||
| * [`IMGPROXY_CACHE_USE`]: the cache storage adapter to use. Can be `fs`, `s3`, `gcs`, `abs` (Azure Blob Storage), or `swift` (OpenStack Swift). When blank, the cache is disabled. Default: blank | ||
| * [`IMGPROXY_CACHE_PATH_PREFIX`]: _(optional)_ a path prefix for the cache files. This can be useful to organize cache files in a specific directory structure. Default: blank | ||
| * [`IMGPROXY_CACHE_BUCKET`]: _(optional)_ the bucket name for cloud storage adapters (S3, GCS, ABS, Swift). When using the filesystem adapter, this can be used as an additional path component. Default: blank | ||
| * [`IMGPROXY_CACHE_KEY_HEADERS`]: _(optional)_ a list of HTTP request headers (comma-separated) to include in the cache key. This allows caching different versions of the same image based on request headers. Default: blank | ||
| * [`IMGPROXY_CACHE_KEY_COOKIES`]: _(optional)_ a list of HTTP request cookies (comma-separated) to include in the cache key. This allows caching different versions of the same image based on cookies. Default: blank | ||
| * [`IMGPROXY_CACHE_REPORT_ERRORS`]: When `true`, imgproxy will report cache errors instead of silently falling back to processing without cache. Default: `false` | ||
|
|
||
| ### Storage configuration | ||
|
|
||
| The internal cache supports all the storage backends that imgproxy can read source images from: local filesystem, Amazon S3 and compatible services (Cloudflare R2, DigitalOcean Spaces, MinIO, etc.), Google Cloud Storage, Microsoft Azure Blob Storage, and OpenStack Swift. | ||
|
|
||
| Configure the storage backend using `IMGPROXY_CACHE_*` variables: | ||
|
|
||
| * For filesystem cache, see [Cache storage: Local filesystem](./internal/local_filesystem.mdx). | ||
| * For S3 cache, see [Cache storage: Amazon S3](./internal/amazon_s3.mdx). | ||
| * For GCS cache, see [Cache storage: Google Cloud Storage](./internal/google_cloud_storage.mdx). | ||
| * For Azure Blob Storage cache, see [Cache storage: Azure Blob Storage](./internal/azure_blob_storage.mdx). | ||
| * For Swift cache, see [Cache storage: OpenStack Object Storage ("Swift")](./internal/openstack_swift.mdx). | ||
|
|
||
| ## Cache key | ||
|
|
||
| The cache key is generated based on: | ||
|
|
||
| * Source image URL | ||
| * Processing options | ||
| * Output format | ||
| * Optional: Request headers specified in `IMGPROXY_CACHE_KEY_HEADERS` | ||
| * Optional: Request cookies specified in `IMGPROXY_CACHE_KEY_COOKIES` | ||
|
|
||
| URL signature is **not** part of the cache key, allowing key rotation without invalidating the cache. | ||
|
|
||
| ## Limitations | ||
|
|
||
| * **No manual cache invalidation**: Currently, imgproxy doesn't provide a built-in means to invalidate the cache. However, imgproxy includes the [cachebuster](../usage/processing.mdx#cache-buster) in the cache key, so you can use it to force cache invalidation when needed. Most storage offerings also support object expiration, so you can set a reasonable expiration time for cached images. | ||
| * **No cache for info requests**: The internal cache is currently used only for image processing requests. Requests to the `/info` endpoint are not cached. | ||
|
|
||
| ## How it works | ||
|
|
||
| When a request comes in: | ||
|
|
||
| 1. imgproxy checks the URL signature (if enabled). | ||
| 2. imgproxy generates the cache key from the request parameters. | ||
| 3. imgproxy checks if a cached image exists in the configured storage. | ||
| 4. If the cached image exists and is valid, imgproxy serves it directly. | ||
| 5. If not, imgproxy processes the image and stores the result in the cache before serving it. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.