-
Notifications
You must be signed in to change notification settings - Fork 182
[DOC-14049]: Document Magma Zstd compression level configuration setting #4120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
RayOffiah
wants to merge
1
commit into
release/8.1
Choose a base branch
from
DOC-14049--8.1--Document-Magma-Zstd-compression-level-configuration-setting
base: release/8.1
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+137
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
132 changes: 132 additions & 0 deletions
132
modules/rest-api/pages/rest-magma-compression-per-bucket.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| = Magma Compression | ||
| :description: Compression algorithms can be set and listed on a per-bucket basis using the REST API. | ||
|
|
||
| [abstract] | ||
| {description} | ||
|
|
||
| Compression settings can be set for an existing bucket. | ||
| This requires `Full Admin`, `Cluster Admin` roles, or the `Bucket Admin` role for the specified bucket. | ||
|
|
||
| == HTTP methods and URIs | ||
|
|
||
| [source] | ||
| ---- | ||
| GET /pools/default/buckets/<bucket-name> | ||
| POST /pools/default/buckets/<bucket-name> | ||
| ---- | ||
|
|
||
| == Parameters | ||
| The following parameters are available to the REST API | ||
| [cols="4,3,3"] | ||
| |=== | ||
| | Parameter | Type | Description | ||
|
|
||
| | `magmaEnableIndexBlockAutotuning` | ||
| | Boolean | ||
| | Enable auto-tuning of Magma index block size based on the compression ratio | ||
|
|
||
| Automatically adjusts the pre-compression block-accumulation size so the post-compression (physical) index block size stays close to the configured target block size. | ||
|
|
||
| |`magmaEnableDataBlockAutotuning` | ||
| | Boolean | ||
| | Enable auto-tuning of Magma data block size based on the compression ratio | ||
|
|
||
| |`magmaIndexCompressionAlgo` | ||
| | `snappy`, `lz4`, `none`, `zstd`, or `zstd_<level>` where `<level>` is an integer from 1 to 22. | ||
|
|
||
| | Configure the block compression algorithm for the index blocks. | ||
|
|
||
| | `magmaDataCompressionAlgo` | ||
| | `snappy`, `lz4`, `none`, `zstd`, or `zstd_<level>` where `<level>` is an integer from 1 to 22. | ||
| | Configure the block compression algorithm for the data blocks. | ||
|
|
||
| | `magmaCompacteddataCompressionAlgo` | ||
| | `snappy`, `lz4`, `none`, `zstd`, or `zstd_<level>` where `<level>` is an integer from 1 to 22. | ||
| | Configure the block compression algorithm for the data blocks after compaction. | ||
|
|
||
| Allows colder data to be compressed with a higher compression ratio using `zstd`. | ||
| This compression also is scheduled in background compaction threads. | ||
|
|
||
| |=== | ||
|
|
||
| == `Curl` Syntax | ||
|
|
||
| [source, shell] | ||
| ---- | ||
| curl -u <username>:<password> -X GET \ | ||
| <ip-address-or-domain-name>:8091/pools/default/buckets/<bucket-name> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd replace the port # with a placeholder. |
||
|
|
||
| curl -u <username>:<password> -X POST \ | ||
| <ip-address-or-domain-name>:8091//pools/default/buckets/<bucket-name> \ | ||
| -d magmaEnableIndexBlockAutotuning=true|false \ | ||
| -d magmaEnableDataBlockAutotuning=true|false \ | ||
| -d magmaIndexCompressionAlgo=snappy|lz4|none|zstd|zstd_<integer> \ | ||
| -d magmaDataCompressionAlgo=snappy|lz4|none|zstd|zstd_<integer> \ | ||
| -d magmaCompacteddataCompressionAlgo=snappy|lz4|none|zstd|zstd_<integer> \ | ||
|
|
||
| ---- | ||
|
|
||
| == Responses | ||
|
|
||
| If the call is successful, `200 OK` is given. | ||
|
|
||
| If the operation is performed on a bucket that does not exist, | ||
| the REST service responds with a `404 (Object not found)` error. | ||
|
|
||
| Sending the request with an invalid parameter value (sending a number when a Boolean value is expected, for example), returns a `400 (Bad Request)` response. | ||
|
|
||
| You can find the precise nature of the problem by examining the payload in the response: | ||
|
|
||
| [source, jsonlines] | ||
| ---- | ||
| { | ||
| "errors": { | ||
| "magmaEnableDataBlockAutotuning": "Value \"ten\" must be a boolean value (true/false, on/off)" | ||
| }, | ||
|
|
||
| } | ||
| ---- | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| Errors in the parameter name result in no change in any of the parameter values, but the REST service responds with `200 (OK)`: | ||
|
|
||
|
|
||
| ==== | ||
|
|
||
|
|
||
| == Examples | ||
|
|
||
| .Retrieve parameter settings for the bucket. | ||
| [source, shell] | ||
| ---- | ||
| curl -u Administrator:password -X GET \ | ||
| localhost:8091/pools/default/buckets/travel-sample | jq | ||
| ---- | ||
|
|
||
| This retrieves all the parameters in the bucket as a JSON string. | ||
| If you want to retrieve a parameter or list of parameters, you can add them to `jq` pipe: | ||
|
|
||
| [source, shell] | ||
| ---- | ||
| curl -u Administrator:password -X GET \ | ||
| localhost:8091/pools/default/buckets/travel-sample | jq \ | ||
| '.magmaEnableDataBlockAutotuning,.magmaEnableDataBlockAutotuning,.magmaIndexCompressionAlgo,.magmaDataCompressionAlgo,.magmaCompacteddataCompressionAlgo' | ||
| ---- | ||
|
|
||
| .Enable auto-tuning of the bucket index block. | ||
| [source, shell] | ||
| ---- | ||
| curl -u Administrator:password -X POST \ | ||
| localhost:8091/pools/default/buckets/travel-sample \ | ||
| -d magmaEnableIndexBlockAutotuning=true | ||
| ---- | ||
|
|
||
| .Set the data compression algorithm | ||
| [source,shell] | ||
| ---- | ||
| curl -u Administrator:password -X POST \ | ||
| localhost:8091/pools/default/buckets/travel-sample \ | ||
| -d magmaDataCompressionAlgo=snappy | ||
| ---- | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe make this a bit more active voice "You can set the compression algorithms..." While the page title makes it clear its for Magma, maybe also add in that these settings are specific to Magma in the description so AI and search picks up on that better.