Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 26 additions & 58 deletions src/content/docs/snowflake/capabilities/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,30 @@ label:

LocalStack exposes various configuration options to control its behaviour.

These options can be passed to LocalStack as environment variables like so:
With `lstk`, these options can be passed as `LOCALSTACK_`-prefixed environment variables when starting the container:

```bash
DEBUG=1 localstack start --stack snowflake
LOCALSTACK_DEBUG=1 lstk start
```

Alternatively, set them as named environment profiles in your config file and reference them from the container block:

```toml
# .lstk/config.toml
[[containers]]
type = "snowflake"
env = ["debug"]

[env.debug]
DEBUG = "1"
```

```bash
lstk start
```

See [Passing environment variables to the container](/aws/developer-tools/running-localstack/lstk#passing-environment-variables-to-the-container) for details.

## Core

Options that affect the core Snowflake emulator functionality.
Expand All @@ -35,11 +53,11 @@ Options that affect the core Snowflake emulator functionality.

By default, the Snowflake emulator accepts requests for hostnames such as `snowflake.localhost.localstack.cloud` and other `*.snowflake.*` hostnames.
If you expose the emulator through a custom DNS name, for example in Kubernetes or behind an ingress, set `SF_HOSTNAMES` to the exact hostnames clients use to reach the emulator.
When you use the `localstack` CLI, add the `LOCALSTACK_` prefix so the CLI passes the variable to the container:
When you use `lstk`, add the `LOCALSTACK_` prefix so the CLI passes the variable to the container:

```bash

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend using the lstk config file to do this: see my reply in https://linear.app/localstack/issue/PRO-327/verify-lstk-behavior-in-capabilitiesconfigurationmd

LOCALSTACK_SF_HOSTNAMES=snowflake.internal.example.com,snowflake.internal,snowflake.localhost.localstack.cloud \
localstack start --stack snowflake
lstk start
```

The first hostname in `SF_HOSTNAMES` is used as the primary hostname for local connection defaults and generated URLs.
Expand Down Expand Up @@ -75,77 +93,27 @@ If your custom hostname also needs a matching TLS certificate, use LocalStack's
LOCALSTACK_SF_HOSTNAMES=snowflake.internal.example.com \
CUSTOM_SSL_CERT_PATH=/var/lib/localstack/custom/cert.pem \
SKIP_SSL_CERT_DOWNLOAD=1 \
localstack start --stack snowflake
lstk start

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

```

The file referenced by `CUSTOM_SSL_CERT_PATH` must contain a certificate and private key that match the hostname used by your Snowflake clients.
For more general guidance on adding trusted certificates to LocalStack, see [Custom TLS certificates](/aws/developer-tools/security-testing/custom-tls-certificates/).

## CLI

These options are applicable when using the CLI to start LocalStack.

| Variable | Example Values | Description |
| - | - | - |
| `LOCALSTACK_VOLUME_DIR` | `~/.cache/localstack/volume` (on Linux) | The location on the host of the LocalStack volume directory mount. |
| `CONFIG_PROFILE` | | The configuration profile to load. See [Profiles](#profiles) |
| `CONFIG_DIR` | `~/.localstack` | The path where LocalStack can find configuration profiles and other CLI-specific configuration |
`lstk` is configured through its config file rather than through environment variables.
See [Configuration](/aws/developer-tools/running-localstack/lstk#configuration) on the `lstk` page for the config file search order, the field reference, and how to define named environment profiles.

## Docker

Options to configure how LocalStack interacts with Docker.

| Variable | Example Values | Description |
| - | - | - |
| `LOCALSTACK_VOLUME_DIR` | `~/.cache/localstack/volume` (on Linux) | The location on the host of the LocalStack volume directory mount. |
| `DOCKER_FLAGS` | | Allows to pass custom flags (e.g., volume mounts) to "docker run" when running LocalStack in Docker. |
| `DOCKER_SOCK` | `/var/run/docker.sock` | Path to local Docker UNIX domain socket |
| `DOCKER_BRIDGE_IP` | `172.17.0.1` | IP of the Docker bridge used to enable access between containers |
| `LEGACY_DOCKER_CLIENT` | `0`\|`1` | Whether LocalStack should use the command-line Docker client and subprocess execution to run Docker commands, rather than the Docker SDK. |
| `DOCKER_CMD` | `docker` (default), `sudo docker`| Shell command used to run Docker containers (only used in combination with `LEGACY_DOCKER_CLIENT`) |
| `FORCE_NONINTERACTIVE` | | When running with Docker, disables the `--interactive` and `--tty` flags. Useful when running headless. |

## Profiles

LocalStack supports configuration profiles which are stored in the `~/.localstack` config directory.
A configuration profile is a set of environment variables stored in a `*.env` file in the LocalStack config directory.

Here is an example of what configuration profiles might look like:

```bash
tree ~/.localstack
/home/username/.localstack
├── default.env
├── dev.env
└── pro.env
```

Here is an example of what a specific environment profile looks like

```bash
cat ~/.localstack/pro-debug.env
LOCALSTACK_AUTH_TOKEN=XXXXX
SF_LOG=trace
SF_S3_ENDPOINT=s3.localhost.localstack.cloud:4566
```

You can load a profile by either setting the environment variable `CONFIG_PROFILE=<profile>` or the `--profile=<profile>` CLI flag when using the CLI.
Let's take an example to load the `dev.env` profile file if it exists:

```bash
localstack --profile=dev start --stack snowflake
```

If no profile is specified, the `default.env` profile will be loaded.
If explicitly specified, any environment variables will overwrite the configurations defined in the profile.

To display the config environment variables, you can use the following command:

```bash
localstack --profile=dev config show
```

:::note
The `CONFIG_PROFILE` is a CLI feature and cannot be used directly with a docker-compose setup.
You can look at [alternative means of setting environment variables](https://docs.docker.com/compose/environment-variables/set-environment-variables/) for your Docker Compose setups.
For Docker setups, we recommend passing the environment variables directly to the `docker run` command.
:::
22 changes: 16 additions & 6 deletions src/content/docs/snowflake/capabilities/init-hooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ CREATE DATABASE test123;
SHOW DATABASES;
```

Mount the script into `/etc/localstack/init/ready.d/` using Docker Compose or the `localstack` CLI:
Mount the script into `/etc/localstack/init/ready.d/` using Docker Compose or `lstk`:

<Tabs>
<TabItem label="docker-compose.yml">
Expand All @@ -51,12 +51,22 @@ services:
- "/var/run/docker.sock:/var/run/docker.sock"
```
</TabItem>
<TabItem label="CLI">
<TabItem label="lstk">
Declare the bind mount and the `DEBUG` profile in your config file, then start LocalStack:

```toml
# .lstk/config.toml
[[containers]]
type = "snowflake"
env = ["debug"]
volumes = ["/path/to/test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql"]

[env.debug]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add an inline comment here saying: "Optionally enable DEBUG" as DEBUG is not required for init hook but helps to diagnose init hook issues

DEBUG = "1"
```

```bash
# DOCKER_FLAGS are additional parameters to the `docker run` command of localstack start
DOCKER_FLAGS='-v /path/to/test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql' \
DEBUG=1 \
localstack start --stack snowflake
lstk start
```
</TabItem>
</Tabs>
Expand Down
13 changes: 6 additions & 7 deletions src/content/docs/snowflake/capabilities/state-management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,13 @@ State Management is an essential feature that supports various use-cases, such a

LocalStack’s Persistence mechanism enables the saving and restoration of the entire LocalStack state. It functions as a **pause and resume** feature, allowing you to take a snapshot of your LocalStack instance and save this data to disk. This mechanism ensures a quick and efficient way to preserve and continue your work with Snowflake resources locally.

To start snapshot-based persistence, launch LocalStack with the configuration option `PERSISTENCE=1`. This setting instructs LocalStack to save all local Snowflake resources and their respective application states into the LocalStack Volume Directory. Upon restarting LocalStack, you'll be able to resume your activities exactly where you left off.
To start snapshot-based persistence, launch LocalStack with the `--persist` command-line option, or the configuration option `PERSISTENCE=1`. This setting instructs LocalStack to save all local Snowflake resources and their respective application states into the LocalStack Volume Directory. Upon restarting LocalStack, you'll be able to resume your activities exactly where you left off.

<Tabs>
<TabItem label="LocalStack CLI">
<TabItem label="lstk">
```bash
export LOCALSTACK_AUTH_TOKEN=<your_auth_token>
PERSISTENCE=1 \
localstack start --stack snowflake
lstk start --persist
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth adding a note that to resume a persisted session after the emulator stopped, or to restart a session while keeping persistence on, you also need this flag, i.e.

lstk stop
lstk start --persist

and

lstk restart --persist

</TabItem>
<TabItem label="Docker Compose">
Expand Down Expand Up @@ -69,17 +68,17 @@ The Export/Import State feature enables you to export the state of your LocalSta
To export the state, you can run the following command:

```bash
localstack state export '<file-name>'
lstk snapshot save '<file-name>'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since file-name here mostly associates with a path, I'd suggest lstk snapshot save <path-to-snapshot-file>

```

You can use the `<file-name>` argument to specify a file path to export the state to. If you do not specify a file path, the state will be exported to the current working directory into a file named `ls-state-export`.
You can use the `<file-name>` argument to specify a file path to export the state to. If you do not specify a file path, the state will be exported to the current working directory into an auto-named snapshot file.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's helpful to link to the snapshot section of the lstk doc for more info on supported save destinations (local and remote).


### Import the State

To import the state, you can run the following command:

```bash
localstack state import '<file-name>'
lstk snapshot load '<file-name>'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you agree with using above then this also changes to lstk snapshot load <path-to-snapshot-file>

```

The `<file-name>` argument is required and specifies the file path to import the state from. The file should be generated from a previous export.
2 changes: 1 addition & 1 deletion src/content/docs/snowflake/tooling/user-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This guide is designed for users new to the Snowflake emulator Web UI. Start you

```bash
export LOCALSTACK_AUTH_TOKEN=<your_auth_token>
localstack start --stack snowflake
lstk start
```

Navigate to [**https://app.localstack.cloud/inst/default/snowflake**](https://app.localstack.cloud/inst/default/snowflake) to access the User Interface.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The code in this tutorial is available on [GitHub](https://github.com/localstack

## Prerequisites

- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [`lstk`](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [LocalStack for Snowflake](/snowflake/getting-started/)
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`awslocal` wrapper](/aws/connecting/aws-cli/#localstack-aws-cli-awslocal)
- Python 3.10 installed locally
Expand Down Expand Up @@ -141,9 +141,9 @@ Start your LocalStack container in your preferred terminal/shell.

```bash showLineNumbers
export LOCALSTACK_AUTH_TOKEN=<your_auth_token>
DEBUG=1 \
LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT=180 \
localstack start --stack snowflake
LOCALSTACK_DEBUG=1 \
LOCALSTACK_LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT=180 \
lstk start --type snowflake
```

> The `DEBUG=1` environment variable is set to enable debug logs. It would allow you to see the SQL queries executed by the Lambda function. The `LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT` environment variable is set to increase the Lambda function's timeout to 180 seconds.
Expand All @@ -170,7 +170,7 @@ awslocal lambda invoke --function-name localstack-snowflake-lambda-example \
--payload '{"body": "test"}' output.txt
```

You will receive a response with the details of the invocation. You can view the output in the `output.txt` file. To see the SQL queries executed by the Lambda function, check the logs by navigating to LocalStack logs (`localstack logs`).
You will receive a response with the details of the invocation. You can view the output in the `output.txt` file. To see the SQL queries executed by the Lambda function, check the logs by navigating to LocalStack logs (`lstk logs`).

```bash
2024-02-07T17:33:36.763 DEBUG --- [ asgi_gw_3] l.s.l.i.version_manager : [localstack-snowflake-lambda-example-b0813b21-ad5f-4ec7-8fb4-53147df9695e] Total # of rows: 3
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The Jupyter Notebook and the dataset used in this tutorial are available on [Git

## Prerequisites

- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [`lstk`](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [LocalStack for Snowflake](/snowflake/getting-started/)
- [Snowpark](/snowflake/integrations/snowpark) with other Python libraries
- [Jupyter Notebook](https://jupyter.org/install#jupyter-notebook)
Expand All @@ -27,7 +27,7 @@ Start your LocalStack container in your preferred terminal/shell.

```bash
export LOCALSTACK_AUTH_TOKEN=<your_auth_token>
localstack start --stack snowflake
lstk start --type snowflake

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply lstk start

```

## Create a Snowpark session
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ With LocalStack's Snowflake emulator, you can create catalog integrations that c

## Prerequisites

- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [`lstk`](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/)
- [LocalStack for Snowflake](/snowflake/getting-started/)
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`awslocal` wrapper](/aws/connecting/aws-cli/#localstack-aws-cli-awslocal)
- Python 3.10+ with `pyiceberg` and `pyarrow` installed
Expand All @@ -29,7 +29,7 @@ Start your LocalStack container with the Snowflake emulator enabled.

```bash
export LOCALSTACK_AUTH_TOKEN=<your_auth_token>
localstack start --stack snowflake
lstk start --type snowflake

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply lstk start

```

## Create S3 Tables resources
Expand Down
Loading