Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions pages/database-management/upgrades.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,20 @@ For a detailed, step-by-step ISSU procedure using Helm charts, see [In-Service
Software Upgrade
(ISSU)](/clustering/high-availability/setup-ha-cluster-k8s#in-service-software-upgrade-issu).

## Roll back an upgrade

Memgraph recovers from durability files written by earlier versions, but files
written by a newer version aren't guaranteed to load in an older one. To go
back to the previous version, restore the backup you created before upgrading:

1. Stop the instance.
2. Install the previous version.
3. Replace the data directory with the backup and start the instance. Changes
made after the backup was taken are lost.

For HA clusters, see
[Rollback](/clustering/high-availability/setup-ha-cluster-k8s#rollback).

## [Version-specific guides](/database-management/upgrades/specific-versions)

Certain Memgraph releases introduce breaking changes or require additional
Expand Down
14 changes: 7 additions & 7 deletions pages/deployment/environments/docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -294,9 +294,8 @@ periodic snapshot creation and create snapshots manually when you need to.

### Dump database

Another approach is to dump the database into a file. That is useful if you want
to move the data between different Memgraph versions that might have
incompatible data formats are used for snapshots or WAL files.
Another approach is to dump the database into a file of Cypher queries, which
you can run on another Memgraph instance to recreate the data.

Here are the steps to dump the database with Memgraph running in a Docker container:

Expand Down Expand Up @@ -329,10 +328,11 @@ import](/data-migration/best-practices#cypher-queries-best-practices).

### Upgrading Memgraph

Between different versions of Memgraph, we might break the
compatibility of the configurations, snapshots, and WAL files. That is done very
rarely, but it is helpful to keep an eye on the [release notes](/release-notes)
to see if there are any breaking changes.
Memgraph recovers from snapshot and WAL files written by earlier versions, so
the new version starts on your existing data directory. Follow the [upgrade
guide](/database-management/upgrades) and check the [release
notes](/release-notes) for breaking changes, such as removed configuration
flags.

## Set up a cluster

Expand Down
15 changes: 7 additions & 8 deletions pages/deployment/environments/linux.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -198,9 +198,8 @@ UNLOCK DATA DIRECTORY;

### Dump database

Another approach is to dump the database into a file. This is useful if you want
to move the data between different Memgraph versions that might have
incompatible data formats are used for snapshots or wal files.
Another approach is to dump the database into a file of Cypher queries, which
you can run on another Memgraph instance to recreate the data.

To dump the database, you can use the `mgconsole` CLI tool. Here is an example
of how to dump the database into a file:
Expand Down Expand Up @@ -237,11 +236,11 @@ import](/data-migration/best-practices#cypher-queries-best-practices).

### Upgrading Memgraph

Keep in mind that between different versions of Memgraph, we might break the
compatibility of the configurations, snapshots, and WAL files. This is done very
rarely, but it is useful to keep an eye on the [release
notes](/release-notes) to see if there are any breaking
changes.
Memgraph recovers from snapshot and WAL files written by earlier versions, so
the new version starts on your existing data directory. Follow the [upgrade
guide](/database-management/upgrades) and check the [release
notes](/release-notes) for breaking changes, such as removed configuration
flags.

## Set up a cluster

Expand Down
54 changes: 37 additions & 17 deletions pages/fundamentals/data-durability.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Data durability
description: Learn about data durability - an important aspect of making sure you keep your data secure, regardless of Memgraph's in-memory storage.
description: Configure snapshots and write-ahead logging (WAL), see how Memgraph recovers on startup, and handle recovery failures.
---

import { Callout } from 'nextra/components'
Expand Down Expand Up @@ -84,12 +84,13 @@ By default, WAL files are located at `/var/lib/memgraph/wal`.

<h4 id="wal-file-lifecycle" className="custom-header">WAL file lifecycle</h4>

**Older WAL files are deleted automatically after a snapshot is created** since
the snapshot contains the full database state up to that point. Only WAL files
containing changes after the latest snapshot are retained.
Memgraph deletes older WAL files when it deletes old snapshots, not after every
snapshot. Once a new snapshot brings the number of snapshots to
`--storage-snapshot-retention-count`, Memgraph deletes the WAL files older than
the oldest snapshot it keeps, except the last WAL file written before that
snapshot.

To control WAL file cleanup indirectly, you can limit the number of snapshots
via `--storage-snapshot-retention-count`.
To keep more snapshots, raise `--storage-snapshot-retention-count`.

**It is not possible to use WAL files exclusively** without snapshots. Memgraph
enforces periodic snapshots when WAL is enabled and will fail to start if WAL is
Expand Down Expand Up @@ -194,7 +195,7 @@ snapshot creation behavior by [defining the relevant
flags](/database-management/configuration#storage). Alternatively, you can make
one directly by running the following query:

```opencypher
```cypher
CREATE SNAPSHOT;
```

Expand All @@ -209,7 +210,7 @@ directory. The `CREATE SNAPSHOT` query will return the path of the newly created
snapshot file.

To query which snapshots currently exist in the data directory, execute:
```opencypher
```cypher
SHOW SNAPSHOTS;
```

Expand All @@ -230,14 +231,26 @@ files store incremental changes and need to be replayed sequentially.

`IN_MEMORY_TRANSACTIONAL` mode supports periodic snapshot creation. The interval
can be set at startup via the `--storage-snapshot-interval` flag or at run-time
via the database settings:
via the database settings. To create a snapshot every 1200 seconds:

```opencypher
```cypher
SET DATABASE SETTING "storage.snapshot.interval" TO "1200";
SET DATABASE SETTING "storage.snapshot.interval" TO "* * 12 * * *";
```

To pause periodic snapshots, set the interval to an empty string:

```cypher
SET DATABASE SETTING "storage.snapshot.interval" TO "";
```

With [Memgraph Enterprise](/database-management/enabling-memgraph-enterprise),
you can set the interval as a CRON expression instead. To create a snapshot
every day at 12:00:00:

```cypher
SET DATABASE SETTING "storage.snapshot.interval" TO "0 0 12 * * *";
```

<Callout type="info">
Changing the configuration settings depends on the way you are using Memgraph,
so please refer to the [configuration
Expand All @@ -250,9 +263,7 @@ in seconds. Interval can also be defined as a 6-field CRON expression (seconds,
hour, day of month, month, day of week). Standard 5-field cron references (e.g.
[crontab.guru](https://crontab.guru/)) describe the last five fields; Memgraph
adds seconds as the first field. By setting the value to an empty string, the background process is
paused and any currently active snapshot creation will finish. Please note that
defining the interval via a CRON expression is an [Enterprise
feature](/database-management/enabling-memgraph-enterprise).
paused and any currently active snapshot creation will finish.

If the database is started in or migrated into `IN_MEMORY_ANALYTICAL` mode, the
background thread will pause and no snapshots will be created as long as that
Expand All @@ -265,9 +276,15 @@ new writes have been committed since the last snapshot. If the periodic snapshot
is skipped it will be logged on INFO level.
</Callout>

<Callout type="warning">
Snapshots and WAL files are presently not compatible between Memgraph versions.
</Callout>
### Compatibility between versions

Memgraph recovers from snapshot and WAL files written by Memgraph 2.0 or any
later version, so after an [upgrade](/database-management/upgrades) you can
start the new version on your existing data directory.

Files written by a newer version aren't guaranteed to load in an older one. To
go back to an earlier version, [restore the backup you took before
upgrading](/database-management/upgrades#roll-back-an-upgrade).

### Parallelized execution

Expand Down Expand Up @@ -454,6 +471,9 @@ deletes them when [`--storage-backup-dir-enabled`](/database-management/configur
is off), leaving a clean single-snapshot directory. As a result, the database **recovers cleanly on the
next restart and does not re-enter the broken state**.

The database then holds only the data in that snapshot. Changes made after the
snapshot was taken, including those in a damaged WAL file, are lost.

`RECOVER SNAPSHOT` is available in both Community and Enterprise editions, so
single-tenant deployments can recover a corrupt default database.

Expand Down
35 changes: 34 additions & 1 deletion pages/help-center/errors/durability.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Durability errors
description: Discover how to maintain the integrity of your graph database by ensuring durability and data reliability. Explore the docs to get to know how to optimize durability settings.
description: Fix durability errors, from missing snapshot or WAL files and a wrong data directory to recovery failures caused by damaged files.
---

import { Callout } from 'nextra/components'
Expand Down Expand Up @@ -48,4 +48,37 @@ be restarted. The default directory is `/var/lib/memgraph`. To learn about all
the configuration options, check out the [reference
guide](/database-management/configuration).

## Why doesn't Memgraph start after a recovery failure?

On startup, Memgraph recovers each database from its snapshot and WAL files. If
a snapshot or WAL file it needs for recovery can't be loaded, usually because a
disk issue damaged it, recovery stops and Memgraph doesn't start. WAL files
that the newest snapshot already covers aren't read, so damage in them doesn't
stop recovery.

To start the instance and recover the database:

1. Restart Memgraph with `--storage-allow-recovery-failure=true`. The database
that failed comes up empty, in a broken state, with its files left as they
were. Other databases on the instance work normally.
2. Switch to the broken database and load a good snapshot from its
`snapshots` directory:

```cypher template
USE DATABASE DATABASE_NAME;
RECOVER SNAPSHOT "SNAPSHOT_PATH";
```

`DATABASE_NAME` is the database that failed (`memgraph` by default), and
`SNAPSHOT_PATH` is the full path to the snapshot file. In Memgraph
Community, skip `USE DATABASE`.

The database then holds only the data in that snapshot. Changes made after the
snapshot was taken, including those in the damaged WAL file, are lost. If you
have a backup of the whole data directory, you can restore it instead.

The `--storage-allow-recovery-failure` flag works only with in-memory storage
modes. For details, see [recovery failure
handling](/fundamentals/data-durability#recovery-failure-handling).

<CommunityLinks/>
15 changes: 12 additions & 3 deletions pages/help-center/errors/snapshots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,18 @@ persistence. Memgraph will try to load the newest snapshot file on startup.

## What to do with corrupt snapshots?

Because snapshots are read-only, any modifications will result in corrupt
files that won't be loaded. The solution is to delete the snapshot files and to
start Memgraph again.
If Memgraph can't load a snapshot it needs for recovery, it doesn't start. You
have two options:

- **Delete the corrupt snapshot file** and start Memgraph again. Memgraph
recovers from the newest remaining snapshot and the WAL files written after
it. If the corrupt file is the only snapshot, deleting it loses the data it
holds.
- **Load a good snapshot.** Start Memgraph with
`--storage-allow-recovery-failure=true`, then load a good snapshot with
`RECOVER SNAPSHOT`. Changes made after that snapshot was taken are lost. See
[recovery failure
handling](/fundamentals/data-durability#recovery-failure-handling).

## Why is data lost when Memgraph is restarted?

Expand Down
Loading