diff --git a/modules/virt-configuring-a-live-migration-policy.adoc b/modules/virt-configuring-a-live-migration-policy.adoc index cd255cdef884..649720cb8be8 100644 --- a/modules/virt-configuring-a-live-migration-policy.adoc +++ b/modules/virt-configuring-a-live-migration-policy.adoc @@ -65,15 +65,19 @@ metadata: name: spec: selectors: - namespaceSelector: <1> + namespaceSelector: hpc-workloads: "True" xyz-workloads-type: "" - virtualMachineInstanceSelector: <2> + virtualMachineInstanceSelector: kubevirt.io/environment: "production" ---- -<1> Specify project labels. -<2> Specify VM labels. - ++ +where: ++ +`namespaceSelector`:: Specifies the project labels. ++ +`virtualMachineInstanceSelector`:: Specifies the VM labels. ++ . Create the migration policy by running the following command: + [source,terminal] diff --git a/modules/virt-configuring-live-migration-heavy.adoc b/modules/virt-configuring-live-migration-heavy.adoc new file mode 100644 index 000000000000..7fd80796ddba --- /dev/null +++ b/modules/virt-configuring-live-migration-heavy.adoc @@ -0,0 +1,70 @@ + +// Module included in the following assemblies: +// +// * virt/live_migration/virt-configuring-live-migration.adoc + +:_mod-docs-content-type: PROCEDURE +[id="virt-configuring-live-migration-heavy_{context}"] += Configure live migration for heavy workloads + +[role="_abstract"] +When migrating a VM running a heavy workload (for example, database processing) with higher memory dirty rates, you need a higher bandwidth to complete the migration. + +If the dirty rate is too high, the migration from one node to another does not converge. To prevent this, enable post copy mode. + +Post copy mode triggers if the initial pre-copy phase does not complete within the defined timeout. During post copy, the VM CPUs pause on the source host while transferring the minimum required memory pages. Then the VM CPUs activate on the destination host, and the remaining memory pages transfer into the destination node at runtime. + +Configure live migration for heavy workloads by updating the `HyperConverged` custom resource (CR), which is located in the `{CNVNamespace}` namespace. + +.Prerequisites + +* You have installed the {oc-first}. + +.Procedure + +. Edit the `HyperConverged` CR and add the necessary parameters for migrating heavy workloads: ++ +[source,terminal,subs="attributes+"] +---- +$ oc edit hyperconverged kubevirt-hyperconverged -n {CNVNamespace} +---- ++ +Example configuration file: ++ +[source,yaml,subs="attributes+"] +---- +apiVersion: hco.kubevirt.io/v1beta1 +kind: HyperConverged +metadata: + name: kubevirt-hyperconverged + namespace: {CNVNamespace} +spec: + liveMigrationConfig: + bandwidthPerMigration: 0Mi + completionTimeoutPerGiB: 150 + parallelMigrationsPerCluster: 5 + parallelOutboundMigrationsPerNode: 1 + progressTimeout: 150 + allowPostCopy: true +---- ++ +where: ++ +`bandwidthPerMigration`:: Specifies the bandwidth of each migration in bytes per second. The default is `0`, which is unlimited. ++ +`completionTimeoutPerGiB`:: Specifies the length of time, in seconds per GiB of memory, at which the migration is canceled if it has not completed and post copy mode is triggered, if enabled. You can lower `completionTimeoutPerGiB` to trigger post copy mode earlier in the migration process, or raise the `completionTimeoutPerGiB` to trigger post copy mode later in the migration process. ++ +`parallelMigrationsPerCluster`:: Specifies the number of migrations running in parallel in the cluster. The default is `5`. Keeping the `parallelMigrationsPerCluster` setting low is better when migrating heavy workloads. ++ +`parallelOutboundMigrationsPerNode`:: Specifies the maximum number of outbound migrations per node. Configure a single VM per node for heavy workloads. ++ +`progressTimeout`:: Specifies the length of time, in seconds, at which the migration is canceled if memory copy fails to make progress. Increase this parameter for large memory sizes running heavy workloads. ++ +`allowPostCopy`:: Specifies whether the post copy mode is enabled. You can enable post copy mode to allow the migration of one node to another to converge, even if a VM is running a heavy workload and the memory dirty rate is too high. Set allowPostCopy to true to enable post copy mode. ++ +. Optional: If your main network is too busy for the migration, configure a secondary, dedicated migration network. ++ +[NOTE] +==== +Post copy mode can impact performance during the transfer, and should not be used for critical data, or with unstable networks. +==== diff --git a/modules/virt-configuring-live-migration-limits.adoc b/modules/virt-configuring-live-migration-limits.adoc index 3957f35fb418..b3fdc906874a 100644 --- a/modules/virt-configuring-live-migration-limits.adoc +++ b/modules/virt-configuring-live-migration-limits.adoc @@ -7,8 +7,12 @@ [id="virt-configuring-live-migration-limits_{context}"] = Configuring live migration limits and timeouts -Configure live migration limits and timeouts for the cluster by updating the `HyperConverged` custom resource (CR), which is located in the -`openshift-cnv` namespace. +[role="_abstract"] +Configure live migration limits and timeouts for the cluster by updating the `HyperConverged` custom resource (CR), which is located in the `{CNVNamespace}` namespace. + +.Prerequisites + +* You have installed the {oc-first}. .Procedure @@ -19,7 +23,8 @@ Configure live migration limits and timeouts for the cluster by updating the `Hy $ oc edit hyperconverged kubevirt-hyperconverged -n {CNVNamespace} ---- + -.Example configuration file +Example configuration file: ++ [source,yaml,subs="attributes+"] ---- apiVersion: hco.kubevirt.io/v1beta1 @@ -29,18 +34,28 @@ metadata: namespace: {CNVNamespace} spec: liveMigrationConfig: - bandwidthPerMigration: 64Mi <1> - completionTimeoutPerGiB: 800 <2> - parallelMigrationsPerCluster: 5 <3> - parallelOutboundMigrationsPerNode: 2 <4> - progressTimeout: 150 <5> + bandwidthPerMigration: 64Mi + completionTimeoutPerGiB: 800 + parallelMigrationsPerCluster: 5 + parallelOutboundMigrationsPerNode: 2 + progressTimeout: 150 + allowPostCopy: false ---- -<1> Bandwidth limit of each migration, where the value is the quantity of bytes per second. For example, a value of `2048Mi` means 2048 MiB/s. Default: `0`, which is unlimited. -<2> The migration is canceled if it has not completed in this time, in seconds per GiB of memory. For example, a VM with 6GiB memory times out if it has not completed migration in 4800 seconds. If the `Migration Method` is `BlockMigration`, the size of the migrating disks is included in the calculation. -<3> Number of migrations running in parallel in the cluster. Default: `5`. -<4> Maximum number of outbound migrations per node. Default: `2`. -<5> The migration is canceled if memory copy fails to make progress in this time, in seconds. Default: `150`. - ++ +where: ++ +`bandwidthPerMigration`:: Specifies the bandwidth of each migration in bytes per second. For example, a value of 2048Mi means 2048 MiB/s. Default: 0, which is unlimited. ++ +`completionTimeoutPerGiB`:: Specifies the length of time, in seconds per GiB of memory, at which the migration is canceled if it has not completed. For example, a VM with 6GiB memory times out if it has not completed migration in 4800 seconds. If the `Migration Method` is `BlockMigration`, the size of the migrating disks is included in the calculation. ++ +`parallelMigrationsPerCluster`:: Specifies the number of migrations running in parallel in the cluster. Default: `5`. ++ +`parallelOutboundMigrationsPerNode`:: Specifies the maximum number of outbound migrations per node. Default: `2`. ++ +`progressTimeout`:: Specifies the length of time, in seconds, at which the migration is canceled if memory copy fails to make progress. Default: `150`. ++ +`allowPostCopy`:: Specifies whether the post copy mode is enabled. You can enable post copy mode to allow the migration of one node to another to converge, even if a VM is running a heavy workload and the memory dirty rate is too high. By default, `allowPostCopy` is set to `false`. ++ [NOTE] ==== You can restore the default value for any `spec.liveMigrationConfig` field by deleting that key/value pair and saving the file. For example, delete `progressTimeout: ` to restore the default `progressTimeout: 150`. diff --git a/modules/virt-live-migration-policies.adoc b/modules/virt-live-migration-policies.adoc new file mode 100644 index 000000000000..a904ae4933a5 --- /dev/null +++ b/modules/virt-live-migration-policies.adoc @@ -0,0 +1,15 @@ +// Module included in the following assemblies: +// +// * virt/live_migration/virt-configuring-live-migration.adoc + +:_mod-docs-content-type: CONCEPT +[id="virt-live-migration-policies_{context}"] += Live migration policies + +[role="_abstract"] +You can create live migration policies to apply different migration configurations to groups of VMs that are defined by VM or project labels. + +[TIP] +==== +You can create live migration policies by using the {product-title} web console. +==== diff --git a/virt/live_migration/virt-configuring-live-migration.adoc b/virt/live_migration/virt-configuring-live-migration.adoc index d3eeebf9b052..a756af113324 100644 --- a/virt/live_migration/virt-configuring-live-migration.adoc +++ b/virt/live_migration/virt-configuring-live-migration.adoc @@ -10,8 +10,6 @@ You can configure live migration settings to ensure that the migration processes You can configure live migration policies to apply different migration configurations to groups of virtual machines (VMs). -include::modules/virt-configuring-live-migration-limits.adoc[leveloffset=+1] - [id="live-migration-policies"] == Live migration policies @@ -22,9 +20,16 @@ You can create live migration policies to apply different migration configuratio You can create live migration policies by using the {VirtProductName} web console. ==== -include::modules/virt-configuring-a-live-migration-policy.adoc[leveloffset=+2] +include::modules/virt-configuring-live-migration-limits.adoc[leveloffset=+1] + +include::modules/virt-configuring-live-migration-heavy.adoc[leveloffset=+1] + +include::modules/virt-live-migration-policies.adoc[leveloffset=+1] + +include::modules/virt-configuring-a-live-migration-policy.adoc[leveloffset=+1] [role="_additional-resources"] [id="additional-resources_virt-configuring-live-migration"] == Additional resources -* xref:../../virt/vm_networking/virt-dedicated-network-live-migration.adoc#virt-dedicated-network-live-migration[Configuring a dedicated Multus network for live migration] +* xref:../../virt/vm_networking/virt-dedicated-network-live-migration.adoc#virt-configuring-secondary-network-vm-live-migration_virt-dedicated-network-live-migration[Configuring a dedicated network for live migration] +