Difference between revisions of "Remus"

From Xen
(Created page with "<!-- MoinMoin name: Remus --> <!-- Comment: --> <!-- WikiMedia name: Remus --> <!-- Page revision: 00000024 --> <!-- Original date: Fri Oct 28 17:09:29 2011 (131982176…")
 
(Xen 4.5)
 
(43 intermediate revisions by 7 users not shown)
Line 1: Line 1:
  +
<!-- The page has been confirmed to be up-to-date for Xen 4.5 -->
<!-- MoinMoin name: Remus -->
 
  +
__NOTOC__
<!-- Comment: -->
 
<!-- WikiMedia name: Remus -->
 
<!-- Page revision: 00000024 -->
 
<!-- Original date: Fri Oct 28 17:09:29 2011 (1319821769000000) -->
 
   
  +
Remus provides transparent high availability to ordinary virtual machines running on Xen. It does this by continually live migrating a copy of a running VM to a backup server, which automatically activates if the primary server fails. Key features:
{{Needs_Refactor|Mixes elements of HowTo, Design Document and Glossary}}
 
   
  +
* The backup VM is an exact copy of the primary VM (disk/memory/network). When failure occurs, the VM continues running on the backup host as if failure had never occurred.
__NOTOC__
 
  +
* The backup is completely up-to-date: even active TCP sessions are maintained without interruption.
= Remus =
 
  +
* Protection is transparent: existing guests can be protected without modifying them in any way.
== Transparent high availability ("Fault Tolerance") for Xen VMs ==
 
Remus provides transparent, comprehensive high availability to ordinary virtual machines running on the Xen virtual machine monitor. It does this by maintaining a completely up-to-date copy of a running VM on a backup server, which automatically activates if the primary server fails. Key features:
 
   
  +
= Host (dom0) requirements =
* The backup VM is an exact copy of the primary VM. When failure happens, it continues running on the backup host as if failure had never occurred.
 
  +
* Xen hypervisor with remus support and tools (included with Xen 4.0+)
* The backup is completely up-to-date. Even active TCP sessions are maintained without interruption.
 
  +
** Note: Remus is not included with XCP, XenServer, or with some of the Linux pre-packaged versions of Xen, so please check your distribution or you may need to build Xen from source
* Protection is transparent. Existing guests can be protected without modifying them in any way.
 
  +
* Xen dom0 kernel that meets the [[Remus dom0 requirements]]
== Remus requirements ==
 
  +
* Shared storage is not required
Remus requires the following components:
 
  +
* DRBD shared storage is supported, allowing faster and automatic re synchronization after a failed host is brought back online
  +
** Otherwise to bring a failed node back online, the VM must be turned off to perform disk re-synchronization
   
  +
= Guest (domU) requirements =
* Xen hypervisor and tools with support for Remus.
 
* Xen dom0 kernel with support for Remus.
+
* Xen PV guests that meet the [[Remus PV domU requirements]]
  +
* Xen HVM guests don't require any changes for Remus
* PV (Paravirtual) guests need to have Remus support in the domU kernel.
 
* Xen HVM guests don't require special changes for Remus. No kernel patches needed in the guest.
 
* For initial testing: Shared storage accessible by both dom0s.
 
* Actual Remus usage doesn't require or support shared storage!
 
* DRBD storage backend is supported, allowing faster and automatic resynchronization after failed host is back online.
 
   
  +
= Installation =
== Remus status ==
 
  +
Installation varies slightly depending upon the host platform, so please see the guides below for examples.
In Xen 4.0.0:
 
   
* Xen hypervisor and tools have Remus support.
+
* [[Install Xen 4.5 with Remus and DRBD on Ubuntu 12.10]]
  +
* [[Install Xen 4.2.1 with Remus and DRBD on Ubuntu 12.10]]
* Only linux-2.6.18-xen is supported as Xen dom0 kernel with Remus.
 
  +
* [[Install Xen 4.1.4 with Remus and DRBD on Ubuntu 12.10]]
* If using a PV domU you need to run linux-2.6.18-xen as domU kernel.
 
In Xen 4.0.1:
 
   
  +
= DRBD support =
* Pvops dom0 kernel support for Remus has been added in Xen 4.0.1-rc4, so it's available in Xen 4.0.1 final release. You can use Linux 2.6.32 based pvops dom0 kernel with Remus.
 
* PV domU kernel still needs to be linux-2.6.18-xen.
 
   
  +
Using DRBD instead of blktap2 for storage replication allows for quick re-synchronization of the disk backend after failed host is back online. Since storage (re)synchronization is done online - while the VM is operational, there is no need to shutdown the VM. Once storage is synchronized, one can start, stop and restart Remus on a running VM anytime.
In Xen 4.2 (xen-unstable development branch):
 
* Many bugfixes to Remus.
 
* Remus support for pvops domU kernels: Linux 2.6.39.2 and later upstream kernel.org versions are now supported as PV domU kernels, in addition to Jeremy's xen.git xen/stable-2.6.32.x branch.
 
* For better Remus performance you should use a domU kernel with "suspend event channel" support, which means linux-2.6.18-xen, or any of the xenlinux forwardports (novell sles11sp1 2.6.32 kernel, for example). pvops domU kernels don't have suspend event channel support yet.
 
* Checkpoint compression for less data to transfer between hosts.
 
   
  +
However, DRBD must be custom built with support for protocol D (see the above install guides), so the normal packaged versions of DRBD are not suitable.
Note that if using linux-2.6.18-xen kernel it needs to be new enough to include Remus support/patches! It's recommended to download the latest version from linux-2.6.18-xen.hg mercurial repository for use with Remus.
 
   
  +
Note that DRBD will be operated in dual primary mode, which carries a number of risks and management issues. Please research this topic to be aware of the potential complications.
== Config options for Linux 2.6.32 pvops dom0 kernel for Remus ==
 
   
  +
* [http://remusha.wikidot.com/#toc4 Install DRBD 8.3.11 (remus version) on Debian Squeeze/Ubuntu 10.04]
Here are the .config options you need to enable for Remus, when using jeremy's xen.git xen/stable-2.6.32.x branch as dom0 kernel.
 
  +
* [[Install Xen_4.2.1 with Remus and DRBD on Ubuntu 12.10#Install_DRBD| Install DRBD 8.3.11 (remus version) on Ubuntu 12.10]]
   
  +
= Limitations =
  +
* For PV domUs, Remus requires "suspend event channel" kernel support. This support has not yet been upstreamed to the Linux kernel [[XenParavirtOps#Upstream_delta_details|at the time of the Xen 4.5 release]]. Otherwise Remus can run most any PV domU in a degraded performance mode. This kernel support is not widely available (not currently available in Ubuntu, for example), but is available with OpenSUSE. See [[Remus PV domU requirements]] for more information.
   
  +
= Release History =
<pre><nowiki>
 
CONFIG_IFB=m
 
CONFIG_IP_NF_IPTABLES=m
 
CONFIG_IP_NF_FILTER=m
 
CONFIG_NET_SCHED=y
 
CONFIG_NET_SCH_PRIO=m
 
CONFIG_NET_SCH_INGRESS=m
 
CONFIG_NET_SCH_PLUG=m
 
CONFIG_NET_CLS=y
 
CONFIG_NET_CLS_BASIC=m
 
CONFIG_NET_CLS_TCINDEX=m
 
CONFIG_NET_CLS_U32=m
 
CONFIG_NET_CLS_ACT=y
 
CONFIG_NET_ACT_MIRRED=m
 
</nowiki></pre>
 
   
  +
== Xen 4.5 ==
  +
* Significant improvements to Remus in Xen have been contributed, such as better [[XL]] integration, better DRDB support, stability and performance improvements and enablement of COLO support
  +
* See [[Remus_Toolkit_Differences#Xen_4.5_and_newer|Remus command line syntax in Xen 4.5]]
  +
* COLO support [although COLO itself is not currently expected to be fully available until 4.7] (see [http://www.xenproject.org/help/presentations-and-videos/video/xpds13v-colo-update.html COLO XPDS13 video], [http://www.xenproject.org/help/presentations-and-videos/video/xpds13-colo.html COLO XPDS13 presentation])
   
  +
== Xen 4.4 ==
== Using Linux kernel v3.x as dom0 kernel for Remus Xen hosts ==
 
  +
* Experimental support for xenlight (xl). Per the man page, there is no support for network or disk buffering at the moment.
   
  +
== Xen 4.2 ==
Upstream Linux v3.x kernel contains Xen pvops dom0 support, but it does not contain "sch_plug" driver which is required for Remus. It's possible to manually add that driver to your custom upstream Linux v3.x kernel build. "sch_plug" driver is available for example from: http://pasik.reaktio.net/xen/remus/linux3x/ .
 
  +
* Many bugfixes to Remus.
  +
* Remus support for pvops domU kernels: Linux 2.6.39.2 and later upstream kernel.org versions are now supported as PV domU kernels, in addition to Jeremy's xen.git xen/stable-2.6.32.x branch.
  +
* For better Remus performance you should use a domU kernel with "suspend event channel" support, which means linux-2.6.18-xen, or any of the xenlinux forwardports (novell sles11sp1 2.6.32 kernel, for example). pvops domU kernels don't have suspend event channel support yet.
  +
* Checkpoint compression for less data to transfer between hosts.
   
  +
== Xen 4.0.1 ==
== DRBD storage backend support for Remus ==
 
  +
* Pvops dom0 kernel support for Remus has been added in Xen 4.0.1-rc4, so it's available in Xen 4.0.1 final release. You can use Linux 2.6.32 based pvops dom0 kernel with Remus.
  +
* PV domU kernel still needs to be linux-2.6.18-xen.
   
  +
== Xen 4.0.0 ==
A Remus version of DRBD can be found from git://aramis.nss.cs.ubc.ca/drbd-8.3-remus . Using DRBD instead of blktap2 for storage replication allows for quick resynchronization of the disk backend after failed host is back online. Since storage (re)synchronization is done online - while the VM is operational, there is no need to shutdown the VM. Once storage is synchronized, one can start, stop and restart Remus on a running VM anytime.
 
  +
* Xen hypervisor and tools have Remus support.
  +
* Only linux-2.6.18-xen is supported as Xen dom0 kernel with Remus.
  +
* If using a PV domU you need to run linux-2.6.18-xen as domU kernel.
   
== Links ==
+
= Links =
  +
* The "xl remus" command has a similar, but not identical, syntax to the traditional Xend based "remus" command. See [[Remus_Toolkit_Differences]]. Also, "xl remus" still has limitations in the 4.4 release, per the man page.
* The Remus project web site is http://nss.cs.ubc.ca/remus/
 
  +
* Research papers about Remus:
* Remus documentation: http://nss.cs.ubc.ca/remus/doc.html
 
* Research/design paper about Remus: http://nss.cs.ubc.ca/remus/papers/remus-nsdi08.pdf .
+
** [http://http://www.cs.ubc.ca/~andy/papers/remus-nsdi-final.pdf Remus - NSDI 2008] Design and fundamental concepts.
  +
** [http://http://www.cs.ubc.ca/~andy/papers/secondsite-vee-final.pdf SecondSite - VEE 2012] Remus in wide-area, with checkpoint compression, replication and storage re-synchronization using DRBD.
* Configuring and installing Remus tutorial: http://remusha.wikidot.com/
 
  +
* Configuring and installing Remus (Xend version) with DRBD, Xen 4.1.2 under Debian Squeeze/Ubuntu 10.04: http://remusha.wikidot.com/
* Latest linux-2.6.18-xen kernel is available from Mercurial tree at http://xenbits.xen.org/linux-2.6.18-xen.hg .
 
  +
* Video demonstration of Remus: http://joburg.eu/en/video/2jV4lOgFJMY/Screen-cast-Remus-High-Availability-based-on-Xen-Hypervisor
  +
* Source code for Remus version of DRBD 8.3.11 http://remusha.wikidot.com/local--files/configuring-and-installing-remus/drbd-8.3.11-remus.tar.gz
   
 
[[Category:Xen]]
 
[[Category:Xen]]
  +
[[Category:Xen 4.5]]
 
[[Category:Users]]
 
[[Category:Users]]
 
[[Category:Developers]]
 
[[Category:Developers]]
Line 88: Line 82:
 
[[Category:HowTo]]
 
[[Category:HowTo]]
 
[[Category:Glossary]]
 
[[Category:Glossary]]
  +
[[Category:High Availability]]

Latest revision as of 14:01, 24 August 2015


Remus provides transparent high availability to ordinary virtual machines running on Xen. It does this by continually live migrating a copy of a running VM to a backup server, which automatically activates if the primary server fails. Key features:

  • The backup VM is an exact copy of the primary VM (disk/memory/network). When failure occurs, the VM continues running on the backup host as if failure had never occurred.
  • The backup is completely up-to-date: even active TCP sessions are maintained without interruption.
  • Protection is transparent: existing guests can be protected without modifying them in any way.

Host (dom0) requirements

  • Xen hypervisor with remus support and tools (included with Xen 4.0+)
    • Note: Remus is not included with XCP, XenServer, or with some of the Linux pre-packaged versions of Xen, so please check your distribution or you may need to build Xen from source
  • Xen dom0 kernel that meets the Remus dom0 requirements
  • Shared storage is not required
  • DRBD shared storage is supported, allowing faster and automatic re synchronization after a failed host is brought back online
    • Otherwise to bring a failed node back online, the VM must be turned off to perform disk re-synchronization

Guest (domU) requirements

Installation

Installation varies slightly depending upon the host platform, so please see the guides below for examples.

DRBD support

Using DRBD instead of blktap2 for storage replication allows for quick re-synchronization of the disk backend after failed host is back online. Since storage (re)synchronization is done online - while the VM is operational, there is no need to shutdown the VM. Once storage is synchronized, one can start, stop and restart Remus on a running VM anytime.

However, DRBD must be custom built with support for protocol D (see the above install guides), so the normal packaged versions of DRBD are not suitable.

Note that DRBD will be operated in dual primary mode, which carries a number of risks and management issues. Please research this topic to be aware of the potential complications.

Limitations

  • For PV domUs, Remus requires "suspend event channel" kernel support. This support has not yet been upstreamed to the Linux kernel at the time of the Xen 4.5 release. Otherwise Remus can run most any PV domU in a degraded performance mode. This kernel support is not widely available (not currently available in Ubuntu, for example), but is available with OpenSUSE. See Remus PV domU requirements for more information.

Release History

Xen 4.5

Xen 4.4

  • Experimental support for xenlight (xl). Per the man page, there is no support for network or disk buffering at the moment.

Xen 4.2

  • Many bugfixes to Remus.
  • Remus support for pvops domU kernels: Linux 2.6.39.2 and later upstream kernel.org versions are now supported as PV domU kernels, in addition to Jeremy's xen.git xen/stable-2.6.32.x branch.
  • For better Remus performance you should use a domU kernel with "suspend event channel" support, which means linux-2.6.18-xen, or any of the xenlinux forwardports (novell sles11sp1 2.6.32 kernel, for example). pvops domU kernels don't have suspend event channel support yet.
  • Checkpoint compression for less data to transfer between hosts.

Xen 4.0.1

  • Pvops dom0 kernel support for Remus has been added in Xen 4.0.1-rc4, so it's available in Xen 4.0.1 final release. You can use Linux 2.6.32 based pvops dom0 kernel with Remus.
  • PV domU kernel still needs to be linux-2.6.18-xen.

Xen 4.0.0

  • Xen hypervisor and tools have Remus support.
  • Only linux-2.6.18-xen is supported as Xen dom0 kernel with Remus.
  • If using a PV domU you need to run linux-2.6.18-xen as domU kernel.

Links