Credit2 Scheduler

This page's purpose is to ease collaboration and coordination when doing Credit2 development. It is also meant as a catch-all for everything relating to Credit2 itself.

Introduction

Design Goals

Credit2 is a general purpose scheduler for Xen, designed with the following goal in mind:

• fairness: fairness is the ability of the scheduler to give each VM its fair share of physical CPU capacity. Such fair share should depend on the VM's scheduling parameters, such as weight, cap, and reservation.
• responsiveness: responsiveness should be intended here as the capability of the scheduler to work well with latency-sensitive workloads.

These design goals have been derived from the more general objective to create a scheduler which can work well in several use cases, with particular attention to:

• server consolidation
• virtual desktop server solutions
• client virtualization

Using Credit2

For using Credit2 as the main scheduler, Xen boot command line should be edited, adding sched=credit2.

Another way of using and/or testing Credit2 is by means of cpupools.

When Credit2 is in use (either as main scheduler or in a cpupool), use xl sched-credit2 to interact with it. For example:

root@tg03:~# xl sched-credit2
Cpupool Pool-node1:
Name                                ID Weight
debian.guest.osstest                 1    256

And:

root@tg03:~# xl sched-credit2 -d 1 -w 128
root@tg03:~# xl sched-credit2 -d 1
Name                                ID Weight
debian.guest.osstest                 1    128

Contributing

Thoughts, suggestions, questions and patches via email to xen-devel. See the list info page for subscription information and the archives.

If asking questions, please refer to Asking Developer Questions. If sending patches, please see Submitting Xen Patches.

For non-developers, even just testing your favorite workload, and observing and reporting any any regressions, improvements, and so on, would be much appreciated. Remember that a scheduler's job is only hard when there's more work to do than there is CPU to do it, so make sure you're focusing on multiple competing workloads.

Readiness Criteria

This is an open issue: when shall Credit2 (but also a scheduler in general) be considered ready for production use, when ready for becoming Xen default scheduler? What we need is a standardized set of regression tests.

Of course, that will greatly help in future development too. In fact, if anyone proposes a change, we can test the change for any regressions in other workloads.

Status

Credit2 is still marked as experimental software. Development is ongoing in order to make it suitable for production usage. Broadly speaking this is the current situation:

• stable and reliable; it is part of Xen upstream and of the OSSTest automated test suite since ages
• benchmarking showed improved fairness, with respect to Credit1, especially in latency sensitive workloads
• lacking extensive testing with many and diverse combinations of workloads
• lacking some optimization; both the crediting algorithm and the load balancing logic are, computationally, rather expensive (more than Credit1)

Missing Features

• CPU Topology:
  • Hyperthreading: avoid sharing threads if possible, and to "discount" sharing of threads. Status: Uma <uma_DOT_sharma523_AT_gmail_DOT_com> is working on this, during her participation to the OPW program. Time horizon: end of OPW9, in March 2015
  • Hard and Soft scheduling affinity: support for vcpu pinning and soft scheduling affinity (aka NUMA aware scheduling is still missing. Status: Justin jtweaver_AT_hawaii_DOT_edu is working on it. Patches sent: old version, v1. Time horizon: April 2015
• Credit Algorithm:
  • Cap and Reservation: both features are available in Credit1, and they should be implemented in Credit2 too. Status: no one working on this. Dario <dario_DOT_faggioli_AT_citrix_DOT_com> will look into this at some point... In case someone is interested, speak up! Time horizon: undefined.
  • Credit math optimization: as of now, for the sake of correctness, every credit update involves a division, which may be slow. Some profiling and (most likely) optimization are required. It may be best to wait on this until the cap and reservation system is working, because the algorithm may change somewhat. Status: no one working on this. Time horizon: undefined

Pending Bugs

• credit2 only uses one runqueue instead of one runq per socket. Status: Dario <dario_DOT_faggioli_AT_citrix_DOT_com> is working on it. Time horizon: really really soon.

Open Issues

Here they are some open questions. Open problems that, although not critical, looks worth investigating:

• How to deal well with mixed workload, e.g., situations where someone is listening to iTunes and there's a flash ad in background burning 100% of the CPU? Can we allow the audio to play well while still giving the VM only its fair share?
• There exists a lot of CPU and/or IO throughput benchmarks, automatically outputting tons of numbers for one to look at. But as far as, e.g., audio and video are concerned, what are ways of automatically and programmatically measuring quality?

Milestones

• 2010 24 December: Simple load balancer checked into xen-unstable; credit2 can now handle multiple sockets.
• 2010 December: Credit2 integrated into the OSSTest automated test suite
• 2010 14 April: Credit2 checked into xen-unstable
• Prototype credit2 scheduler designed for latency-sensitive workloads and very large systems included in Xen 4.1

References

• "In Perfect Xen, a Performance Study of the Emerging Xen Scheduler" (Dec 2013) text
• XenSummit Asia 2009: a brief summary of the design target, goals, interface, credit design, and load balancing design. (text, slides)