Code Review Dashboard

The Xen project has developed a code review Dashboard in collaboration with Bitergia. There are two variants of this dashboard

• kibana.bitergia.com/xen/app/kibana: User version
• kibana.bitergia.com/xen-edit/app/kibana: Administrator version (allows to create extra customizations]

This document provides a version of the documentation for the*user version* of the dashboard, that is editable and easily consumable

Panels

The Xen Project Code Review dashboard has 4 panels: One for documentation, a Community Panel, a Performance Panel and a Backlog Panel.

Community Panel

In this panel, several widgets are grouped by use-case.

Use case: Identify top reviewers (for both individuals and companies)

Evolution 'Reviewed-by' flag (no patchseries, no patches)

Shows the evolution of the Reviewed-by flag over time. Allows to create a time filter.

Top People and Domain Reviewing Patches

Counts the number of people and organisations who commented on other people’s patches (by counting the Reviewed-by attributes) in the selected time period. Allows to filter by developer (e-mail address and e-mail domain).

Use case: Identify Imbalances between reviewers and contributors

Evolution Patches (no patchseries, no comments)

Shows the evolution of the patches posted over time. Allows to create a time filter.

Top People and Domain Imbalance Analysis

Over the selected time period, the following fields are aggregated per person or domain * Balance: Difference between patches posted and and comment activity. A positive number implies that the person or organisation comments more than she posts patches. * Patches: Number of patches posted * Reviews: Number of Reviewed By flags

Use case: Identify post-ack comments on patches

Evolution Comments Activity (no patchseries, no patches)

Shows the evolution of comments made on patches (both real and self-comments) over time. Allows to create a time filter.

Top People and Domain Commenting on Patches

Over the selected time period, the following fields are aggregated per person or domain * Comments: The number of review comments on other people’s patches. * Post ACK comments: The number of comments made after a patch has been ACK’ed

Use case: Drill into Review activity information

The following widgets are of particular interest to code submitters, reviewers and maintainers. A similar view, which is optimised to identifying which code reviews are almost complete are visible in the Backlog and Data-Explorer panels.

Evolution Email Activity (patchseries, patches, comments)

Shows the evolution of review related e-mail activity over time. Allows to create a time filter.

Evolution Patch Series (no patches, no comments)

Shows the evolution of patch series posted over time. Allows to create a time filter.

Patch Series

Lists the patch series in an exportable table that satisfy selected filters.

Reviews (using Reviewed-by flag)

Lists emails that set the Reviewed-by flag that satisfy selected filters.

Comments

Lists emails comments that satisfy selected filters.

Performance Panel

The data presented in this panel allows us investigate merge time, complexity of patches, number of review comments and other related metrics which indicate complexity and correlate the different properties.

Important note: this panel only shows patch series that were merged, as many of the metrics in this panel only apply to merged series.

Each section below, gives a brief overview of the widgets that are present in this panel.

PatchSeries

Counts the number of patch series

PatchSeries per Year

The pie chart represents the total number of patch series that were merged per year. When clicking on some of the portions of the pie, this will create a time based filter for that specific year.

Evolution NumPatches per PatchSeries

Total number of patches for selected patch series over time, arranged in percentiles. An indirect measure for complexity of patch series submitted over time.

Several charts (including this one), show data which are divided into 3 percentiles. The datasets are ordered by a specific metric, (in this case the total number of patches merged) and are grouped up to the 25%, the 50%, the 75% percentile and in some cases the 95% percentile.

This approach is easiest to explain by example: if your 3-month-old baby is in the 25th percentile for weight, that means 25 percent of 3-month-old babies weigh the same as or less than your baby, and 75 percent weigh more. The higher the percentile number, the bigger your baby is compared to other babies the same age.

Ability to create a time based filter.

Time to merge per number of patches per patch series

A histogram that shows the time to merge on the y axis and the number of patches per series on the x-axis. The histogram is sorted in order of descending population (like a Pareto chart), with values that occur most often on the left and values that occur least often on the right.

Creates a filter for the num_patches = <selected column on y axis>

Evolution NumVersions per PatchSeries

Total number of re-posted versions for selected patch series over time, arranged in percentiles. An indirect measure for the degree of actionable feedback and possible quality/complexity issues on the patch series submitted over time.

Ability to create a time based filter.

Time to merge per versions of patches per patch series

A histogram that shows the time to merge on the y axis and the number of reposts per series on the x-axis. The histogram is sorted in order of descending population (like a Pareto chart), with values that occur most often on the left and values that occur least often on the right.

Creates a filter for the num_versions = <selected column on y axis>

Evolution NumVersions per PatchSeries

Total number of re-posted versions for selected patch series over time, arranged in percentiles. An indirect measure for the degree of actionable feedback and poss

Backlog Panel

Evolution Merged(1) and Non Merged (0) PatchSeries

Shows the evolution over time of merged and un-merged patch series. Note that we cannot currently identify patch series to other projects (e.g. Linux, QEMU, …) that are cross-posted on xen-devel@ as merged, even if they have in fact been merged.

Top People and Domains that got merged

Lists the number of people and organisations that were merged.

Patch Series Merged and Not Merged (Backlog)

Lists the patch series that were merged and not merged

Use case: Identify recently active patch series

Use one of the following pre-defined time filters to identify review activity in a certain time period (as defined below):

Hot      = [now to 1st week}
Warm     = [1st week to 1st month}
Tepid    = [1st month to 2nd month}
Cold     = [2nd month to 5th month} 
Freezing = [5th month to 1 year}
Dead     = [1 year to *} 

Check the Patch Not Merged (Backlog) widget to identify series that were not merged.

Use case: Identify abandoned patch series (those older than 1 year)

Use the Dead time filter and check the Patch Not Merged (Backlog) widget to identify series that were not merged.

Use case: Identify patch series in progress

Check the Patch Not Merged (Backlog) widget or merged:0 filter to identify series that were not merged.

Use case: See whether the backlog is increasing

Use the Evolution Merged(1) and Non Merged (0) PatchSeries widget together with the merged:0 and merged:1 filter to identify trends

Use case: Focus attention on nearly completed reviews

Use the following filters to identify patches that are nearly complete. Note that these filters can also be used in the Data-Explorer Panel: * patchserie_percentage_ackedpatches (number): contains the % of acked patches in a specific patch series. Numbers may range from 0 to 100. Use the [90 TO 100} notation for filters. * patchserie_numackedpatches (number): contains the number of acked patches for a particular patch series * patchserie_numpatches (number): contains the number of patches that make up a particular patch series

Data-Explorer Panel

This view is tailored towards code submitters, reviewers, maintainers, committers and release managers. Documentation is available in the panel itself. It can also be used as a more detailed view into review related messages using the techniques outlined in the Backlog panel.

Index-Review and Index-Time Panel

This view provides raw access to the raw xen-patchseries-reviewers and xen-patchseries-timefocused indexes and can be used to explore the raw data behind other panels.