6.3.3 Rankings

An admissible interpretation is therefore to sort the model elements by a metric, and review the design of the model elements with the highest values: Are those high values justified, should the model element be considered critical?

There is no definitive answer to the question how many elements to choose for review from the top of the sorted list. One strategy could be to select so-called "outliers". Another strategy could be to select a certain number or percentage of model elements from the top, based on available resources for reviewing.

SDMetrics supports this kind of interpretation of measurement data. The histograms in the metric view allow you to visually identify outliers (Section 4.5 "The View 'Histograms'"). In the table view, you can sort the elements by a metric, and highlight elements in the upper percentiles for a metric (Section 4.4 "The View 'Metric Data Tables'").

What about thresholds?

One recurring suggested use of design metrics is that they can be used to build simple quality benchmarks based on thresholds. If a design metric exceeds a certain threshold, the design element is either rejected and must be redesigned, or at least flagged as "critical". It is difficult to imagine why a threshold effect would exist between, for example, size metrics and fault-proneness. This would imply a sudden, steep increase in fault-proneness in a certain size value range, something that would be difficult to explain. Also, empirical data does not support this idea [BEGR00].