MediaMonkey Feature/Bug Process

[rusty]

This summarizes how the MediaMonkey team uses the Forum and Mantis to help plan which features/bugfixes get done, and track whether what's been implemented is satisfactory or not.

Community Feedback:
Most community input comes via the Forums (Wishlist, Need Help, Bug Reports, and Beta Forum) and eSupport (our ticketing system). Forum moderators add forum feedback to Mantis, while MediaMonkey support personnel add relevant support issues to Mantis. When issues are added into Mantis, they include bidirectional links between the source and Mantis, along with status information.

For example:
1) Forum members raise an issue in the [Needs Help] or [Beta] Forums.
2) Other members/Moderators help resolve the issue, if possible.

• -If Moderators confirm that it's a bug, they add BUG#xxxx to the title, and enter it in Mantis, along with bi-directional links between the forum report and the bug.
  -If the issue is reported bug but is determined not to be one, then NOTREP is added to the report title in the forum.
  -In addition, the following icons are used to denote bug status
  • Confirmed bug:
    Resolved bug or not an issue:
    Help:
    Information:

Note: Having 2 input systems is a bit onerous, and we are looking for ways to combine them (e.g. using getsatisfaction.com). In addition, we're contemplating the use of ideascale.com (or similar service), to assist in the prioritization of wishlist items.


Mantis - Product Version/Build and Severity:
When items are entered into Mantis, they are assigned a Severity (Feature, Trivial, Text, Tweak, Minor, Major, Crash, Block), to indicate whether the issue is a new Feature, or a bug--along with the perceived severity of a bug to users). Bugs also have attached Version/Build# information indicating what version of the software is affected.


Mantis - Target Version and Priority:
At the beginning of any release cycle, Product Managers and Developers create a bucket of Features and Bugs and assign it a particular Target version. Priority is determined based on a combination of severity and the effort/risk associated with a particular feature/bug, and may be re-evaluated as needs/priorities/resources change over the course of a release:

• None: the issue has not yet been triaged. i.e. its priority for the release has not yet been evaluated.
  Immediate: the issue should be resolved for the next build
  Urgent: must-have for the release
  High: nice to have for the release
  Normal: nice to have, but unlikely due to risk
  Low: no longer planned for this release

Note: Up until MediaMonkey 3.1, Target version was used only intermittently, and only the Priority field was used to track whether an issue would make it into the 'current' release. At a certain point in the release cycle we decided that we would limit bugs that we'd tackle for 3.1 to those Prioritized as 'Immediate'. In other words, we're not exactly adhering to the process at the moment.


Mantis Workflow:
The Reporter enters an issue into Mantis. The issue Status is set to 'New' and Priority to 'None'.

The Product Manager/Dev Manager are notified of Untriaged issues (those with no Priority), and either:

• -Set Priority and Assign the bug to a Developer
  -Set Status to 'Resolved' as 'Duplicate' or 'Won't fix'

The developer reviews 'Assigned' bugs and either:

• -Sets status to 'Resolved' as 'Fixed' or 'Cannot Reproduce' as the case may be. Fixed bugs should have both a Fixed in Version value and a Build#.
  -Sets status to 'Feedback' to request clarification from the Reporter or the Manager
  • -Reporters, Dev/Product Managers comment on issues with Status='Feedback', and set status to 'Feedback' or 'Resolved' or 'Assigned', as needed.
    -Developers comment on issues with Status='Feedback' and set status to 'Feedback' or 'Resolved', as needed.

Testers verify all issues that have been set to Status='Resolved', and either:

• -Change Status to 'Closed' with comment 'Verified in build xxxx', if the implementation meets users needs.
  -Change Status to 'Reopened':'Feedback', if the implementation does not meet users needs.
  In addition:
  -Testers should tag the bug with 'todoc' if additional documentation is required
  -Testers should update the bug description, if necessary since it will be appear directly in the changelog
  -If the issue is resolved but there are associated issues that are somewhat independent of the original bug, then a new bug should be opened.

Notes:
-If an issue is a regression from a previous version, '(regression)' should be appended to the Title.
-For any Status change except to Status='New', an issue should have an Owner. e.g. a bug should not be changed to Status='Feedback' without also setting who the feedback is required from. Note that Developers/Managers will typically overlook issues with Priority set to 'High' or lower, so if feedback is absolutely required in such cases, Priority can be raised with an indication in the comments that 'Priority is being raised from X to Y for discussion purposes only'.
-It's generally a good idea to keep discussions in Mantis, since it lets others see the rationale for decisions that are eventually made and because it makes it clear who 'owns' the issue at any point in time. In cases where an offline discussion is more productive, it's a good idea to summarize the discussion in mantis so that the rationale for a decision is clear.
-When deciding whether to create a new bug or enter information within an existing bug, use common sense and enter bugs in whatever manner will help developers most quickly resolve them. e.g. if two similar behaviors have different underlying causes, two bugs should be opened; but if they have a common underlying cause that will be resolved by a single fix, it's usually preferable to track them within a single bug. On the other hand if two completely distinct behaviors are caused by a common problem, a common parent bug can be used to track the 2 child bugs.
-Regarding the question of whether it's better to re-open a bug or create a new one; Bugs should only be 'Re-opened' if the original issue is not resolved--otherwise it's preferable to open a new issue, and set a Relationship to the previous issue. This ensures that the new issue will be Triaged correctly and that the new issue will be reflected correctly in the changelog.
-Note that the term 'bug' refers to both design and implementation deficiencies. In other words, if a specification yields an implementation that doesn't satisfy user needs, the issue should be 'Reopened' even though the requirements defined in the specification were met.
-The development environment also includes an automated test suite than is used independently of the blackbox testing described above.

[chrisjj]

-If Moderators confirm that it's a bug, they add BUG#xxxx to the title

How about state Peke's suggestion http://www.mediamonkey.com/forum/viewto ... 86#p356286 of a leading space? And adopt his suggestion of [#xxxxx] without "BUG" - this conserves scant space.

[chrisjj]

I'm seeing broken graphics here.