Yes, it would be a useful plugin if someone is managing test cases.

Eric

Hi Jens,

I think you have a wonderfull idea. Unfortunately I can't help with any coding, since I don't know anything about ruby (except gem install or rake script.rb ;-), but as a QA guy, I can really help you write down some sort of specifications, usecases, desired flows, etc. (and of course w/ testing).
I did take a short look at your attachment: most probably will post back tomorrow with my thoughts/observations.

best,
marius

(well, only today I found some time for looking at your mindmap, sorry for the delay)
Hi Jens,

All the considerations below come from the design ideas and workflow in other testcase management tools such as Testopia or TestLink.
- there should be a 'common pool' of testcases to be (re)used in any project;
- there should be a way to copy but also reference testcases from common pool; (referencing a testcase is a powerful feature: when someone reviews a common testcase, all it's references are updated);
- testcases can be grouped in named suites (for instance when you test a form in a web page, you'll want to execute as a group the testcases for username, password, email)
- when copying/referencing a testcase, user should be able to change the summary, in order to reflect the current project/subproject, flow/window under test (say you have a generic suite that tests a numeric entry. While copying it to the current suite/plan you may change the name);
- the hierarchy may be: testplan > test suites -> testcases
a testplan is tied to: a sub/project, or an issue (a task, but also a bug)
- a testplan is also tied to a given version of a project. Creating a new testplan for a new version should be as easy as possible (like import the plan for an earlier version then edit);
- use meaningful names: project name (or it's identifier), suite name (maybe define an identifier also for this level). In Manage Testplan page, all the identifiers should be stripped so in the associated table, there's enough room to display the entire summary (this summary should be the internal storage name for the testcases). Obviously, when printing testplans or test reports, the name of the testcases should be prepended with the project identifier and the test suite identifier
- the execution order of the testcases should be easily modifiable;
- testruns are execution instances for testplans;
- testcases may have keywords associated, and users may search testcases by keywords;
- a test activity (most probably a Task associated to the test team), should be present at project level; this way you ensure that a project is not marked as finished, and not delivered to customer unless tests are done (should this be a preference while creating a project version?);
- testcases may depend on other testcases / suites;
- testcases may block other testcases / suites;
- (for simplicity) testcase importance is reflected by the order in which they are present in the testplan. Users should be able to easily modify the order (drag & drop?);
- the result of a testcase should be Pass / Fail. What about having also PassWithProblems? (for example if the test passes but it takes way too much time to receive the result, or the computer memory/CPU are exhausted during run);
- consider also an import feature for testcase/testsuite/testplan;
- as with other issues, for each testcase consider having time estimates, spent time, % done. These should be summed at testsuite or testplan level and presented in reports. (what about due date? should it be inherited from project or task?);
- (regarding your observation for unit tests and dependencies) a testcase should have a setup prerequisite? What about a tear-down activity?
- anyways, a testsuite should contain a prerequisites section. This should also be true for a testplan.

I am truly sorry that the above ideas are written in a totally unordered way, but hopefully they help.

best regards,
marius

Very useful idea!

When I see such a big "requirements" list, I get scared.

Maybe a simpler approach would be better.

Maybe could be created a button to create a test case for some issue, that wold test an issue, e when created it could be updated just as the time entry.

The information for a test case are some steps and expected results, for each step a Pass/Fail checkbox.

After the creation of a test case, associated to an specific issue, many executions cold be done, and they wold be registered for later view, like the time report.

If you start a plugin with this simplistic view, you could get something done in a short time, and then, many people will test and give you feedback where to go.

my 2 cents

Hi Enderson,

I do agree w/ you about implementing something in short time and get some feedback. On the other hand, one should have in mind the 'entire picture'. If you implement something minimalistic, and receive very good feedback, but do not pay attention to the final target, you may end up by having to re-write everything just because you did not think about the architectural structure.

adding 2 cents more,

Hi Jens

I think it makes total sense to integrate it into Redmine. I agree that as a start point it is best to keep it simple but the bigger picture will need to be kept in mind for future development.

Am happy to help with any testing you need

Just +1 on this idea.

Regarding the scope of the project which could lead to no success eventualy - could there be a way to "just" make hooks and integrate with 3rd party QA / testing suite? Say Bromine ?

Hi Ivan,

the aim was to add testcase management functionality in redmine. "Just making hooks" would need to add such in different places like: an entire project (maybe a testplan for each version), a subproject, a task, a bug or a feature. Note that I did not say "issue", although in redmine terminology task / bug / feature are all issues.

Hi Jens,

is this plugin still in development?

Ole

Hello Ole,

I have not the time to work on it. The only work I have done is the proposal from the first post, sorry.
Jens

Hi-

I'm about to undertake the process of implementing a test management tool. Is there still interest or another redmine plugin for test case management that someone knows about?

I'm willing to take the reins, though I have slightly different requirements and opinions and am open to suggestions and help.

My requirements are:

Tests have a tracker type of Test
Tests have a title (subject) and description.
Tests can have comments.
Tests can have additional attributes
Tests can be versioned - ideally tied to the source code for the system under test.
Tests can be grouped into suites (attribute) -- possible tagging
Tests can be tied to a feature or bug (attribute) -- or support (issue)
A test execution can be tied to a milestone/iteration (version)
A test execution can include a specific revision of a test
A test can be associated with an 'implementation' e.g., automation

It would be nice to plug in reporting and automation. I'm picturing CI integration and a Fit-style "go" button.

-Aaron Evans
aarone_at_one-shore.com

I think I'm going to follow Enderson's suggestions.

I'll keep it simple for now, with the option of integrating external tools later.

TestCases with a title and description
TestResults with a simple status (P/F) and a text note; associated to a TestCase and a TestRun
TestRuns are integer counts associated to a Project & Version

I also like the idea of being able to associate tests to an issue. This could be used to create a new test case or associate existing tests cases.

hi guys. Yes. i'm also quite interested in such a plugin. I'm a developer myself, but I dont know ruby/rails. Despite this i'm using redmine because, well, it's very well done :-). But it's hard not to be able to tweak my redmine or participate in such plugins... Anyway i'm ready to help testing whatever would come.

If that may help, here are what would be my requirements, or "ideal specifications". I think i'm really focused on triaging results rather than specifying tests. That's because most of my tests would be things like "subdir x Compile", "Unit tests for subdir x pass",... That is, tests already exist elsewhere and are well described and self contained.

Data objects¶

Tests¶

For each project, there is a list of tests, quite simple (subject, description). Ideally we could copy (part of) this list to other projects. An improvement would be to be able to group tests togethers.

Environement¶

An 'environement' could be a platform, a compiler, a couple of those.. ("tested with gcc-4.3.3 on freebsd"). Just a plain title would be enough for this.

Test run¶

• date
• a version/tag (either a version in the redmine meaning or something else)
• who did it
• a list of Test results, as described in the next section.

Test result¶

• result (pass,fail)
• note ("almost pass, but...")

Requests/views¶

Now, the real interesting thing is what to do with all of this.

Create testrun¶

First, we need to create a test result. The page would offer all possible tests, date and 'who did it' would be set automatically, the user would just enter the environment and the version/tag tested.
For all possible tests, there would be a dropbox (passes/fails/untested, default to 'untested' of course), and a (not mandatory) lineedit to enter the comment.
This testrun could be modifiable

Display of data¶

The most obvious ones are

• For a test, show all results, sorted by date(default) and a summary (#pass, #fail, #tested. Ideally some graphics showing results by environments. Possibility to only show success or failure. This is, imho, the most important one.
• List of all testruns, with a summary (date, who, x pass, x fail, x tested), ideally with graphics.
• For a testrun, display all results, sorted by result

Improvements¶

For the environments, a title is enough, but in an ideal world, we could have any number of criteria ("os"/version, "compiler"/version, "dependency"/version), and an environment would be a choice for all those criteria. So that we could perform requests like "when was the last success for this test using "gcc" whatever version on "linux" whatever version. But this is quite more complicated to specify and implement, and i'm afraid this is not general enough.

I'm sure some people would be interested in providing relationships between tests, like "depends on".

Thomas-

It looks like we have fairly similar ideas.

I'm thinking of this as more of a functional test framework, but there shouldn't be too much conflict with your needs.

From my perspective, an Environment should be an attribute of a Test Run. There should probably exist an abstraction to duplicate all tests from a Test Run. Maybe this is just a copy, or maybe it's a Test Suite, or maybe it's a different object -- I've seen some refer to it as a "Test Plan".

I hadn't thought of the historical view of individual tests, but this should be fairly trivial, excepting that I want test implementations to be able to evolve over time -- to track the requirement, although I realize requirements change over time as well, as does the accuracy and relevance of the test.

I'm working slowly on developing this in my spare time. I'm also relatively new to Rails and Redmine development which slows me down further.

This past week I've got a very basic Redmine plugin where you can enter a test name and description. Over the next couple weeks you should be able to create Test Runs and manually pass/fail them and optionally add

Any help is welcome, though there's no infrasturcture for collaboration yet. I have a personal SVN repository, and can take some time to set up on rubyforge, github, sourceforge, or somewhere if there's a desire. Otherwise, I'll just publish what I've got here occasionally, and on my blog at http://fijiaaron.wordpress.com

I'd expect a very basic (but usable) version 0.1 to be ready within a month.

Right now you can see what I'm working on (uptime not guaranteed) at http://redmine-tcm.one-shore.com/test_cases and I'll work on getting a repository up that you can look at. Otherwise, email me at aarone at one-shore.com

We are developing similar plugin
see http:\\www.qaproject.net - "Redcase"

Now it's available here for downloading:
http://sourceforge.net/projects/redcase/files/

But there is only alpha version and there are many problems there (in code and in actions), so we don't recommend using this one in the real work for a while.

If you enable git on your sourceforge project, I might be able to give a hand.

Hi,

I am currently trying out the Redcase plugin. I must say it is a pretty impressive job well done.
However, I am getting difficulties in using it.

I have created test cases under ‘Issues’ and I can see the new test cases appearing under the “unsorted” ‘Test Suites’ section.

However, I am unable to add the test cases into the ‘Execution Suite’ section by the Drag & Drop method.

Am I doing something wrong? Or am I missing any step(s)?

Can anyone please advise.

Thanks.

Kevin

Each test case has specific life cycle New - > InProgress -> Obsolete .
Only test case with state "InProgress" can be executed.

2Kevin

When your test cases (issues) are in any state apart from 'InProgress' you can't execute them or move to execution tree.

To do that you need to change test case issue state to 'InProgress' and then you should see it in test case tree with green icon (instead of red) - it will mean you can now move test case to execution tree and then execute it.

Thank you Evgeniy and Maxim for explaining. I have understood and shall give it a try.

Thanks again. :)

Cheers,
Kevin

Hey Maxim and Evgeniy,

I would like to check out redcase, but I am having some trouble installing. I downloaded the tarball, copied it into my vendors/plugins folder, ran

rake db:migrate:plugins RAILS_ENV=production

Cheers
Ole

Hey,

sorry, that was a simple mistake on my part: I had wrong permissions on the plugin folder.
After that it showed up in the plugins list in the Administration panel. I then changed the permissions in redmine as desired and added the checkmark to assign to the tests tracker, after which finally the tab showed up.
This is where I am currently stuck: The tab is there but it is empty. What do I have to do to have the system recognize my unit tests? Or does it only work with a certain framework? I am using CxxTest for C++.

Cheers
Ole

You can see video tutorial http://www.youtube.com/watch?v=VmI93EreB_w
Now this plugin supports only manual tests.