Test Management tools compatible to MS Visual Studio Team System 2008

Replies to This Discussion

Permalink Reply by Rob Lambert on April 27, 2009 at 8:18pm

It's not entirely clear what you are wanting to do.

When you say 'cope' with Team System 2008 what do you mean?

I have experience of VS 2008 and may be able to help you but I probably need you to clarify what you are wanting to do.

Do you use TFS? Which bits of VS 2008 do you use? Do you publish results to TFS? Do you use work items? Are you using the manual test case tool?

There are so many dimensions to VS 2008 that to be perfectly honest - Google is your best friend.

I have some blogs on TFS, Visual Studio and Testing here:
http://blogs.imeta.co.uk/RLambert/Default.aspx

There are also some more at the iMeta home page as well as some MS and TFS knowledge.
http://blogs.imeta.co.uk/

They are good places to start but really I'd suggest you pin down what you want to do with Team Suite, then redefine the question on here. Or alternatively, check out Microsofts own help here:
http://msdn.microsoft.com/en-us/teamsystem/default.aspx

At the moment it appears MS website is down though....

Hopefully I'll be able to help you further.

Rob..

▶ Reply to This

Permalink Reply by Jog on April 28, 2009 at 1:14pm

Hi Rob,

Many thanks for your reply and the links on this topic.

Actually, we use TFS to store the requirements, i.e Work Items. We have extended the basic functionality by adding risks/data requirements/approval process....that's entirely managing, documenting requirements.
We use .Net framework (C#), C++ (little) with SQL DB server.2005.
We don't use any specific tools for testing. We use the basic build provided by MS VS 2008, however we have amended the build process with additional bits to check DB upgrades, data integrity checks, DB installs, Schema comparision...most of them are done thru SQL scripts generated automatically during the build process based on the changes made to source since the execution of the last build .
As of now we don't use TFS to manage our testing.
We test things manually..
1). I'm in the process of identifying any open source tools to automate out testing (mainly functional testing), because we need to test the functionality for every build that happens...which will make the life difficult to test again and again....!
2). Test Management tools to manage the test cases/conditions/results etc.

Kindly let me know if you have any exposure in the above areas...and dealt with similar issues...!!
Many thanks for your help in advance.

Jog.

▶ Reply to This

Permalink Reply by Rob Lambert on April 28, 2009 at 1:32pm

Hi Jog,

Cool. Thanks for the information. In my experience any of the open source test case management tools will work assuming they can export and import excel sheets.

As for the management of results, well, to be honest I would stick to the test case management tool for that. The only real way to get this working with TFS is to duplicate the tests as work items. I.e. create a new one like you have with requirements. But this is double the work.

Testlink is pretty good. As is Testopia.

But I'd stick with them for the whole management process. Until Visual Studio 2010 comes along the manual testing side of things is proving quite tricky to manage and report on.

There is nothing to stop you storing the test cases within TFS though to give you the version control, or even in the Source Control of Visual Studio, but for reporting purposes I reckon you may find it is more hassle than it is worth. a simple excel spreadsheet can often be the most accurate and simple solution.

As for automating tests, well since you don't appear to have the Test Edition of Visual Studio then I would suggest you use Selenium for the functional automation. again, we have some good information on our website on integrating Selenium with MS build to build, deploy and run the tests.

Here is a blog posting by one of my colleagues Mike, who is often on here:
http://blogs.imeta.co.uk/mblake/archive/2008/12/19/542.aspx

And here is one I did with a process based look at the same solution:
http://blogs.imeta.co.uk/RLambert/archive/2009/01/15/selenium-tfs-a...

Selenium is well supported, easy to get started with and very powerful. However, I'm assuming here that your application under test is a webapp. If not then a google search should shed some light on the best open source apps.

Although visual studio is very powerful and the ability to link work items is really cool, for testing it requires creating a few too many work items to track and this can be better done in excel, in my opinion :)

Good luck and I hope this helps

Rob..

▶ Reply to This

Permalink Reply by Jog on April 29, 2009 at 2:11pm

Hi Rob,

Thanks so much for kindly suggesting different ways to solve the testing problems that I'm facing now...
Some time back, I thougt about Selenium to automate the functionality and I didn't have enough information about selenium to proceed with (when I started)..
Now I'm going thru the information given by you and it really helps...

Also, in TFS we tried to make a new type of Work Item which can be used to write and store the test condition and results etc., also it can be linked to the corresponding Requirement/Technical Specification/Work Item - however the problem here is that, each Work Item can hold one test condition....the disadvantage here is that if we have 1000 test conditions..we will end up creating 1000 Work Items and it is difficult to manage various things like Test summary/Scope/coverage in a Project level perspective.
So I decided to go for the Traditional Excel spread sheet where we can have one tab for each Topic/Sub Topic/Work Item without any restrictions on the number of Test conditions and we can see the MI summary in one tab which pulls all the information together from all the tabs of the Excel sheets...I found it works for the time being....!
I'm just going through the Selenium stuff that you gave me....This is the main task for me to automate our testing process....(Some how)...!
I will let you know how I'm getting through in the due course....I hope I can come back to you if I get any problems during this process.
Many thanks for your courtesy and kind assistance in this matter.

Jog.

▶ Reply to This

Permalink Reply by Rob Lambert on April 29, 2009 at 2:56pm

Hi Jog,

Glad you are finding the information useful.

You said "..each Work Item can hold one test condition". This is exactly why we stopped using the manual tests in Visual Studio Test Edition as although they were essentially word documents, you could only pass/fail at the test case level and not step level. Very frustrating.

TFS work items are good for storing the high level title which can be used for reporting and linking and other metrics. To start creating a template for a real test case is not easy and may not even be possible.

At the moment, until VS 2010 comes out, am only using the web and load tests in Visual Studio and on one project using the work items for tasks and defects.

We run several projects using agile techniques so we have moved more towards other tools. I do like VS for the power of work items but for manual tests, it is tricky.

A couple of the guys here have just completed a cool multi threaded selenium test which is pretty cool. We are very much just using what is right for that context and mixing and matching where possible.

Let me know if you need more help.

P.S - I don't think you can go wrong with Excel. Best test management tool....ever.

Rob..

▶ Reply to This

Permalink Reply by Jog on April 30, 2009 at 9:39am

Hi Rob,

Many thanks for your reply on this.
Yes - Excel is the ever green test management tool..!
Just a quick thing, if you don't mind...I was just trying to download Selenium IDE from their web site and it is down loaded as .XPI file extension, I don't know which program can be able to access this to install.
Also, I forgot to ask you something before...
My application does involve in selecting options(some times configuring) from drop down list and entering (little) data on forms, drag and drops etc. are involved....I really don't know Selenium can record these actions , however I'm just going to try .....So could you please kindly suggest me how can I download Selenium IDE...
Thanks so much for your courtesy and kind assistance in this matter.

Kind Regards,
Jog.

▶ Reply to This

Permalink Reply by Rob Lambert on April 30, 2009 at 10:07am

Hi Jog,

Selenium IDE is an extension for the web browser Firefox. If you open up the latest version of Firefox and then go to Tools > Add-Ons > Get Add-Ons and search for Selenium you will get the option to download it there. Only takes a few minutes.

Selenium is then accessed from Tools > Selenium IDE.

It will open up a new window which contains the Selenium recorder. You can then hit the red record button at the top right to start and stop recording. (I think it is recording by default when opened).

You can then browse around your application under test and see what and how it records. That's the easiest way to get familiar with it.

After you get used to it, you can then start to look at ways to make your tests robust, use XPath to locate stuff, enter more generic steps to select links etc.

One thing to be aware of is the use of click / select. Most web apps will require these (as they are when recorded) to be replaced with clickandwait / selectandwait as you may be waiting for a page to respond.

The playback speed may also cause you some issues, so have a play with it and slow it down if needed.

Once you get more familiar you can step out in to code and look at using Selenium Remote Control.

The OpenQA website has some really good guides on it.

http://seleniumhq.org/

Drop downs, text entering etc etc are all cool and easy to do. I've never tried drag and drop but I know there are 'ways' to support it. Not sure what they are though.

Rob..

▶ Reply to This

Permalink Reply by Jog on April 30, 2009 at 12:14pm

Hi Rob,

Many thanks once again for the prompt reply on this. I knew I will get prompt reply :-), thanks ever so much. Now I know where to call in case of an Emergency ...! I really appreciate your help..!
I was able to download the tool as you mentioned...one thing I understood is that, it is used to test purely web (browser based) applications...(and I'm sure whether it is limited to Web based testing or not as I started going thru this yesterday only ...)
, my applications functionality involves in (less than) 5 % of Web application (thru the web browser,) however 95 % is custom software built through C# (.Net framework), C++ (little) with SQL Server DB and scripts....so lot of things as I told drag and drops ...selecting the options...entering the text etc...is on our custom Software front end(not browser), selecting options from menu and tools supported for us to process some functionality.... built with our own grids to pull the data from the DB tool. Display the information from DB in terms of Charts or in terms of tree structure etc...(Now I believe it gives you better picture)...I would have explained you like this earlier...I'm sorry)...
However when I was going through Selenium....I started thinking it doesn't support such functionality testing...... Because I tried testing through browser and Selenium doesn't recognise any other Software...
I'm sure you might have worked in such environment...( I think)...
Now again back to Google...to check open source tools..for my testing needs....!
Could you please kindly advice on this.... If you know any such tools for me..
Many thanks for your courtesy and kind assistance in this matter.

Kind Regards,
Jogesh.

▶ Reply to This

Permalink Reply by phil kirkham on April 30, 2009 at 12:40pm

I really think you might be better off starting another discussion as we seem to be doing the entire project for you and this thread is going off on too many tangents...

Permalink Reply by Rob Lambert on April 30, 2009 at 12:43pm

Hi Jog,

Yep - web testing only. for client testing try a Google search. I'm sure there are threads on the Software Testing Club about such tools.

I don't use them anymore as I'm all web testing. When I did use them, I used Winrunner which has been replaced (as far as I am aware) with Quick Test Pro - but these aren't cheap. No doubt there are some cheaper tools out there.

Good luck

Rob..

Permalink Reply by Suhas Patil on April 29, 2009 at 7:09am

Take a look at Nuevosoft Test Manager (http://www.nuevosoft.com). We used this test management tool in-house for testing Assetlink application (upwards of 4500 testcases) and on a weekly build schedule.

Nuevosoft Test Manager gives you:
1. an ability to export / import testcases, features and test results using spreadsheets.
2. highly intuitive and information rich dashboards.
3. Traceability from feature to testcase to bug.
4. automatically uploading test results from automated tests.
5. canned reports for almost all of your testing needs.
6. highly extensible due to web services
7. full visibility and ability to audit testing to keep senior management and your customers confident about testing outcome

▶ Reply to This

Permalink Reply by Jog on April 29, 2009 at 4:57pm

Hi Suhas,

Many thanks for your reply on this. I was looking Nuevosoft Test Manager the other day and I didn't know how good it was...I'm currently looking at various opensource Test Management tools as well before I finalise which tool to be used for test management.
As you know that MS Visual Studio is going to release their new edition MS VS 2010 by the end of this year or early next year which enables us to manage the testing as well through the same.
If I don't have any other options in open source...I will probably think to go for Nuevosoft.
Which Software environment do you work ? and do you use any automated tools currently ??
Many thanks for help and for your kind advice.

▶ Reply to This

• ‹ Previous
• 1
• 2
• Next ›
• Page