Test Driven Development – Patterns & Practices

I have posted the different sections of the term paper Test Driven Development – Patterns & Practices that I have submitted for Software Engineering course of MS-CS. The published section of the paper are as under.

Introduction: Test Driven Development & Units Testing
TDD/Unit Testing Practices
TDD Patterns
TDD Anti-Patterns
References.

Feel free to share your feelings/comments. Try to use Firefox 3.0+ while reading the post. IE7.0 sucks .

Test Driven Development – Patterns & Practices: References

[Beck02] Addison Wesley. Test-Driven Development by Example: Kent Beck 2002.

[Astels03] Prentice Hall. Test Driven Development: A Practical Guide by David Astels 2003.

[Lesse07] Manning. Test Driven TDD and Acceptance TDD for Java Developers by Lasse Koskela. 2007.

[Mar06] The impact of Test-Driven Development on Design Quality: Maria Sinaalto.

[Beck04] Addison Wesley. Extreme Programming Explained, Second Edition: Embrace Change.

[Gera07] Addison Wesley xUnit Test Patterns – Refactoring Test Code: Gerard Meszaros. May 2007

[Beedle05] Kent Beck, M. Beedle, et al., Manifesto for Agile Software Development, 28.12.2005,

http://www.agilemanifesto.org.

[Janzen] Software Architecture Improvement through Test-Driven Development: David S. Janzen.

[Martin06] Prentice Hall – Agile Principles, Patterns, and Practices in C# By Martin C. Robert, Martin Micah 2006.

[Prag03] Pragmatic Programmer – Pragmatic Unit Testing 2003.

[TDD1] Introduction to Test Driven Design (TDD) http://www.agiledata.org/essays/tdd.html

[TDD2] Stepping Through the Looking Glass: Test-Driven Game Development www.gamesfromwithin.com/articles/0502/000073.html

[TDD3] The Three Rules of TDD. http://butunclebob.com

[TDD4] Test-driven development. http://en.wikipedia.org/wiki/Test-driven_development

[Pancur] Pancur, M., Ciglaric, M., M., T., and Vidmar, T. (2003). Towards empirical evaluation of testdriven development in a university environment. In EUROCON 2003, Computer as a Tool, The IEEE Region 8, volume 2, pages 83–86.

[Deursen] van Deursen, A. (2001). Program comprehension risks and opportunities in extreme programming. In Proceedings of the Eight Working Conference on Reverse Engineering (WCRE’01), pages 176–185.

[Constantine] Constantine, L. (2001). Methodological agility. Software Development, pages 67–69.

[George04] George, B. and Williams, L. (2004). A structured experiment of testdriven development. Information and Software Technology, 46(5):337–342.

[refa01] The Process of Database Refactoring.

http://www.agiledata.org/essays/databaseRefactoring.html

[agil01] Agile Database Best Practices

http://www.agiledata.org/essays/agileDataModeling.html

[FOW01] Continuous Integration – Martin Fowler http://martinfowler.com/articles/continuousIntegration.html

[Shun01] Test Driven Development – Self Shunt Pattern

http://www.objectmentor.com/resources/articles/SelfShunPtrn.pdf

[Fow07] Mocks Aren’t Stubs – Martin Fowler

http://martinfowler.com/articles/mocksArentStubs.html

[Wor01] JUnit best practices

http://www.javaworld.com/jw-12-2000/jw-1221-junit.html

[Wor02] Best practices for test-driven development

http://www.javaworld.com/javaworld/jw-05-2004/jw-0510-tdd.html

[Vin04] Manning – JUnit in Action, Vincent Massol, Ted Husted 2004

[Ant01] JUnit Antipatterns

http://www-128.ibm.com/developerworks/opensource/library/os-junit/

[Ant02] TDD Anti-Patterns by James Carr

http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/

[surv01] Agile Adoption survey

http://www.ambysoft.com/downloads/surveys/AgileAdoption2007.ppt

[surv02] TDD Survey

http://www.stelligent.com/content/view/159/61/

[TDD3] Yahoo TDD group

http://tech.groups.yahoo.com/group/testdrivendevelopment/

Test Driven Development – Patterns & Practices: Anti-Patterns

The problem while finding the TDD Anti-Patterns was that there is not a single definitive source of information for anti-patterns; instead you have to rely on group/forums/blogs to get the information from [Ant02, TDD3]. Following is a list of common anti-patterns.

· The Giant: A unit test that is testing too many targets and contains too many test cases in there. Just like any other class, a unit test should have only one responsibility and test cases inside it should help it achieving that bit of functionality. This kind of a test cast show very low cohesion as many of its test methods are unrelated.

· The Mockery: Mocking should be used when a unit test require a resource that is too expansive (in term of time and space); it’s a need base object. If a developer try to mock out thing that is not being tested then unit test is actually testing the mocks not objects.

· The Inspector: Agile way of development promotes just enough design philosophy, but it does not mean that design principles (encapsulation, low coupling, and high cohesion) are not applicable. Code that does not have these attributes will be very difficult to refactor later on; infect it will break some code in an attempt to do so. A test case that breaks the encapsulation principle in an attempt to get the required visibility of the target is just doing that. Very difficult to maintain and refactor that kind of code.

· The Local Hero: A test case/unit test that has a dependency on something specific to a particular environment (may be in IDE or platform) will show a green bar only when it is tested under those condition and will show a red else where.

· The Nitpicker: A unit test case should focus on the part of the target that is required; if it is testing some part of it that is not important for the overall goals of the systems. This kind of a test case will required that target should be kept in state that is unimportant for the system.

· The Secret Catcher: A test case that appears to be doing nothing as it is not asserting something e.g. if a test case is relying on an exception to be thrown. This kind of a test case should make it explicit by naming the exception variable expected and add a true assertion to make its intention absolutely clear to the reviewer of code.

· The Loudmouth: A unit test the write too many messages/logging to the console/log files. If the test is passing, a test case should keep its mouth shut. See Tangled Vision Pattern.

· Excessive Setup: A unit test that requires a lot of time/code to setup and does not make it’s intention entirely clear to every one or require a lot of time to do so. This kind of a test case normally tests too many things, try refactoring to make it manageable.

· Hidden Dependency: A test relies on some external data to be available before it gets executed; test case will pass when data is available and fail when it will not be there.

· The Greedy Catcher: A test case that relies on silent exceptions to pass its test cases or log down lesser informative messages to console/log file.

· The Liar: A unit test that pretends to test to target but upon inspection it’s found that it was not able to do so. This kind of a test uses all your system resources & time but does not do any provide any useful functionality.

· The Enumerator: The unit test that does make its intentions entirely clear as the names of the test cases is not following the Legible Naming best practice. The name of the test case uses a sequence as test1, test2 etc. The only way to find the intention is to look at the test case code.

· The Slow Poke: A test case that run very slow, it may require an expansive resource. Try creating mock to simulate the resource behavior.

· The Dodo: A functionality which was probably outdated and not required by the application now. As the functionality was built using the test first, it should have a test case for it which should be deleted.

· The Stranger/The Distant Relative: A unit test is designed to test a particular target object, if a test case insides it is not testing the intended target object and test a another object (most likely an object used by the target object). This kind of a test case create unit test with low cohesion.

· The Operating System Evangelist: A test case that has a dependency on operating system specific features. This will affect the portability of the test case severely.

· Success against All Odds: A unit test should be designed in a way that if one of the test case/part of the test is failed the entire unit test is failed; infect when ever a assertion failed in a test case, unit test should halt the processing of the test case with a red bar. The opposite of this rule is anti-pattern.

· The Free Ride: A test for a functionality/feature was written inside an existing test case; instead of writing a new test case for the functionality. Make the test case less cohesive.

· Silver Bullet: A test case that tests entire set of functionality provide by an object or group of objects. It has only one test case method which tests. A common indicator is that the test method is often the same as the unit test name, and contains multiple lines of setup and assertions.

· Love Birds: A test case which is so closely coupled with a particular class under test that it distracts from the need to test important interaction between classes. Commonly found in a test package structure created for Homing Pigeons where there is no obvious place to put tests for class interaction

· The Web Surfer: A unit test that requires a connection to internet with access to the outside world in order to run.

Test Driven Development – Patterns & Practices: Patterns

This is a list of most common of the TDD patterns. Most of those are from the seminal work of Kent Beck [Beck02].

Testing Patterns: The patterns are not confined to Test Driven way of development, but are helpful for Unit Testing in general.

· Child Test: You have created a large test case that is not easy to run. What you do? You should think before writing a large test case that how I can make it smaller; found a way – then apply it. Otherwise try creating a new test case (smaller in size) based on the large test case with only broken parts of the test case in it. Now make that test case working, it will be relatively easy to make it work. After the smaller test work, bring in the rest of the test case.

Try dividing the larger test case into smaller more manageable test case and make them work. If you will be able to make the smaller test cases to work then making larger test-case to will be a matter of integration.

· Mock Object: As mentioned previously, a mock object mocks an expensive/complicated resource. That resource might not be available at the start of the project or even if it is available will require a lot of time to run e.g. Database Server, Web service etc. Using mock object we can have the same interface as the actual resource provides and using it we can run our test cases. Again it’s well aligned with quick red-green-refactor philosophy.

While using Mock object you are also considering the level of visibility that they have and how your application classes will interact with them. That results in systems that are loosely coupled. To find whether you are loosely coupled with the mock object with the actual object, if the transition is smooth then it’s an indication of loose coupling. This technique also helps in reducing the risks that are associated with using mock objects.

· Self Shunt: If two of your classes are collaborating with each other, self shunt helps you in finding whether the collaboration is correct or not. For this purpose we will create our test compatible with other object and the first object will call one of the methods in our test case and verify whether the collaboration is expected.

How do you test that one object communicates correctly with another? Have the object under test communicate with the test case instead of with the object it expects. Our test case is impersonating one the objects in the collaboration [Shun01].

· Log String: Log string helps in determining the sequence of method calling in a sense that it maintain a string and add to it every time a method is called. And at the end (tearDown()), we compare the string with the sequence we require. If sequence is fine we move onto the next test case if it is not than we fix it to make it work. This pattern normally works well in conjunction with the Self Shunt pattern.

· Crash Test Dummy: Crash test is similar to a Mock object; a mock object simulates a valuable resource where as crash test simulates a condition that has a rare chance to happen. Its work in tandem with the philosophy that is its not tested than its not working, so we test it any way.

· Broken Test: This pattern tells that a programming/TDD session should be broken with a red bar. It means if your time is up for the day, you should leave a test case broken. The next day when come back you will have an obvious place to start from. It has a psychological effect that something is not finished yet and it will give me place to continue from. The technique is useful when you are working on things that are not affecting anyone else – mean a solo project.

· Clean Check-in: When working in an environment where your have a team of developers, you should leave a session with a green bar. If you leave a session with a red bar, it might conflict with some one else’s work. You cannot predict what has to the code since you last saw it, so there can be merging issues that you have to resolve. And merging related issues are the most annoying for a programmer. You need to have a certainty and confidence in your code, it will be easier a session if you have the confidence.

Red Bar Patterns: What to test? Where to start? How to move forward? When to stop? These are the question answered by these patterns.

· One Step Test: You have devised a rough list of test cases based on the requirement, you have started working on those and implementing a few obvious starters, but now you are not sure which test case to work on. That can happen even if you are just starting to write code for the test cases that you have.

Choose a step that will take you one step closer to the overall objective that you are targeting. Off course the experience will help you, the knowledge of the domain will let you sail easily, but if you don’t have extensive knowledge of the problem at hand, you can still draw your conclusion based on the last step you have taken and that single step will always be helpful in achieving the transition from unknown—known.

· Starter Test: This pattern is somewhat related to the One Step Test and is about finding the test to start with. Start with simplest of the test cases and test with very simple and output parameters. By doing so you will get a feeling of how things are and what next to be done. One Step Test will help you in answering this question.

· Explanation Test: The pattern is about how you are going to develop a TDD culture in the environment that you have working right now. Manager and team leader are normally responsible for developing such a culture in your organization where people can see the benefits of TDD right away. Forcing someone to do something will offend him and would not serve the purpose well.

Interpersonal skill plays an important role when you are convincing some one about the benefits of the TDD. A simple technique is when one of the member is explaining something like a piece of code or a design diagram; start talking in terms of the test case, is we have this function/and call it using this object it should show us the value ‘abc’. The good part of this technique is that you can apply it at different levels

· Learning Test: This pattern is about when you use component off the shelf solutions, and you understand the functionality provided the Component using the documentation that they have provided. We normally expect that something will work the way written as expected in the docs, why not write test case to find it out whether the functionality exists or not. This technique will help in achieving the understanding as well as verifying the functionality that you requires out of the components.

Kent beck shares the following experience from a project where this pattern was helpful [Beck02]. “Jim Newkirk reported on a project in which Learning Tests were routinely written. When new releases of the package arrived, first the tests were run (and fixed, if necessary). If the tests didn’t run, then there was no sense running the application because it certainly wouldn’t run. Once the tests ran, the application ran every time”.

· Another Test: The pattern helps in achieving the productive level desired, sometimes during the course of the daily routine; you got an idea in mind and start wondering about — how you are going to implement that idea. Thinking about new idea is good but it will stray you off the topic under discussion. Write down the idea and few points that you have in mind about it and keep focusing on the problem at hand.

· Regression Test: Regression test require foreknowledge of the problem, they provide a way whether due a new feature everything in intact. While writing a test case you should have the same mind set as were fixing a regression defect. These tests add the variability missing in other methodologies. We should have regression tests both at the system level as well as at the lower levels. Regression tests at system level are just like user playing with the system, whereas small scale regression tests will be helpful in finding the design errors. When you find a red bar due to regression test, you should know that something is still missing from the design. Again refactoring will help in reaching to the green bar.

Green Bar Patterns: These patterns help in the transition from a red bar to a green bar. Think red bar as an enemy in those arcade games and you will make every effort to turn it into green.

· Fake It (‘Til You Make It): The philosophy of TDD is to execute Red-Greed-Refactor cycle in quick successions. If a method has no obvious implementation then you can fake its implementation by returning a value that will satisfy the test e.g. returning the same value the test requires. In this way you will reach at the green bar and then move to the next step. The fake implementation will eventually get updated as the other associated test will begin to crumble. At that time you have a better understanding of what is required out of the method and you will be in a better position to implement it.

This has a psychological affect attached to it as having a green bar feels different from having a red bar. It also has an impact on controlling the scope better. A good example of this is given in [Beck02].

· Triangulate: This pattern helps you with question. When should we apply abstraction (data & procedural)? Solution to it is; when you have two or more examples; abstract it out. Triangulation will also help in fake implementation. If you started to have duplication assertions inside you test cases for similar cases, try refactoring the code to achieve abstraction and removing duplication.

· Obvious Implementation: The implementation using Fake It and Triangulation might not be the one to remain in the system for long as they are sometimes used to pass a failing test, if you are sure about the implementation and it’s obvious from what you have done so far; then you should go and implement it. Remember that the implementation should not take a long time as it will go against the philosophy of test driven which require quick red-green-refactor cycle.

· One to Many: The pattern is helpful while working with collection of object. An obvious question to ask is how to test collection of objects? Solution is to test it without the collection and then make it work with collections. The patterns also help in isolating the change.

xUnit Patterns: These are the patterns for the xUnit family of tools. So these patterns are very specific in describing what to do. My suggestion is to read in conjunction with the best practices in the previous section.

· Assertion: Checking whether the test work or not? It’s a Boolean operation. xUnit family of tools provide to a set of assertions (assertNN()) to make it happen. While working with the assertion, be very specific about why are you using. Right general assertion will not be very useful. It is also a good practice to provide description of what this assertion is for in the first parameter of the assertion.

· Fixture/External Fixture: There can set of common data or resources that will be used by more than one test. One way is to create data/resources in each of the methods. Other is to create it at a common place from where all the test cases can access and use the data/resources. Another important aspect of using resources is that after using we have relinquished them so the others can use the resources. Creating resources without freeing those will eventually brought your testing system down to its knees.

For these purposes xUnit provide setUp() and tearDown() methods, one is called before a group of tests are executed while the other is executed after the tests finish their execution. You can hook your initialization and freeing up code in these methods. Another important point while working with fixtures is that there is no one on one correspondence between your application and test classes. Sometimes one fixture corresponds to more than one class and some time more than one fixture test a single class.

· Test Method: A single test case is represented by a single method. If two or more methods are part of the same fixture, combine those together in a single class. For naming of the test cased, check out the ‘Legible naming’. Write small unit test that will make them stand out in the code and will also serve as documentation for your code.

Start by creating a list to TODO comment for each test inside a fixture, you can categorize the test cases in nester TODO comments.

· Exception Test: How do you test for expected exceptions? Catch expected exceptions and ignore them, failing only if the exception isn’t thrown. Be very specific while dealing with exceptions; if method is throwing a specific exception, check for that exception not anything the hierarchy of that exception. Check Legible Exception practice for other useful tips.

· All Tests: How to test more than one unit tests together? xUnit provide a suite method to put them together. Possibly create a different suite for different packages and create Master test that combine all the test suites from the entire package. It’s is ultimate technique to achieve automated unit testing. Just click the unit test button and here it goes.

Test Driven Development – Patterns & Practices: Practices

Following are TDD and Unit testing best practices.

· Legible Naming: A method name should suggest it purpose, test cases are just like methods we have in our application code. So, same rules apply to them as well. A good practice while naming a test case is start its name with ‘test’ should be the name of the method under test ‘testNameOftheTestee’. If we have more that one test cases for the same method, instead of numbering them add the purpose of the test case at the end as ‘testNameOftheTesteePurpose’.

· Exceptional Legibility: A test case that is written to check an exception should make this exception clear by naming the exception variable ‘expected’. Name of the variable should give indication of its purpose and that what it is doing here.

· Failed due to: Each failure should have a description attached to it. So, just by looking at the console, a programmer can find out what’s wrong with the code and quickly go there are fix. A description is normally attached to the assertion in xUnit. This practice resulted in fast red-green-refactor cycle.

· Same Namespace yet Disparate: How to structure you test classes along with the application classes? This practice gives us the answer to this question. A general practice is to place test in the same namespace where you have your application code being tested. You can create a child namespace for the tests but parent namespace should remain the same. That will normally be required for protected and package access in some programming languages.

· Single Point of Focus: A fine grained test is always desired, the more fine grained our test are the greater the level of coverage we can achieve. In Object Oriented realm a fine grained test case will focus on single object at a time. As we are focusing on a single object at a time, the things are in our control and not much can go wrong. A test case that tries to cover too many things will eventually become a maintenance hazard.

· Continuous Melioration: A unit test behave as a user of your code, it give you instant feed back in term of failure and successes. As the application is solely based on the test cases we have and test cases are based on user requirement; so, a test case is just like a user telling us what to do next. As maxim goes “user is always right”. This approach leads to robust and correct solutions.

Another point worth mentioning here is that unit test help us improve the code, normally while unit testing we try to all possible paths and the only way to know about the paths is know the number of branches it has. The more branches that you have the more difficult it will be to write unit test code for that particular code. It is good indication of a bad design, apply refactoring to minimize the number of branches that your code take and then try again.

· Refactor, as they say: This practice has several facet attached to it, let me discuss those one by one.

- Improve the Design: Agile and XP methodologies promoted the term of just enough design that will work. We design as we move along in an iterative manner; if a conflict comes in the picture, we refactor the code to resolve the conflict. Now refactoring is just a way to improve the design without making any functionality related changes. How can we make sure that we just did that and everything else stays intact? Just run the test cases.

- Improve the Test Code: If you are working with mock objects and mock object take has huge setup method then it’s an indication of wrong number of Mock Objects. Try merging the expanding the mock object using refactoring.

- Try to refactor the long setup and teardown methods and use extract method to extract the common code and creating utility classes to share the same code between test classes.

· Layer Supertype: Create a super class of all your test classes and place the common test code in that super class, this will help in increasing the reusability of the code and well as you have a place to put something that will be applicable to all the test classes e.g. You can write interceptor for you test classes.

· Regress as you go: One of benefits of approach like TDD is test automation. A change is made to the systems, may be a new feature is added or an existing feature is modified. Just run the test cases to confirm that everything stays where it was before the changes. It is kind of on the go regression mechanism for your application code. A new change can be made with confidence as you know if something goes wrong I’ll have red bar indication.

· One = One: A single unit test should correspond to a single test method in the test class, if you have two unit tests; create a separate test for each of those. It is generally a bad practice if you try to test more that one unit test in the same test method. In other words we can say that a single test method has a single responsibility of testing a whole/part of a unit test. And a single test class has a single responsibility of testing a single fixture. Same Object orientation principles are application here, just the context is different.

· Exceptional absentee: While working with mocks, you sometimes need a few methods for the test cases. A mock object might be mimicking an object with a lot methods e.g. Database Driver. Good IDE will create a mock object based on the interface of the actual object in seconds but you need only a few those. So, make your intension absolutely clear by throwing an exception from methods that are not useful for you purpose instead of returning null from those. Leave methods with no return type as it is.

· Restricted Mockery: Mock object are just dumb object, they are mimicking something. Mock objects are not the place to put your business test code into; instead if you have to test business method, try creating a stub for that purpose [Fow07].

· Keep an eye on: A general practice in unit testing is that that if a test should fail for a specific case then it should fail for it. In TDD we write test before the application code, so this happens automatically as we go along. A technique to check the validity of a test case is to test it against a skeletal class; if it is not failing for that class then something is wrong wit the test case. Try reviewing it.

· Just enough Design: The simplest thing that could possibly work is the principle of XP. This is opposite to the upfront design methodology. This principle works hand in hand with the test first style of development. In TDD we only require the bare minimum application code that will make a failing test pass and improve as we move along the way.

· Immutability is the key: While working with an approach like TDD, where test code drives your application classes; always strive for application (or ever test classes) that are immutable. Immutability will not only simplify the design of the application but will also have no side effects at all. Immutable class designs are generally more maintainable [Wor01].

· Pairing is sharing: Coming from a code first back into the realms of TDD, there is always a temptation in mind to switch back to code first approach (may be due to fatigue). There comes in the partner to keep you focused. In pair programming, one person is on the driving seat (at keyboard) and the other person look at the code can come up with a suggestion; this will help in improving the design of the application in continuous manner. Pair Programming is indispensable when we talk about TDD [Wor02].

· Explicit Ordering: Never rely on the sequence in which the xUnit family of tools may call your code. These tools rely on reflection (introspection) techniques to find the methods out of you test cases, the behavior of these mechanism may vary from platform to platform (or ever IDE to IDE). If you write you your test cases with this assumption then you might have portability issues with your code. If you want explicit sequence, add test cased in the order in which you like to the test suite; xUnit will call it in that particular sequence [Wor02].

· Say No to Side effects: Avoid writing test cases with side effects. A test case will have side effects if it interferes with execution of other test cases in the system. Side effect can cause the following problems.

- It will affect the data used by another test case

- We cannot repeat the test case without manual intervention.

Automation of the test case will be severely effected by the manual intervention and data corrupting problems are very hard to diagnose. Avoid test cases with side effects [Wor01].

· Tangled Vision: Avoid test cases that require a Visual inspection for a human being to verify the test case generated data; instead a test case should be complete in its working and should have the mechanism to inspect the data. Visual inspection requires patience and an ability to inspect a large quantity of data; they qualities are not common in normal human beings

Test Driven Development – Patterns & Practices: Introduction

Abstract: Test-Driven Development (TDD) is a novel approach to software engineering that consists of short development iterations where the test case(s) covering a new functionality are written first and then the necessary code is development to make that test pass. Unit TDD focuses on first writing a small unit test and using that to drive development of a small solution fragment. As unit testing the critical part of the TDD cycle, so we should be able to answer the following questions concerning a unit test; what properties (A-TRIP) a good unit test should exhibit? What to test and what not to test and how to find it (Right BICEP)? What options do we have to facilitate automatic unit testing (Mock Objects) and independence of test? – Primary goal of TDD.

While TDD focuses on improving the internal quality of the software, the external quality of the software is measure using another test driven concept – Acceptance TDD. Acceptance TDD involves writing one or more systems-level acceptance tests (or “customer tests”) for a customer-centric feature, before the solution. As member of the team develop difference part of the software, the overall quality of the software cannot be assessed until we have tested it on the integrated code and this integration is achieved using the continuous integration. Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates daily – leading to multiple integrations per day.

As the test driven community is evolving, practitioner of the technique have found effective/Successful ways to achieve a task. These ways are communicated in the form Patterns, Guidelines/Best Practices. Unsuccessful ways to achieve something or practices that tend to have a negative impact on the overall success of the project are communicated as AntiPatterns.

Introduction: Test-Driven Development/Test First Development/Test Driven Design (TDD) is a software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is refactored to accommodate changes. The goal of TDD can be two-faced. On one side it can be viewed as specification technique focus on the functional as well as implementation details (Design) [Martin06], on the other side it is viewed as programming technique as quoted by Ron Jeffries “the goal of TDD is to write clean code that works”.

Test Driven development (TDD) is a core part of the agile code development approach derived from Extreme Programming (XP) [Beck04] and the principles of the Agile Manifesto [Beedle05]. Despite its name, TDD is rather a development and design technique than a testing technique. When the test is passed, the code is refactored to improve the internal structure of the code. This incremental cycle presented below is repeated until all functionality is implemented [Beck02]. TDD should be seen as complementary to Agile Model Driven Development (AMDD) approaches and the two can and should be used together. TDD does not replace traditional testing, instead it defines a proven way to ensure effective unit testing [TDD1].

Following are the benefits of using TDD.

· In literature, TDD is proposed to guarantee testability and to reach exhaustive test coverage [Astels03].

· It is also claimed to increase developer confidence. If a bug is found, the developer should create a test to reveal the bug and then modify the production code so that the bug goes away and all other tests still pass. On each successive test run, all previous bug fixes are verified. [Beck02]

· Enable integration (continuous integration) are often that allows larger teams of programmers to work on the same code base, because the code can be checked in more often.

· TDD is said to affect the design of the code by helping to produce highly cohesive and loosely coupled systems (Modular Systems & Usable Design).

· It is also said to encourage the explicitness about the scope of the implementation while it helps separating the logical and physical design, when only the code needed at a certain time is implemented. [Beck02].

· The suite of unit tests provides constant feedback that each component is still working. [TDD2]

· The unit tests act as documentation that cannot go out-of-date, unlike separate documentation, which can and frequently does. [TDD2]

· When the test passes and the production code is refactored to remove duplication, it is clear that the code is finished, and the developer can move on to a new test (helps in calculating the percentage work done and future schedules). [TDD4]

· Test-driven development forces critical analysis and design because the developer cannot create the production code without truly understanding what the desired result should be and how to test it. [TDD4]

· The test suite acts as a regression safety net on bugs. [TDD2]

· Reduced debugging time.

· WYGIWYM (What You Get Is What You Meant) What TDD gives you is the confidence that the code you just wrote does what you wanted it to do. [TDD2]

The life cycle of typical Test Driven approach can sum up as follows [Beck02]

1. Add a test.

2. Run all tests and see the new one fail.

3. Make a little change.

4. Run all tests and see them all succeed.

5. Refactor to remove duplication.

6. Repeat

Red-Green-Refactor is an alternative mnemonic for the TDD cycle of writing a test, making it pass, and making it pretty. Red and Green colors are associated with the xUnit generation of tools where Red represents test failure and green represents a success.

When we begin the TDD cycle by writing a test, it fails. It fails because our system is broken right now; it doesn’t have all the functionality we want it to have and then we make it pass, we implement the required functionality—and as a result red bar turns to green. The last part of the cycle, refactor, is just that—refactoring. As we improve the design of the code without altering its external behavior, all tests should pass, thus, we should remain green. [Lesse07]

Practitioners of TDD described the following three rules that should be kept in mind while following a test driven approach. [TDD3]

1. You are not allowed to write any production code unless it is to make a failing unit test pass.

2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.

3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

A few Shortcoming of TDD, as identified by practitioners/researchers of the area are summarized below.

· Test-Driven Development is difficult to use in situations where full functional tests are required to determine success or failure. Examples of these are user interfaces, programs that work with databases and some that depend on specific network configurations. TDD encourages developers to put the minimum amount of functional code into such modules and maximize the logic that is extracted into testable library code, using fakes and mocks to represent the outside world.

· Management support is essential. Without the entire organization believing that Test-Driven Development is going to improve the product, management will feel that time spent writing tests is wasted.

· The tests themselves become part of the maintenance overhead of a project. Badly written tests, for example ones that check hard-coded error strings or which are themselves prone to failure, are expensive to maintain. There is a risk that tests that regularly generate false failures will be ignored, so that when a real failure occurs it may not be detected

· Some other shortcomings as identified in literature by Researchers of the area are:

§ Lack of design & Documentation. [Pancur] [Deursen]

§ Difficulty in applying unit tests and scaling the application. [Constantine]

§ Its reliance on refactoring and skill of the Developer. [George04]

If TDD is a technique than Unit testing is the methodology we adopt to achieve it. Unit testing is performed using the xUnit Generation of tools (JUnit, VBUnit, NUnit etc). TDD is just another way to Unit test and we can unit test without being Test Driven. Unit testing tests the internal meaning of the code a.k.a White box testing. Most of the guidelines & practices that are applicable for a units testing are applicable to TDD (except for the one that relies on code availability). We will discuss more about unit testing in general in the next section to come.

Some related concepts like Acceptance/Customer TDD and continuous integration are essential for the overall success of a TDD based project and both of the techniques are covered in the next section.

Unit testing:

A unit test is a piece of code written by a developer that exercises a very small, specific area of functionality of the code being tested. [Prag03]

A few of the benefits associated with Unit Testing.

· Facilitates change: Run the test once you make a change to ensure everything stays intact. You are always confident at what you are doing as now you have a kind of device for verification.

· Simplifies integration: As we have the tested the units alone and we are confident in what they are up to. Using this approach we have divided larger problem into small problem and then we have to only test those part that are affected when these parts are merged.

· Documentation: Unit Test cases are kind of living documentation for what you are coding. You can deduce by looking at the code what we are expecting out of this code.

· Design: Simplify design and help in improving the design as you move along. We will see in the practices section how unit test improves the design and help in the reduction of cyclomatic complexity of our code

An obvious question while unit testing. What to test? Right BICEP. The answer to this question is

· Right: Are the results right?

· B: Are all the boundary conditions CORRECT?

· I: Can you check inverse relationships?

· C: Can you cross-check results using other means?

· E: Can you force error conditions to happen?

· P: Are performance characteristics within bounds?

So, what are these Boundary Conditions? CORRECT. The following is list of boundary conditions for a unit test.

· Conformance: Does the value conform to an expected format?

· Ordering: Is the set of values ordered or unordered as appropriate?

· Range: Is the value within reasonable minimum and maximum values?

· Reference: Does the code reference anything external that isn’t under direct control of the code itself?

· Existence: Does the value exist (e.g., is non-null, nonzero, present in a set, etc.)?

· Cardinality: Are there exactly enough values?

· Time (absolute and relative): Is everything happening in order? At the right time? In time?

Mock Object is another important concept while unit testing. Mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A computer programmer typically creates a mock object to test the behavior of some other object. In a unit test, mock objects can simulate the behavior of complex, real (non-mock) objects and are therefore useful when a real object is impractical or impossible to incorporate into a unit test.

Good tests possess some properties. What are those properties? A-TRIP

· Automatic: Units testing process should be automatic. Just press the button and here it goes.

· Thorough: A good unit test should be thorough enough to test every possible way a code can break.

· Repeatable: We should be able to run the unit test again and again and should get the same result; an important point while designing a repeatable unit test is that it has no side effects.

· Independent: Units test are not coupled with a particular environment or other unit tests, if you want to run more than one unit test cases, create a suite of it.

· Professional: Unit Tests are just like any other code you write; it follows the same design principles as the rest of your code do.

Hear are a few of the limitations and shortcoming of unit tests.

· Integration & Functional Level Testing: Testing cannot be expected to catch every error in the program – it is impossible to evaluate all execution paths for all but the most trivial programs. The same is true for unit testing. Additionally, by definition unit testing only tests the functionality of the units themselves. Therefore it will not catch integration errors, or broader system level errors (such as functions performed across multiple units, or non-functional test areas such as performance).

· Combinatorial Problem: Software testing is a combinatorial problem. For example, every Boolean decision statement requires at least two tests: one with an outcome of “true” and one with an outcome of “false”. As a result, for every line of code written, programmers often need 3 to 5 lines of test code.

· Continuous Integration & Version Control: To obtain the intended benefits from unit testing, a rigorous sense of discipline is needed throughout the software development process. It is essential to keep careful records not only of the tests that have been performed, but also of all changes that have been made to the source code of this or any other unit in the software. Use of a version control system is essential. If a later version of the unit fails a particular test that it had previously passed, the version-control software can provide a list of the source code changes (if any) that have been applied to the unit since that time.

Acceptance/Customer TDD:

“Doing the right thing” that the philosophy of the Acceptance testing. User involvement is required for this process to achieve its goals. Acceptance testing is a team activity and a team process and relies on the team related concepts. In TDD, tests are not just a verification tool but also an integral part of the requirement and specification and customer collaboration. In TDD the unit tests drive the functionality at the code level while is Acceptance TDD the test drive the functionality at the feature level. Acceptance tests are generally comes out the user stories, requirement specification and customer collaboration. The involvement of the customer makes this process an important activity for requirement verification [Lesse07].

Four important steps for the test driven process are picking up a user story, write acceptance test for it, automating the test and then implementing those test to make them pass. This four step process is repeated throughout the agile process and iteration.

Continuous Integration:

Best definition I can found of continuous integration is; “Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily – leading to multiple integrations per day, each part of the integration is verified by an automated build (including test) to detect integration errors as quickly as possible”. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly [FOW01].

Test driven development, Acceptance TDD and continuous integration can go hand it hand with one another; infect they can reinforce each other. As each member of team integrate his/her part of the code, in Test first approach he will make sure that he get the green light before he commits the code and integrate it with main source. So, he is making it sure that his/her part work according to the specs (test cases). We can write a set of test cases to test the integration of these parts; infect we can use these test cases to drive out our integration. In a larger team this option can become cumbersome and start losing the effectiveness associated with fast TDD iterations. Another approach is to write a set of test cases specifically targeted towards these integration issues and whenever a member integrates his/her code he run the those test cases and make it sure everything stays intact (red-green-refactor cycle). Hence lesser problem related to the integration plus loosely coupled cohesive solutions.