views:

198

answers:

5

Hello,

I've recently asked a question regarding best ways of separating business logic from data access to make application testable (here). Thanks to Jeff Sternal, I've created interface for the data access and passing its concrete implementation from the application top level to BL. But now I'm trying to figure out how can I separate file access methods from business logic.

Let's say I have a function that loads data from a database into dataset, formats loaded data (formats are stored in some external xml file) and finally serializes dataset to a file. So, to support testability I need to move all the functions that access file system to some interface. But first - I find it quite handy to just call dataset.WriteXml(file), but for the testability I have to create interface and move dataset.WriteXml() to its implementation, which looks to me as unnecessary layer and makes code less obvious. And second - if I move all methods that access file system to a single interface, it will violate SRP principle, because serializing\deserializing datasets and reading data formats from a file seems to be different responsibilities, correct?

A: 

It's not completely clear from your description what your code is doing. What is the most important thing your code is doing? Is it the fomatting? I would say don't bother testing the writing of the xml. How would you verify that anyway?

If you must read from and write to files, you might be better altering your code so that you pass in a streamreader/streamwriter or textreader/textwriter, with the calling code injecting an instance of something like memorystream for testing and filestream for real i/o in production.

IanT8
A: 

You could use the versions of WriteXml() that takes a Stream or TextWriter and set up your code such that this object is passed into your code, then for testing pass in a mock object.

Michael Borgwardt
Stream may work, but then I will have to open FileStream in the function caller and pass it to my function, so it will be opened all the time data is processed.
Dev
+2  A: 

I think you need to split up your code a little bit more...

You say: Let's say I have a function that

  1. loads data from a database into dataset,
  2. formats loaded data (formats are stored in some external xml file) and
  3. finally serializes dataset to a file.

That sounds like 3-4 jobs at least...

If you split your code some more, then you can test each of those functions without all the other goo around them.

If you just want to do Dataset.WriteXML, Then you dont have to test that. Thats a Framework function thats working quite nicely. Try getting some mocks in there to fake it out. How exactly depends on your solution...

Answer to comment:

Creating all those small classes with their own tests will make testing easy, and it will also make your functions small and compact (-> easy to test) You would test if the content of the Dataset is exactly what you require, not if the dataset gets serialized correctly into an xml file. You would also test if your formater would perform its function correctly, without any dependencies to any other logic. You would also test the Data access, but without accessing the DB(Stubs/Mocks again)

After you know that all this works like it should, you "just" verify that the propper method on the dataset will get called, and that should satisfy you, since you tested the other parts in isolation already.

The tricky part about unittesting is getting meaningfull tests. They should be -fast- and -simple-

To make the tests fast, you should not touch stuff you have no control over:

  • Webservices
  • Filesystems
  • Databases
  • Com Objects

To make them simple, you keep you clases focused on a single task, that where the SRP comes in, which you already mentioned. Take a look at this answer... It will also point out the other principles of "SOLID" development

http://stackoverflow.com/questions/1423597/solid-principles/1423627#1423627

Heiko Hatzfeld
+1, why did someone downvote this?
Jeff Sternal
Do I understand correctly that it is not necessary to create test for the function itself, but for its "parts" (points 1 and 2)?
Dev
@Heiko - Thanks for the answer!
Dev
+1  A: 

For a quick summary, if you really want to make this testable, I recommend:

  1. Extracting the data formatting code into a new class (which implements an interface you can use to mock).
  2. Passing this class your DataSet.
  3. Make the new class return the DataSet as an IXmlSerializable that you can also mock.

Without seeing the current code, I have to make some assumptions (hopefully not too many!) - so perhaps the current code looks something like this:

public class EmployeeService {

    private IEmployeeRepository _Repository;

    public EmployeeService(IRepository repository) {
        this._Repository = repository;
    }

    public void ExportEmployeeData(int employeeId, string path) {

        DataSet dataSet = this._Repository.Get(employeeId);
        // ... Format data in the dataset here ...
       dataSet.WriteXml(path);
    }
}

This code is simple and effective, but not testable (without side-effects). Additionally, having all that logic in one place is a violation of the single-responsibility principal.

Depending on your needs, that's probably fine! Fulfilling the SRP is always just a goal, and we always need to balance testability with other factors that influence our designs.

However, if you want to segregate responsibilities a bit more and make it testable, you could do so by moving the formatting logic into its own class (which will implement IEmployeeDataSetFormatter), then inject an IEmployeeDataSetFormatter into this method call (alternately we could inject it into the service's constructor, just like the IEmployeeRepository). The method that formats the data will return an IXmlSerializable so we can mock it for safe, isolated testing:

public interface IEmployeeDataSetFormatter {
    IXmlSerializable FormatForExport(DataSet dataSet);
}

public class EmployeeDataSetFormatter: IEmployeeDataSetFormatter {
    public IXmlSerializable FormatForExport(DataSet dataSet) {
        // ... Format data in the dataset here ...
        return (IXmlSerializable) dataSet;
    }
}

public void ExportEmployeeData2(int employeeId, string path, IEmployeeDataSetFormatter formatter) {

    DataSet dataSet = this._Repository.Get(employeeId);

    IXmlSerializable xmlSerializable = formatter.FormatForExport(dataSet);

    // This is still an intermediary step - it's probably worth
    // moving this logic into its own class so you don't have to deal
    // with the concrete FileStream underlying the XmlWriter here
    using (XmlWriter writer = XmlWriter.Create(path)) {
        xmlSerializable.WriteXml(writer);
    }
}

This has definite costs. It adds an additional interface, an additional class, and a bit more complexity. But it's testable and more modular (in a good way, I think).

Jeff Sternal
What is confusing me in all these is that testability requires me to create twice more types than I already have... For the current example IXmlSerializable will work, but it may be hard to use it if want to have some additional processing of the dataset after applying formatting, also it will not work if I need to save data to some custom format. I'm probably stepped on the wrong way trying to find universal silver bullet.
Dev
Testability influences the design, yes. And yes, you might have to create more interfaces and classes, with the benefit that they will be more robust and easier to test.
Bluebird75
@Dev - I agree with Bluebird75, though remember: you don't have to test *everything*! (Unless you're practicing TDD.) If the cost of making it testable exceeds the payoff, punt for now!
Jeff Sternal
I see.. I'm not fan of TDD, just attempting to start making more flexible and testable solutions. Perhaps I need some time to adapt to not being confused by inventing much more interfaces than usual "straight" approach requires.
Dev
+1  A: 

Go with the extra classes. When testing it its easier to:

  • For the dataset exporter/writer - check the modified data is passed to the writer
  • For the formatter - check the formatting logic

This doesn't mean you don't:

  • Use dataSet.WriteXml, you just do so in the v. simple class ... as a one liner, you don't need to do add a focused integration test for it ... but if u l8r on store say in an external system, u might choose to do so in that other implementation, without affecting the other code
  • Use separate classes for a dataset exporter/writter than for a simple file writer.

As long as you are using v. simple dependency injection, you are really just keeping everything simple ... which will get you better results both in the short and in the long run.

Ps. not hitting the file system during tests is important, as when your number of tests grow you want them to run v. fast, so don't be misguided to what appears at first "quick" accesses for a test

eglasius
Looks like extra types are unavoidable :)
Dev
They are not evil, do it. Go back later on and compare with any old code you have around and you will see is simpler. Also as was suggested read about SOLID. This ebook is also some good info / examples on it: lostechies.com/content/pablo_ebook.aspx/… ps. upvotes are welcomed :)
eglasius
Wow, cool book. I really like example of SRP with the Form and ListView populated from xml file, it shows exactly what makes me feel lost - several lines of simple code were refactored to 12(!) types.
Dev
@Dev - it can seem like overkill on small or simple projects whose requirements are not likely to change a lot. From your original post (linked above) its sounds like your application fits that profile - "a simple utility that allows to import data from a file with special format to the database." But when you're dealing with volatile requirements and a large, complex domain, those numerous tiny classes that only do one thing are a godsend. They're easy to understand (in isolation), change, and test (though the interaction *between* the classes can get more complex).
Jeff Sternal
@Jeff - I've already added 6 tiny classes and interfaces to my application and there will be more. So far I like the changes, though it's not so easy to find the right balance.
Dev
+1 on creating purpose-driven classes as needed. It took me a while to figure out, but having responsibilities clearly divided make testing and changes much easier.
Grant Palin