GPXReader - Clearing TDD Tests

Introduction

As you might probably read at yesterdays article I've finally found how I shoud follow TDD principle and "flow".

It basically means, I will not create a bunch of tests, but instead I'll create a bunch of not-implemented methods/functions that are documented about their responsibilities but has no code inside of them.

After that I'll start working on those methods with a TDD.

Prerequisites - Clearing up pre-existing tests.

1. Why your unittests should be fast?

I'll make an example of how my testing failed in real-life example. I was working on a python project. In that project I didn't follow the TDD rules, but instead, once the "feature" has been created I created tests that "covered" expected behaviour of source-code.

These tests were not time-optimized, they used real-time data and needed access to internet service.

That made them a lot time consuming. It also pushed me to not run all of the "unit-tests" but only those that I was working currently, and then before "pushing" work of the tests, check if other tests are working properly.

Now as I recall it was not such a bad thing, since I didn't use TDD properly and source-code of other features was not changed in some new "features".

But in some occasional cases I needed to change behaviour of features that had already test-coverage, and then I needed to run all tests that took a bit of time.

Then, instead of working on tests I was waiting actually until tests will complete with success or failure. To make a more expressive example, please check following:

  • Each run of all tests took around 120 seconds (2 minutes) (yeah .... SECONDS... :( ) - in earlier phase of project, at the end of project it was close to 300 seconds (5 minutes... )

  • Each feature per day might took at least 5 runs of tests-checking to find if any regression is not on source-code.

This takes 120 * 5 = 600 seconds = 10 minutes of (waste? ) time (in early phase of project)

In the end it took: 300 * 5 = 1500 seconds = 25 minutes of overall interrupted idle-waiting periods.

Of course I took this time to re-read source-code, but either way I was in a "waiting" stage.

It's an optimistic prognosis. What If your 8h work day makes about 20-25 changes and each change you need to run all tests ?

Easy to measure : 25 * 300 = 125 minutes! = more then 2 hours of (wasted?) time.

2. What I've found out in code?

In my GPXReader Tests I've found a bunch of tests that were not implemented. Those were just information for myself to speed-up development with TDD and pomodoro-technique.

I've figure-out that this information is more-less self-explanatory and It's more a documentation for source-code than test-documetation.

Check below how it look like before changes :

Link to full source-code at github

class BaseGPXReaderTest(object):
    """
    A Prof Of Concept for creating one generalized GPXReader test class.
    It will then be inherited by other classes that will use TrackPoints example data,
    Route Points data and Way Points data using pytest-fixtures.
    """

    def test_get_points(self):
        """
        Tests if there will be data output from get_points.

        Test if:
        - points exists for different types.
        - what will happen if types exists, but no data?
        """
        raise NotImplementedError()

    def test_get_elevations(self):
        """
        Tests if all elevations will be returned from input element.
        Test should check if elements are TrackPoints Route Points or Way Points.
        """
        raise NotImplementedError()

    def test_get_elevations_len(self, elevations_len):
        """
        Test if proper amount of elevations are read by get_elevations method.
        """
        raise NotImplementedError()

    def test_get_track_points(self):
        """
        Test if all track points are gathered
        """
        raise NotImplementedError()

    def test_get_route_points(self):
        """
        Test if all route points are gathered
        """
        raise NotImplementedError()

    def test_get_way_points(self):
        """
        Test if all way points are gathered
        """
        raise NotImplementedError()

    def test_get_lowest_elevation(self):
        """
        Checks method for:
        - If no elevations in route, method should return None.
        - If elevations are equal or only one available, return None.
        - If more then one elevation (different) available, return lowest.
        """
        raise NotImplementedError()

    def test_get_highest_elevation(self):
        """
        Checks method for:
        - If no elevations in route, method should return None.
        - If elevations are equal or only one available, return None.
        - If more then one elevation (different) available, return highest.
        """
        raise NotImplementedError()

    def test_animal_figure_route(self):
        """
        Tests if method named "animal_figure_route" will return proper name of animal
        for proper biking route.
        """
        raise NotImplementedError()

class AbstractGPXReaderTest(BaseGPXReaderTest):
    """
    Abstract Implementation of Base GPX Reader Test
    """

    def test_get_points(self):
        """
        Tests if there will be data output from get_points.

        Test if:
        - points exists for different types.
        - what will happen if types exists, but no data?
        """
        points = []
        for point in gpxreader().get_points():
            points.append(point)
        assert len(points) > 0

    def test_get_elevations(self):
        """
        Tests if all elevations will be returned from input element.
        Test should check if elements are TrackPoints Route Points or Way Points.
        """
        elevations = gpxreader().get_elevations()
        print len(elevations)
        assert len(elevations) > 0

    def test_get_elevations_len(self, elevations_len):
        """
        Test if proper amount of elevations are read by get_elevations method.
        """
        elevations = gpxreader().get_elevations()
        assert len(elevations) == elevations_len

    def test_animal_figure_route(self):
        """
        Test implementation for animal figure route
        """
        raise NotImplementedError()

@pytest.mark.usefixtures("gpxreader")
class TestTrackPointGPXReader(AbstractGPXReaderTest):
    """
    Specific version of Abstract test-class that uses mainly TrackPoints gpx file for tests.
    """
    pass

3. What changes I've made to make this more clear?

So I've change this source-code and ripped it apart.

I've changed the BaseGPXReaderTest with functions into a Abstract class for GPXReader with functions that needs to be implemented:

Link to full source-code of this file at github

class AbstractGPXReader(object):
    """
    AbstractGPXReader - An Abstract Class containing most of methods that needs to be implemented.

    I.e.
    get_track_points(self, item_nb=0):
    """

    def get_track_points(self, item_nb=0):
        """
        Track points are gathered
        """
        raise NotImplementedError()

    def get_route_points(self, item_nb=0):
        """
        Route points are gathered
        """
        raise NotImplementedError()

    def get_way_points(self, item_nb=0):
        """
        Way points are gathered
        """
        raise NotImplementedError()

    def get_lowest_elevation(self):
        """
        Returns information about lowest elevation from route
        Returns None if not found any elevations!

        Checks method for:
        - If no elevations in route, method should return None.
        - If elevations are equal or only one available, return None.
        - If more then one elevation (different) available, return lowest.
        """
        raise NotImplementedError()

    def get_highest_elevation(self):
        """
        Checks method for:
        - If no elevations in route, method should return None.
        - If elevations are equal or only one available, return None.
        - If more then one elevation (different) available, return highest.
        """
        raise NotImplementedError()

    def animal_figure_route(self):
        """
        Uses Points to compare route with animal shapes.
        Returns information if this route looks similar to some animal or not.
        Returns Name of animal from enumeration or None if none animal found.

        Tests if method named "animal_figure_route" will return proper name of animal
        for proper biking route.
        """
        return NotImplementedError()

Final result

For now final result does not look "much" better. Instead it's a fast implementation! :)

It follows TDD technique in at least point of "fast-unit-tests".

To make unit-tests faster for now, I"ve changed the .gpx example file smaller and faster to run.

I've also find that the Makefile is not best optimized in terms of timing.

Makefile contained pip install -r requirements additionally at the "make unittest" flow , that is not needed at all if you already have pip-installed all dependency.

So changes I've made in Makefile:

unittest_all: prepare_db unittest

unittest:
    cd bikingendorphines && python manage.py test

Now all I have to do is follow TDD technique and make tests for our GPXReader.

Code commits done for this post:

Tools and applications used:

Acknowledgements

  • Tomatoid - a Pomodoro-Technique web-based(and not only) web-service.

Accomplished:

1. Add even more tests to GPXReader.

Created additional markdown_commits script that transforms git-log into Markdown links :) check HERE

What's next

2. Add test-cases for business logic to GPXReader - endorphines-algorithms :)

3. Python backend - initial REST-API.

4. Making API documentation - research for best API documentation!

Comments

comments powered by Disqus