An Inverted Test Pyramid

I believe that most of us are familiar with the concept of "Test Pyramid", a simple heuristic originally described by Mike Cohn, which states a project should aim rather to have a larger number of unit tests instead of end-to-end tests. In fact I’ve been working on a project suffering from an inverted pyramid, and I think it’s worth mentioning some of the downsides that relying solely on end-to-end tests might have.

Don't slow me down

If we take developing a web app as an example, then we might end up using something like Selenium to build UI tests. Even with tools or frameworks like Capybara, or SitePrism if you want to use page objects, that make building these kind of tests more accessible, they still are usually harder and take longer to write than unit tests. Moreover, they tend to be brittle; when new features are added or UI elements change slightly, they may break, meaning more time and work spent onto them.
They are also much slower to run, and as the number of tests grows, so does the time it takes to run the whole test suite, slowing everything down further. Of course, this is expected, but you will reach bothersome run-times sooner when you have a larger number of UI tests.
All of this adds up to a more frustrating working environment for the developer, and also to a slower development process, which translates into higher development costs.

Lack of feedback

Apart from the disadvantages already mentioned, I feel the biggest drawback is the lack of feedback the developer gets. Unit tests are usually self-contained, so when one fails, it leads the developer straight to the problem. On the contrary, when a UI test fails, it will probably need some tracking down to be done before finally figuring out what needs to be fixed. So if a UI test fails without a unit test failing alongside, it’ll probably mean there is one missing.
As Roman mentions on his post "Tests: Paving Our Way", tests not only help us achieve better designs and abstractions, they also double up as documentation for our code. Unit testing helps document different concepts in the domain we are working on, and more importantly, how these concepts are represented in our code. On the other hand, end-to-end testing only tells us how a specific user might interact with the system in a specific use case scenario, which is important but not as useful for the developer, especially new team members who might not be familiar with the domain or code base.

Final thoughts

I’m not saying that end-to-end testing should be avoided completely as it helps catching errors that might not be detected otherwise. They are especially important in testing critical use case scenarios. Let’s say you have a use case where an error might cost you thousands of dollars over the time it takes for it to be detected and fixed, of course you would want this tested as thoroughly as possible, and end-to-end testing provides a mandatory additional level of protection.
Nevertheless, it’s important to keep in mind they test different things and their purpose is different. We should take into account their higher cost when evaluating how to test the system we are working on. The value they add should balance the time and effort that goes into them. Finally, we should also never forget unit tests even if there are higher level tests that cover the same code.