C# vs JavaScript for Selenium tests

A few weeks ago I had a choice to be made and this choice was not easy for me. Here I would like to share my decision and some reasoning behind. I have decided to add some automated GUI testing to the project I am working on. This is a web application and therefore, the choice of the testing tool was simple – Selenium. However, I could not decide which language to use for coding the tests and testing framework. The thing is that I am currently bilingual guy (as probably every non-Node web developer) and I regularly use both C# and JavaScript. So I started coding first tests and inception of the test framework in both. After few days of coding using each language, I have finally chosen C# over JavaScript. Let me tell why and probably my reasoning will help someone else to make a decision.

I have started with JavaScript (Node, Jasmine, WebDriverJs, VS Code). Well, because:

  • Traditionally people write tests in scripting languages, which should add to the productivity and code simplicity
  • Node is true cross platform technology and if needed I would be able to run tests on any platform I like
  • I wanted to learn how to code for Node (doing modules and stuff)

At first, everything went well and smoothly, but then problems started to creep in. The root of these problems was JavaScript’s Selenium library asynchronous nature while tests are synchronous by nature. Test steps must execute sequentially because order matters. Perform “Save” button click before the form is filled and the test is miserably failed. Here are the problems:

  • Whole WebDriverJS API is asynchronous and every command results in a promise! Promise! Well, they are meant to be a simplification of callbacks, but they are still callbacks and for a sequential guy like me, haveing a promise after every API call is too much. To simulate synchronous test execution they use the ControlFlow library, which implicitly synchronizes asynchronous actions, making it so you only have to register a promise callback when you want to catch an error or access a return value. The brain just explodes. But it is just testing! It should be simple to be valuable. Resulted code with promises just promised to become maintenance nightmare;

  • Sometimes tests did not work as expected and I had to debug. You know, just put a breakpoint in the place where some call to web driver API happens. Like here, say, right before clicking the button:

But you know what? When execution stops at the breakpoint it is basically the middle of registration of callbacks. You are stopped even before Selenium started to search for “inputEmail”! I just can’t imagine where to put a breakpoint to have the test stop right before clicking the button (probably there is some way to do it but I don’t know it)

  • In the test itself, I did not want to use any variables for simplicity sake. So that business analyst or QA guy could easily understand what happens in the test. However because of
    promises I just had to introduce a variable and complexity, because there was no way to ensure that test assertion happened in the very end of the test:

While the code is probably not very complex, but it is definitely not for non-programmers.

  • Since Javascript is dynamic and WebDriverJS API is also dynamic, while coding you should constantly look at documentation or run debugger to see what kind of methods are available and how to call them correctly

After some fight, I gave up and started to do the same with C#. Wow! It was a just different world, all these problems were gone. Static typing gave fantastic auto-completion, debugging was just great! I could stop the test at any place and then go through it step by step observing the effects in the browser. Because there was no need to deal with promises while asserting, I was able to write test without variables:

Conclusion: sometimes your language and its library are not suitable for the task. Although I really like JavaScript (good parts) and it shines in the browser it turned out it is not the best option for doing test automation. Did I discover Atwood’s Law break? 🙂

Atwood’s Law: any application that can be written in JavaScript, will eventually be written in JavaScript.

Facebooktwittergoogle_pluslinkedinmailFacebooktwittergoogle_pluslinkedinmail
4 Comments

    Adam

    Thank you for this post. As a C# developer in a new node.js shop, trying to write end-to-end synchronous tests using asynchronous javascript (Protractor, Jasmine, Selenium, node.js) has made long for my days with C#. I totally agree. I want to be able to focus on writing tests, not getting hung up on the syntax of writing tests with precious little debugging ability. Plus, if I was using C#, I would be able to use the power of generics, enums, overloaded test methods etc in C# to do all sorts of better test-code organization. And, when the UI we are testing changes, as it does constantly, refactoring our strongly typed tests in C# would be a breeze and with much fewer errors and pain than refactoring Javascript. Glad to hear a kindred spirit. There is zero chance the shop I’m at will consider C# so I’m stuck making the best of it. Thankfully, with the Visual Studio Code free IDE and using ES6 class syntax which is natively understood by Visual Studio Code, I can mimic an OOP approach to testing, and Jasmine 2 expectations handle the Promises portion a little easier, but still, it isn’t C#

      OleksandrNechai

      Yeah, Adam… all that is unfortunate. But today, I believe, you could alleviate this problem with TypeScript, which now has more interesting type system than C# + async await. Still, promices but feels like sequential flow.

    Misconceptions and Facts: Lies and Truth About the Business of Modeling

    Misconceptions and Facts: Lies and Truth About the Business of Modeling

    The truth of the web business market is greater than it appears to be. Let’s see it?

    Andriy

    I would say I simply love automating with Protractor+Jasmine+TypeScript. I think this is the best solution for Angular. Protractor knows how to correctly wait for Angular, TypeScript gives us types, enums, etc. Jasmine has nice and clear test structure, expects. Stuff that can’t be achieved with Selenium can be done with nodejs. Overall the only problems we may have to deal with are promises but if handled correctly with then we are good to go.

Leave a Reply to Adam Cancel reply

Your email address will not be published.