Using Playwright E2E Tests With C# .NET

This is a post in a series about the automated E2E testing framework Playwright. While you can start anywhere, it’s always best to start right at the beginning!

Part 1 – Intro
Part 2 – Trace Viewer

These days, end to end browser testing is a pretty standard practice amongst modern development teams. Whether that’s with Selenium, WebDriver.IO or Cypress, realistically as long as you are getting the tests up and running, you’re doing well.

Over the past couple of years, Cypress had become a defacto end to end testing framework. I don’t think in the last 5 years I’ve been at a company that hasn’t atleast given it a try and built out some proof of concepts. And look, I like Cypress, but after some time I started getting irritated with a few caveats (Many of which are listed by Cypress themselves here).

Notably :

• The “same origin” URL limitation (Essentially you must be on the same root domain for the entire test) is infuriating when many web applications run some form of SSO/OAuth, even if using something like Auth0 or Azure AD B2C. So you’re almost dead in the water immediately.
• Cypress does not handle multiple tabs
• Cypress cannot run multiple browsers at the same time (So testing some form of two way communication between two browsers is impossible)
• The “Promise” model and chaining of steps in a test seemed ludicrously unwieldy. And when trying to get more junior team members to write tests, things quickly entered into a “Pyramid of doom“.

As I’ll talk about later in another post, the biggest thing was that we wanted a simple model for users writing tests in Gherkin type BDD language. We just weren’t getting that with Cypress and while I’m sure people will tell me all the great things Cypress can do, I went out looking for an alternative.

I came across Playwright, a cross platform, cross browser automation testing tool that did exactly what it says on the tin with no extras. Given my list of issues above with Cypress, I did have to laugh that this is a very prominent quote on their homepage :

Multiple everything. Test scenarios that span multiple tabs, multiple origins and multiple users. Create scenarios with different contexts for different users and run them against your server, all in one test.

They definitely know which audience they are playing up to.

Playwright has support for tests written in NodeJS, Java, Python and of course, C# .NET. So let’s take a look at the latter and how much work it takes to get up and running.

For an example app, let’s assume that we are going to write a test that has the following test scenario :

Given I am on https://www.google.com
When I type dotnetcoretutorials.com into the search box
And I press the button with the text "Google Search"
Then the first result is domain dotnetcoretutorials.com

Obviously this is a terrible example of a test as the result might not always be the same! But I wanted to just show a little bit of a simple test to get things going.

Let’s get cracking on a C# test to execute this!

Now the thing with Playwright is, it’s actually just a C# library. There isn’t some magical tooling that you have to download or extensions to Visual Studio that you need to get everything working nicely. You can write everything as if you were writing a simple C# unit test.

For this example, let’s just create a simple MSTest project in Visual Studio. You can of course create a test project with NUnit, XUnit or any other testing framework you want and it’s all going to work much the same.

Next, let’s add the PlayWright nuget package with the following command in our Package Manager Console. Because we are using MSTest, let’s add the MSTest specific Nuget package as this has a few helpers that speed things up in the future (Realistically, you don’t actually need this and can install Microsoft.Playwright if you wish)

Install-Package Microsoft.Playwright.MSTest

Now here’s my test. I’m going to dump it all here and then walk through a little bit on how it works.

[TestClass]
public class MyUnitTests : PageTest
{
    [TestMethod]
    public async Task WhenDotNetCoreTutorialsSearchedOnGoogle_FirstResultIsDomainDotNetCoreTutorialsDotCom()
    {

        //Given I am on https://www.google.com
        await Page.GotoAsync("https://www.google.com");

        //When I type dotnetcoretutorials.com into the search box
        await Page.FillAsync("[title='Search']", "dotnetcoretutorials.com");

        //And I press the button with the text "Google Search"
        await Page.ClickAsync("[value='Google Search'] >> nth=1");

        //Then the first result is domain dotnetcoretutorials.com
        var firstResult = await Page.Locator("//cite >> nth=0").InnerTextAsync();
        Assert.AreEqual("https://dotnetcoretutorials.com", firstResult);
    }
}

Here’s some things you may notice!

First, our unit test class inherits from “PageTest” like so :

public class MyUnitTests : PageTest

Why? Well because the PlayWright.MSTest package contains code to set up and tear down browser objects for us (And it also handles concurrent tests very nicely). If we didn’t use this package, either because we are using a different test framework or we want more control, the set up code would look something like :

IPage Page;

[TestInitialize]
public async Task TestInitialize()
{
    var playwright = await Playwright.CreateAsync();
    var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
    {
        Headless = false
    });
    Page = await browser.NewPageAsync();
}

So it’s not the end of the world, but it’s nice that the framework can handle it for us!

Next what you’ll notice is that there are no timeouts *and* all methods are async. By timeouts, what I mean is the bane of every selenium developers existence is “waiting” for things to show up on screen, especially in javascript heavy web apps.

For example, take these two calls one after the other :

//Given I am on https://www.google.com
await Page.GotoAsync("https://www.google.com");

//When I type dotnetcoretutorials.com into the search box
await Page.FillAsync("[title='Search']", "dotnetcoretutorials.com");

In other frameworks we might have to :

• Add some sort of arbitrary delay after the GoTo call to wait for the page to properly load
• Write some code to check if a particular element is on screen before continuing (Like a WaitUntil type call)
• Write some custom code for our Fill method that will poll or retry until we can find that element and type

Instead, Playwright handles that all under the hood for you and assumes that when you want to fill a textbox, that eventually it’s going to show and so it will wait till it does. The fact that everything is async also means it’s non-blocking, which is great if you are using playwright locally since it’s not gonna freeze everything on your screen for seconds at a time!

The rest of the test should be pretty self explanatory, we are using some typical selectors to fill out the google search and find the top result, and our Assert is from our own framework. Playwright does come packaged with it’s own assertion framework, but you don’t have to use it if you don’t want to!

And.. That’s it!

There are some extremely nifty tools that come packaged with Playwright that I’m going to write about in the coming days, including the ability to wire up with Specflow for some BDD goodness. What I will say so far is that I like the fact that Playwright has hit the right balance between being an automation test framework *and* being able to do plain browser automation (For example to take a screenshot of a web page). Cypress clearly leans on the testing side, and Selenium I feel often doesn’t feel like a testing framework as much as it feels like a scripting framework that you can jam into your tests. So far, so good!

Next up, I wanted to take a look at the Playwright inbuilt “Trace Viewer”, check out that post here : https://dotnetcoretutorials.com/2022/05/24/using-playwright-e2e-tests-with-c-net-part-2-trace-viewer/