This is a post in a series about the automated E2E testing framework Playwright. While you can start anywhere, it’s always best to start right at the beginning!
Part 1 – Intro
Part 2 – Trace Viewer
A massive selling point in using a automated test runner such as Cypress.io, is that it can record videos and take screenshots of all of your test runs right out of the box. It comes with a pretty nifty viewer too that allows you to group tests by failures, and then view the actual video of the test being executed.
If we compare that to Selenium, I mean.. Selenium simply does not have that feature. It’s not to say that it can’t be done, you just don’t get it out of the box. In most cases, I’ve had automation testers simply take a screenshot on test failure and that’s it. Often the final screenshot of a failed step is enough to debug what went wrong, but not always. Additionally, there is no inbuilt tool for “viewing” these screenshots, and while MS Paint is enough to open a simple image file, it can get confusing managing a bunch of screenshots in your downloads folder!
Playwright is somewhere in the middle of these. While it doesn’t record actual videos, it can take screenshots of each step along the way taking before and after shots, and it provides a great viewer to pinpoint exactly went wrong. While it works out of the box, there is a tiny bit of configuration required.
I’m going to use our example from our previous post which uses MSTest, and add to it a little. However, the steps are largely the same if you are using another testing framework or no framework at all.
The full “traceable” MSTest code looks like so :
[TestClass]
public class MyUnitTests : PageTest
{
[TestInitialize]
public async Task TestSetup()
{
await Context.Tracing.StartAsync(new TracingStartOptions
{
Title = TestContext.TestName, //Note this is for MSTest only.
Screenshots = true,
Snapshots = true,
Sources = true
});
}
[TestCleanup]
public async Task TestCleanup()
{
await Context.Tracing.StopAsync(new TracingStopOptions
{
Path = TestContext.TestName + ".zip"
});
}
[TestMethod]
public async Task WhenDotNetCoreTutorialsSearchedOnGoogle_FirstResultIsDomainDotNetCoreTutorialsDotCom()
{
//Our Test Code here. Removed for brevity.
}
}
Quite simply :
- Our TestInitialize method kicks off the tracing for us. The “TestContext” object is a MSTest specific class that can tell us which test is under execution, you can swap this out for a similar class in your test framework or just put any old string in there.
- Our TestCleanup essentially ends the trace, storing the results in a .zip file.
And that’s it!
In our bin folder, there will now be a zip file for each of our tests. Should one fail, we can go in here and retrieve the zip. Unlike Cypress, there isn’t an all encompassing viewer where we can group tests, their results and videos. This is because Playwright for .NET is relying a bit on both MSTest and Visual Studio to be test runners, and so there is a bit of a break in tooling when you then want to view traces, but it’s not that much leg work.
Let’s say our test broke, and we have retrieved the zip. What do we do with it? While you can download a trace viewer locally, I prefer to use Playwright’s hosted version right here https://trace.playwright.dev/
We simply drag and drop our zip file and tada!
I know that’s a lot to take in, so let’s walk through it bit by bit!
Along the top of the page, we have the timeline. This tells us over time how our test ran and the screenshots for each time period. The color coding tells us when actions changed/occurred so we can immediately jump to a certain point in our test.
To the left of the screen, we have our executed steps, we can click on these to immediately jump in the timeline.
In the centre of the page we have our screenshot. But importantly we can switch tabs for a “Before” and “After” view. This is insanely handy when inputting text or filling out other page inputs. Imagine that the test is trying to fill out a very large form, and on the very first step it doesn’t fill in a textbox correctly. The test may not fail until the form is submitted and validation occurs, but in the screenshot of the failure, we may not even be able to see the textbox itself. So this gives us a step by step view of every step occurring as it happens.
To the right of the screen you’ve got a bunch of debug information including Playwright’s debug output, the console output of the browser, the network output of the browser (Similar to Chrome/Firefox dev tools), but importantly, you’ve also got a snapshot of your own code and which step is running. For instance, here I am looking at the step to fill a textbox.
This is *insanely* helpful. If a test fails, we can essentially know exactly where in our own code it was up to and what it was trying to do, without thinking “Well, it was trying to type text into a textbox, let me try and find where that happens in my code”.
And that’s the Playwright Test Trace Viewer. Is it as good as Cypress’ offering? Probably not quite yet. I would love to see some ability to capture a single zip for an entire test run, not per test case (And if I’ve missed that, please let me know!), but for debugging a single test failure, I think the trace viewer is crazy powerful and yet another reason to give Playwright a try if you’re currently dabbling with Selenium.
