VIDEO TRANSCRIPTION
AI Jason
Cursor + Browser control = Self improving coding agent
Description generated by Skrybot
The user has been utilizing Playwright MCP for automated testing. Playwright allows simulating browser interactions for testing web applications. With Playwright MCP, a cursor agent can access the browser, identify UI bugs, and self-iterate to improve the application. The user demonstrated how to set up Playwright MCP, create UI tests, and automate testing workflows. Additionally, the user mentioned utilizing AI models and prompting techniques from a free playbook by Mindstream. The process involves setting up Playwright MCP, creating specific configurations, and running automated UI tests for tasks like login authentication and adding to-do items. The user highlighted the ability to create reusable UI tests and automate testing processes using Playwright MCP.
Transcription generated by Skrybot
I've been using Playwright MCP for the past few days. To get Cursor self-improve the UI and application and become my automated QA assistant that can run sophisticated testings by itself. So today I want to show you what my workflow is and how can you adopt it for your own project. Nowadays, you normally hear Playwright for those browser agents, but initially it is introduced to do end-to-end testing for web applications.
It can simulate browser interaction like text input, mouse click, scrolling the page and extract data so that we can actually write some scripts to simulate how a real user is going to interact with the website for example i might want to build a test for the normal sign up and checkout flow for the ai builder club where a user should be able to click a button type in the email address joining receive the code and log into the platform to do that i can just write a test script with playwright where it can open the browser go to the website url click on the sign up button typing the email address and clicking on the button In the end, we can validate if the user actually received the code to log in.
And this test can be run every time before I try to push some change to the production code so it can capture the error before it goes to production. And this all sounds great, but building tests is not exactly the fun part of building software. To build this kind of testing script, you normally need to build very specific selectors to be able to identify UI elements by using CSS or XPath, even though there are tools like CodeGen.
which is a feature from playwright that allow you to do a command line like this which will open up the browser as well as the recorder on the right side so that you can just click through the ui flow like how a real user gonna do and on the right side it will automatically capture and generating the testing code for you based on the action you've taken but still it is quite a manual process and if you change the elements or layout of the web location this task gonna break and you need to update your test again but playwright released its own mcp it basically gave cursor agent access to the browser and that means a few things one is that instead of you defining very static tests yourself you can get a cursor to use playwright to run the browser and become your qa tester on the other side because cursor now has access to the web browser it can see what is going on on the web page identify any ui bugs or flaw in the flow and self iterate until the application works exactly like what expected and this to me is such a game changer and open up so much possibilities You can even turn cursor into a general web scraper where you can open the browser, take a snapshot of content available on the page, and do complex interaction with the web page to get the specific information that you want.
So today I'm going to tell you what are different ways you can utilize Playwright MCP to do automated testing and get a cursor to self-fixing the issue. But before we dive into that, I know many of you are building AI applications or integrating AI into your own business. And choosing the right model, prompting techniques is often a challenging task. Since there are so many new stuff every month, that's why I want to introduce you to this free playbook from Mindstream. It breaks down everything from which AI model is best for which task to advanced prompting frameworks like GRWC that can boost your results up to 55%. It features compared top models like GPT 4. 5, Cloud 3.
7, Gorg 3, plus practical tips on using specialized model for business tasks like how Cloud 3. 7 Sonnet is best for thoughtful analysis across long documents. while make sure large focus on mass problems. So no more guessing work about which model fits your specific task best. And what I love the most is their advanced prompting frameworks and power techniques for better prompts. Those are exact strategies that I use when working with AI to create better code and automate testing workflows. Those frameworks and techniques transform media work outputs into exceptional ones and can be applied for scenarios like prompting Tertiary to use PlayWrite MCP to do automated testing better. And you can download this playbook for free.
I have put a link in the description below so you can click and check out. And now let's get back to how you can use Playwright MCP. Firstly, let's set up the Playwright MCP. So to use Playwright MCP, you actually can't use the latest version for now because the latest version somehow has some issues. I will go to the npm package. The last version that I found quite stable is this 0. 0. 2. So if you go to cursor, add MCP, this is the setup I will have. So Playwright and run mpx at playwright mcp at 0. 0.
2 and there are also some additional configurations i have here one is this dash dash config to a specific configuration file that i created here so play right allow you to define configuration file that including all the different settings you have like which type of browser it should be using what a device it should simulate the user profile and whether it should be using vision stuff like that i specifically use this because i have a few different settings that are often used to making sure it can go through certain website that has anti-bot setup like those special arguments the user agent initial scripts if you're interested you can join the ai build club i have specific content that dive deep into how to handle anti-bot setup as well as websites that require login and authentications I have put the link in the description below for you to join.
So for this specific case, I will create this config file and do dash dash config and point to this specific file. I also do dash dash vision to turn on the vision ability so that cursor can actually see what's going on the web page. And I will actually create two different MCP servers, one with vision, one without vision. That's because vision model is particularly good at certain scenarios, like when you wanted to view what's going on the page and iterate the UI.
but it is less good at allocating specific position to click or ui elements to interact with and that is quite essential when you want playwright to do the testing so i will have another mcp called playwright test which don't have the vision model and now when i go back to mcp i can see there are two mcps added at any given time i will just turn on one of them and firstly i will use the one with vision model to show you how can you use playwright to run the application look at ui and self-iterate to improve the interface so here we have a basic to-do app the ui individual component looks actually pretty good but all together just something looks a bit off this is a very common scenario when you do ai coding it's that 20 or 30 percent missing that makes the ui just not as sleek so i'm going to give prompt please use playwright mcp to view the ui identify area to improve for ui and iterate it until it looks perfect and here just put a small text here make sure the width of browser in mcp to be 700 pixel.
So it's easier for me to just show you guys side by side. So it will open the browser and resize to be smaller one as you can see here. Then it will do this browser screen capture to look at what's on the UI here. Then it start doing the iterations. The first thing it will do is try to refine the car spacing and input field. So spacing does look a little bit better, especially around this part of UI. So you change to something like this, which does actually looks better. And the last bit is that the task item background and spacing could be adjusted.
for even more notion style and we could improve alignment cool so after a few rounds of iteration and feedback the ui is a lot more clean now much better than what we have before and this is all thanks to playwright mcp ability to look at the screenshots directly and make more accurate iterations next let's talk about how to use playwright mcp to do automated ui testing let's assume i just turn this to do app into a fully functional application that requires some login and once people log in they can start adding to do tasks like prepare the scripts assume the testing we want to do is we want to making sure the login authentication works and we also want to test whether users can add tasks successfully and meanwhile i also add this cursor rules so i'll give a prompt we have a simple to-do app set up in next.
js now let's test application using the playwright mcp firstly let's test if you can log in successfully and then test if users can successfully add to do and mark things as completed so you can see if firstly try to inject the rules that we just written here which is that two specific things one is that making sure you always resize screen to this width so it's easier for us to view and also the select tab index start from one instead of zero so this is one kind of problem i often see they have for some reason you keep failing to focus on the specific input and i think this is because we are using the vision which is great for ui iterations but not as great for doing the tests.
I'm going to disable the original one but use the playwright test instead and just to making sure I'll also restart the cursor and it successfully put in the email and password sign in and then start testing adding the tasks change the label click add and the task has been added and also check if the show complete function works as well great this you can see it's pretty sophisticated it did pretty complex test flow what's really really cool is that obviously this test is really smart but it does consume token and credits every time because it is going through large learning model to make it reusable every time i think there are two approaches one is that since this prompt already works you can create a folder called tests and then just save all those test prompts and every time it just goes through this flow to do the tests as an ai agent but obviously the downside of that is that it does cost money so what you can also do is that give it prompt after it successfully complete the flow that you expected and then say now let's create a reusable playwright ui test based on the flow above so i can run as automated test every time so it will install playwright tests set up the playwright configuration which will store the information about which url to go what's the browser it should be using for testing and then it created ui tests that we can use and now you can see it write a bunch of different tests for testing logging and adding tasks complete tasks and then it tried to run the task itself which it passed but three of them failed I can just prompt it to continue which will reflect what are the errors in the test and update the test here.
Now all the tests have been written for your ToDo app and it also has a readme that including how to run those tests.
It even created a GitHub action that if you push this repo to GitHub, every time when you want to merge some new change to your GitHub repo, it will have to go through those tests and only after those tests are passed then you will be able to push the production which will make the whole process fairly automated and all you need to do is really just to open the terminal if you want to test manually all you need to do is just run this NPM run test end-to-end and it will just automatically run through all those tasks and tell you if they're passed or not So this is how you can use Playwright MCP to do more AI driven tests either using large range model agent or let it articulate specific testing scripts that you can use for any new change that you are trying to push in production If you're interested, you can join the AI Builder Club where I have a step-by-step setup guide for everything I show in this video, a more in-depth course of how to best set up to avoid anti-bot, as well as websites that require login and authentication, alongside with many other in-depth content from either me or other industry experts I talk to around AI coding and building large-language model applications in production.
We also provide premium tools that will really boost up your AI coding process, like a platform that converts your ideas into a product requirement dot that can steer AI coding agent better. as well as LaunchKit that already set up authentication, Superbase, and Stripe. But most importantly, you will be joining a community of top AI builders who are launching and building their own AI startups, who might already have the answer to the challenge that you are facing today. I have put a link in the description below for you to join. I hope you enjoyed this video. Thank you, and I'll see you next time. .
S K R Y B O T
Skrybot. 2023, Video transcriptions from YouTube
By visiting or using our website, you agree that our website or the websites of our partners may use cookies to store information for the purpose of delivering better, faster, and more secure services, as well as for marketing purposes.