Selenium Grid 4 Distributed Setup Tutorial and Guide
After reading numerous articles and watching countless repeated tutorials to understand the latest of Selenium Grid, I think it's time to present the setup in a quick & efficient manner.
The below-mentioned information is not easy to find unless you decide to spend 60–70 hrs of research just to connect some missing pieces present in the official documentation here.
YES! This means we should start contributing to the above link or you can start by liking & commenting on this article below to spread love & learning.
With that said,
“Yeah, here we go for the hundredth time
Hand grenade pins in every line
Throw ’em up and let something shine!”
Step 1: Getting Started
Read the official documentation to know about Selenium Grid 4 and what is new in it. How does it differ from the old version?
Step 2: Pre-requisite checks
Now, that you know what is Selenium Grid 4, it's time to do a hands-on setup. I will be sharing steps directly about the distributed setup and not about the standalone or hub way because that is something pretty straightforward.
Download. Run. Observe.
https://www.selenium.dev/documentation/grid/getting_started/
Before proceeding to the below-mentioned steps, in this step you need to evaluate
a. Whether you even need a distributed setup?
b. Do you even have sufficient resources(pre-requisite) to run a distributed setup (2 VMs on any of the cloud(AWS/GoogleCloud/DigitalOcean))
If the answer to both of the above is a YES, go ahead.
Step 3: Downloading necessary files
Download the latest JAR file from here and upload them to both the Virtual Machines you are going to use for setting up distributed selenium grid.
Alternatively, you can also run to download via command line
wget https://github.com/SeleniumHQ/selenium/releases/download/selenium-4.0.0/selenium-server-4.0.0.jarmv selenium-server-4.0.0.jar se4.jar // Renaming for ease of run
Step 4: Managing & storing JARs
As a matter of practice, I would recommend moving this download jar to somewhere accessible by multiple users. e.g. /opt directory
cd /opt
mkdir /selWeb
mv /Downloads/selenium-server-4.0.0.jar .Do this in both the machines.
Step 5: Naming the VMs
For the sake of understanding, let's name the two VMs as
Machine1: DisMaster
Machine2: DisNode
As per the nomenclature, we will be setting up all queues and the router on DisMaster, whereas DisNode will act as the node responsible for running actual tests.
Step 6: Port Awareness & Relaxation
In distributed Selenium Grid Setup, we need to set up two-way communication between DisMaster and DisNode on multiple ports. This calls out for relaxation of certain ports for incoming & outgoing traffic.
For the starters,
a. Enable/allow incoming connections from DisNode on port numbers 4442, 4443 on the DisMaster
b. Enable/allow incoming connections for the port ranging from 5550 to 5559 on both DisMaster, DisNode.
c. Enable incoming connections on the grid URL node of your choice. It can be any port, and the default is 4444. In our examples, we will use 5550 as a port, because 4444 was running an old instance of Selenium Grid 2.
Step 7: Selenium Grid Components Glimpse
This is important to understand the flow of data and establish an understanding of the underlying components.
Must read: https://www.selenium.dev/documentation/grid/components/
Step 8: Start the Event Bus
Assuming you are in the same directory where the selenium jar is present. Create a TMUX session and run
tmux new -s ebjava -jar se4.jar event-bus
It serves as a communication path to other Grid components in subsequent steps.
Runs on port 5557 by default.
Note: I will be using tmux sessions before every command so that we can maintain sessions in parallel and observe what’s happening whenever needed.
Step 9: Session Map
tmux new -s sessionsjava -jar se4.jar sessions
A session map is responsible for mapping session IDs to the Node where the session is running.
Runs on port 5556 by default.
Step 10: Let’s form a queue of sessions!
tmux new -s sqjava -jar se4.jar sessionqueue
Session Queue adds the new session request to a queue, then the distributor processes it.
Runs on port 5559 by default.
Step 11: The DISTRIBUTOR
tmux new -s distributorjava -jar se4.jar distributor --sessions http://localhost:5556 --sessionqueue http://localhost:5559 --bind-bus false
Nodes register to it and assign a Node for a session request.
Runs on port 5553 by default.
Any mishandling of the same can lead to…
Step 12: The Mighty Routing Begins
It’s now time to run the router which is the grid entry point, the address which will be used in your automation code.
The router is in charge of routing requests to the correct component.
Runs on port 4444 by default.
A different one can be specified by -p parameter, 5550 in our case.
tmux new -s routerHubjava -jar se4.jar router --sessions http://localhost:5556 --distributor http://localhost:5553 --sessionqueue http://localhost:5559 -p 5550
Once the router has started, you will see such a screen.
So the grid has started with the final step of the router but there are no nodes yet. This means if you try to run a test, you will encounter SessionNotStarted exceptions & similar.
Step 13: Connecting a Node
Well as scary as the number 13 sounds, the same horror was there in order to properly master this step of connecting a node properly to the DisMaster.
DisNode → DisMaster
So login/ssh into the DisNode machine.
Pre-requisite steps: Step 3 and Step 4 above.
tmux new -s nodejava -Dwebdriver.chrome.driver=/usr/local/bin/chromedriver -jar se4.jar node --publish-events tcp://{DisMasterPublicIPAddress}:4442 --subscribe-events tcp://{DisMasterPublicIPAddress}:4443 --grid-url http://{DisMasterPublicIPAddress}:5550
{DisMasterPublicIPAddress} would be the public IP address of the VM running the router.
The Node needs to connect to the router machine and subscribe to both publish and subscribe events running on ports 4442 and 4443 respectively. If you don’t perform this mapping, NOTHING will work. Trust me!
I have spent countless hours researching, reading the readme of numerous projects including Selenium, and then somewhere on a youtube tutorial I found a reference & bang it worked. (Of course, if you had the ports open correctly, which was another nightmare because I simply cannot request the Ops team to make multiple ports open & accessible.)
Step 14: Selenium 4 Grid in action
Visit http://{DisMasterPublicIPAddress}:5550/ui/index.html#/ in your favorite browser and witness the node, the router, the GRID4 in action
Well, this is it. This is your moment of glory.
. >
. >>
. >>>
. >>>>
. >>>>>
. >>>>>>
.>>>>>
.>>>>
.>>>
.>>
.>
Step 15: And you thought it was done.
Update your code with the new URL and with Grid4 you don’t need to mention /wd/hub in the remote driver URL
String oldCodeOldHub = "http://localhost:4444/wd/hub"; // Notice there is no need of /wd/hub
String newCodeNewLife = "{DisMasterPublicIPAddress}:5550"; public WebDriver driver;
driver = new RemoteWebDriver(new URL(newCodeNewLife), caps);
Step 16: Sing along like Adele would do.
“This is the end
Hold your breath and count to ten
Feel the Earth move and then
Hear my heart burst again
For this is the end
I’ve drowned and dreamt this moment
So overdue, I owe them
Swept away, I’m stolen.
Let the tests RUN,
When it crumbles
We will stand tall
Face it all together, let the tests run”
