Triggering Remote Jenkins jobs from another Jenkins

Continuous integration and delivery is a very crucial part of software life cycle for automatic execution or deployment of the code. Usually we have single Jenkins for deployment, automation scripts etc. But many times different teams like Ops, Dev and QA create their own Jenkins for their own purposes.

Now if we talk about integrating everything at same place, it is not feasible to manage and re-create jobs. So solution is to make communication between two Jenkins servers and trigger build accordingly.

In this blog post, I will talk about how can you trigger a JOB-A (on remote-jenkins ) from JOB-B (on local-jenkins). The real scenario of this would be; when we are trying to trigger an automation script on Jenkins1 after successful completion of code deployment on Jenkins2.

For understanding this let assumes few things below-

  • We have a local job-  Job-B (local-jenkins) on server local-jenkins:8080
  • We have a remote job – Job-A (remote-jenkins) on server – remote-jenkins:8080

Now we want to trigger Job-A from Job-B. For achieving this we need to install few plugins in our local Jenkins (from which we want to trigger the job- local-jenkins in this case) –

Go to manage-Jenkins->configure system->Parameterized Remote Trigger Configuration, and do configuration as stated below-

remotetrigger1

You can add many remote servers. Now you have to do following changes in your local Jenkins job i.e. Job B.

buildsteps

buildinfo

Now save the configuration of your job and build your local job i.e. Job-B. Console of local job looks like below- local job

Console of Remote job looks like below – remotejob

You can see it says that started by local-Jenkins. So it has triggered this job on remote Jenkins from local Jenkins.

Similarly you can link multiple jobs to be build across different Jenkins server. I hope now you can easily integrate multiple Jenkins. Please add comments in case of any issues encountered. 🙂

 

Docker Training Program – [Build, Ship, and Run Any App, Anywhere]

In the automation driven industry, we are way more advance in automating our test cases, deployments etc. but automating your infrastructure, setup of environments is still a pain. We all have seen situations when something is working on a machine but it is not working on other machine. Sometimes a QA files a OS specific defect , but developer is no longer able to re-produce. So solution of all these problem is one single thing – DOCKER

I have recently started working on Docker and utilized this very efficiently in Automation Testing and Dev-ops; specially in setting up the execution environment. Following are the major benefit of docker-

  • Build, Ship, and Run Any App / automation script , Anywhere
  • Setup of execution environment for dev/testing is a matter of seconds
  • Docker hub – A cloud of docker images which provides image for every possible software you are looking for, and you can also push your own images and use it form anywhere
  • Continuous integration and fast deployment

Going forward, I would be going through following topics-

  1. Introduction to Docker container technology, how is it different from virtual machines
  2. Installing and Setting up Docker on Windows
  3. Installing and Setting up Docker on Linux (Ubuntu)
  4. Understanding Docker components like Docker, Docker-Machine and Containers
  5. Understanding Docker images and containers
  6. Hooking your local source code in to container
  7. Understanding major docker commands and shortcuts
  8. Executing your local selenium test inside the container
  9. How to use Selenium-Grid with docker
  10. Building custom images from dockerfile
  11. How to minimize the size of your docker images
  12. Managing your containers with docker compose
  13. How to scale your execution environment with docker and multi-threadinng
  14. Using docker containers as Jenkins slaves
  15. Docker on AWS

So far, I will  be writing about each topic mentioned above. I will also keep adding new topics to the list. Stay tuned to this post for all docker related stuff.

You can reach out to – gauravtiwari91@yahoo.com for more details and personal training with live projects.

Shifting left with DevOps and Continuous Integration

Adopting Continuous delivery helps to achieve rapid application development throughout the software application life cycle. It is a methodology, a mindset change,a shift-left approach and a leadership practice to streamline manual processes and enforce consistency and repeat-ability in software delivery pipeline. This is about enhancing collaboration and shared matrices and processes across Developers, QA and Ops team.

Read time – 10 minutes

In order to establish a Continuous Delivery environment, the most important requirement is the implementation of an automated Continuous Integration (CI) System. The CI process involves all stages – right from code commits to a version control system (done on the CI server) that serves as a kick off for a build to compile, run tests, and finally package the code. DevOps is playing a major role in going towards defect-preventive approach.

This blog will demonstrate the steps and advantages of implementing a Continuous Integration System using Jenkins and a group of virtual machines. You can utilize your CI system for automatic infrastructure setup and execution of suitable automated scripts whenever a new build or commit happens in that automation script code.

Highlights

  1. Setting up a Jenkins Master machine
  2. Setting Jenkins slaves for distributed execution
  3. Creating new Jenkins job for new scripts
  4. Creating execution pipeline for automating the build steps
  5. Scheduling script execution

 

  • Setting up a Jenkins Master Machine – Installation of Jenkins is very easy; it is just about executing a jar file or executable. Once this is up, it can be accessed from any machine available on that network through a web browser. Jenkins Master is responsible for re-directing all commands and execution to Slave machines. For set up instructions, go through Setting up Jenkins in 5 minutes
  • Setting Jenkins Slave for distributed execution – Jenkins Slave machine can be a real machine, a virtual machine or a dockerized container which has capability of an operating system. Jenkins slave can also be set up by just executing a jar file and registering that machine as a slave against Master Jenkins machine. When Jenkins master receives instructions, it processes that and decides which script would be executed on which slave machine. So if multiple slaves are connected, execution can be done in distributed manner and will result in multi processing + multi threading of execution.
  • Creating new jobs for script execution – A job in Jenkins is about defining a series of action for successful execution of script. First it fetches the latest script code from sub-version, and then it builds that code on some selected slave or master (which is decided by Jenkins Master). Once the code is built on slave machine, Scripts get started and executed. usually, we use Selenium, Java and Maven for automated build process. Once a job is created, it has a web URL. This web URL is helpful in sharing execution results, build info etc.
  • Creating execution pipe line – Execution pipe line is a visual representation of different job executions in sequence. This helps in automating the whole process of script execution. So once all Jenkins jobs are set up, you can decide the order of their execution based on the type of script e.g. Smoke, Regression, release validation, web services test etc. So it creates a pipe line of execution which has a single trigger point. That trigger is initiated whenever a new build / release happen on platform or there is any requirement of executing it. Once a single job in this pipeline gets finished, it automatically triggers the next job in the pipe line. QA/Developer keeps getting the continuous feedback of automated results, which helps us in identifying the defects. Below diagrams represents a pipe line which shows headless scripts execution which is done using plugin – Build pipeline plugin

build-pipeline

  • Scheduling script execution – Jenkins job is that these can be scheduled for future executions and can provide results of nightly test build script executions. It helps a lot of time spent in manual triggering and monitoring of execution and enhance the automated process of script execution. There are two ways to do this-
  1. Jenkins plugin for scheduling – Build schedule plugin
  2. Select the option for build periodically in build steps and define a regular expression which is a 5 digits separated number define the execution timing and cycle. for e.g. below images showing that this job will be executed at 22:00 (10:00 PM) daily.schedule

I have setup this kind of infrastructure for automated test script execution, Next step is to do the same at build deployment level. I will keep posting more stuff related to Devops and continuous integration. Happy Testing 🙂

Selenium 3: Firefox with Gecko Driver

Simon Stewart had announced Selenium 3 release on 25th May 2013 and it has finally beta released to use on 2nd August 2016. In this blog post I will be discussing the changes happened on implementation level when you will actually be writing code using Selenium 3.0

If you are still not aware with API level changes, then I suggest you to go through my last blog Way to Selenium 3.0 and then get back to this one 🙂

Following are some implementation level changes i have observed after writing Selenium 3.0 first program

  • You need JAVA 8+ to run Selenium 3 test
  • You need Gecko driver (like  chrome and IE driver) to run scripts on Mozilla Firefox.
  • Jar library size is now minimized to 10 MB
  • leg-rc jar is no more bundled in main selenium jar, you need to separately download it.
  • Official support for IE requires version 9 and above
  • Detailed changes in Selenium 3.0.0 can be found at link – Selenium 3 change log

So now when you run your current script with Selenium 3.0 jar files on Firefox browser,you might see below error

java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.gecko.driver system property; for more information, see https://github.com/mozilla/geckodriver. The latest version can be downloaded from https://github.com/mozilla/geckodriver/releases at com.google.common.base.Preconditions.checkState(Preconditions.java:199) at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:109) at org.openqa.selenium.firefox.GeckoDriverService.access$100(GeckoDriverService.java:38) at org.openqa.selenium.firefox.GeckoDriverService$Builder.findDefaultExecutable(GeckoDriverService.java:91) at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:296) at org.openqa.selenium.firefox.FirefoxDriver.createCommandExecutor(FirefoxDriver.java:245) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:220) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:215) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:211) at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:124)

So now you need Gecko driver to execute scripts on Firefox, Gecko driver exe can be downloaded from – GeckoDriver. Now you need to specify the system property with gecko path

System.setProperty("webdriver.gecko.driver","path of geckodriver.exe");
WebDriver driver = new FirefoxDriver();

What is Geckodriver – A Proxy for using W3C WebDriver-compatible clients to interact with Gecko-based browsers. Geckodriver provides HTTP API described by the WebDriver protocol to communicate with Gecko browsers, such as Firefox (Version after 47). Even if you are working with older versions of Firefox browser, Selenium 3 expects you to set path to the driver executable by the webdriver.gecko.driver.

P.S. If you have automated build system dependent on Selenium scripts(critical cases), I recommend you to wait for official non beta release for selenium and then update the libraries of your build system. Its always better to use stable version of any software.

Happy Coding 🙂

 

Selenium WebDriver : An ecosystem of browser standards

Yes!.. You read that right, Selenium WebDriver : An ecosystem of browser standards. We says that WebDriver is a tool which automates browser. But looking at its W3C standard being implemented by almost every browser vendor, It seems to me a Creator of  an ecosystem for browser standards. 

Just have a look at the evolution happening around WebDriver –

  1. Selenium 3.0 is soon going to be released with most of the W3C standard
  2. Apple has announced that Safari 10.0 would have official WebDriver support
  3. Microsoft now has official support for WebDriver with Edge browser
  4. Mozilla has released Marionette(an automation driver for Mozilla’s Gecko engine)
  5. WebDriver is not able to launch FF47.0 (but that’s not an issue with WebDriver, that is a Firefox issue 😀 )

All above facts clearly stats that all the browser vendors are focusing so much to implement W3C standard , so that doing cross-browser automation testing is no more a hectic task.

Now lets spot some light on all the points discussed above.

  • Selenium 3.0 – The details about selenium 3 can be found in my previous blog post – Way to Selenium 3.0
  • Apple’s Safari 10.0 release –  WebDriver is mentioned as one of the feature of Safari 10.0 on its official release page –Apple’s Safari 10.0. It says that  – Safari on OS X supports WebDriver, which lets you automate web-content testing. It provides a set of interfaces to manipulate DOM elements and control the browser’s behavior. You can enable Remote Automation in the Develop menu and then launch the server using /usr/bin/safaridriver
  • Microsoft’s Edge release – “The Microsoft Edge implementation of WebDriver supports both the W3C WebDriver specification and the JSON Wire Protocol for backwards compatibility with existing tests”. More details can be found at –Microsoft Edge WebDriver guide
  • Marionette driver by Mozilla – Marionette is an automation driver for Mozilla’s Gecko engine. It can remotely control either the UI or the internal JavaScript of a Gecko platform, such as Firefox. It can control both the chrome (i.e. menus and functions) or the content (the webpage loaded inside the browsing context), giving a high level of control and ability to replicate user actions. In addition to performing actions on the browser, Marionette can also read the properties and attributes of the DOM.
  • Recently one of my colleague had upgraded Firefox 46.0 to 47.0 and existing WebDriver scripts was breaking with error “Firefox 47 – Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms“. After a lot of debugging and going through the link – Issue With FF47.0, I got to know that this is some issue with FF 47.0 and fixed in FF 48.0 which is not yet released. I just felt proud that an automation tool is identifying issues in a Browser. 😀

WebDriver has actually created an ecosystem for all browser. And when this thing would be implemented in all browsers, cross browser testing would be just about changing the browser driver.

Looking forward to see all the browsers implementing this W3C standard and making our cross browser automation work so transparent, error free and simple 🙂

Way to Selenium 3.0

 

Selenium 3 is on its way to be shipped. Simon Stewart (Selenium Project Lead) along with Applitools has given a sneak peek about Selenium3.0 on 25th May 2016. The Webinar was just great as all other presentations by Simon Stewart. I have identified few points and will try to summaries in this blog post-

Main aim for Selenium 3 is to be – “a tool for user-focused automation of mobile and web apps“. this one liner explains it all about Selenium 3 . So they are going to focus on automation of web and mobile apps only (No support for any other kind of application). For Selenium 3, They have primarily worked on making the technology behind Selenium as stable and capable as possible.

For understanding the changes happening in Selenium3, you really need to understand how selenium has been evolved since the time Selenium-Core was released. During this blog post, I will be going  through following points-

  1. The past of Selenium project – how it evolved
  2. Where is Selenium 3.0
  3. User visible changes
  4. Back-end changes

Evolution of Selenium  – Json huggins from throughworks has created a javascript test runner which was capable of injecting Javascript in html pages and perform the required action. User need to create their script in html which is not really a programming language. Due to JavaScript’s same origin policy, user was not able to perform action on the different domain (websites). Then RC came in picture, where your script can actually communicate to a proxy server(RC server) and trick your browser to believe that all the requests are coming from same domain. It was really hard to maintain the Selenium RC/Core tool for the Selenium developer, so they have started working on creating a standard api which can perform all action required on a web page irrespective of the language in which scripts are written.

Simon Stewart then discovered WebDriver api which works on JSON wire protocol over http, and don’t have any language specific dependency. Then this api was wrapped in to language specific bindings to support multiple languages and this was called Selenium 2.0

Where is Selenium 3 – There are two level changes; back-end level changes and user visible changes.

User visible changes

  1. Selenium Core would completely be removed.
  2. Selenium RC would be deprecated, there would be no active development to support Core or RC except the very urgent fixes.
  3. The RC api would be backed by WebDriver, so you can execute your existing RC script (WebDriver is recommended for all active development for new scritps)
  4. Client bindings for other languages except for JAVA are still same
  5. Using WebDriver after quit() would be now an illegal State Exception
  6. For mobile users, the Selenium 3 will be hosting a suite of tests to facilitate interoperability between the many different projects available (i.e. Appium, ios-driver, Selendroid etc.) that are extending the WebDriver API to also cope with mobile.
  7. New Java binding structure for Selenium3 is as follows-
  • Selenium – JAVA : just the webdriver classes
  • Selenium 3 server : Lighweight, command line compatible remote server
  • leg-rc : the old selenium client side classes, WebDriverBackedSelenium, Remote end point for Se3 server

As there would be no support for RC and core, existing scripts would be executed by leg-rc’s “WebDriverBackedSelenium”. Selenium RC server would be replaced by Selenium 3 server which would be command line compatible that means you can access its features through command line. it would really help in continuous integration. Selenium3 server might also have capability to hook in to already opened browser windows.

Backend level changes – Due to support for many browsers by Selenium 2.0 , it was really hard for developer to fix selenium specific issues. For example fixing an issue for one browser (chrome) trigger some other issue for different browser(Firefox). The reason for this inconsistency is that every browser follows different approaches for Xpath engines and other browser mechanism. So Selenium has come up with W3C standard and asked every browser owner to follow these standard while implementing browser. Selenium developer community has also recommended every browser vendor to own the drivers as they are more familiar with their browser. So apart from these following can be changes from back-end side-

  1. Migrate all drivers to use the status strings rather than status codes in responses
  2. All actions to have a single end point for JSON wire protocol service
  3. Migration to netty or webitt server

There is no changes in terms of additional features. They are targeting to have new features specific to mobile in Selenium 4.0. Selenium 5.0 would be completely W3C compatible. Selenium3 and Selenium4 would have interoperability i.e. client bindings of Selenium3 can communicate to Selenium 4 server and vice – versa.

If we look in terms of stability and back-end changes for Selenium , it is a big change. That is why they have changed a major version from 2.0 to 3.0

The old versions will still be available as a separate download, but active development will cease, except for very urgent fixes.

Hopefully , it would be available to download on official Selenium site. I hope i have covered all the features and points related to Selenium 3, feel free to add in comments if anything is missed. 🙂

EDIT 1 : For Q&A session with Simon Stewart during this webinar, please refer to comments of this blog post. 🙂

EDIT 2 : Selenium 3 beta is now officially released, please visit my latest blog – Selenium 3: Firefox with Gecko Driver

Symptoms that your mobile automation is going to fail

Smartphone users are increasing every day and mobile world is growing so rapidly. People now share and access mobile apps more frequently than web apps.

Now talking about their creation, release cycles are so quick that you have to do a lot of work in testing product and that too should be quick. What choice we left ? Automation!. But are we really utilizing that in good manner or we are just making simple things too complex by introducing automation.

Based on my experience their are few symptoms which indicates that you are going towards failure in mobile automation. Agenda for sharing this info is to make you enable in judging whether you are going on right track or not.

So be aware of following 10 symptoms which might lead you to failure if you will not find alternative for them-

  1. Deploying the build (APK/IPA) is a manual process– If you are still doing it manually, stop it. Mobile automation should be as smooth as automating a web app and accessing it through a browser. To speed up the process this part of automation should be automated. There are several tools e.g. Jenkins, Team City. You can refer to my blog Continuous integration with Jenkins in 5 minutes for Continuous integration setup.
  2. No Process where test can execute 24/7 – In the agile world, release cycles are too quick and a tester hardly get some time to perform Regression, functional testing for each feature. You should have a process where your automation test can be triggered anytime and can give you quick validation results. You can achive it by setting up your own mobile devices cloud, or you can use existing paid services like Saucelabs, Testdroid cloud etc.
  3. Scrolling the device screen is co-ordinate based – If this is the case it is very hard to stable your test. As an automation developer, our primarily goal is to make sure that a test can successfully executed on any device of any size. So make sure all actions are object based rather than co-ordinate based. your script might not work for a device with different size.
  4. Wasting more time in stabilizing test rather than building them – It happens when we start creating script, we don’t really realize the factors like – locator strategies, target devices, screen sizes, complex actions involved. As a result, after certain point we waste our time in making current script stable and cant really look forward for new creation. So advice is before starting a project do a little bit of POC(proof of concept), identify the complex situations and take care of them from the starting itself.
  5. You can’t remember the last time when your tests ran successfully – I have seen such situations that when people have small numbers of cases, they have a control to easily identify the false failures and modify the script accordingly. then slowly false failures keeps growing up as the number of cases increases. I would say don’t let you get into that situations. Take each false failure seriously and try to handle all exceptions, so that you can have confidence in your scripts when it comes to releasing the app.
  6. Using indexes at too many places to identify the elements on screen – If you had worked on selenium, you might related when we create xpath based on location in html DOM, It is similar to that. Avoid using indexes in your locators. Mobile app web views, text box’s positions might change in future and you will again end up with modification of scripts.
  7. Using Image recognition or OCR extensively for identification – I have seen many times, people uses OCR or image based technique for identification of elements on screen. Chances of changing the UI design in mobile app are very high. Sometimes not the design but the images might get changes with minor pixel size changes and you need to replace all images for image based identification.
  8. Working for several months but have only dozen cases – A manual tester always try to create test-cases where he can achieve coverage in minimum cases, which result in few complex cases. When we take them for automation, we don’t really divide in to many smaller meaning full cases and start creating complex work flows. I suggest to break cases in smaller one, so that by using parallel execution you can minimize the execution time and get the more effective visibility in terms of coverage. So its good to have many small test scripts rather than few complex scripts.
  9. Running the same script on different size devices, never works on first try – Primary part of testing a mobile app is to test it across multiple devices and screen sizes. If this is not happening in single attempt, means you are not really taking care of all possible devices and screen sizes. You end up modifying your script as soon as you run the script on some new devices. So its advisable to be aware of device scope and functionality future in advance. Be in touch with business analyst is very important
  10. You are not controlling the targeted devices and versions – This is one of the most important aspects in terms of infrastructure for mobile script execution. As an automation engineer, you should be responsible for handling devices and their versions, installing apps in those devices. If someone else is doing it then you should be in sync with that team.

If you are facing anyone of above mentioned problem, start implementing best alternative. Lets not hamper your app quality and focus on being efficient in testing. Hope these point will help you.

If anyone has gone through some other complex situation in mobile automation, feel free to add your symptom in comment 🙂