Load And Performance Testing Tool

Load and Performance Test Tools :     Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program concurrently. As such, this testing is most relevant for multi-user systems; often one built using a client/server model, such as web servers. However, other types of software systems can also be load tested. For example, a word processor or graphics editor can be forced to read an extremely large document; or a financial package can be forced to generate a report based on several years' worth of data. The most accurate load testing simulates actual use, as opposed to testing using theoretical or analytical modeling.

Load and performance testing analyzes software intended for a multi-user audience by subjecting the software to different amounts of virtual and live users while monitoring performance measurements under these different loads. Load and performance testing is usually conducted in a test environment identical to the production environment before the software system is permitted to go live.
As an example, a web site with shopping cart capability is required to support 100 concurrent users broken out into following activities:
  • 25 Virtual Users (VUsers) log in, browse through items and then log off
  • 25 VUsers log in, add items to their shopping cart, check out and then log off
  • 25 VUsers log in, return items previously purchased and then log off
  • 25 VUsers just log in without any subsequent activity
A test analyst can use various load testing tools to create these VUsers and their activities. Once the test has started and reached a steady state, the application is being tested at the100 VUser load as described above. The application’s performance can then be monitored and captured.

The specifics of a load test plan or script will generally vary across organizations. For example, in the bulleted list above, the first item could represent 25 VUsers browsing unique items, random items, or a selected set of items depending upon the test plan or script developed. However, all load test plans attempt to simulate system performance across a range of anticipated peak workflows and volumes. The criteria for passing or failing a load test (pass/fail criteria) are generally different across organizations as well. There are no standards specifying acceptable load testing performance metrics.
A common misconception is that load testing software provides record and playback capabilities like regression testing tools. Load testing tools analyze the entire OSI protocol stack whereas most regression testing tools focus on GUI performance. For example, a regression testing tool will record and playback a mouse click on a button on a web browser, but a load testing tool will send out hypertext the web browser sends after the user clicks the button. In a multiple-user environment, load testing tools can send out hypertext for multiple users with each user having a unique login ID, password, etc.
The popular load testing tools available also provide insight into the causes for slow performance. There are numerous possible causes for slow system performance, including, but not limited to, the following:
  • Application server(s) or software
  • Database server(s)
  • Network – latency, congestion, etc.
  • Client-side processing
  • Load balancing between multiple servers
Load testing is especially important if the application, system or service will be subject to a service level agreement or SLA.

Below mentioned are some of the most used Load And Performance Testing tools in this very aspect :

Multi-Mechanize - Multi-Mechanize is an open source framework by Corey Goldberg for web performance and load testing. It allows you to run simultaneous python scripts to generate load (synthetic transactions) against a web site or web service. Results can be saved in CSV format along with an HTML report containing stats and graphs. Proficiency with Python, HTTP, and performance/load testing is recommended to use Multi-Mechanize successfully.

Load2Test - Performance/load/stress/high availability testing tool from Enteros Inc. Can capture real production workload for playback simulating thousands of users for top down testing or back-end testing at the component layer. Also available in the Amazon EC2, RackSpace or PlatForm Labs cloud environments. Integrated performance management and root cause analysis system automatically collects performance metrics across load test infrastructure and applies cross-component correlation to identify performance degradations and bottlenecks. Targeted platforms/OS's: Load2Test Controller: Windows, Linux; Load2Test Test Nodes: Windows, Linux. Load2Test performance monitors and root cause analysis data collectors - OS: Windows, Linux, HP/UX, SUN Solaris, IBM AIX Databases: Oracle, DB2, SQL Server, Sybase, MySQL, PostgreSQL; App: .NET, JBoss, WebLogic, WebSphere, Oracle Application Server (OAS), GlassFish, Tomcat, JettyStorage: NetApp Filers, IBM DS8300

Xceptance LoadTest - Load testing and regression tool from Xceptance Software Technologies, Inc for web and Java and other app load testing. Includes recording capabilities. XLT Cloud Service available. Tests implemented as JUnit 4 test cases. For web-based tests, the framework provides a (headless) browser that can emulate Internet Explorer or Firefox behaviour. Can execute client-side JavaScript in the emulated web browsers and that way it simplifies the creation of test cases for Web 2.0 applications. Platform independent due to tool being implemented in Java; test scripting in Java or Ruby. Free for up to five virtual users.

SiteBlaster - Web site load and stress testing tool; shareware. Can be used to rapidly submit requests to a site, or can pause a random amount of time between submissions, approximating user behavior. During testing the pages being tested will be displayed. Reports created on test completion. Designed to be very easy to use; intended for software developers and architects who want some early indication about performance characteristics of the web sites they create. Simulates MS IE web browsing functionality; a web page that is well behaved in IE should be well behaved in SiteBlaster. Best used to test those sites that use URL query strings to pass data to its web page(s). PDF user guide available. For Windows.

Load-Intelligence - Affordable load-testing “Software as a Service” from Cloud-Intelligence. Software and unlimited hardware all included. JMeter users can execute their test-scripts in an unlimited, pre-configure, distributed environment. Neither setup nor installation are required. Immediate access to JMeter logs, reports, test script, CSV files and more.

LoadStorm - A web-based load testing tool/service as a distributed application that leverages the power of Amazon Web Services to scale on demand with processing power and bandwidth as needed. As the test loads increase to hundreds or thousands of virtual users, LoadStorm automatically adds machines from Amazon's server farm to handle the processing. Tests can be built using the tool in such a way as to simulate a large number of different users with unique logins and different tasks.

BrowserMob - On-demand, self-service, low-cost, pay-as-you-go service from Neustar enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.

Load Impact - Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes - server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.

fwptt - Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

Gomez RealityLoad - An on-demand load testing service (no licenses) from Gomez.com/Compuware. Utilizes Gomez’s Active Network providing on-demand active monitoring from 150+ enterprise-grade servers running in world-class, high-bandwidth data centers around the globe, and Gomez’s Active Last Mile which provides on-demand active monitoring from 150,000+ real, consumer-grade desktops in 168+ countries around the globe.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder - A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurb├╝ro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

JKool Online - Performance measurement and monitoring service from Nastel Inc. for web-based J2EE and SOA applications; start and stop live data monitoring whenever needed; drills down to JMS, JDBC, method calls, servlets and sessions with simple one-click option to view live session details; built-in support for jBoss, WebLogic and IBM WebSphere Application Server.

Funkload - Web load testing, stress testing, and functional testing tool by Benoit Delbosc written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.

QEngine Performance Tester - Automated testing tool from Zoho Corp. for performance testing (load and stress testing) of web applications and web services; J2EE, .NET, AJAX, PHP, Ruby on Rails, SOAP Web Services etc. Supports multiple browsers on Linux and Windows.

NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.

Test Complete Enterprise - Automated test tool from SmartBear Software includes web load testing capabilities.

QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

Visual Studio Test Professional 2010 - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio. Includes Lab Management capabilities.

OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.

Neustar Webmetrics Load Testing - Remote, consultative load testing service by Webmetrics/Neustar. Simulates end-user activity against designated websites for performance and infrastructure reliability testing.

Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad - Java-based multi-platform performance testing and analysis tool from PureLoad Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

Web Performance Load Tester - Load test tool emphasizing ease-of-use, from Web Performance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

Optima Quality Studio - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizer load testing module supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QALoad - Tool from Microfocus (formerly from Compuware) for load/stress testing of web, database, and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.

IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer - Enterprise-class load-testing tool from Microfocus (formerly from Borland, formerly from Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad - Load testing tool from Radview Software. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.


Vbscript-coding-in-qtp

Posts-on-using-qtp-efficiently


Posts-on-object-properties-in QTP


Posts-on-qtp-vbscript-functions


Posts-on-coded-ui-testing


For gaining more insights in the automation using QTP log on to below url :
Automation Testing Using QTP 

10 comments:

  1. Hi,

    First of all want to Thank You for your useful post.
    This Software Testing article is very useful for me. I would like to introduce another good site which is having Software testing and QTP content, Have a look.
    QTPbook

    ReplyDelete
  2. Nice article, I read your post form the beginning. It’s sound interesting to read. Thanks for sharing the useful blog.
    Selenium Training|Selenium Course in Chennai

    ReplyDelete
  3. The blog gave us idea about the load and performance Testing Tool My sincere Thanks for sharing this post and please continue to share this kind of post
    Software Testing Training in Chennai

    ReplyDelete
  4. This article is so informatic and it really helped me to know more about the Selenium Testing. This selenium article helps the beginners to learn the best training course. So keep updating the content regularly.
    Selenium Training in Chennai | Best Selenium Training institute in Chennai | Selenium Course in Chennai

    ReplyDelete
  5. really you have been posted an informative blog. really this will be very helpful to many peoples. so keep on sharing such kind of an interesting blogs.
    software testing training in chennai

    ReplyDelete
  6. Thanks for this blog. Provided great information. All the details are explained clearly with the great explanation...
    Software Testing Training in Bangalore
    R Programming Training in Bangalore

    ReplyDelete
  7. The blog was absolutely fantastic! Lots of great information which is very useful for us. Thanks for sharing. are you looking for Software Testing Training in Chennai reach us besant technologies. Selenium Training in Chennai

    ReplyDelete
  8. Really awesome blog. Your blog is really useful for me. Thanks for sharing this informative blog. Keep updating your blog.
    Software Testing Training in Chennai | Software Testing Training Institute in Chennai

    ReplyDelete
  9. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information..
    Software Testing Training in Bangalore
    Devops Training in Bangalore

    ReplyDelete