JMeterのテストプラン自体を並列で実行したいケースもある
はじめに
そのようなケースで使えないか、ということでCA TechnologiesのBlazeMeterチームが開発しているOSSの負荷テスト自動化ツールのTaurusを試してみた
※BlazeMeterはJMeterのSaaS
対象バージョン
- Taurus : v1.14.0
Taurusとは
- JMeterを拡張、抽象化したOSSの負荷テスト自動化ツール
- Taurus = Test Automation Running Smoothly
- YAMLかJSON形式でテストを記載できるので読みやすくバージョン管理も容易
- 既存のJMeterやその他ツールのテストスクリプトを実行可能
- JMeter等の複数のテストを一つのシナリオにまとめることができる
Taurusのインストール
公式ページの案内に従いInstallerを取得し実行する
pipでもインストール可能だがWindowsの場合は必要なソフトウェアがパッケージされたインストーラがおすすめの模様
インストール後はまず公式のLearning TaurusページのGetting StartedからBeginner's Courseをみてみるとよい
Learning Taurus
Beginner's Course
Guide PDFとサンプルファイルが入手可能なのでそちらをもとに試してみる
※プロキシ環境で実行する場合はYAMLファイルにプロキシの設定が必要となるので下記ページを参考に設定する
Taurus Configuration Syntax
Taurusで負荷テストを実行する
- Beginner's Courseに付属のYAMLファイルを少し変えてみる
- taurus_jmeter_script1.yml
---
execution:
- concurrency: 3
ramp-up: 3s
iterations: 10
scenario: Thread Group
scenarios:
Thread Group:
requests:
- label: blazedemo
method: GET
url: http://blazedemo.com/
- body:
fromPort: Paris
toPort: Buenos Aires
label: reserve
method: POST
url: http://blazedemo.com/reserve.php並列度3、ランプアップ期間3秒、繰り返し数10回でBlazeMeterのデモサイトにアクセスするシナリオ
- Taurusを開き上記ファイルを配置したフォルダに移動して下記コマンドでテストを実行する
bzt taurus_jmeter_script1.yml- 結果を確認する
経過がコマンドプロンプトに表示される
01:06:25 INFO: Screen 140x35 chars:
lqqqqq 1 3 users, 2 ~2 active qqqqklqqqqqqqqqqqqqqqqq Latest Interval Stats at 01:06:18 qqqqqqqqqqqqqqqqklqwqk │
x ...oo.ooo.o.o...xx Average Times: Percentiles: Response Codes: x x │
x ...oo.ooo.o.o...xx Elapsed: 1.084 0.0%: 0.785 200: 100.00% (2)x x lqqqk w w wqqk w w lqqqk │
x ...oo.ooo.o.o...xx Connect: 0.137 50.0%: 0.785 All: 100.00% (2)x x lqqqu x x x x x mqqqk │
x o.ooooooo.o.ooooxx Latency: 0.413 90.0%: 1.383 x v mqqqv mqqqv v mqqqv mqqqj │
x o.ooooooo.o.ooooxx 95.0%: 1.383 x - v1.14.0 by BlazeMeter.com - │
x o.ooooooo.o.ooooxx 99.0%: 1.383 x │
x ooooooooo.o.ooooxx 99.9%: 1.383 xJMeter: Thread Group │
x ooooooooo.o.ooooxx 100.0%: 1.383 x 100 % │
x ooooooooo.o.ooooxx xElapsed: 00:00:29 Overtime:│
x ooooooooo.o.ooooxx x 00:00:26│
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj │
lqqqqqqq 1 2 hits, 2 0 fail qqqqqqklqqqqqqqqqqqqqqqqqqqqq Cumulative Stats 00:00:16 qqqqqqqqqqqqqqqqqqqqk local │
x . xx Average Times: Percentiles: Response Codes: x mem: 48.700 │
x . xx Elapsed: 1.266 0.0%: 0.778 200: 100.00% (38)x disk-write: 18,159 │
x . . . xx Connect: 0.154 50.0%: 1.283 All: 100.00% (38)x engine-loop: 0.022 │
x . . . xx Latency: 0.426 90.0%: 1.826 x bytes-sent: 6,779 │
x . ... . . xx 95.0%: 1.887 x disk-space: 8.900 │
x . ... . . xx 99.0%: 2.108 x cpu: 2.500 │
x ....... . ....xx 99.9%: 2.108 x bytes-recv: 389,503 │
x ....... . ....xx 100.0%: 2.108 x disk-read: 0 │
x ................xx x conn-all: 0 │
x ................xx x │
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjx xqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq│
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqkx Labels Hits Failures Avg Time x │
x . xxblazedemo 20 0.00% 1.682 x │
x .. . . xxreserve 18 0.00% 0.803 x │
x .. . . . xx x │
x o. . . . . ... xx Errors: x │
x @.... ... . ....xxNo failures occured x │
x @...............xx x │
x @o..o.o.o.......xx x │
x @@oo@o@o@oooooooxx x │
x @@@@@o@@@o@o@@@@xx x │
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj │終了後は以下のような表示となる
01:06:29 WARNING: Please wait for graceful shutdown...
01:06:29 INFO: Shutting down...
01:06:29 INFO: Post-processing...
01:06:30 INFO: Test duration: 0:00:33
01:06:30 INFO: Samples count: 60, 0.00% failures
01:06:30 INFO: Average times: total 1.240, latency 0.412, connect 0.140
01:06:30 INFO: Percentiles:
+---------------+---------------+
| Percentile, % | Resp. Time, s |
+---------------+---------------+
| 0.0 | 0.776 |
| 50.0 | 0.835 |
| 90.0 | 1.826 |
| 95.0 | 1.887 |
| 99.0 | 2.108 |
| 99.9 | 2.108 |
| 100.0 | 2.108 |
+---------------+---------------+
01:06:30 INFO: Request label stats:
+-----------+--------+---------+--------+-------+
| label | status | succ | avg_rt | error |
+-----------+--------+---------+--------+-------+
| blazedemo | OK | 100.00% | 1.682 | |
| reserve | OK | 100.00% | 0.799 | |
+-----------+--------+---------+--------+-------+
01:06:30 INFO: Artifacts dir: D:\app\Taurus\Lab\2020-01-22_01-05-48.657155
01:06:30 INFO: Done performing with code: 0TaurusでJMeterのテストプランを実行する
- スクリプトの確認
こちらはBeginner's Course付属のものをそのまま利用
- taurus_jmeter_script2.yml
---
execution:
- executor: jmeter
scenario:
script: jmeter_script1.jmx
- executor: jmeter
scenario:
script: jmeter_script2.jmxJMeterのテストプランを２つ並列で実行するのみ
- jmeter_script1.jmx, jmeter_script2.jmx
jp@gc Dummy Samplerでリクエストを偽造するだけのもの
1は125ループに適当なRandomタイマーが入っており約2分半程度、2は数秒で完了するものになっている
- 実行する
こちらもbztコマンドにYAMLスクリプト名を渡すだけ
bzt taurus_jmeter_script2.yml- 結果を確認する
経過がコマンドプロンプトに表示される
並列で実行されており先にjmeter_script2.jmxが完了しているのがわかる
01:17:49 INFO: Screen 140x35 chars:
lqqqqq 1 5 users, 2 ~1 active qqqqklqqqqqqqqqqqqqqqqq Latest Interval Stats at 01:17:44 qqqqqqqqqqqqqqqqklqwqk │
x.................................xx Average Times: Percentiles: Response Codes: x x │
x.................................xx Elapsed: 0.520 0.0%: 0.409 200: 100.00% (3)x x lqqqk w w wqqk w w lqqqk │
x.................................xx Connect: 0.000 50.0%: 0.433 All: 100.00% (3)x x lqqqu x x x x x mqqqk │
x.................................xx Latency: 0.041 90.0%: 0.718 x v mqqqv mqqqv v mqqqv mqqqj │
x.o......o.o......o...............xx 95.0%: 0.718 x | v1.14.0 by BlazeMeter.com | │
x.o......o.o......o...............xx 99.0%: 0.718 x │
x.o.o...oo.o.oooooo..o.oooo.ooo.o.xx 99.9%: 0.718 xJMeter: jmeter_script1.jmx │
x.o.o...oo.o.oooooo..o.oooo.ooo.o.xx 100.0%: 0.718 xRunning... │
x.oooooooooo.ooooooooo.oooooooooooxx xElapsed: 00:02:27 ETA: N/A│
x.oooooooooo.ooooooooo.oooooooooooxx x │
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjJMeter: jmeter_script2.jmx │
lqqqqqqq 1 3 hits, 2 0 fail qqqqqqklqqqqqqqqqqqqqqqqqqqqq Cumulative Stats 00:02:16 qqqqqqqqqqqqqqqqqqqqkFinished │
x . . xx Average Times: Percentiles: Response Codes: xElapsed: 00:00:07 │
x .. . . . . xx Elapsed: 0.540 0.0%: 0.101 200: 100.00% (572)x │
x . . .. . . . . . . . xx Connect: 0.000 50.0%: 0.529 All: 100.00% (572)x local │
x . . .. . . . . . . . xx Latency: 0.052 90.0%: 0.903 x bytes-recv: 163 │
x . . .... . ...... .. ........ . xx 95.0%: 0.938 x cpu: 0.600 │
x ........ . ......... ........ ..xx 99.0%: 0.990 x engine-loop: 0.068 │
x ........ . ......... ........ ..xx 99.9%: 0.998 x disk-space: 8.900 │
x.................................xx 100.0%: 0.998 x bytes-sent: 217 │
x.................................xx x conn-all: 0 │
x.................................xx x mem: 46.700 │
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjx xqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq│
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqkx Labels Hits Failures Avg Time x │
x . . . xxSampler #1 572 0.00% 0.540 x │
x . . . . . . xx x │
x .... .. . .... . . ... xx Errors: x │
x ..... .... .. ...... ........ .xxNo failures occured x │
x ................................xx x │
x.................................xx x │
x.................................xx x │
x.................................xx x │
xoooooooooooooooooooooooooooooooooxx x │
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj │終了後は以下のような表示となる
01:18:02 WARNING: Please wait for graceful shutdown...
01:18:02 INFO: Shutting down...
01:18:02 INFO: Post-processing...
01:18:02 INFO: Test duration: 0:02:40
01:18:02 INFO: Samples count: 626, 0.00% failures
01:18:02 INFO: Average times: total 0.536, latency 0.051, connect 0.000
01:18:02 INFO: Percentiles:
+---------------+---------------+
| Percentile, % | Resp. Time, s |
+---------------+---------------+
| 0.0 | 0.101 |
| 50.0 | 0.52 |
| 90.0 | 0.905 |
| 95.0 | 0.94 |
| 99.0 | 0.989 |
| 99.9 | 0.998 |
| 100.0 | 0.998 |
+---------------+---------------+
01:18:02 INFO: Request label stats:
+------------+--------+---------+--------+-------+
| label | status | succ | avg_rt | error |
+------------+--------+---------+--------+-------+
| Sampler #1 | OK | 100.00% | 0.537 | |
| example 2 | OK | 100.00% | 0.431 | |
+------------+--------+---------+--------+-------+
01:18:02 INFO: Artifacts dir: D:\app\Taurus\Lab\2020-01-22_01-15-09.240701
01:18:02 INFO: Done performing with code: 0レポートを出力する
BlazeMeter Reporting Serviceを利用してレポートを出力する
- テストを実行してレポートを出力する
レポートの出力方法はコマンドラインオプションに-reportをつけるだけ
bzt taurus_jmeter_script1.yml -report- レポートを確認する
テストの実行が完了後ブラウザが自動的に起動しレポートが表示される
サマリリポートや
リクエスト統計など
とくにBlazeMeterのアカウントがなくとも生成は可能で、その場合7日間保持される模様
詳しくは下記参照
BlazeMeter Reporting Service
終わりに
Taurusを使用してみて、手軽に既存のJMeterのテストプランを並列実行できる、BlazeMeterの美麗なレポートを無償でも利用可能といって点は魅力を感じた。
一方でマニュアルや下記参考情報など見る限り現状ではJMeter標準のhtmlレポート作成がサポートされていなそうな点は大きなマイナスポイント。また（これはJMeter自体もではあるが）YAMLの設定をもとにJMXを生成しているため、テストの起動がやや遅い点も微妙。前者の点がクリアされないと現状ではお仕事での利用は難しいか。
参考）
JMeter : Generating the default html report not possible with Taurus
