$Id: USING,v 1.4 2003/02/08 17:26:57 dpuryear Exp $

This document describes how to use JabberTest.

Questions or comments? Send them to me at dustin@puryear-it.com.

I wrote jabbertest using the 'small, focused toolset' approach. This means
that each tools does one thing, and only one thing. For comprehensive testing
you will need to create scripts or wrapper programs to use my tools. I have
included examples of scripts in the tests/ subdirectory, which will be 
discussed at the end of this document.

:: TOOL SET ::

userreg - User Registration
=========
The userreg tool is used to create a set of n users and to return the amount
of time it takes to register each user. At a minimum you need to supply the
Jabber server and the number of users you wish to register:

[dpuryear@crack testsuite]$ ./userreg -h crack -u 6
0     998319213.325422 998319213.362014 0.036592
1     998319213.370257 998319213.398563 0.028306
2     998319213.410227 998319213.438734 0.028507
3     998319213.450251 998319213.478475 0.028224
4     998319213.490252 998319213.518468 0.028216
5     998319213.530215 998319213.558429 0.028214

In this example I have create 6 users on the Jabber server crack. It is easy
to see that each registration took less than 0.04 seconds. In addition, in
columns 2 and 3 you can see exactly when the registration was initiated and
when it completed--this information will allow you to compute actual 
registrations per second statistics.

The userreg tools uses a prefix of test_ to create each user account. Thus, if
you are creating 6 accounts userreg will create: test_0, test_1, test_2,
test_3, test_4, and test_5.

If you already have n user accounts beginning with test_ and wish to create m
more, than use the --start-number option:

[dpuryear@crack testsuite]$ ./userreg -h crack -u 3 -n 6
6     998319468.990000 998319469.024580 0.034580
7     998319469.036889 998319469.065149 0.028260
8     998319469.076917 998319469.105398 0.028481

As you can see we continued creating users where we left off in the previous
example. If we had not used -n the result would have been:

[dpuryear@crack testsuite]$ ./userreg -h crack -u 3                 
registration failed

This occurs because userreg attempts to register test_0, but test_0 already
exists in the Jabber user database.

** NOTE **
It is important to know that userreg is used to create the test accounts that
will later be used by the other jabbertest tools to log into Jabber. Until you
run userreg with the appropriate number of users the other tools will fail
unless you manually create the test accounts.

pasvlogin - Passive Logins
=========
The pasvlogin tool allows you to create n concurrent sessions. This tool can
actually be quite useful. It allows you to test how many concurrent sessions
your Jabber server can handle at once (perhaps to test the 1024 limit in most
Linux installations), or to create passive sessions that can be used during
other testing.

The easiest way to use pasvlogin is to just specify a Jabber server and the
number of test sessions:

[dpuryear@crack testsuite]$ ./pasvlogin -h crack -u 6
0     998320640.079130 998320640.092117 0.012987
1     998320640.099936 998320640.107248 0.007312
2     998320640.120114 998320640.128465 0.008351
3     998320640.141298 998320640.150445 0.009147
4     998320640.160059 998320640.170083 0.010024
5     998320640.180169 998320640.191179 0.011010

Using pasvlogin in this way simply creates 6 sessions and then close them all
after they have all been opened, as you can see in the jabberd record.log:

20010820T15:17:20 test_0@crack login ok 192.168.155.210 work
20010820T15:17:20 test_1@crack login ok 192.168.155.210 work
20010820T15:17:20 test_2@crack login ok 192.168.155.210 work
20010820T15:17:20 test_3@crack login ok 192.168.155.210 work
20010820T15:17:20 test_4@crack login ok 192.168.155.210 work
20010820T15:17:20 test_5@crack login ok 192.168.155.210 work
20010820T15:17:20 test_0@crack session end 0 0 0 work
20010820T15:17:20 test_1@crack session end 0 0 0 work
20010820T15:17:20 test_2@crack session end 0 0 0 work
20010820T15:17:20 test_3@crack session end 0 0 0 work
20010820T15:17:20 test_4@crack session end 0 0 0 work
20010820T15:17:20 test_5@crack session end 0 0 0 work

Notice that pasvlogin, like msgloadrec and msgloadsnd (below), use the user
accounts created by userreg for testing.

You can also specify that pasvlogin maintain the concurrent sessions until a
keypress:

[dpuryear@crack testsuite]$ ./pasvlogin -w -h crack -u 6

0     998320756.461671 998320756.480350 0.018679
1     998320756.487440 998320756.494999 0.007559
2     998320756.507259 998320756.515579 0.008320
3     998320756.527236 998320756.536575 0.009339
4     998320756.547212 998320756.557165 0.009953
5     998320756.567289 998320756.578304 0.011015
Enter quit to close sessions and quit..

In this mode the sessions are not closed until a key is pressed, as seen in
the jabberd record.log:

20010820T15:19:16 test_0@crack login ok 192.168.155.210 work
20010820T15:19:16 test_1@crack login ok 192.168.155.210 work
20010820T15:19:16 test_2@crack login ok 192.168.155.210 work
20010820T15:19:16 test_3@crack login ok 192.168.155.210 work
20010820T15:19:16 test_4@crack login ok 192.168.155.210 work
20010820T15:19:16 test_5@crack login ok 192.168.155.210 work

To close the sessions hit a key, and the sessions will close:

20010820T15:19:16 test_0@crack login ok 192.168.155.210 work
20010820T15:19:16 test_1@crack login ok 192.168.155.210 work
20010820T15:19:16 test_2@crack login ok 192.168.155.210 work
20010820T15:19:16 test_3@crack login ok 192.168.155.210 work
20010820T15:19:16 test_4@crack login ok 192.168.155.210 work
20010820T15:19:16 test_5@crack login ok 192.168.155.210 work
20010820T15:20:03 test_5@crack session end 47 0 0 work
20010820T15:20:03 test_4@crack session end 47 0 0 work
20010820T15:20:03 test_3@crack session end 47 0 0 work
20010820T15:20:03 test_2@crack session end 47 0 0 work
20010820T15:20:03 test_1@crack session end 47 0 0 work
20010820T15:20:03 test_0@crack session end 47 0 0 work

msgloadrec and msgloadsnd - Message Load Testing
=========
The msgload tools have a very simple objective: to create a load on the server
by sending n messages per second. 

The msgloadsnd tool is used to send the messages to the recipient client,
msgloadrec:

msgloadsnd --> jabberd --> msgloadrec

The communication is entirely one way: from msgloadsnd to msgloadrec. However,
msgloadsnd will watch for error and bounce messages from jabberd.

To use msgload you will normally launch one msgloadrec for each msgloadsnd.
The two instances will use a 'user pair'. For example, if msgloadsnd logs in
using the x account then msgloadrec should log in using y, and the
two accounts, x and y, are considered a pair. This is important to 
know because msgloadsnd will direct it's message to user y.

Launching the tools is simple. For msgloadrec, specify the Jabber server, and 
tell msgloadrec who it is:

[dpuryear@crack testsuite]$ ./msgloadrec -h crack -t test_1 

At this point msgloadrec will wait for any messages sent to the user test_1.
It is expecting a specially formatted message that is sent from msgloadsnd.
If you send other messages to msgloadrec it will not print timing data.

Next, launch msgloadsnd by telling it the Jabber server, it's user name, and 
the user to send messages to, and the number of messages to send:

[dpuryear@crack testsuite]$ ./msgloadsnd -h crack -t test_1 -f test_0 -m 3

This is great, and traffic is generated, but msgloadrec will be operating in 
silent mode:

$ ./msgloadrec -h crack -t test_1 

This means you won't see any output. However, you can cause msgloadrec to
print out delivery timing data using the -d (timing data) option:

[dpuryear@crack testsuite]$ ./msgloadrec -h crack -t test_1 -d
0     0     998322923.815560 998322923.816672 0.001112
1     1     998322923.824633 998322923.834102 0.009469
2     2     998322923.834467 998322923.854103 0.019636

This output was produced when msgloadsnd was invoked as:

[dpuryear@crack testsuite]$ ./msgloadsnd -h crack -t test_1 -f test_0 -m 3

Notice that msgloadrec has not stopped. Rather, msgloadrec will continue 
waiting for message indefinately unless you use the -a (abort-after) option,
which specifies that msgloadrec abort after n seconds of inactivity:

[dpuryear@crack testsuite]$ ./msgloadrec -h crack -t test_1 -d -a 3
0     0     998323036.529014 998323036.535219 0.006205
1     1     998323036.535521 998323036.555166 0.019645
2     2     998323036.545074 998323036.573484 0.028410
[dpuryear@crack testsuite]$ 

So what does the output mean? Left to right the columns are as follows:

total received
message-id
message-sent sec
message-sent usec
message-received sec
message-received usec
diff

The good ones to watch are message-id and diff. If message-id is out of 
order, such as follows:

0     0     998323036.529014 998323036.535219 0.006205
1     2     998323036.535521 998323036.555166 0.019645
2     1     998323036.545074 998323036.573484 0.028410

Then you got messages 1 and 2 out of order. In other words, message 2 
arrived before message 1 did, even though message 1 was sent first. The diff
column shows you the difference in time between a message was sent and
when it was received. Obviously, you need to ensure the clocks of the
sending and receiving machines are synced for this to be accurate.

:: SUPPLIED SCRIPTS ::

There are two scripts supplied with jabbertest to demonstrate how the toolset
can be used: nusers.sh and nusers2.sh. They are very ROUGH. In other words,
you will most definately want to tweak them for your use. Better yet, sit down
and figure out what you are trying to accomplish and write your own.

NOTE: The scripts depend on bash features. Update #!/bin/bash to match your
installation directory. If you do not have bash installed make alterations
as needed. I do plan on using something a little cleaner for these scripts
when I have time. - Dustin

nusers.sh - Test n Users
=========
The nusers script will create n sessions using n user pairs (n*2 actual 
connected users) by loading msgloadsnd and msgloadrec on the local machine.

nusers2.sh - Test n Users
=========
The nusers2.sh script is similar to nusers.sh but offers more advanced testing
and reporting capability. With nusers2.sh you can specify a Jabber server
in addition to remote testing clients. Unlike, nusers.sh, which has the
following data flow:

tester ---> jabberd
       <-----|

nusers2.sh uses:

sender (machine 1) ---> jabberd (machine 2) --> receiver (machine 3)

This allows you to truly swamp Jabber, which makes for informative testing. 
To use nusers2.sh specify the Jabber server, number of simulate user pairs, 
messages per second that each user pair will attempt to generate, the total
number of seconds to send messages, and the user name to use:

[dpuryear@crack tests]$ ./nusers2.sh crack 5 3 5 test_

Start time: 11:02:53
Hostname:   crack
User Pairs: 5 (10 real users connected)
Rate:       3 msg/sec
Duration:   5 seconds

*** Copying clients..
*** Eating old messages.. o
*** Starting receivers..
*** Starting senders..
*** Waiting to collect data.. oooo
*** Creating reports..
*** Done

Notice that there is little data in the output. Rather, most of the data is in
the form of reports (.rpt files), and output files on the receiving and testing
clients. For example, in this test a report file named 5x3x5.rpt was generated 
with the contents:

[dpuryear@crack tests]$ cat 5x3x5.rpt 
UserP MinDelTime MaxDelTime AvgDelTime MsgCnt ExpMsg MsgLossRate
1     0.00281    0.00816    0.00377    15     15     0.00000
3     0.00263    0.00622    0.00364    15     15     0.00000
5     0.00249    0.00933    0.00375    15     15     0.00000
7     0.00233    0.01004    0.00328    15     15     0.00000

Note that the UserP field specifies the user pair. In this instance the user 
pair 1 contained users test_0 (sending) and test_1 (receiving). In addition,
each test contributes to the summary.rpt file, which is a summary of each test
that is run:

[dpuryear@crack tests]$ cat summary.rpt 
UserPs Rate  Duration MinDelTime MaxDelTime AvgDelTime MsgCnt ExpMsg MsgLossRate
5      3     5        0.00233    0.01004    0.00361    75     75     0.00000

There are addition files on the testing clients:

Recieving Client
e.* files = stderr of msgloadrec during it's 'eat-messages' phase
r.* files = stderr of msgloadrec during it's receive messages phase
output.user-name = The timing data provided by the -d option.

Sending Client
s.* files = stderr of msgloadsnd during it's sending messages phase

In addition, jabberd provides good information in jabber-x/error.log and 
jabber-x/record.log.

Note that nusers2.sh must be modified not only via the command-line but via
variables set in the script itself. 

