/*****************************************************************************
 *
 *  Copyright (C) 2004,  Bill Pierce 
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; either version 2 of the License, or
 *  any later version.
 *
 *  This program is distributed in the hope that it will be useful,
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *  GNU General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License
 *  along with this program; if not, write to the Free Software
 *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 *
 *  Bill Pierce can be contacted at:
 *  7862 NE 148th St. 
 *  Kenmore, WA 98028
 *  systems_r_up@yahoo.com
 *
 *****************************************************************************/

This is io_profile V1.00 (Beta) 7-12-04

Thanks for downloading io_profile. I hope you find it useful.  If so,
feel free to drop me a line.

                        - Bill Pierce

***  Building io_profile  ***

The io_profile utility is written in C and can be compiled
on Solaris, Linux and Win32 systems. Porting it to other flavors
of UNIX should not be difficult, and ports can be expected in
future versions. 
The code in this kit is distributed under the GNU
General Public License and the GNU Lesser General
Public License, so io_profile is guaranteed to remain
a free, open-source tool.

To build the kit, simply unpack the zip file, cd to the src  
directory, copy the appropriate makefile to a file called
makefile, type "make" ("nmake" on Win32).

for example, on Linux:
%>unzip io_profile.zip
%>cd io_profile/src
%>cp makefile.linux makefile
%>make

I have built io_profile on Linux (Red Hat 8 & 9) and Solaris 
(2.7 & 2.8 ) using gcc V2.9X, and on Windows 2000 using 
MS Visual C++ 6.0.  The build should take less than half a minute.

***  Compile-time configuration ***

Most of the configuration of how io_profile is to run is done 
at run time from command-line arguments.  However, for the
adventurous, there are 
a few parameters that can be configured at compile time in the
header file io_profile_config.h. Among these are the 
default values of some of the 
command-line parameters and also the number of levels and number
of files per level to the directory structure created when 
the program operates in multi-file mode.

***  Installing io_profile  ***

Once you have compiled a binary image of io_profile, 
copy it to a directory in your search path.  io_profile
does not depend on external libraries other than 
standard system libraries, so you should be able
to copy the binary anywhere you like and run it, 
even to other computers.

***  Binary forms  ***

In case you don't want to build io_profile yourself,
this kit includes a bin directory containing precompiled binary
forms of io_profile for Win32, Linux, and 32bit Solaris. 
Simply copy the io_profile binary executable into your path
and run it.

***  Running io_profile ***

io_profile creates a set of test files in which to
read and write in the current working directory. Before it does so,
it does a very simple check to make sure the filesystem that is
to be written to has sufficient space for the test.
The amount of space required depends on whether you are using
single (-s) or multi- file mode and the size of the test file 
(-l) you choose. It then creates a buffer of random data 
in memory to use for the test. Finally, it performs some 
exercise of the I/O channel to try to bring the channel to a 
steady state of operation before timed measurements begin.

After creating the test files and initializing, io_profile 
measures the write profile and then the read profile.

Profiles are measured by starting at the smallest (-m) read/write
and doubling it until the largest (-M) read/write is reached.
At each read/write size, a number (-n) of repetitions of the following sequence
occurs and the whole operation is timed against the system time:
1) A random file offset is chosen and it is verified that size will
fit within the file</LI>
2) A random file is chosen and opened (in multi-file mode) using
fopen()</LI>
3) and a random location within that file is chosen and seeked using
fseek()
4) the buffer is read (written) using fread() (fwrite())
5) the io stream is flushed by calling fflush() (on writes)
6) The file is closed by calling fclose()

If buffer cache bypass is enabled (-b) equivalent low-level,
OS-specific system functions
are used instead of the C-library stdio functions above.
See the source code for details.

First a sweep of writes is performed, then a sweep of reads. 
After all the repetitions are complete at each read/write size, the size,
number of operations, duration, throughput and io_rate are output 
as comma-separated values to stdout. Errors are reported to stderr.

After the test is complete, the file or filesystem created 
during the test is left behind and can be examined or deleted manually.

The whole thing looks like this when it runs:

[bill@linus sda3]$ io_profile
IO profile test parameters
--------------------------
Date: Sun Mar  7 17:12:11 2004
Location: linus:/mnt/sda3
File mode: Multi-file filesystem, 3 levels, 4 files per level
Synch mode: Using libc with no buffering
File size: 4194304 bytes
Reads (Writes) from 2 to 65536 bytes
Repetitions per read (write) size: 1000

...

Conducting write test...
I/O size [bytes], operations, time [s], throughput [bytes/s], I/O rate [1/s]
2,1000,0.078000,25641.025641,12820.512821
4,1000,0.176000,22727.272727,5681.818182
8,1000,0.075000,106666.666667,13333.333333
16,1000,0.073000,219178.082192,13698.630137
32,1000,0.072000,444444.444444,13888.888889
64,1000,0.073000,876712.328767,13698.630137
...
65536,983,1.037000,62123324.975892,947.926712
Done with write test

Conducting read test...
I/O size [bytes], operations, time [s], throughput [bytes/s], I/O rate [1/s]
2,1000,0.061000,32786.885246,16393.442623
4,1000,0.061000,65573.770492,16393.442623
8,1000,0.061000,131147.540984,16393.442623
16,1000,0.065000,246153.846154,15384.615385
32,1000,0.061000,524590.163934,16393.442623
64,1000,0.061000,1049180.327869,16393.442623
...
65536,985,0.864000,74714074.074074,1140.046296
Done with read test

***  Discussion of Command-line options ***

Two of the operational modes of io_profile warrant additional discussion.

Either a single test file (-s) (io_profile_file.out) or a small
filesytem starting with dir0 are created and used in the current working 
directory.  The size of the filesystem may be configured at 
compile-time in the io_profile_config.h header file. The size
of the data that is maintained is the number of levels times
the number of files per level times the file size. 

The -b mode of operation attempts to elminate the effects of the 
operating system's buffer cache by using lower level system 
calls and indicating in these calls that reads and writes 
should be synchronous and should not use the buffer cache.
The influence of the buffer cache is very significant because
it decouples the I/O operations the application performs
and the I/O operations the OS is performing on the 
physical devices.  Buffer cache can make a fast RAID 
device look equivalent to an IDE drive under some workloads.
Buffer cache effects also make tests
less reproducible because the results depend on the
state of the cache.  Assuming your application does not
bypass the cache, you'll want to run your tests with the cache
to get the applicaion's view of the IO performance with 
the normal caching the OS will perform.  If you want a more
direct measurement of the I/O subsystem on the other side of
the cache, however, use the -b mode.

This seems to work as designed on Windows, however it appears
the buffer cache is not completely eliminated on Solaris 
and Linux.  Writes appear to be synchronous with operations
to the physical device, but reads go so fast  

*** Known issues ***

Issue #1 Logged: 3/29/04  Status: Open
Reads still use appear to use caching even when -b is specified on Solaris 
and Linux.  Even when running with -b you'll notice that reads 
are spectacularly faster than writes, indicating that caching 
is still being used somewhere in the IO stack.  I was unable
to find a way to defeat this in this release.  See the FAQ
for workaround tips.

Issue #2 Logged: 3/29/04  Status: Fixed 5/9/04 
Reads are exceptionally slow using buffered I/O (not -b) on Solaris.
Solution: This was due to setvbuf causing the std C library
implementation to break buffer reads down into one byte
read calls.  Found this using truss. I don't know why they do this.
It did not work this way on Linux (strace).
The fix was my decision to remove calls to setvbuf in 
std C (ie. not -b mode) mode and let the C library do
its buffering.

***  Known bugs ***

*** Reporting bugs ***

Please report bugs by email to systems_r_up@yahoo.com.
I'll usually be able to get back to you within a day or two.

*** New Features in Version 1.0  ***

What?! It's the first version. It's all new.

*** Future Features ***

Support for other UNIX flavors.
Support for larger filesystems and test file sizes.
A quiet mode that outputs just the CSV data for easy parsing.

*** Acknowledgments ***

I'd like to thank my wife Lisa for supporting my need to do 
some good in the world on my own terms. I'd also like to thank TeraCloud 
for their sponsorship, allowing io_profile to remain an 
Open Source project, and providing a home address for the io_profile 
project on the Web.

***  How to get involved  ***

If you would like to contribute to io_profile development, please
drop me an email.  I am gauging interest in opening development
to the community.  Until then, I'll be the sole maintainer of the
code and will be rolling in bug-fixes and upgrades on a case-by-case
basis.


