mirror of
https://github.com/yasuhirokimura/db18
synced 2026-06-08 18:29:23 +00:00
Yasuhiro KIMURA
248 lines
10 KiB
HTML
248 lines
10 KiB
HTML
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
|
||
<title>Transaction throughput</title>
|
||
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
|
||
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
|
||
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
|
||
<link rel="up" href="transapp.html" title="Chapter 11. Berkeley DB Transactional Data Store Applications" />
|
||
<link rel="prev" href="transapp_tune.html" title="Transaction tuning" />
|
||
<link rel="next" href="transapp_faq.html" title="Transaction FAQ" />
|
||
</head>
|
||
<body>
|
||
<div xmlns="" class="navheader">
|
||
<div class="libver">
|
||
<p>Library Version 18.1.40</p>
|
||
</div>
|
||
<table width="100%" summary="Navigation header">
|
||
<tr>
|
||
<th colspan="3" align="center">Transaction
|
||
throughput</th>
|
||
</tr>
|
||
<tr>
|
||
<td width="20%" align="left"><a accesskey="p" href="transapp_tune.html">Prev</a> </td>
|
||
<th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th>
|
||
<td width="20%" align="right"> <a accesskey="n" href="transapp_faq.html">Next</a></td>
|
||
</tr>
|
||
</table>
|
||
<hr />
|
||
</div>
|
||
<div class="sect1" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h2 class="title" style="clear: both"><a id="transapp_throughput"></a>Transaction
|
||
throughput</h2>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<p>
|
||
Generally, the speed of a database system is measured by the
|
||
<span class="emphasis"><em>transaction throughput</em></span>, expressed as
|
||
a number of transactions per second. The two gating factors
|
||
for Berkeley DB performance in a transactional system are
|
||
usually the underlying database files and the log file. Both
|
||
are factors because they require disk I/O, which is slow
|
||
relative to other system resources such as CPU.
|
||
</p>
|
||
<p>
|
||
In the worst-case scenario:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
Database access is truly random and the database is
|
||
too large for any significant percentage of it to fit into
|
||
the cache, resulting in a single I/O per requested
|
||
key/data pair.
|
||
</li>
|
||
<li>
|
||
Both the database and the log are on a single
|
||
disk.
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<p>
|
||
This means that for each transaction, Berkeley DB is
|
||
potentially performing several filesystem operations:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
Disk seek to database file
|
||
</li>
|
||
<li>
|
||
Database file read
|
||
</li>
|
||
<li>
|
||
Disk seek to log file</li>
|
||
<li>
|
||
Log file write
|
||
</li>
|
||
<li>
|
||
Flush log file information to disk
|
||
</li>
|
||
<li>
|
||
Disk seek to update log file metadata (for example,
|
||
inode information)
|
||
</li>
|
||
<li>
|
||
Log metadata write
|
||
</li>
|
||
<li>
|
||
Flush log file metadata to disk
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<p>
|
||
There are a number of ways to increase transactional
|
||
throughput, all of which attempt to decrease the number of
|
||
filesystem operations per transaction. First, the Berkeley DB
|
||
software includes support for <span class="emphasis"><em>group
|
||
commit</em></span>. Group commit simply means that when the
|
||
information about one transaction is flushed to disk, the
|
||
information for any other waiting transactions will be flushed
|
||
to disk at the same time, potentially amortizing a single log
|
||
write over a large number of transactions. There are
|
||
additional tuning parameters which may be useful to
|
||
application writers:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
Tune the size of the database cache. If the Berkeley
|
||
DB key/data pairs used during the transaction are found in
|
||
the database cache, the seek and read from the database
|
||
are no longer necessary, resulting in two fewer filesystem
|
||
operations per transaction. To determine whether your
|
||
cache size is too small, see <a class="xref" href="general_am_conf.html#am_conf_cachesize" title="Selecting a cache size">Selecting a cache size</a>.
|
||
</li>
|
||
<li>
|
||
Put the database and the log files on different
|
||
disks. This allows reads and writes to the log files and
|
||
the database files to be performed
|
||
concurrently.
|
||
</li>
|
||
<li>
|
||
Set the filesystem configuration so that file access
|
||
and modification times are not updated. Note that although
|
||
the file access and modification times are not used by
|
||
Berkeley DB, this may affect other programs -- so be
|
||
careful.
|
||
</li>
|
||
<li>
|
||
Upgrade your hardware. When considering the hardware
|
||
on which to run your application, however, it is important
|
||
to consider the entire system. The controller and bus can
|
||
have as much to do with the disk performance as the disk
|
||
itself. It is also important to remember that throughput
|
||
is rarely the limiting factor, and that disk seek times
|
||
are normally the true performance issue for Berkeley
|
||
DB.
|
||
</li>
|
||
<li>
|
||
Turn on the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> or
|
||
<a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> flags. This changes the Berkeley DB
|
||
behavior so that the log files are not written and/or
|
||
flushed when transactions are committed. Although this
|
||
change will greatly increase your transaction throughput,
|
||
it means that transactions will exhibit the ACI
|
||
(atomicity, consistency, and isolation) properties, but
|
||
not D (durability). Database integrity will be maintained,
|
||
but it is possible that some number of the most recently
|
||
committed transactions may be undone during recovery
|
||
instead of being redone.
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<p>
|
||
If you are bottlenecked on logging, the following test will
|
||
help you confirm that the number of transactions per second
|
||
that your application does is reasonable for the hardware on
|
||
which you are running. Your test program should repeatedly
|
||
perform the following operations:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
Seek to the beginning of a file
|
||
</li>
|
||
<li>
|
||
Write to the file
|
||
</li>
|
||
<li>
|
||
Flush the file write to disk
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<p>
|
||
The number of times that you can perform these three
|
||
operations per second is a rough measure of the minimum number
|
||
of transactions per second of which the hardware is capable.
|
||
This test simulates the operations applied to the log file.
|
||
(As a simplifying assumption in this experiment, we assume
|
||
that the database files are either on a separate disk; or that
|
||
they fit, with some few exceptions, into the database cache.)
|
||
We do not have to directly simulate updating the log file
|
||
directory information because it will normally be updated and
|
||
flushed to disk as a result of flushing the log file write to
|
||
disk.
|
||
</p>
|
||
<p>
|
||
Running this test program, in which we write 256 bytes for
|
||
1000 operations on reasonably standard commodity hardware
|
||
(Pentium II CPU, SCSI disk), returned the following
|
||
results:
|
||
</p>
|
||
<pre class="programlisting">% testfile -b256 -o1000
|
||
running: 1000 ops
|
||
Elapsed time: 16.641934 seconds
|
||
1000 ops: 60.09 ops per second</pre>
|
||
<p>
|
||
Note that the number of bytes being written to the log as
|
||
part of each transaction can dramatically affect the
|
||
transaction throughput. The test run used 256, which is a
|
||
reasonable size log write. Your log writes may be different.
|
||
To determine your average log write size, use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to
|
||
display your log statistics.
|
||
</p>
|
||
<p>
|
||
As a quick sanity check, the average seek time is 9.4 msec
|
||
for this particular disk, and the average latency is 4.17
|
||
msec. That results in a minimum requirement for a data
|
||
transfer to the disk of 13.57 msec, or a maximum of 74
|
||
transfers per second. This is close enough to the previous 60
|
||
operations per second (which was not done on a quiescent disk)
|
||
that the number is believable.
|
||
</p>
|
||
<p>
|
||
An implementation of the previous <a class="ulink" href="writetest.cs" target="_top">
|
||
example test program</a> for IEEE/ANSI Std 1003.1
|
||
(POSIX) standard systems is included in the Berkeley DB
|
||
distribution.
|
||
</p>
|
||
</div>
|
||
<div class="navfooter">
|
||
<hr />
|
||
<table width="100%" summary="Navigation footer">
|
||
<tr>
|
||
<td width="40%" align="left"><a accesskey="p" href="transapp_tune.html">Prev</a> </td>
|
||
<td width="20%" align="center">
|
||
<a accesskey="u" href="transapp.html">Up</a>
|
||
</td>
|
||
<td width="40%" align="right"> <a accesskey="n" href="transapp_faq.html">Next</a></td>
|
||
</tr>
|
||
<tr>
|
||
<td width="40%" align="left" valign="top">Transaction tuning </td>
|
||
<td width="20%" align="center">
|
||
<a accesskey="h" href="index.html">Home</a>
|
||
</td>
|
||
<td width="40%" align="right" valign="top"> Transaction FAQ</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</body>
|
||
</html>
|