
Issue #1
Spring, 1992
back to DBAdvisor Main page
In
This Issue:
Getting the
Most from the New Features in VAX/DBMS v4.3
By Bryan Holland
The developers of VAX/DBMS Version 4.3 have
added many features to an already feature-rich product.
Knowing how and when to implement these new features can
yield significant benefits for your applications. Deciding
which of the many new features are right for your
application, and effectively implementing them is not a
trivial task!
Global Buffers
One of the most exciting enhancements to
VAX/DBMS, is the introduction of global buffering. This
feature allows multiple users of a database on a node to
share buffers read by other database users. Depending on your
workload, this could significantly reduce the I/O bottlenecks
in your database applications. However, global buffering
requires additional synchronization, so those in an
environment constrained by CPU may not benefit from this new
feature. To fully take advantage of global buffering, you
need to adjust the GBLPAGFIL and GBLPAGES SYSGEN parameters,
as well as the VIRTUALPAGECNT authorize parameters.
Knowing how and when to implement
these new features can yield significant benefits for your
applications. Deciding which of the many new features are
right for your application, and effectively implementing them
is not a trivial task!
Fast Commits
Applications with relatively short
transactions and frequent updates to the same records will
probably benefit from Fast Commits. Fast Commits is
implemented using the UNDO/REDO recovery technique, whereby
committed transactions are committed (written) only to the
AIJ (After Image Journal). Periodically, a checkpoint is
taken in which all updated pages are flushed to the database
storage areas, and a checkpoint record is written to the AIJ.
The frequency of the checkpoints is configurable by the DBA
to occur based on AIJ file growth, elapsed time, or based on
transaction counts. Since database updates are written only
to the AIJ, Fast commits can improve performance. Standard
commit protocol requires that updates be written to the RUJ
(Run Unit Journal), the database storage areas, and the AIJ
before completing the commit. To take advantage of this new
feature, you must have AIJs enabled for your database.
Commit to Journal
The Commit to Journal optimization is
really an extension of the fast commit enhancement. When
COMMIT_TO_JOURNAL is enabled, a process allocates a range of
TSN (Transaction Sequence Numbers) during the bind to the
database. Commits are only recorded in the AIJ (rather than
to the root file), which can reduce root file I/O and
contention. This enhancement also allows for parallel commits
to occur from multiple nodes. To take advantage of this
feature, SNAPSHOTS must be DISABLED or DEFERRED. However, if
you disable SNAPSHOTS, you will be unable to perform /ONLINE
backups.
AIJ Backup Quiet Points Eliminated
VAX/DBMS V4.3 adds many
features to an already feature-rich product. Knowing how and
when to implement these new features can greatly benefit your
application. However, deciding which of the many new features
are right for your application, and effectively implementing
them is not a trivial task.
Before version 4.3, when backing up the
AIJ, DBMS would suspend transactions attempting to ready the
database until all active transactions have committed (thus
achieving a quiet point). Waiting for quiet points for
database backups has always been a problem for 7 by 24 sites
(at least since V2.0 when /ONLINE backups were first
implemented). Although a complete solution to this problem is
not being offered for version 4.3 of DBMS (stay tuned for
future releases), version 4.3 does allow AIJs to be backed up
without the need for a quiet point. This can significantly
improve concurrency, especially for those environments with
long-running transactions and high transaction volumes.
AIJ Optimization
In Version 4.3, it will be possible to
optimize the AIJ by using the DBO/OPTIMIZE/AFTER_JOURNAL
command. This minimizes the time required to roll forward a
database using DBO/RECOVER. This command creates a new
(optimized) AIJ file from an existing AIJ file, eliminating
all rolled back transactions, duplicate updates to the same
records, and updates to the database are ordered by physical
DBKEY. I strongly recommend that the ORIGINAL (un-optimized)
AIJ file always be included as part of your backup strategy,
optimizing the AIJ only when needed for a recovery
operation. An ideal time to perform this would while the
DBO/RESTORE operation is restoring your database to the last
backup.
Security Enhancements:
Version 4.3 brings NCSC Class
"C2" level security to VAX/DBMS. The primary impact
of these changes, is the ability to audit events that affect
your database. If you are in a DOD related industry, or a
pharmaceutical industry, you may want to consider the
benefits that C2 level security provides.
The new audit features of DBMS use the VMS
Auditing services. These VMS auditing services store the
audit records in the VMS audit file, and send audit alarms to
operator terminals with SECURITY enabled.
Restructuring Utility
The DRU (Database Restructuring Utility)
has undergone several changes, which allow greater
flexibility the types of changes you can now make to your
VAX/DBMS database. Starting with version 4.3 of DBMS, you
will be able to change the data type and lengths of items,
change the allocation mode (STATIC/DYNAMIC) in the storage
schema, and remove unused DEFAULT clauses from the schema.
DRU gives you the ability to make database changes directly
to the data in your database without performing an
unload and load. There are several other, less
significant changes to DRU, which help to make the product a
complete offering.
While DRU has not eliminated the need for
periodic database reloads, it certainly provides a reasonable
option in many cases.
AIJ Sequencing
The panic surrounding a typical database
recovery is not an ideal time to guess at which AIJ files to
apply to the database, and in what order. Provided that you
have implemented a solid backup and recovery strategy for
your database and your AIJs, AIJ sequencing information may
not be important to you. However, many sites will benefit
from the new AIJ sequencing information available with V4.3
that will help the DBA identify the AIJ backup media, help
correlate the AIJ backups to database backups, and allow AIJ
recovery to determine the appropriate recovery point.
DBO/SHOW LOCKS
Lock conflicts have always been a source of
frustration and confusion for VAX/DBMS users. Although this
new command will not help eliminate the frustration, it may
help alleviate some of the confusion. DBO/SHOW/LOCKS performs
a detailed analysis of database locking, including blocking
processes. This command replaces the undocumented DBO/SHOW
GETLKI command that provided a dump-like view of all database
locks on a node. The VAX/DBMS developers should be commended
on their implementation of this new command.
Multi-threaded Backup Enhancements
The multi-threaded backup utility will
support area-specific backup and restores. This is useful for
those sites who run a periodic archive/purge routine on large
areas, but don't require a full backup of the entire database
after such an operation. Performing area-specific backups can
save considerable time.
Large VAX/DBMS sites will benefit from the
additional device support provided by the multi-threaded
backup facility. These enhancements include support for the
following devices: TA91s, Data compression for TA90-E drives,
and stack loader support for TF-857 tape drives.
DBO/VERIFY enhancements
DBO/VERIFY is the DBA's first line of
defense against database corruption. This utility performs
internal integrity checks on your database to detect most
types of internal corruption.
Starting with version 4.3, DBO/VERIFY will
verify the contents of the ROOT file, check for inconsistent
and corrupted areas, and the new /SUMMARY command-line
qualifier will produce summary messages.
Unfortunately version 4.3 still does not
perform a complete check for orphaned records, or validate
errors in page free space calculations.
DECtrace V1.1
If you are one of the sites who are
benefiting from the use of DECtrace to analyze performance
bottlenecks, you will want to make certain that you upgrade
to version 1.1 of DECtrace before installing version 4.3 of
DBMS.
Version 4.3 brings NCSC Class
"C2" level security to VAX/DBMS.
Fewer DBO/MODIFY commands require
exclusive access
Ask a 7 by 24 show when they can shutdown
the database to perform certain maintenance functions, and
the response is NEVER. Realistically, you should anticipate
some down time. Fortunately, DBMS is moving in the right
direction, allowing the DBA greater concurrent access to the
database. Starting with V4.3, you will be able to perform the
following DBO/MODIFY commands without requiring exclusive
access to the database:
- /AIJ_OPTIONS...
- /BUFFERS
- /DBR_BUFFERS
- /EXPANSION
- /EXTENSION
- /GLOBAL_BUFFERS...
- /OPEN
- /READ_ONLY
- /RECOVER_JOURNAL
- /SNAPSHOTS=ALLOCATION
- /THRESHOLD
RUJ naming and placement
To alleviate the problems caused by
directory version limits, and to avoid the potential for RUJs
being purged, RUJs (Recovery Unit Journals) V4.3 will have a
time stamp embedded in their names. The default location for
RUJs will be SYS$LOGIN_DEVICE:[DBM$RUJ]. If you are running
in a cluster environment, you will want to make certain that
this device is available CLUSTER-WIDE for recovery purposes.
Database Statistics can be disabled
The new DBO qualifier, /[NO]STATISTICS,
added to the CREATE and MODIFY commands to enable and disable
database statistics reporting. Although these statistics
contain a wealth of useful information for optimizing a
database, it requires small amount of overhead to gather
these statistics. Those shops who want to reduce DB overhead
to a minimum may want to disable database statistics for
their production databases. When necessary, you can enable
statistics, allowing the gathering of statistics for
performance monitoring and analysis.
SPAM algorithm
When the space management (SPAM) technology
was first introduced in V2.0, it significantly improved the
performance of storing new data in partially full areas.
However, it was still possible to exceed the T3 threshold
(the point at which a database page is considered
"full") of a page when storing new records. This
could lead to record fragmentation problems when inserting
records into sets or when dynamic storage items expanded.
Get the most
from the new features of VAX/DBMS v4.3
call the VAX/DBMS specialists:
Software Concepts Internatioal
(603) 879-9022
Starting with V4.3, the DBMS will not
exceed the T3 threshold when storing new records. This is not
to say that this change will eliminate all record
fragmentation. However, with careful tuning of the SPAM
thresholds, you can significantly reduce the possibility of
record fragmentation.
In general, the assumptions you made
regarding the threshold values will continue to be fine.
However, considering these changes, it makes sense to review
your SPAM thresholds before reloading the new database.
DBO Indirect list file processing
As DBMS matures, the complexity of the DBO
commands increase with each version. With the added features
and added complexity the DBO command length is
forever-increasing. To help us deal with this trend, starting
with V4.3, any DBO command that accepts a list of values, can
have those values processed from an indirect list file. To
use an indirect list file, you type "@filename" in
place of the command list.
Contents
By Bryan Holland
Is your corporate data really safe? Before
you answer this question, you must first ask "Safe from
what?"
Database security is often confused
with data secrecy. To fully evaluate whether your
database is secure, the definition of database security must
include secrecy, integrity, and availability.
There are three essential aspects to data
security; secrecy, integrity, and availability. Therefore, to
answer the simple question, "Is you data safe?" you
need to look at each aspect separately. Different sites have
different requirements for secrecy, integrity and
availability. Therefore, it is impossible to solve all
"security" issues with a single solution. The
security solution must match the unique privacy, integrity
and availability requirements of each site.
Security from the database perspective
Computer security is a highly complex topic
because there are so many different aspects to it. The
purpose of this article is to help you understand basic
database security issues. This article is not intended to be
an exhaustive reference of securing computer systems; such an
article requires significantly more space than is available
here. Therefore, I will assume a generally secure computing
environment, and will focus on security from the database
perspective.
VAX/DBMS has many features that ensure a
high level of database security, in each of the aspects
mentioned above.
Secrecy
Database secrecy refers to the protection
of data against accidental or intentional disclosure to
unauthorized persons, or unauthorized modifications or
destruction. DBMS provides many tools to prevent unauthorized
persons from accessing your data.
Database security is often confused
with data secrecy. To fully evaluate whether your database is
secure, the definition of database security must include
secrecy, integrity, and availability.
VMS Protection
The security features of VMS influence the security
mechanisms of VAX DBMS. The file protection of database files
(.ROO, .DBS, .SNP) should be set to prevent access by VMS
utilities, such as delete, rename, copy, backup, etc. Proper
VMS protection of your database files, guards against
corruption or illicit access to data through non-DBMS
utilities. AIJ files should be placed in a directory
accessable only to the DBA.
The database monitor log file (DBMMON.LOG) provides
information about all database attaches. An unscrupulous user
could use the information contained in this file to determine
the location of other databases on your system. Armed with
this information, they may try to exploit any holes in your
system security to gain unauthorized access to your database
files.
Environment with very high security requirements may also
want to restrict access to the DBO image. DBO/SHOW USERS
shows information about who is running what images, and to
which database they are attached.
Unless your database is secured using security schemas,
DBQ provides unbridled DML access to your database.
Restricting access to the DBQ image to ensure that your
database is being accessed only through the controls built
into your application programs.
Security Schemas
Security Schemas provide a way to control DML access to
sensitive data. A user can access a data entity only if it is
included in their subschema view and only to the extent
allowed by the security schema mapping.
In versions of VAX/DBMS before v4.3, every database was
created with an empty user execution list (UEL) in the root
file. As long as the UEL is empty, the database is unsecured
against DML access. Starting in v4.3, all newly created
databases will have only the creator's UIC placed in the UEL,
thus allowing only the creator DML access to the
database. Likewise, all securable DBO commands will only be
granted to the creator of the database.
Command Authorization Lists (CALs)
While security schemas provide access control over DML
access to the database, CALs provide access control over DBO
access to the database.
If used carelessly or maliciously, some of the DBO
commands can destroy data or corrupt the integrity of the
database, while others may have a negative impact on database
performance or could be used to illicitly gain access to the
information contained in the database. CALs protect the
database against unauthorized DBO access. Secure those DBO
commands that access only the metadata in the CDD through the
CDD security mechanisms.
The RUJ recovery mechanisms provide
the most crucial functions for maintaining database
integrity. ...Updating your database in BATCH UPDATE mode
(disabling RUJs) is like engaging in unsafe...computing!
If a CAL contains one or more UIC entries, then only those
users with matching the entries in the CAL will have access
to the DBO command. If the CAL is empty, then the DBO command
for that CAL is unsecured. Version 4.3 changes the way CALs
are placed on newly created databases -- By default, only the
creator will have access to all DBO commands.
Auditability
The DBMMON log file has always provided limited audit
history of database attaches. Version 4.3 greatly extends the
audit capabilities through the DBO/AUDIT command. Event
audits for ACL checks, ACL changes, DBO usage and DML access,
can be enabled on a per-database basis. Alarms and/or journal
records can be produced for each type of event.
Integrity
Database integrity refers to the logical
and physical soundness of the data stored in a database. If
the data is physically corrupt, DBMS will not be able to
access the data. The DBO/VERIFY utility checks the database
for physical corruption and reports any inconsistencies.
Verification of the logical integrity of data is a more
difficult problem. Schema check clauses provide some level of
logical integrity control, but can not come close to
providing a complete validation.
DBMS maintains database integrity by
automatic detection, rollback, and recovery of incomplete
transaction through user based recovery-unit journaling,
enabled after-image journaling, automatic use of the VMS
distributed lock manager, and automatic database recovery
following a complete system failure.
Distributed Lock Manager
VAX/DBMS uses the distributed lock manager extensively to
maintain database integrity. The lock manager controls
virtually all aspects of database integrity from managing
process failure, cluster fail over, concurrent access,
snapshot sequencing, buffer management, and much more.
Recovery Unit Journals
By default, all database transactions record a
"before image" of all database updates to the
Recovery Unit Journal (RUJ) file. The RUJ recovery mechanisms
provide the most crucial functions for maintaining database
integrity. If the transaction is aborted (due to a system
failure, software failure, or user-initiated termination
[Ctrl-Y]), DBMS automatically rolls back (undoes) the
partially completed transaction. This helps to ensure both
the logical integrity and physical integrity of your
database.
I noted above, that DBMS logs all updates to the database
to the RUJ file by default. I implied in this statement, that
it is possible to "disable" RUJs. It is possible to
"disable" writing to the RUJ by readying the
database for BATCH UPDATE. Updating your database in BATCH
UPDATE mode (disabling RUJs) is like engaging in
unsafe...computing! If there is any failure during a
BATCH UPDATE transaction, your database will be marked
CORRUPT!
To guard against unintentional use of BATCH UPDATE, I
strongly recommend that all sites implement at
least one security schema that denies BATCH UPDATE to all
users -- For those rare times when you require batch update
mode, grant an alternative security schema to the user's UIC
for the duration of the BATCH UPDATE transaction.
DBO/Verify
DBO/VERIFY is your best means of early detection of
physical corruption in your database. Verify performs
physical integrity checks of the internal page format, record
structures, index structures, and most set pointers in your
database.
Frequent use of DBO/VERIFY will help to
ensure that you can recover your database from your most
recent backup(s) and After Image Journals.
If you can't recover your database from a
backup, DBO/VERIFY provides a skilled technician with
valuable information about the location and type of
corruption in your database. They can use this information to
perform low-level patches to the database using the DBO/ALTER
utility.
DBO/BACKUP & DBO/RESTORE.
If updating your database without using RUJs is like
practicing unsafe computing, then failure to adequately
DBO/BACKUP your database is like playing Russian-Roulette.
DBO/BACKUP and DBO/RESTORE provide the single most important
mechanism for recovering your database in the event of
database corruption.
While I include media failure (where the disk that
contains your database has been completely obliterated by a
head crash) as an extreme form of database corruption, the
corruption could be more subtle such as an
"isolated" pointer problem. In either case, being
able to successfully use DBO/RESTORE to restore your database
to the last known corruption-free point is critical to being
able to successfully restore your database with confidence.
Availability
Database availability refers to the
accessibility of the data within the database. No matter how
accurate, free of corruption, and logically organized your
data is, it is useless if it is not available when
needed. VAX/DBMS provides many mechanisms to ensure high
availability to the data in your VAX/DBMS database.
VAX/Cluster support
VAX/DBMS allows concurrent, multi-processor access to your
database when running in a VAXcluster environment. If one of
the nodes on your VAXcluster fails, DBMS will automatically
recover your database on one of the surviving nodes.
DBO/BACKUP and
DBO/RESTORE provide the single most important mechanism for
recovering your database in the event of database corruption.
In a properly configured VAXcluster environment, it is
possible to achieve virtually uninterrupted access to your
VAX/DBMS database. Proper file placement is one key to high
availability in a VAXcluster environment.
In a VAXcluster environment, you want to make certain that
all database files (.ROO, .DBS, .SNP, .RUJ, .AIJ) are
available to all nodes sharing the database. The RUJ
files are the most critical, since they must be accessible to
in order for automatic recoveries to occur if a node fails.
By default, your RUJ files are placed in your sys$login
directory (except in v4.3, where the default is [dbm$ruj] on
the sys$login device). To override the default location of
the RUJ files, define the system logical name DBM$RUJ to
point to the directory to contain the RUJs. The directory
that contains the RUJ files should NOT have version
limits enabled.
Place your AIJ files on a separate device from all other
database files. DBO/CREATE defines the placement for the
AIJs, while DBO/MODIFY modifies the AIJ file placement.
Dual-pathed HSC disks provide you with the highest level
of availability -- If one HSC were to fail, the other can
take over. Dual-pathed MSCP-served disks also provide high
availability, but do not provide for high performance, except
for the node to which the disks are connected.
DBO/BACKUP
There are four primary ways in which DBO/BACKUP improves
database availability: Use of the /ONLINE features for
ONLINE backups; Multi-threaded features for maximizing
throughput; /INCREMENTAL backups to minimize the
amount of data to be backed up; and by-area features
to further minimize the amount of data being backed up.
Performing ONLINE backups can significantly improve
availability, since the ONLINE backups allow full concurrent
access to the database during the backup. Thus, it is
possible to obtain a consistent backup of your database, even
while other processes update the database. You must have
snapshots allowed and enabled to perform online
backups.
Multi-threaded backups optimize database backups by
concurrently backing up multiple areas to multiple tape
devices. Thus, in an ideal environment, the time required to
backup your database could be reduced to the time required to
backup up your largest area file.
INCREMENTAL backups save time by backing up only those
database pages that have changed since the last full
backup.
DBO/RESTORE
If a database or media failure occurs, you will need to
quickly restore your database. The multi-threaded features of
DBO/RESTORE allow you to maximize the throughput of your
database restores by concurrently restoring multiple database
files from multiple tape devices. In addition, it is also
possible to use by-area restores, to restore only those areas
that were on the failed device(s).
The most stringent database
security levels may be too costly for some organizations.
Likewise, the cost of recovering from system or media
failures, or a breach of privacy may be too great to justify
applying only minimal security features.
After Image Journals
After image journaling (AIJ) is your database's primary
protection against media failures (disk crashes). If you have
AIJs enabled and you experience a media failure, you can
fully recover your database back to the last completed
(committed) transaction -- Without loosing a single update!
To achieve this level of protection, place your AIJ file on a
separate device from any of your database files.
In general, you can expect about a 4% performance penalty
for using AIJs. However, starting in V4.3, if you use AIJs
and have "FAST COMMITS" enabled, you may actually improve
database performance with AIJs enabled.
Bound shadow sets
Bound shadow sets also provide high availability, and can
improve performance in read-intensive environments.
Summary
VAX DBMS provides secure environments with
all the tools necessary to achieve level C2 security.
VAX DBMS security is UIC based. Therefore,
a user with BYPASS privilege is not subject to UIC checking
and can therefore access any file, including those of a DBMS
database. If you need a very secure environment, you will
want to strictly control privileges such as BYPASS, SETPRV,
SECURITY, and READALL.
VAX/DBMS provides many facilities to tailor
your database security requirements to meet the needs of your
organization. The most stringent database security levels may
be too costly for some organizations. Likewise, the cost of
recovering from system or media failures, or a breach of
privacy may be too great to justify applying only minimal
security features.
Database
Corruption
got you down?
Call Software Concepts Internatioal.
Discover how to protect your database from corruption,
unauthorized access
or hardware/software failures.
(603) 879-9022
Determining the proper level of security
features to implement in your environment requires a
cost/benefit analysis to determine the cost of implementing
security options versus the cost (risk) of breaching database
secrecy, integrity or availability.
Contents
By Bryan Holland
Verify--It's the tool we love to hate! We
love it, because it gives us confidence that our database is
free of internal corruption. We hate it because it is a
strain on our system resources.
The case for running DBO/VERIFY
Despite the integrity mechanisms built
into VAX/DBMS, database corruption is still possible.
Database corruption can result from hardware failures, media
failures, software failures, or user actions.
Running DBO/VERIFY
is NOT an option.
VAX/DBMS maintains integrity controls
across all VAX/VMS environments using automatic detection,
rollback and recovery of incomplete transactions with
recovery unit journals, the distributed lock manager, and
database recovery processes if there is a complete system
failure.
Despite the integrity mechanisms built into
VAX/DBMS to safeguard your database, corruption is still
possible. Database corruption can result from hardware
failures, media failures, software failures, or user actions.
Recovering from corruption
Although the focus of this article is not
about "recovering from corruption," a few words on
this topic are worth mentioning. The most critical aspect of
recovering from database corruption is quick identification
of the corruption, access to valid backups, and the use of
After Image Journaling (AIJ). The performance impact of AIJs
is negligible when compared to the cost and risk associated
with manually reentering transactions (since the last valid
backup) or low-level patching of corrupted databases.
Verifying your VAX/DBMS database is NOT
AN OPTIONIt must be done! However, performing a
complete verification of your database using the
"/SETS" option can prove to be a time and resource
drain on your system. Therefore, it seems only reasonable
that besides optimizing our application, we must also
optimize the database verifications.
Improving VERIFY performance
On the surface, DBO/VERIFY DOES not offer
any tuning or optimization parameters. You ask, "How can
we optimize DBO/VERIFY performance?" This is the topic
of this article.
Step 1:
The first step in tuning DBO/VERIFY is to
make certain that you start with a well-tuned database. If
your database is performing poorly, and it is inefficient to
access the data in your database, DBO/VERIFY will perform
poorly. That is not to say that you cannot improve the
performance of DBO/VERIFY for a poorly tuned database, but
you will never achieve an optimal level of performance.
Increase buffering
Verify is generally I/O intensive. One of
the easiest ways to improve the performance of DBO/VERIFY is
to use a large number of buffers. Increasing the number of
buffers increases the likelihood that a page will already be
in the buffer pool if it is accessed a second time.
Although there are many ways to increase
the number of buffers for verify, the preferable method for
increasing buffers (for verify) is to define the process
logical name DBM$BIND_BUFFERS. The goal is to verify the
database with the fewest I/Os using the fewest buffers.
A second aspect to buffering, is modifying
the length of the buffers used by verify. Increasing the
buffer length will cause more database pages to be read into
the buffer pool with each I/O, and can significantly improve
the performance of large sequential reads. Depending on your
database environment, increasing the size of buffers using
DBO/MODIFY/LENGTH can improve database performance.
Unfortunately, there is not a process
logical name that will allow you to modify the buffer length
at bind-time. Therefore, unless you can run your
verifications standalone, you should set the value of LENGTH
to that will optimize the overall performance of your
database application.
Multithread verify
Partitioning the verify by area, and
performing the verifications in parallel can yield
significant performance improvements. This allows you to
tailor the performance of verify to fully use the computing
power and bandwidth available. In general, you will want to
submit additional verifications (in parallel) until you
saturate the I/O bandwidth or the CPU resources across your
cluster. Submit parallel verifications in descending order of
the time required to verify the area.
One of the easiest ways to improve
the performance of DBO/VERIFY is to use a large number of
buffers. The goal is to verify the database with
the fewest I/Os using the fewest buffers.
System and process quotas
Improving performance of any application,
is really a matter of efficiently using all available
resources. This typically means using more memory and CPU to
minimize I/O. To fully realize these benefits, we must first
make certain that our process can use the available memory.
System (SYSGEN) parameters:
- GBLPAGCNT
- GBLPAGFIL
- VIRTUALPAGECNT
- WSMAX
Process (UAF) parameters
- ASTLM
- BYTLM
- DIOLM
- ENQLM
- PGFLQUOTA
- WSEXTENT
- WSQUOTA
Boost process priority
Thus far, we have looked at ways to improve
verify performance by reducing I/Os at the expense of memory
and CPU usage. If we accomplished our previous objectives, we
should begin reaching a CPU bottleneck. To give verify an
edge over other processes on the system, increase the
priority of the verification process to use a larger portion
of the CPU cycles.
ONLINE verify
If your verifications still run too long,
or if you require 7-by-24 up time, ONLINE verifications allow
you to verify your database concurrently with other processes
accessing the database.
Reduce verification level
As a last resort, you can try
reducing the integrity level of the verification. By
verifying only the PAGE or SEGMENT structures, you can
significantly reduce the overhead associated with verify.
However, PAGE and SEGMENT verifications also significantly
reduce the integrity level of verify by completely bypassing
the validation of the set-pointers. Although your verifies
will complete faster, you failed to check the most critical
aspect of a VAX/DBMS database -- the set pointers.
Incremental verifies
Incremental verifies are a quick and
efficient way to verify those database pages that have
changed since the last full or incremental verification.
Incremental verifies take considerably less time than a full
verify, allowing you to perform incremental verifies more
frequently than you would full verifies. If possible, it
makes sense to perform incremental verifies daily, and full
verifies before your full backups.
Monitoring VERIFY performance
It is possible to monitor the performance
of DBO/VERIFY with such tools as:
- VMS monitor to determine possible CPU
bottlenecks.
- DCL/SHOW PROCESS/MEMORY to monitor
memory usage
- DBO/SHOW STATS to analyze data file
I/O operations and stalls to determine possible disk
bottlenecks.
Using these tools, it is possible to
determine what are the bottlenecks when running DBO/VERIFY.
You can then take corrective action to improve the
performance of VERIFY. As in any tuning exercise, this
requires that you make changes incrementally, and monitor and
record the results.
Still not enough...
At some point in the process, you may
determine that you are fully using all CPU cycles all
available I/O bandwidth. If you still require greater
throughput for your verifies, you will need to consider
buying more memory and CPU cycles. As your local DEC sales
representative will tell you, memory and CPU cycles are
cheap.
Summary:
Although DBO/VERIFY does not offer any
tuning parameters, you still have significant control over
the performance of your verifications, including:
- Start with a well-tuned database.
- Increase the number of buffers --
DBM$BIND_BUFFERS
- Increase the size of the buffers
DBO/MODIFY LEN=n
- Multi-threaded VERIFY.
- Tune system and process quotas for
optimal efficiency.
- Increase the priority of the
verification process.
- Perform incremental verifies.
- Use /ONLINE verifications to allow
concurrent access to the database during
verification.
- Reduce the level of verification and
perform "/SET" verification only on those
pages with checksum or segment errors.
Although DBO/VERIFY does not offer any
tuning "parameters", knowing how to exploit the
general tuning capabilities of VAX/DBMS gives you tremendious
control over DBO/VERIFY performance.
IS
DBO/VERIFY
STILL RUNNING ???
Software Concepts Internatioal
can help!
Their VAX/DBMS specialists can
identify and solve your
database bottlenecks.
(603) 879-9022
Contents
INTOUCH 4GL
Adds VAX/DBMS Interface
By Dan Esbensen
Perhaps one of the most frustrating aspects
of working with a network style database, is the apparent
lack of productivity tools. Touch Technologies, Inc., of San
Diego, CA, recently announced their VAX/DBMS interface to
their high performance 4GL language, INTOUCH.
The INTOUCH interface to VAX/DBMS provides
4GL access to the VAX/DBMS database engine. The design goals
for this project were to provide a high performance interface
to VAX/DBMS that provides the ease-of-use of a 4GL while
maximizing database concurrency and minimizing locking.
Set types and currency indicators are
concepts that are unique to VAX/DBMS. Although they provide
VAX/DBMS with efficient access paths among records, they also
serve as a source of confusion for 3GL programmers. A set
type is a defined relationship among different record
types in a database. A set occurrence is a single
instance of an owner record that has zero or more member
record occurrences. Currency indicators serve as place
holders in the database. Currency indicators mark your
position within the database, each realm (one or more files),
each record type, and each set-type.
From a performance and locking standpoint,
each currency indicator places a lock on a record in your
database, thus restricting access to that record. Excessive
active currency indicators mean excessive record locking.
The 3GL problem...
The biggest problems faced by most VAX/DBMS
development environments are low productivity and excessive
locking. Productivity suffers as a result of dealing with the
complex issues of set types, set occurrences, and currency
indicators, all of which are necessary to navigate a network
style database using a 3GL programming language.
Traditionally, most VAX/DBMS development
environments have been weak in their management of currency
indicators and thus become burdened by excessive lock
conflicts and unnecessary overhead. Mismanagement of currency
indicators is a natural result of dealing with the
complexities of currency indicators when programming in
traditional 3GL/Data Manipulation Language (DML) languages.
This results in poor database performance, low transaction
throughput, and in extreme cases complete database deadlocks.
Currency indicators, a necessary evil...
Programming for a VAX/DBMS database would
be impossible without the use of currency indicators.
However, each currency indicator places a read-only lock on
your records. By default, VAX/DBMS updates all possible
currency indicators for each DML access -- thus locking
records with unused currency indicators. Programming around
this in traditional 3GL languages requires significant
expertise in the use of currency indicators, and additional
code.
Most other 4GLs either do not support DBMS,
fail to make use of its sets, or are so cavalier with the use
of currency indicators as to render them unacceptable for
multiple user environments.
Most other 4GLs either do not
support DBMS, fail to make use of its sets, or are so
cavalier with the use of currency indicators as to render
them unacceptable for multiple user environments.
The INTOUCH solution...
INTOUCH solves these problems by providing
an easy-to-use 4GL interface to VAX/DBMS that minimizes the
programmers' need to manage complex currency issues. INTOUCH
makes full use of DBMS sets, including indexed access by key
values, range retrievals, and access to the owner record of a
set occurrence. INTOUCH understands database set
types, and automatically handles' currency indicators.
In fact, INTOUCH automatically establishes currency
indicators only when necessary, and releases
them as soon as they are no longer needed.
Mixed database environments
Most application environments require
access to more than just one data source. Using INTOUCH, it
is easy to access data from different databases from within a
single application, all sharing the common INTOUCH syntax!
Thus, programmers don't need to know to access RMS, DBMS,
Rdb/VMS, or ADABASE -- INTOUCH provides a common interface to
your data. Within a single program, it is easy to mix and
match data from all of these sources!
Programmer Productivity.
Because INTOUCH understands set types and
currency indicators, INTOUCH relieves the programmer from
dealing with these complex issues. Programmers spend more
time solving business problems and less time thinking about
how to navigate the database.
Most application environments
require access to more than just one data source. Using
INTOUCH, it is easy to access data from different databases
from within a single application, all sharing the common
INTOUCH syntax!
Using INTOUCH it is very easy to make
sophisticated user interfaces and screens, including pop-up
windows, for interactive applications. While many languages
require the use of external libraries or outside routines,
the INTOUCH language has all screen handing and keyboard
mapping tools built right in. The powerful formatting
capabilities of the INTOUCH language make report generation
equally easy. Creating complex reports is easy, since
programmers concentrate on the report format, rather than how
to access the data needed for the report.
Locks Less, works great!
Because INTOUCH only updates those currency
indicators that are required to navigate your database,
INTOUCH locks less. This vastly improves database
concurrency, allowing more users to simultaneously access the
database with greater throughput.
Can INTOUCH really help?
To find out how much INTOUCH can improve
your development productivity, call Touch Technologies for a FREE
30 day trial.
To understand the significance of INTOUCH's
lock management, try the following example on your VAX/DBMS
database.
The following two DBQ* examples illustrate
the reduced locking achieved when programming in INTOUCH
versus typical 3GL programming. These examples illustrate the
lock reduction possible when using INTOUCH. They are not
examples of the INTOUCH language syntax.
Typical locking in a 3GL
dbq> FREE ALL CURRENT
dbq> LOOP
dbq> FIND NEXT
dbq> END
dbq> SHOW LOCKS
Typical locking using INTOUCH:
(This is NOT an example of INTOUCH code)
dbq> FREE ALL CURRENT
dbq> LOOP
dbq> FIND NEXT RETAINING
dbq> END
dbq> SHOW LOCKS
Compare the number of locks held after each
test. INTOUCH locks less!
Get INTOUCH !
Discover the difference with
INTOUCH, the high-performance 4GL for VAX/VMS systems!
Call Touch Technologies,
Inc.
for additional information or
a FREE evaluation copy of INTOUCH!
Call Touch Technologies,
Inc. at
(800) 525-2527 or (619) 455-7404
* DBQ is a database query utility included
with VAX/DBMS. It is a utility that lets you interactively
retrieve, update, and store database records using COBOL-like
DML syntax.
Dan Esbensen is an associate VAX/VMS
consultant with System Management Software, Inc. and
President of Touch Technologies, Inc. Dan has over 11 years
experience with VAX/VMS.
Contents
By Bryan Holland
Unloading and reloading a VAX/DBMS database
is often a lengthy and risky process. The process is lengthy
due to the huge volume of data involved, and risky due to the
complexity of the relationships defined in the SCHEMA and the
potential for error in defining the set significant
information. Creating and maintaining reload scripts was
expensive due to the expertise required to design a reload
strategy, and the time involved in manually developing the
reload scripts!
To solve these problems, Software Concepts
Internatioal developed ReloadExpert, an optimizing
Unload/Reload script generator for VAX/DBMS databases.
ReloadExpert is the most efficient and cost-effective tool
for generating all files required to efficiently reorganize
your VAX/DBMS database safely!
ReloadExpert minimizes the
time required to perform database unload and load operations
by using its extensive knowledge base to optimize the
database load scripts, and by generating multiple unload
scripts that allow for multi-threading the unload.
ReloadExpert minimizes the
time required to perform database unload and load operations
by using its extensive knowledge base to optimize the
database load and unload scripts.
ReloadExpert minimizes
the risk of introducing errors into the database during a
restructuring operation, by carefully analyzing every set in
the database and extracting all set significant information
during the unload operation. The ease in which reload scripts
are generated virtually eliminate the expense of creating and
maintaining reload scripts!
Impressed the Experts!
I've had a lot of experience unloading and
loading databases. What impresses me the most about ReloadExpert,
are the creative solutions it developes when solving complex
reload problems. In a matter of seconds, it created better
reload scripts than it took me hours to write by hand!
Consider the following segment from a database Bachman
diagram:
When I first looked at this problem, the
solution seemed obvious. Load record-types "B" and
"C" hierarchically under "A", and then
load "D", using logical keys for "B" and
"C" to make the proper set connections.
ReloadExpert showed me that
this is not the optimal load strategy! The preferred
strategy, and the one choosen by ReloadExpert,
is to load "B" hierarchically under "A"
and then load "D" hierarchically under
"C", using the logical keys for making the proper
connections to sets "a_c" and "b_d".
ReloadExpert produces all
files needed to successfully unload and load your VAX/DBMS
databases using the standard VAX/DBMS unload and load
facilities. All script files are fully commented, describing
how each set relationship will be maintained during the load
operation.
Ease of generation:
The extensive knowledge base used by ReloadExpert
analyzes both the logical and physical data definitions in
the database to minimize user input, while generating a truly
optimized reload script. Only when the schema definition does
not guarantee a unique key to the most efficient access paths
does ReloadExpert prompt for input. The ease of generation
can save you thousands of dollars in development and
maintenance costs over manually developed scripts!
Why are ReloadExpert scripts so efficient?
Several components of ReloadExpert ensure
that the reload scripts are truly optimized.
- Every record-type is analyzed to
ensure that it is loaded using the most efficient
access path in the database.
- Load routines adjust database
buffering to assure maximum use of available system
memory for optimum load performance.
- Unload scripts may be multi-threaded,
to maximize throughput of the unload process.
- The unload and load facilities enter
the DBCS at a lower level than high level application
programs, resulting in more efficient code than high
level languages!
Reorganizing of your database is a
time-critical function. You can not afford an inefficient
unload and reload strategy! With ReloadExpert,
your reload scripts are always up-to-date, and always
optimized!
ReloadExpert licenses are only $5,000
per-CPU-- the efficient scripts that ReloadExpert generates
can be run on any node in the customers organization! Thus,
multiple node sites realize even greater savings!
Fast
reloads
without
writing
a single line of code!
ReloadExpert automatically
generates optimized unload
and reload scripts without writing a single line of code!
Call today for additional information
about
ReloadExpert and never write another reload!
(603) 879-9022.
Contents
by Bryan Holland
Is your VAX/DBMS database performing
optimally? Are you concerned with the security of your
database? Are you finding strange new error codes being
reported by DBO/VERIFY? Is your system disk being filled with
DBMBUGCHK.DMP files? Perhaps your business has changed, but
your database hasn't! Now is the time to bring the two back
together!
VAX/DBMS is a high-performance database
management system. Achieving optimal performance from
VAX/DBMS requires a thorough understanding of how and when
each feature is to be implemented and the relationships
between them.
VAX/DBMS is a high-performance
database management system. Achieving optimal performance
from VAX/DBMS requires a thorough understanding of how and
when each feature is to be implemented and the relationships
between them.
Trying to understand all of the intricacies
of such a powerful and complex product can be overwhelming.
To achieve optimal performance from these complex features,
you need people that specialize in VAX/DBMS.
Let's face it, DBMS is not a
"new" technology. While everyone seems to be
scrambling for the newer style relational and object-oriented
databases, finding true VAX/DBMS specialists is becoming
increasingly more difficult.
Vendor
Profile
Software Concepts Internatioal was
founded in 1986,
and has been serving the specialized needs of VAX/DBMS
sites ever since. Bryan Holland, the principal of SCI
has over 9 years of experience in building VAX/DBMS
based applications. He is a frequent presentor of technical
VAX/DBMS material at national conferences. Bryan is
considered
one of the top VAX/DBMS specialists.
However, the good news is, that there are
still many applications that fit the time-proven
network-style database model (upon which VAX/DBMS is based).
Applications such as Concilium's Workstream, ASK MAN-MAN, and
Arthur Anderson's FMD and MACPAC thrive on the efficiency of
VAX/DBMS' imbedded pointers.
When your factory and business rely on
VAX/DBMS, it is important to develop a relationship with a
reliable vendor that understands your special needs, and can
provide support for your database and your business.
In today's competitive business environment, you can't afford
to be struggling with your database while your shop floor
remains idle, or suffers from sluggish database performance.
You need DBMS expertise!
Software Concepts Internatioal
specializes in VAX/DBMS applications. SCI provides
VAX/DBMS support in the following areas:
- Repairing corrupted databases.
- VAX/DBMS database performance tuning.
- VAX/DBMS database optimization
analysis and implementation.
- Analysis of logical and physical
database design options.
- Developing database modification
strategies to optimize performance and flexibility
while minimizing the impact on existing application
code.
- Database security and risk analysis.
- Developing and implementing database
backup and recovery strategies.
- Designing and implementing alternative
VAX/DBMS database UNLOAD and LOAD strategies.
- Database administration training.
- Database application development
training.
- Application development support.
Whether you developed your VAX/DBMS
application in-house, or you purchased it from a software
vendor, your best source for database support is from the
VAX/DBMS experts. Call Software Concepts International at
(603) 879-9022 for all your VAX/DBMS support needs!
Contents
| Phone |
USA Toll-free (Sales) |
Fax |
e-mail |
| (603) 879-9022 |
1-888-CODASYL |
(603) 879-9023 |
Copyright © 1996 Software Concepts
International Inc. All rights reserved.