NexusDB

Please note: This article contains old NexusDB V3 beta information and might contain outdated, wrong or misleading information.

As most of our customers know from comments on the newsgroup, we have a test application in place which is able to run thousands of fully independent SQL and Pascal scripts. The application has manual and batch modes. The first we use for debugging purposes, the second is run automatically with different test configurations on a regular base to keep regressions at bay.

We have finally found some time to clean up the application and will now make it available as "NexusDB Test Runner" to all our customers for creating and running their own tests.

This should make testing for regressions that we missed much easier for you. At the same time we would also want to encourage you to try and supply us with test case that can be used with the application. As a matter of fact, the first step after we get to work on a confirmed issue from a customer is to create a test case that represents the issue. We use the manual mode of the application to debug most of the issues. Every resolved issue gets it's own test in our framework, to avoid the problem to come back as side effect of any other changes.

As mentioned, the Test Runner is capable of running SQL scripts as well as Pascal scripts.

SQL script tests

Let's first take a look at SQL scripts.

Here's a simple example:

-- PATH = {BASEDIR}\0001

drop table if exists t1;
create table t1 (f1 autoinc);

insert into t1 values(0);
insert into t1 values(0);
insert into t1 values(0);

declare v1 integer;
set v1 = (select max(f1) from t1);

if v1 <> 3 then
  signal 'Expected value = 3, actual value = '+cast(v1 as varchar(10));
end if;

Please note the first line, which is telling the Test Runner where the data for the test is (to be) located. {BASEDIR} can be set in the Test Runner, in fact for batch mode you can supply a list of base directories either as relative path or as fixed path. The rest of the example is just a standard SQL script, which does some things and throws a SQL exception if there's a problem. The Test Runner simply executes this scripts in an internal ServerEngine / Database / Query combination that points to the given path as AliasPath. It wraps the Query.Open command with a try except and reports the error either via a ShowMessage (in manual mode) or by adding the problem to the report log (in batch mode).

The Test Runner also adds some more functionality to test maximum response times or allows the usage of parameter. Please refer to the examples supplied with the Test Runner zip file (in the MyTests sub folders). These examples also show some techniques to test for expected errors, or to test the actual query result set against a known correct result set.

Pascal script tests

The second type of tests are Pascal Scripts. The Test Runner uses RO PascalScript in combination with NexusDB "headers" for the scripts. This allows to write complex tests for most of the components of NexusDB.

Example:

begin
  // first set the alias dir 
  Database.AliasPath:=BaseDir+'\0006';
  Table.TableName:='t1';
  // create new table
  Table.FieldDefs.Clear;
  Table.FieldDefs.Add('f1', ftInteger, 0, true);
  // this overrides an existing table
  Table.CreateTable(tsPersistent, false);
  Table.Open;
  try
    Table.Edit;
    Table.FindField('f1').AsInteger:=1;
    Table.Post;
    Table.First;
    if not Table.FindField('f1').AsInteger=1 then
      raise(-1, 'Error');
  finally
    Table.Close;
  end;
end.

The scripting engine exposes several classes and methods to the scripts:

function BaseDir: String

This is function returns the base directory from which the script is run. It should be used to correctly set any Alias or output of the test.

Database: TnxDatabase

This represents a TnxDatabase instance created for convenience by the Test runner connected to an internal Session/ServerEngine. You can create your own instance of a TnxDataBase if you wish to do so, in fact you can create your complete chain of components from Transport up and thus created automated tests against a remote server.

 class=codeTable: TnxTable

This represents a TnxTable instance connected to Database created for convenience by the Test runner. You can create your own instance of a TnxTable if you wish to do so.

Query: TnxQuery

This represents a TnxQuery instance connected to Database created for convenience by the Test runner. You can create your own instance of a TnxQuery if you wish to do so.

ServerEngine: TnxServerEngine

Points to the internal server engine of the Test Runner

procedure Log(aLine: String) 

A method to write text into the Execution log

Again please look into the examples for tests 0006 and 0007, which are easy to follow and show a lot more.

Batch mode

The Test Runner batch mode is a very powerful tool for automated testing. Yet it is very simple in it's functionality. It allows you to enter a number of Base directories, then does a recursive search for files named *.sql (sql test scripts) and *.nxtest (pascal test scripts) and executes one after the other, writing any exceptions to the report and at the end saving the report to a file.

Just give it a try, I'm sure you're gonna like it.

How can you help?

We are constantly extending our tests, and would like to include as many tests of our customers as possible. For that purpose we're asking everyone of you to supply us with as many tests that are relevant to your project's requirement. Please stick to the following guidelines to allow easy integration into our already huge number of tests.

1. create a test folder structure {BaseDir}\YourName\xxxx (with xxxx being a four digit number padded with 0)
2. name the scripts xxxx.sql or xxxx.nxtest with xxxx representing the same number as in the step above.
3. put each test into it's own sub directory
4. try to make tests completely self contained; if you need to supply a complete table with data, please put it into the correct subdirectory. If a complete table is used by several tests, please reference the same table (see test 0008 in the Test Runner zip file how to do that).
5. only use relative paths within your folder structure. Any access to paths outside will result in failure.
6. recursively compress (zip, arj, 7z) the {BaseDir}\YourName folder and send it to support at nexusdb.com

Once we've received your file, we will then test whether all requirements for inclusion into our test suite are met. If they are we will let you know that from this point on all your tests are run before any official release of a new NexusDB build. You can thus rest assured that nothing of the things that your tests cover is broken when you upgrade to the new build. It goes without saying that the more your tests cover the better for you and of course for ourselves. Less problems means happier customers and that's at the end what we want to achieve.

What do I do if I have questions about the Test Runner or tests I want to make?

That's simply answered. Just use the support newsgroup to post any queries and we will try to answer as quickly as possible. Please don't use the comments of this blog for questions about the Test Runner, as it will likely only be read your self and occasionally by me.