Using the Bulk Import and Export Utilities,
Page 3 of 3


[top]
[prev]
[next]
Documentation Top
Index for This Book

How to Use the Utilities

Bulk Import and Export with Cloudview

To bulk-import data into a table with Cloudview, select the table, then choose File->Import.

To bulk-export data from a table with Cloudview, select the table, then choose File->Export.

To bulk-import and replace existing data with Cloudview, select the table, then choose File-Import. Select Replace Existing Data on the advanced tab.

Cloudview also allows you to create a control file to control the format of the imported and exported data.

NOTE: Cloudview’s on-line help has more details.

Bulk Import and Bulk Export with the Stand-Alone Utilities

The syntax for the standalone bulk-import utility is:

java [-DimportProperties ] COM.cloudscape.tools.FileImport
databaseConnectionURL tableName inputOrExportFileURL
[controlFileURL]

The syntax for the standalone bulk-export utility is:

java COM.cloudscape.tools.FileExport databaseConnectionURL
tableOrViewName inputOrExportFileURL [controlFileURL]

NOTE: You can set system properties such as cloudscape.system.home on the command line using parameters to the java command.

Arguments to Import and Export Utility Commands

  • databaseConnectionURL
    The Cloudscape database connection URL (string) for the database containing the table.
  • tableName
    The name of the table into which you want to bulk-import the data. Qualify the table name with the schema name if it is not in the default schema APP.
  • tableOrViewName
    The name of the table or view from which you want to bulk-export the data. Qualify the table name with the schema name if it is not in the default schema APP.
  • inputOrExportFileURL
    The path or the URL of the ASCII file from which to bulk-import or bulk-export the data. If a controlFileURL is not specified, the import or export file must be of the default file format (see “Default Input or Output File Format”). If a controlFileURL is specified, it must be of the input format described by the control file.
    The path separator in the path or the URL is / (forward slash), per the standard file:// URL protocol. Absolute or relative paths are accepted.
    For local files, do not use the file:// protocol; omit the protocol altogether.
    For export, only the http:// protocol is not allowed. Import supports http://.
  • controlFileURL
    The path or URL of the control file specifying the file format of the ASCII file for the bulk import or export.
  • importProperties
    The properties for the bulk load. See “Import Properties”. By default, the utility sets the property insertMode to bulkInsert. This property is ignored if the requirements for fast mode are not fulfilled. See “Requirements for Fast Mode”. (Where appropriate, you can also specify other properties such as cloudscape.system.home.)

Bulk Import and Export Utility Examples

java COM.cloudscape.tools.FileImport jdbc:cloudscape:toursDB APP.myints c:/mydata/mydata.dat

imports c:\mydata\mydata.dat into the table myints in the schema APP in the database toursDB.

java -DinsertMode=replace COM.cloudscape.tools.FileImport jdbc:cloudscape:wombat APP.abc ../mydata/mydata2.dat ../mydata/ mydata.ctl

Imports mydata2.dat into the table in abc in the schema APP, replacing existing data and running in fast mode in the database wombat using the control file c:\data\myData.ctl.

java -Dcloudscape.system.home=c:\mydatabases COM.cloudscape.tools.FileImport jdbc:cloudscape:toursDB Flights http://airlines.com/data/flight.info

Imports flight.info via HTTP into the table Flights in the database toursDB, which is located in the specified system directory.

java COM.cloudscape.tools.FileExport jdbc:cloudscape:wombat APP.abc c://mydata/mydata3.dat

Exports data from table or view abc in database wombat to file c:\mydata\mydata3.dat, using the default format (since no control file is specified).

java COM.cloudscape.tools.FileExport jdbc:cloudscape:wombat APP.abc c:/mydata/mydata4.dat c:/mydata/mydata.ctl

Exports from table or view abc in database wombat to file mydata4.dat using the file format described in mydata.ctl.

Bulk Import and Export Within an SQL-J Statement

You can use the bulk import and bulk export utilities within an SQL-J statement, which is executable from ij, Cloudview, or any Java program.

To Bulk-Import

There are two ways to bulk-import within an SQL-J statement:

  • Create a new instance of the COM.cloudscape.tools.FileImport class (aliased as FileImport) in the FROM clause of an insert. Such an instance constitutes an ExternalVirtualTable (described in the Cloudscape Reference Manual) that is allowed in a FROM clause instead of a tableExpression.
INSERT INTO tableName [ PROPERTIES propertyList ]
SELECT selectItems
FROM NEW FileImport(constructorParameters)
[ AS ] correlationName [ derivedColumnList ]

NEW: The alias FileImport is new in Version 3.0. In previous versions, you had to use the full class name.

  • Call the static method Import, specifying the table name, file name, and control file name as parameters. (The method constructs and executes the SQL-J statement used above.)
CALL FileImport.Import(staticMethodParameters)

Arguments to Import Command

  • tableName
    The name of the table into which you want to bulk-import the data (qualified by the schema name if necessary).
  • propertyList
    Properties for the bulk import. See “Import Properties”.
  • selectItems
    Projects one or more columns to be inserted in the form of derivedColumnName [, derivedColumnName]*.
  • constructorParameters
    The parameters taken by the constructor for COM.cloudscape.tools.FileImport. See “Parameter Sets for COM.cloudscape.tools.FileImport Constructor”.
  • staticMethodParameters
    The parameters taken by static Import method of COM.cloudscape.tools.FileImport. See “Parameter Sets for Static Method Import in COM.cloudscape.tools.FileImport”.
  • correlationName
    A correlation name for the table, required by the FROM clause syntax.
  • derived column list
    Optional names of the columns returned by the utility. By default, the import utility names the columns COLUMN1, COLUMN2, , COLUMNX. These are the names that you reference in selectItems.

Parameter Sets for COM.cloudscape.tools.FileImport Constructor

Parameter Sets for Static Method Import in COM.cloudscape.tools.FileImport

For examples, see “Bulk Import and Export Within SQL-J Statement Examples”.

To Bulk-Export

Use an instance of the COM.cloudscape.tools.FileExport class (aliased as FileExport), specifying the table or view from which to export and the name of the output file. If you need to use a format different from the default format, also specify the name of the control file created with Cloudview.

NEW: The alias FileExport is new in Version 3.0. In previous versions, you had to use the full class name along with the CLASS keyword.

To export the entire table, specify a table name. To export data from any select statement, define a view as the SQL-J statement from which you want to export data. For example, to export data from a join, create a view to represent the join:

CREATE VIEW joinViewForExport AS
SELECT country, city.getName()
FROM Countries JOIN Cities
USING (country_ISO_code)

Then export from the view.

CALL FileExport.Export(constructorParameters)

COM.cloudscape.tools.FileExport.Export Parameter Sets

  • (java.sql.Connection, TableOrViewNameString, OutputFileURL)
  • (java.sql.Connection, TableOrViewNameString, OutputFileURL, ControlFileURL)
    java.sql.Connection is the connection to the Cloudscape database. TableOrViewName is the name of a table or a view in Cloudscape from which to export data. Qualify the table or view name with the schema name if it is not the default schema, APP.
    OutputFileURL is the URL (of type file:// only) of the ASCII output file accessible to the machine running Cloudscape without going through any network transport protocol. The output file must be of the default file format (see “Default Input or Output File Format”). OutputFileURL must be enclosed in single quotes.
    ControlFileURL is the URL of the control file. It must be accessible to the machine running Cloudscape without going through any network transport protocol. ControlFileURL must be enclosed in single quotes.
    See “About the Control File”.
    TableOrViewName, OutputFileURL, and ControlFileURL must be enclosed in single quotes if they are constants.

For examples, see “Bulk Import and Export Within SQL-J Statement Examples”.

Import Properties

You can specify the insertMode property during bulk import in order to use Cloudscape’s fast import mode, which allows Cloudscape to use minimal logging and under-the-cover performance enhancements. Working in fast import mode results in a significant performance improvement.

If the table is empty, you only need to explicitly set this property when working within the context of an SQL-J statement. Otherwise, the utility sets this property by default. If the table is not empty, you must replace existing data to work in fast import mode. You always must explicitly set this property in that case.

The legal values for this property are bulkInsert and replace. If any other value is specified, an exception is thrown. Use bulkInsert to insert data into an empty table. Use replace to replace data in an existing table.

You can set this property in the following circumstances:

  • When using the command-line utility, set the property on the command line as a -D parameter.
  • When using an INSERT INTO SQL-J statement, set the property in a PROPERTIES clause after the table name.
  • When using a CALL SQL-J statement, set the property in an ImportPropertiesObject.

Syntax

insertMode={ bulkInsert | replace }

When the insertMode property is set to either of the legal values, Cloudscape creates a new physical container for the table, which is more efficient than re-using the existing container.

NOTE: Cloudscape ignores the insertMode=bulkInsert property if any of the requirements for fast import mode are not met. See “Requirements for Fast Mode”. Cloudscape throws an exception if you specify insertMode=replace, and any of the requirements for fast import mode are not met.

If the insertMode property is set to bulkInsert, you can also specify an integer value for the bulkFetch property. This property sets the size of the bulk fetch for rebuilding any indexes associated with the table; the default value is 16.

NOTE: You can turn on bulkInsert even when you are not using the import utilities. For example, you may want to turn on bulkInsert when selecting data from another table or selecting data from an external virtual table. See “INSERT statement” in the Cloudscape Reference Manual.

Syntax

bulkFetch=intValue

Setting this property to a higher value may speed up index rebuilding but requires more memory. Use the -mx parameter to the java command line to allocate more memory.

Bulk Import and Export Within SQL-J Statement Examples

-- This imports all columns and rows from
-- mydata3.dat into table1 in slow import mode
INSERT INTO myints SELECT *
FROM NEW COM.cloudscape.tools.FileImport
('/usr/share/data/mydata3.dat')
AS t
-- this imports the first two columns returned by
-- the file importer (whose columns are given the
-- correlation names a, b, and c) using the specified input
-- and control files. The example uses fast import mode.
INSERT INTO table2 PROPERTIES insertMode=bulkInsert
SELECT a, b FROM NEW COM.cloudscape.tools.FileImport(
'file:////Jeeves/share/data/data2.asc',
'file:////Jeeves/share/mapping/data2.ctl')
AS t (a, b, c)
-- Imports the first four columns from the specified file
-- (which are given correlation names)
-- into WorldCupStatistics using the default file
-- format in slow import mode
INSERT INTO WorldCupStatistics
SELECT a, b, c, d FROM NEW COM.cloudscape.tools.FileImport(
    '../cloudscape/demo/programs/tours/scripts/wc_stat.dat')
AS FI(a, b, c, d, e, f, g, h)
-- Imports the first four columns from the specified file
-- (which are given correlation names)
-- into WorldCupStatistics using the default file
-- format replacing the current data
INSERT INTO WorldCupStatistics PROPERTIES insertMode=replace
SELECT a, b, c, d FROM NEW COM.cloudscape.tools.FileImport(
    '../cloudscape/demo/programs/tours/scripts/wc_stat.dat')
AS FI(a, b, c, d, e, f, g, h)
-- use the static method to import data
-- into the WorldCupStatistics Table
CALL FileImport.Import(getCurrentConnection(), 'WORLDCUPSTATISTICS', 'wc_stat.dat');
-- Exports data from the tabletable3 into the file
-- table3.dump using the default file format.
CALL FileExport.Export(getCurrentConnection(), 'APP.table3', '/ temp/table3.dump');

Default Input or Output File Format

When no control file is specified, the input or output file takes on the default file format.

The default file format is a delimited ASCII file with the following characteristics:

  • Rows are separated by a new line.
  • Fields are separated by a comma (,).
  • Field start is indicated by a double quote (").
  • Field end is indicated by a double quote (").
  • Field start and end are optional unless the field itself contains a field separator (comma) or row separator (CR-LF). For example, if a field contains the string “Hey, look over here!” the field start and end must be indicated by double quotation marks.

An example file of four rows and four columns using default file format:

1,abc,22,def
22,,,"a is a zero-length string, b is null"
13,hello,454,"world "
4,b and c are both null,,

With no control file specified, this is what COM.cloudscape.tools.FileExport outputs:

"1","abc","22","def"
"22",,,"a is a zero-length string, b is null"
"13","hello","454","world "
"4","b and c are both null",,

To create a file in any other format, use Cloudview to create a control file that specifies the format. Cloudview allows you to specify the following:

  • Fixed-width instead of delimited file. You will need to specify the width of each field.
  • Field separator characters.
  • Field start and end characters.
  • Row separators.
  • If you are using a fixed-width file, the default output value for NULLs.

About the Control File

When you use Cloudview to specify a format other than the default, Cloudview allows you to name the control file and save it in the location of your choice.

NOTE: The control file is a text file generated and edited by Cloudview. Use Cloudview to maintain this file, because the format of this file is subject to change. Cloudview will handle upgrades to the file format if this format changes.

Treatment of NULLS

In a delimited file, a NULL value is exported as an empty field. The following example shows the export of a four-column row in which the third column is NULL:

7,95,,Happy Birthday

Import works the same way; an empty field is imported as a NULL value.

When you customize the control file for import or export, you can choose fixed-width instead of delimited files. In fixed-width files, a NULL value is exported as the word NULL.

In fixed-width export, the example shown above would appears as follows:

7 95 NULL Happy Birthday

During import, NULL is imported as a NULL value.

When using Cloudview, you can specify a different import or export value for NULL, which you will have to do if the width of the column that contains the NULL value is smaller than 4 (the number of characters in the word NULL).

[top]
[prev]
[next]


Cloudscape Version 3.0
For technical support, go to: www.cloudscape.com and click Support.
Copyright © 1998 and 1999 Cloudscape, Inc. All rights reserved.