Invantive UniversalSQL Grammar

sqlBatch:

Compatibility

The Invantive implementation of SQL is based upon ANSI SQL, extended by aspects from popular SQL implementations such as PostgreSQL, MySQL, Oracle, Teradata and Microsoft SQL Server. It is topped of with Invantive-specific extensions, especially for procedural SQL, distributed SQL and distributed transactions. The basis is to implement functions such that as little as possible changes are necessary to run a SQL statement originating from another SQL implementation on Invantive UniversalSQL. For instance, to retrieve the current time you can use 'sysdate', 'now', 'getdate()' and 'sysdatetime' to name a few. The same holds for the procedural extension Invantive Procedural SQL, which reflects SQL/PSM and makes it easy to port Oracle PL/SQL or PostgreSQL PL/pgSQL statements.

Important Safety and Usage Information

Intended Use and Limitations: This software, developed by Invantive, is designed to support a variety of business and information technology data processing functions, such as accounting, financial reporting an sales reporting. It is important to note that this software is not designed, tested, or approved for use in environments where malfunction or failure could lead to life-threatening situations or severe physical or environmental damage. This includes, but is not limited to:

• Nuclear facilities: The software should not be used for operations or functions related to the control, maintenance, or operation of nuclear facilities.
• Defense and Military Applications: This software is not suitable for use in defense-related applications, including but not limited to weaponry control, military strategy planning, or any other aspects of national defense.
• Aviation: The software is not intended for use in the operation, navigation, or communication systems of any aircraft or air traffic control environments.
• Healthcare and Medicine Production: This software should not be utilized for medical device operation, patient data analysis for critical health decisions, pharmaceutical production, or medical research where its failure or malfunction could impact patient health.
• Chemical and Hazardous Material Handling: This software is not intended for the management, control, or operational aspects of chemical plants or hazardous material handling facilities. Any malfunction in software used in these settings could result in dangerous chemical spills, explosions, or environmental disasters.
• Transportation and Traffic Control Systems: The software should not be used for the control, operation, or management of transportation systems, including railway signal controls, subway systems, or traffic light management. Malfunctions in such critical systems could lead to severe accidents and endanger public safety.
• Energy Grid and Utility Control Systems: This software is not designed for the control or operation of energy grid systems, including electrical substations, renewable energy control systems, or water utility control systems. The failure of software in these areas could lead to significant power outages, water supply disruptions, or other public utility failures, potentially endangering communities and causing extensive damage.
• Other High-Risk Environments: Any other critical infrastructure and environments where a failure of the software could result in significant harm to individuals or the environment.

User Responsibility: Users must ensure that they understand the intended use of the software and refrain from deploying it in any setting that falls outside of its designed purpose. It is the responsibility of the user to assess the suitability of the software for their intended application, especially in any scenarios that might pose a risk to life, health, or the environment.

Disclaimer of Liability: Invantive disclaims any responsibility for damage, injury, or legal consequences resulting from the use or misuse of this software in prohibited or unintended applications.

Distributed SQL, Databases and Data Containers

It is easy to exchange and/or combine data across the supported platforms with data. To each platform (such as Salesforce or Exact Online Belgium) multiple connections can be active with the same or different platform-specific connection settings. Each open connection to a platform is named a 'data container'.

All opened connections together are named a 'database'.

When multiple data containers have been opened, each one has an alias to refer it by in Invantive UniversalSQL statements. For instance, a connection can be open for two different customer accounts on Exact Online Netherlands aliased as 'eolnl_comp1' and 'eolnl_comp55') and one for an Exact Online Belgium custom, aliased as 'eolbe_my_new_company'. The aliases can be freely chosen as long as they are valid identifiers and defined in the databases configuration file 'settings.xml'.

Service Providers

A number of special connections are always made, each of which can occur at most once. These are the 'service providers' such as:

• 'datadictionary': metadata of the current database, such as list of tables and executed SQL statements performance.
• 'os': information on the operating system running the SQL engine, such as reading file contents.
• 'smtp': synchronously send mails through SMTP.

Partitioning

Especially online platforms have a multi-tenant structure, in which the data is partitioned per customer, company or person. When the data model is identical across tenants, Invantive UniversalSQL considers them 'partitions'. SQL statements can run across multiple or one partitions, often in parallel. This enables consolidation scenarios across partitions (such as Exact Online or Nmbrs companies) as well as high-performance in MPP environments.

The partitions to be used can be specified with the 'use' statement, either through an explicit list of partitions to be selected across data containers, or through a SQL select statement returning the list of partitions to use. Please note that although the 'use' statement resembles the 'use DATABASE' statement on Microsoft SQL Server or PostgreSQL you can on Invantive UniversalSQL have multiple partitions active at the same time in one user session.

Identifiers

For identifiers, the regular conventions hold for the set of allowed characters. Depending on the platform, the identifiers are case sensitive or not. In general, it is best to assume that the identifier are case insensitive. There is no length limit on an identifier imposed by Invantive UniversalSQL.

Procedural SQL

Invantive Procedural SQL (or "PSQL" for short) is a procedural extension on top of Invantive UniversalSQL. It is based on the ISO-standard 9075-4:2016 (SQL/PSM) and extends Invantive UniversalSQL with procedural options like blocks, variables, conditional execution and loops. The procedural code is - together with the Invantive UniversalSQL contained - as a whole into pseudo-code and then executed.

The procedural code does not lean on the procedural options of the platforms being used, so it is easy to retrieve and change data in all supported cloud, file and database platforms. The pre-compiled procedural code does not perform context switches between procedural and SQL logic.

Licensing

Features

The available functionality of Invantive UniversalSQL features is based upon the licensed features. For instance the free implementation of Invantive UniversalSQL is limited to 15.000 rows and no access to group functions. Please consult the data dictionary contents for your license features.

Usage Fees

For paid products, the fee depends on a number of factors: users, devices and billable partitions. All fees depend on actual use. Additionally, the number and volume in KB of reads and writes is registered to enable communication with the platform partners on the performance of their platform.

A rough estimate is that the billable partitions reflect the product of number of legal entities times number of source systems. The number of billable partitions is determined as follows, where using twice or more often the same billable partition doesn't influence the number:

• Accessing a provider which does not have a concept of "partition" typically counts as one partition per different connection. A so-called data container ID is determined which reflects the backend connected to. For instance, connecting to my-ftp-host.com and another-ftp-host.com counts as two partitions. Examples of such providers are FTP, SQL Server and cbs.nl.
• Accessing a provider which has a concept of "partition" counts as the number of partitions used during the connection. For instance, using the Exact Online company 234 and 456 counts as two partitions. Examples of such providers are Exact Online, NMBRS, Loket and XML Audit File Financieel.
• Accessing data through xmltable, csvtable, internettable, jsontable and exceltable counts as the number of different parameter combination used. For instance, accessing an Excel range 'Sheet1' and a named range 'mydata' counts as two partitions.

settings.xml

The file settings.xml defines for a user or program the list of defined databases. Databases are grouped in 'database groups' for visual display. Database groups have no further functionality. Each database consists of one or multiple data containers.

The file 'settings.xml' is most often found on Microsoft Windows in your '%USERPROFILE%\invantive' folder, such as 'c:\users\john.doe\invantive\settings.xml'. It is shared across all Invantive UniversalSQL product installations for the user.

There are many scenarios to share database specifications across a user community, such as WAN-scenarios with Invantive Web Service, large corporate scenarios using DNS-entries as well as file shares, included files as well as single user solutions. Please involve a consultant when you want to deploy across thousands of users or more.

For user communities of up to 10 users, we recommend that company-specific settings are grouped per role in a separate file named 'settings-ROLE.xml' and placed in the default folder. Invantive UniversalSQL will automatically merge these files in the main settings.xml file.

Group Functions

The Invantive implementation of SQL is based upon ANSI SQL, extended by aspects from popular SQL implementations such as PostgreSQL, MySQL, Oracle, Teradata and Microsoft SQL Server. It is topped of with Invantive-specific extensions, especially for distributed SQL and distributed transactions. The basis is to implement functions such that as little as possible changes are necessary to run a SQL statement designed for use with another SQL implementation on Invantive UniversalSQL. For instance, to retrieve the current time you can use 'sysdate', 'now', 'getdate()' and 'sysdatetime' to name a few.

Popular group functions such as 'stddev' are available. However, currently you can not combine in one unnested SQL statement both group functions as well as expressions on the variables. In that case use an inner (nested) SQL statement to apply the expressions on the data, and execute the group functions in the outer SQL statement with the syntax 'select group() from ( select ... from ... )'.

Locking

An Invantive UniversalSQL statement can work with many traditional and online platforms. There are no locking features on data and objects, since few online and traditional platforms connected provide these and the typical use of distributed transactions leave even less opportunity for data and object locking.

Transactions

Invantive UniversalSQL has limited support for transactions. DML is forwarded to a platform and depending on the platform an error can cause part of the work to be registered or everything to be rolled back. Within the SQL engine, multiple changes can be collected and forwarded to the platform at once. For instance, when creating an EDIFACT message you need to combine an invoice header with invoice lines into one EDIFACT message. Collection of multiple changes is done using the 'identified by' and 'attach to' syntax, optionally preceded by 'begin transaction'.

Format Masks

Operations such as to_char and to_date support a limited number of format masks:

• YYYY: 4-digit year.
• YYYYMM: 4-digit year, plus 2-digit number month within year.
• YYYYMMDD: 4-digit year, 2-digit month number within year, plus 2-digit day number within month.
• YYYYMMDDHH24MI: 4-digit year, 2-digit month number within year, 2-digit day number within month, 2-digit 24-hour clock hour number, plus 2-digit minutes number within hour.
• YYYYMMDDHH24MISS: 4-digit year, 2-digit month number within year, 2-digit day number within month, 2-digit 24-hour clock hour number, 2-digit minutes number within hour, plus 2-digit seconds number within minute.
• YYYY-MM-DD: 4-digit year, dash ('-'), 2-digit month number within year, dash ('-'), plus 2-digit day number within month.
• YYYY-MM-DD HH24:MI:SS: 4-digit year, dash ('-'), 2-digit month number within year, dash ('-'), plus 2-digit day number within month, 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• YYYY/MM/DD: 4-digit year, slash ('/'), 2-digit month number within year, slash ('/'), plus 2-digit day number within month.
• YYYY/MM/DD HH24:MI:SS: 4-digit year, slash ('/'), 2-digit month number within year, slash ('/'), plus 2-digit day number within month, 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• MMDDYY: 2-digit month number within year, 2-digit day number within month, plus 2-digit year.
• MMDDYYYY: 2-digit month number within year, 2-digit day number within month, plus 4-digit year.
• DDMMYYYY: 2-digit day number within month, 2-digit month number within year, 4-digit year.
• DD-MM-YY: 2-digit day number within month, dash ('-'), 2-digit month number within year, dash ('-'), 2-digit year.
• DD-MMM-YY: 2-digit day number within month, dash ('-'), short month name, dash ('-'), 2-digit year.
• DD/MM/YY: 2-digit day number within month, slash ('/'), 2-digit month number within year, slash ('/'), 2-digit year.
• DD/MMM/YY: 2-digit day number within month, slash ('/'), short month name, slash ('/'), 2-digit year.
• DD-MM-YYYY: 2-digit day number within month, dash ('-'), 2-digit month number within year, dash ('-'), 4-digit year.
• DD-MMM-YYYY: 2-digit day number within month, dash ('-'), short month name within year, dash ('-'), 4-digit year.
• DD/MM/YYYY: 2-digit day number within month, slash ('/'), 2-digit month number within year, slash ('/'), 4-digit year.
• DD/MMM/YYYY: 2-digit day number within month, slash ('/'), short month name within year, slash ('/'), 4-digit year.
• DD-MM-YYYY HH24:MI:SS: 2-digit day number within month, dash ('-'), 2-digit month number within year, dash ('-'), 4-digit year, space (' '), 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• DD/MM/YYYY HH24:MI:SS: 2-digit day number within month, slash ('/'), 2-digit month number within year, slash ('/'), 4-digit year, space (' '), 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• MM/DD/YYYY HH24:MI:SS: 2-digit month number within year, slash ('/'), 2-digit day number within month, slash ('/'), 4-digit year, space (' '), 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• DD/MMM/YYYY HH24:MI:SS: 2-digit day number within month, slash ('/'), short month name within year, slash ('/'), 4-digit year, space (' '), 2-digit hour on 24 hour clock, colon (':'), 2-digit minutes within hour, colon (':'), 2-digit seconds within minutes.
• YYYYIW: 4-digit year, plus ISO week number.
• YYYY-IW: 4-digit year, dash ('-'), plus ISO week number.
• IW: ISO week number.
• D: day of week according to ISO 8601.
• MMMM DD, YYYY HH24:MI TT: long month, 2-digit day number within month, comma (','), space (' '), 4-digit year number, space (' '), 2-digit hour on 24 hour clock, colon (':'), 2-digit minute within hour, space (' '), time zone.

Release History

The following significant changes were made to the Invantive UniversalSQL grammar:

• Release 17.29 (BETA) adds:
• 20181031: allow named expressions as table function parameters, using => as association operator.
• 20181123: add WHEN EXCEPTION to PSQL BEGIN..END block, with exception being WHEN OTHER or WHEN exception name.
• 20181123: extend SYNCHRONIZE statement.
• 20181224: extend SYNCHRONZIE statement with IGNORE NULLS clause.
• Release 17.30 (Production): All changes sofar.
• Release 17.31 (BETA) adds:
• 20190116: addition of language code parameter to TRANSLATE_RESOURCE.
• 20190117: addition of IS_NUMBER function.
• 20190124: addition of TO_BOOLEAN, IS_BOOLEAN, IS_DATE, IS_GUID functions.
• Release 17.32 (Production): All changes sofar.
• Release 17.33 (BETA) adds:
• 20190131: texts as partition names .
• 20190310: expression as billing id and reference.
• 20190322: select all columns 'except' syntax.
• 20190401: months_between SQL function.
• 20190401: excel_day SQL function.
• 20190403: sqlrowcount, sqlerrm and sqlcode functions.
• 20190410: 'exists' expression.
• 20190601: 'when obsolete within' variant of 'alter persistent cache refresh'.
• 20190605: gzip and ungzip functions.
• 20190624: 'alter session set roles' syntax for connectors with row-level security.
• 20190701: ascii_to_blob and unicode_to_blob functions.
• 20190706: 'alter session set iuid source' syntax.
• 20190821: add htmltable.
• 20191007: add 'except' to synchronize statement for insert and update.
• 20191008: add 'ignore changes to' to synchronize statement for update.
• 20191209: multiple PSQL batches in one statement.
• 20200102: add internettable.
• Release 17.34 (Production): All changes sofar.

A SQL batch consists of one or more Invantive UniversalSQL and/or PSQL statements separated by the semi-colon batch separator (';').

no references

sqlOrPSqlStatement:

A number of SQL and PSQL statements can be used to compose a batch.

referenced by:

• sqlBatch

sqlEmptyStatement:

referenced by:

• sqlOrPSqlStatement

sqlStatement:

An Invantive UniversalSQL statement can retrieve or exchange data for many traditional and online platforms. Many platforms also support the use of DML (Data Manipulation Language) statements to change the data contained. On a few platforms you can execute DDL (Data Definition Language) statements to create new data structure or objects such as tables, procedures or sequences.

Popular Invantive UniversalSQL extensions are the synchronize statement to make sure multiple data sources contain the same data and the use statement to run SQL across multiple partitions.

referenced by:

• pSqlStatement
• sqlOrPSqlStatement

selectStatement:

A SQL select statement retrieves data from one or multiple data containers. A select statement can be composed of multiple data sets retrieved from many platforms, combined by set operators such as 'union'.

Often the performance of cloud platforms is less than traditional database platforms. With the 'limit' clause a limited number of rows can be retrieved quickly from a table or view after applying sorting as specified by the possibly present 'order by'. An alternative for a 'limit' clause is to use the 'top' clause.

A sequence of Invantive UniversalSQL statements, separated by the semi-colon separator character.

Each statement in the SQL batch will be executed consecutively. Execution will be stopped when an error occurs during execution of a statement.

referenced by:

• createOrReplaceViewStatement
• createTableStatement
• embeddedSelect
• expression
• insertStatement
• pSqlForRecordLoopStatement
• sqlStatement
• useStatement

setOperatorSelectStatement:

SQL is based upon a solid mathematical foundation named 'set theory' with some exceptions. The set operators of Invantive UniversalSQL enable you to combine sets of data sets such as merging two sets of data. Please note that SQL actually uses 'bags', which opposed to 'sets', allow duplicates. To change bags of data into sets, either use 'distinct' or the 'union' set operator without 'all'. In general, the extensive use of 'distinct' signals bad database design.

The 'union' set operator returns the union of the data on the left and right side of the union while removing duplicate rows. The 'union all' set operator returns the union of the data on the left and right side of the union without removing duplicate rows. The 'minus' set operator returns all rows from the left side which do not occur in the right side. The 'intersect' set operator returns all rows that occur both in the left and right side.

The 'union' set operator has an extension to facilitate sieving overlapping data sets of decreasing quality using 'distinct on'. The 'union distinct on' set operator returns the union of the data sets listed, but only the first match on the column list is returned. The first match is defined as a fallthrough from left to right. In general, the preferred source of a set of data is listed first, followed by one or multiple 'union distinct on' set operators, each with a data set of less preference.

referenced by:

• selectStatement

uniqueSelectStatement:

Retrieves a data set from one or more data containers.

referenced by:

• selectStatement
• setOperatorSelectStatement

dataSource:

A data source can be a table, a table with parameters or a nested select (an 'inline view'). Also, a data source can be a binary or text string being interpreted as XML, CSV, JSON or binary Excel Open XML format.

referenced by:

• joinStatement
• uniqueSelectStatement

select:

referenced by:

• uniqueSelectStatement

executionHints:

Execution hints allow you to control individually the execution of SQL statements. Whenever possible, the hints will be used. In contrary to other platforms, Invantive UniversalSQL requires a hint to be valid according to the grammar when specified. This reduces the engineering risk that hints become invalid by accident.

referenced by:

• deleteStatement
• uniqueSelectStatement
• updateStatement

httpDiskCache:

The http_disk_cache-hint specifies whether messages may be cached on disk when the provider uses HTTP to exchange data with the backing platform. This typically holds only for cloud-based platforms such as Exact Online, Teamleader or Salesforce. The default setting is false. The first parameter is a boolean whether data may be taken from the disk cache, the second parameter is a boolean whether data retrieved must be stored also in the disk cache and the third parameter is an interval that specifies the timespan before a disk cache hit found is to considered stale, such as "interval '4 hours'".

The use of the http_disk-cache-hint is recommended for data which is known to change seldom such as seeded or reference data. The contents of the disk cache are persistent across Invantive UniversalSQL sessions.

The disk cache is located in the Cache folder of the Invantive configuration folder.

referenced by:

• executionHints

httpMemoryCache:

The http_memory_cache-hint specifies whether messages may be cached in memory when the provider uses HTTP to exchange data with the backing platform. This typically holds only for cloud-based platforms such as Exact Online, Teamleader or Salesforce. The default setting is false. The first parameter is a boolean whether data may be taken from the memory cache, the second parameter is a boolean whether data retrieved must be stored also in the memory cache and the third parameter is an interval that specifies the timespan before a memory cache hit found is to considered stale, such as "interval '4 hours'".

The use of the http_memory-cache-hint is recommended for data which is known to change seldom such as seeded or reference data. The contents in the memory cache are forgotten across Invantive UniversalSQL sessions.

The memory cache is located in the Cache folder of the Invantive configuration folder.

referenced by:

• executionHints

materializeResultSet:

referenced by:

• executionHints

ods:

The ods-hint controls the use of the Invantive Data Cache stored in a relational database. The Invantive Data Cache is also the basis of the Operational Data Store managed by Invantive Data Replicator and the data warehouses managed by Invantive Data Vault. The ods-hint specifies the maximum age data from the data cache eligible for use.

The boolean specifies whether the Data Cache may be used to answer a query. Set it to false to disable use of Data Cache for the duration of the query. Keep it on the default true to use Data Cache.

The interval specifies the period of time during which cached results are considered sufficiently fresh for use, such as '30 minutes'.

When no interval is present, the actual platform is consulted. The default with Invantive Data Cache enabled is to always use the data cache contents when not stale according to the metadata of the data cache. In general, that defaults to a maximum age of 7 days.

When use of data cache is requested and a filter is applied, all data will be retrieved from the actual platform and filtered on the Invantive UniversalSQL engine. This also applies when a filter is applied that can be forwarded to the actual platform. When the data is requested a limited number of times and the filter severely reduces the amount of data to be transported, it can be faster to not use the data cache.

referenced by:

• executionHints

resultSetName:

referenced by:

• executionHints

resultSetSerialization:

referenced by:

• executionHints

joinSet:

Control join approach between two data sources. A column-indexed lookup will be used instead of a full table scan when the number of rows on the left-hand side does not exceed the maximum number of rows specified in the hint. When not specified, a hash lookup will only be used when the number of rows on the left-side does not exceed 5.000.

The actual implementation of a hash lookup depends on the platform on which the data container runs. For instance with OData, a number of requests will be made using an in-construct with a limited number of in-values. With a relation database platform, a native SQL 'in' will be used.

The first identifier is the alias of the table on the right-hand side of the join. The second identifier is the name of the column used to join upon in the right-hand side. The numeric constant specifies upto what number of rows on the left-hand side of the join will allow the join set hint to be used. When the number of rows exceeds the numeric constant, a full table join is made.

The following example takes for instances 5.000 sales invoices from an Exact Online environment with 100.000 sales invoices. Each sales invoice has 4..10 lines. The join does not retrieve all sales invoices nor all invoice lines, but instead fetches the 5.000 sales invoices using the where-clause, and then retrieves the related invoice lines using a column-indexed lookup by invoiceid. Since Exact Online is an OData source, the approximately 30.000 invoice lines will be retrieves in 300 session I/Os each having an in-construct for 100 lines on invoiceid.

referenced by:

• executionHints

noJoinSet:

The no_join_set hint disables the use of hash-joins. It can be enabled using the join_set hint.

referenced by:

• executionHints

lowCost:

The low_cost-hint specifies that the select with the hint must be considered a select with low execution costs. Low execution costs trigger early evaluation during parsing. By default, select statements using solely in memory storage, dummy and data dictionary are considered low cost and evaluated early. The evaluation of all others is delayed as long as possible.

The use of the low_cost-hint is recommended when the select is used with a 'in ( select ... )' syntax and the developer knows beforehand that it will evaluate fast to values and that the use of these values will allow the use of server-side filtering for the outer select.

referenced by:

• executionHints

distinct:

Addition of the 'distinct' keyword to a SQL select statement de-duplicates the rows returned. Rows are considered duplicates when the values in all selected columns are identical, with two null-values considered equal.

referenced by:

• avgAggregateFunction
• countAggregateFunction
• listAggAggregateFunction
• productAggregateFunction
• stdDevAggregateFunction
• sumAggregateFunction
• uniqueSelectStatement

topClause:

With the 'top' clause a limited number of rows can be retrieved quickly from a table or view after applying sorting as specified by the possibly present 'order by'.

referenced by:

• uniqueSelectStatement

limitClause:

With the 'limit' clause a limited number of rows can be retrieved quickly from a table or view after applying sorting as specified by the possibly present 'order by'.

referenced by:

• deleteStatement
• selectStatement
• updateStatement

forClause:

The 'for' clause replaces the results from a query by a single row or a few rows, single column embedding the full results in the respective format.

referenced by:

• selectStatement

forCsvClause:

The 'for csv' clause replaces the results from a query by few rows and asingle column named 'CSV' embedding the full results in an CSV format. Depending on the data volume, multiple rows can be returned. By default, each output row contains at most 1.000 rows from the query. The actual number of rows from the query can be set using 'OUTPUT PER ... ROWS'.

By default, the first output row starts with a technical header of the respective column names. The headers can be replaced by labels using 'INCLUDE HEADERS'. The headers can be fully disabled by adding 'EXCLUDE HEADERS'. The output of headers in each output row can be enabled using 'REPEAT HEADERS' with 'OUTPUT PER ... ROWS'. In that case, 'OUTPUT PER ... ROWS' must always be specified even with the default of 1.000 query rows per output row.

The CSV-output can use other texts than CR/LF for row delimiter, ',' for column delimiter, '"' for quoting character and '\"' for escaped quoting character. These can be set using, respectively, 'ROW DELIMITER', 'COLUMN DELIMITER', 'QUOTING CHARACTER' and 'ESCAPED QUOTING CHARACTER'.

referenced by:

• forClause

forExcelClause:

The 'for excel' clause replaces the results from a query by a single row, single column named 'Excel' embedding the full results in an Excel xlsx-file format. The Excel xlsx-file has an Excel worksheet with a name (by default 'resultset1') with the rows.

By default, the first Excel worksheet starts with a header with all column names. No header is added when 'EXCLUDE HEADERS' is specified. Labels can be used as headers by specifying 'INCLUDE HEADERS'.

referenced by:

• forClause

forJsonClause:

The 'for json' clause replaces the results from a query by a single column named 'JSON' embedding the full results in a JSON format. Depending on the data volume, multiple rows can be returned. By default, each output row contains at most 1.000 rows from the query. The actual number of rows from the query can be set using 'OUTPUT PER ... ROWS'.

One of the naming configurations is 'AUTO'; the rows return one JSON object, with column names and their values as, respectively, JSON property names and values. A period ('.') in a column name will not influence the generated JSON differently from any other character. Casing of the column names will be reflected in the JSON property names. Casing is upper-case by default in Invantive UniversalSQL. Escape column names using square brackets to accurately specify casing of JSON property names.

As an alternative, the naming configuration can be switched to 'PATH'. With 'PATH', the JSON property names are identical to 'AUTO' except for a period ('.') in a column name. Each period in a column name introduces a new nesting level in the generated JSON.

A wrapper can be specified using 'ROOT'. The default root JSON property name is 'root'. A deviating name can be specified as a text constant between parentheses.

Properties are excluded for column null values. JSON properties are generated even for null values when 'INCLUDE_NULL_VALUES' is present.

The JSON objects are combined into a JSON array unless 'WITHOUT_ARRAY_WRAPPER' is present. With 'WITHOUT_ARRAY_WRAPPER' present, the output becomes NDJSON, where each individual output row represent a single source row.

referenced by:

• forClause

forXmlClause:

The 'for xml' clause replaces the results from a query by a single row, single column named 'XML' embedding the full results in an XML format.

referenced by:

• forClause

forXmlClauseCommonDirectives:

referenced by:

• forXmlClause

embeddedSelect:

An embedded select, also known as an 'inline view', retrieves rows using the specified select statement. These rows are consumed by the outer select as were it the results of retrieving the rows from a table.

Invantive UniversalSQL does not allow grouping rows with expressions as columns. An embedded select is typically used to evaluate expressions to rows with solely constants. After applying the embedded select the group operators can be applied.

referenced by:

• dataSource

pipelinedTableFunctionSpec:

referenced by:

• dataSource

tableSpec:

A table specification without parameters. The optional alias after the at-sign specifies a specific data source to be used, such as 'exactonlinerest..journals@eolbe' specifying the use of Exact Online Belgium when 'eolbe' is associated by the database definitions in settings.xml with Exact Online Belgium.

A number of special so-called 'service providers' are always present, such as 'datadictionary' for use by an alias.

referenced by:

• alterPersistentCacheConfigureWebhooksStatement
• alterPersistentCacheDropStatement
• alterPersistentCacheSetTableOptions
• alterPersistentCacheTableRefreshStatement
• applyToClause
• createTableStatement
• dropTableStatement
• synchronizeStatement

tableSpecWithRotator:

referenced by:

• deleteStatement
• insertStatement
• updateStatement

tableOrFunctionSpec:

A table specification requiring a comma-separated list of parameters to determine the rows to be retrieved.

Traditional SQL syntax did not provide for parameterized queries, matching set theory. Modern variants such as pipelined table functions allow a stored procedure or other imperative language-based approaches to generate rows based upon parameter values. Many data containers support queries that returns rows based upon parameter values. This holds especially for SOAP web services. Table specifications with parameters ease queries on such data containers.

The optional alias after the at-sign specifies a specific data source to be used, such as 'exactonlinerest..journals@eolbe' specifying the use of Exact Online Belgium when 'eolbe' is associated by the database definitions in settings.xml with Exact Online Belgium.

no references

tableOrFunctionSpecWithRotator:

referenced by:

• dataSource

tableFunctionSpec:

A list of parameter value expressions identified by position or name to determine the rows to be retrieved by a tableOrFunctionSpec.

referenced by:

• tableOrFunctionSpec
• tableOrFunctionSpecWithRotator

numberedOrNamedExpressionList:

referenced by:

• pSqlFunctionOrProcedureStatementNS
• pSqlPackageProcedureStatementNS
• pSqlRaiseStatement
• tableFunctionSpec

expressionList:

An ordered comma-separated list of value expressions.

referenced by:

• functionExpression
• listAggAggregateFunction
• zipAggregateFunction

namedExpressionList:

An unordered list of value expressions, identified by the parameter name.

no references

namedExpression:

A value expression, identified by the parameter name, the association operator '=>' and the value expression.

referenced by:

• namedExpressionList
• numberedOrNamedExpressionList

distributedAliasOrRotatorDirective:

referenced by:

• tableOrFunctionSpecWithRotator
• tableSpecWithRotator

rotatorDirective:

referenced by:

• distributedAliasOrRotatorDirective

distributedAliasDirective:

The distributed alias after the at-sign specifies a specific data source to be used, such as 'exactonlinerest..journals@eolbe' specifying the use of Exact Online Belgium when 'eolbe' is associated by the database definitions in settings.xml with Exact Online Belgium.

A number of special so-called 'service providers' are always present, such as 'datadictionary' for use by an alias.

referenced by:

• pSqlPackageProcedureStatementNS
• partitionIdentifierWithAlias
• setIdentifier
• tableOrFunctionSpec
• tableSpec

dataContainerAlias:

When multiple data containers have been defined in settings.xml for a database, each one is assigned an alias. An alias typically takes the form of a limited number of characters. The presence of an alias allows Invantive UniversalSQL to precisely determine to what data container forward a request for data.

referenced by:

• alterPersistentCacheRefreshStatement
• distributedAliasDirective
• distributedAliasOrRotatorDirective
• rotatorDirective

passingSourceOrPathExpression:

The passing option specifies the source of the contents to be interpreted. The contents can be specified as the outcome of an expression such as from a previous read_file() table function, a URL downloaded using httpget() or a string concatenation. The contents can also be specified as to be taken from a specific file identified by it's file name and path as an expression.

referenced by:

• csvTablePassing
• excelTablePassing
• htmlTablePassing
• jsonTablePassing
• xmlTablePassing

xmlTableSpec:

Data stored in XML format can be interpreted as a data source using the xmltable keyword.

The expression specifies a master XPath expression within the context of which the rows are evaluated using the column specifications.

The passing option specifies the source of the data in XML format. The source is often the outcome of a read_file() table function, a URL download using httpget() or a previous xmlformat() SQL function.

The columns are specified using their XPath relative to the master path.

referenced by:

• dataSource

xmlTablePassing:

referenced by:

• xmlTableSpec

xmlTableLiteral:

A literal value containing a valid XML document.

referenced by:

• xmlTableSpec

xmlTableColumns:

A list of XML table column specifications.

referenced by:

• xmlTableSpec

xmlTableColumnSpec:

The columns are specified using their XPath relative to the master path.

referenced by:

• xmlTableColumns

jsonTableSpec:

Data stored in JSON format can be interpreted as a data source using the jsontable keyword.

The expression specifies a master JSON expression within the context of which the rows are evaluated using the column specifications.

The passing option specifies the source of the data in JSON format. The source is often the outcome of a read_file() table function or a URL download using httpget().

The columns are specified using their JSON path relative to the master path.

referenced by:

• dataSource

ndjsonTableSpec:

referenced by:

• dataSource

jsonTablePassing:

referenced by:

• jsonTableSpec
• ndjsonTableSpec

jsonTableLiteral:

A literal value containing a valid JSON document.

referenced by:

• jsonTableSpec
• ndjsonTableSpec

jsonTableColumns:

A list of JSON table column specifications.

referenced by:

• jsonTableSpec
• ndjsonTableSpec

jsonTableColumnSpec:

The columns are specified using their JSON path relative to the master path.

referenced by:

• jsonTableColumns

csvTableSpec:

Data stored in CSV format can be interpreted as a data source using the csvtable keyword.

The passing option specifies the source of the data in CSV format. The source is often the outcome of a read_file() table function or a URL download using httpget().

The interpretation process can be controlled on table level using the table options and the specification ends with the CSV columns being mapped to data source columns using their relative position within a row.

referenced by:

• dataSource

csvTableOptions:

The interpretation process can be controlled on table level using the table options.

The row delimiter is a text that separates two CSV rows, such as "chr(13) || chr(10)" or simply a pipe character "'|'". The default is the operating system-specific variant of new line.

The column delimiter is a text that separates two CSV columns such as "';'". The default is comma.

Data in CSV will typically have one or more header CSV rows labeling the individual columns. The 'skip lines' statement excludes the first number of CSV rows from processing.

referenced by:

• csvTableSpec

csvTableLiteral:

A literal value containing a valid CSV document.

referenced by:

• csvTableSpec

csvTablePassing:

referenced by:

• csvTableSpec

csvTableColumns:

A list of CSV table column specifications.

referenced by:

• csvTableSpec

csvTableColumnSpec:

Each CSV column is mapped to a data source column using it's relative position within the CSV row. Position 1 represents the first CSV column. A position can be represented using an integer, or by specifying the keyword 'NEXT'. The keyword 'NEXT' specifies that the column is located one position higher than the previous column. When there is no previous column, the column is located at position 1.

referenced by:

• csvTableColumns

excelTableSpec:

Excel file contents in Open XML, such as used with the file extensions 'xlsx' and 'xlsm', can be interpreted as a data source using the exceltable keyword.

The rectangle specifies the rectangular area from which should be taken. The first worksheet is used when no rectangle is specified or a null value as worksheet name. Rows in Excel are interpreted as rows from the data source, whereas columns in Excel are interpreted as columns.

The passing option specifies the source of the (binary) contents in zipped Open XML format such as used with the file extensions 'xlsx' and 'xlsm'. The source is often the outcome of a read_file() table function or URL download using httpget().

The interpretation process can be controlled on table level using the table options and the specification ends with the Excel columns being mapped to data source columns using their relative position within the rectangular area.

referenced by:

• dataSource

excelDataRectangle:

The rectangle specifies the rectangular area from which data should be taken. Rows in Excel are interpreted as rows from the data source, whereas columns in Excel are interpreted as columns. All cells within the rectangle are considered data; headings should be excluded from the rectangle specification.

A rectangle can be either:

• The contents of a complete worksheet, specified by an expression returning the worksheet name. Specify null to select the first worksheet independent of name. Alternatively, a worksheet can be selected by specifying an integer indicating the relative position, including hidden worksheet.
• A named range within a specific worksheet, specified by expressions for optional worksheet name and named range name.
• A cell range identified by an expression with it's dimensions including optional worksheet name and a range definition. The range definition can have various forms: 'A1:Z999' or '$A$1:$Z$999' for an area, 'A:Z' or '1:26' for all cells in column(s) or 'A1' to refer to a specific cell.
• An Excel table identified by an expression with the table name.
• A named range identified by an expression with the named range name.
• Unspecified, selecting the first worksheet independent of name.

referenced by:

• excelTableSpec

excelTablePassing:

referenced by:

• excelTableSpec

excelTableOptions:

The interpretation process can be controlled on table level using the table options.

A position can be represented using an integer, or by specifying the keyword 'NEXT'. The keyword 'NEXT' specifies that the column is located one position higher than the previous column. When there is no previous column, the column is located at position 1.

Empty rows will typically be found when consuming a rectangular area larger than the actual data, such a complete worksheet. The 'skip empty rows' statement eliminates all completely empty rows from the output.

referenced by:

• excelTableSpec

excelTableColumns:

A list of Excel table column specifications.

referenced by:

• excelTableSpec

excelTableColumnSpec:

Each Excel column is mapped to a data source column using it's relative position within the rectangular area. Position 1 represents the first Excel column falling within the Excel rectangular area.

referenced by:

• excelTableColumns

htmlTableSpec:

Data lossy stored in HTML format can be interpreted as a data source using the htmltable keyword.

The expression specifies a master path within the context of which the rows are evaluated using the column specifications.

The passing option specifies the source of the data in HTML format. The source is often the outcome of a read_file() table function or a URL download using httpdownload(). Full adherence to the HTML format is not required; the parser often automatically corrects many HTML format errors.

The columns are specified using their path relative to the master path.

referenced by:

• dataSource

htmlTableLiteral:

referenced by:

• htmlTableSpec

htmlTablePassing:

referenced by:

• htmlTableSpec

htmlTableExpression:

referenced by:

• htmlTableSpec

htmlTableColumns:

A list of HTML table column specifications.

referenced by:

• htmlTableSpec

htmlTableColumnSpec:

Each HTML table column is mapped to a data source column using the path relative to the master path.

referenced by:

• htmlTableColumns

stringSplitSpec:

Splits a text upon another text, return one row per element.

Parameters:

• Input: The text to split.
• Splitter: Text being split upon.

Returns: a number of rows with one column named 'value'; one per element in the input.

referenced by:

• dataSource

internetTableSpec:

Data stored on the Internet accessible through the HTTP protocol can be interpreted as a data source using the internettable keyword. A depth-first scan is done in which solely unique URLs are returned. Starting of at one starting URL, URLs are downloaded from the Internet consisting of webpages and content. Contents is made available with no further deeper inspection. Webpages in HTML format are scanned for more URLs by default for the following paths:

• //a[@href]: all hrefs in anchors;
• //script[@src]: all sources of scripts;
• //link[@href]: all hrefs of links.
• //img[@src]: all hrefs of images.

The startAtExpression specifies the initial webpage to retrieve data for.

A pre-defined list of columns is available per retrieved URL:

• URL: URL of page;
• Contents_char: the character contents, converted from the original character set into UTF-8;
• Contents_blob: the binary contents;
• Mime_type: MIME-type returned by the web server;
• Http_status_code: numeric HTTP response status code;
• Date_retrieval_utc: date/time when the response was received (UTC);
• Retrieval_duration_ms: time between the request and complete response in milliseconds;
• Bytes_retrieved: number of bytes retrieved;
• Depth: recursion depth, starting at 1 for the initial URL;
• Retrieval_successful: indicator whether the response was completely successful retrieved;
• Last_modified: date/time when the response's content was last modified;
• Etag: ETAG on the content as returned by the web server;
• Content_disposition: preferred file name and encoding to be used;
• Cache_Control: contents of cache-control HTTP response header;
• Expires: contents of the Expires HTTP response header;
• Error_message_code: Invantive UniversalSQL engine error message code if any occurred;
• Error_message_text: Invantive UniversalSQL engine error message code if any occurred.

referenced by:

• dataSource

startAtExpression:

referenced by:

• internetTableSpec

sitemapExpression:

referenced by:

• internetTableSpec

excludeExpression:

referenced by:

• internetTableSpec

internetTableOptions:

The process can be controlled using options:

• Stay on site: when present, recursion restricts to URLs on the same host name as the starting URL. Default behaviour is to branch out to other sites too.
• Maximum depth: limit the depth of the recursion to a specific number.
• Ignore errors: do not stop on the first error but continue. Default behaviour is to stop on the first error or result completion, whatever comes first.

referenced by:

• internetTableSpec

sqlDataTypeExtended:

referenced by:

• castExpression
• createTableArgument
• csvTableColumnSpec
• excelTableColumnSpec
• htmlTableColumnSpec
• jsonTableColumnSpec
• xmlTableColumnSpec

sqlDataType:

referenced by:

• pSqlDataType
• sqlDataTypeExtended

groupBy:

Grouping of multiple rows into groups is specified by the groupBy. A group will be introduced for each distinct combination of column values for the columns listed. The values of grouped columns can be used in the select clause. Columns not being grouped upon can only be used within the context of a group function listed as 'aggregateFunction'.

referenced by:

• uniqueSelectStatement

orderBy:

Sort the rows returned as specified by the list of columns. Values are either sorted ascending (the default) or descending.

referenced by:

• listAggAggregateFunction
• selectStatement

pivotClause:

referenced by:

• dataSource

sortDirection:

A sort direction can be either 'asc' for 'ascending' (the default) or 'desc' for 'descending'.

referenced by:

• sortedColumn

columnList:

A comma-separated list of columns.

referenced by:

• allColumnsSpecId
• groupBy
• identifiedByMultipleClause
• insertFieldList
• setOperatorSelectStatement
• synchronizeInsertStatement
• synchronizeUpdateStatement

sortedColumnList:

An ordered comma-separated list of column values to sort upon.

referenced by:

• orderBy
• resolveByClause

sortedColumn:

A column values to sort upon.

referenced by:

• sortedColumnList

column:

A column is identified by an identifier, possibly prefixed by the name of the table or the alias of the table from which the column is to be taken.

referenced by:

• columnList
• pivotClause
• sortedColumn
• updateValue

columnNoAliasList:

referenced by:

• pivotClause

columnNoAlias:

referenced by:

• columnNoAliasList

whereClause:

The where-clause restricts the number of rows in a result set by applying one or more boolean condiditions which rows must satisfy.

referenced by:

• deleteStatement
• uniqueSelectStatement
• updateStatement

joinStatements:

A list of join statements.

referenced by:

• uniqueSelectStatement

joinStatement:

A join statement combines two result sets. Only combinations of rows taken from both result sets are returned when they meet the join conditions.

referenced by:

• joinStatements

joinCategory:

The join category specifies what combinations of rows are considered. The following variants can be used:

• inner join, as indicated by 'join' or 'inner join': an inner join returns all combinations of rows from both result sets that meet the join conditions.
• left outer, as indicated by 'left outer join': a left outer join returns the same rows as an inner join, extended by one row for each row in the left result set having no matching rows in the right result set. Each column that originates from the right result set is assigned a null value.
• right outer, as indicated by 'right outer join': a right outer join returns the same rows as an inner join, extended by one row for each row in the right result set having no matching rows in the left result set. Each column that originates from the left result set is assigned a null value.
• full outer, as indicated by 'full outer join': a full outer join returns the same rows as an inner join, extended by one row for each row in the right result set having no matching rows in the left result set. Each column that originates from the left result set is assigned a null value. The results are also extended by one row for each row in the left result set having no matching rows in the right result set. Each column that originates from the right result set is assigned a null value.
• cross join, as indicated by 'cross join': a cross join returns a Cartesian product of the rows from both result sets. A 'Cartesian product' is a term from set theory, which indicates that all combinations are returned.

referenced by:

• joinStatement

joinSubCategory:

The join sub-category refines the join category. Please see 'joinCategory' for an explanation.

referenced by:

• joinCategory

join:

referenced by:

• joinStatement

inner:

referenced by:

• joinCategory

outer:

referenced by:

• joinCategory

left:

Extracts a substring from a value with the given length from the left side. Keyword is also used with joins.
Parameters:

• Input: Text to extract substring from.
• Length: Maximum length of the substring.

referenced by:

• functionExpression
• joinSubCategory

right:

Extracts a substring from a value with the given length from the right side. Keyword is also used with joins.
Parameters:

• Input: Text to extract substring from.
• Length: Maximum length of the substring.

referenced by:

• functionExpression
• joinSubCategory

full:

referenced by:

• joinSubCategory

cross:

referenced by:

• joinCategory

sum:

Group function to sum together individual numerical values. Occurrences of null are considered 0, unless there are only null values. In that case the outcome is null.

referenced by:

• sumAggregateFunction

product:

Group function to multiply together individual numerical values. Multiplying large values can quickly exceed the range of the resulting Decimal data type. The product group function is typically used in financial and probability calculations with values near 1.

referenced by:

• productAggregateFunction

min:

Group function to find the minimum value from a group of numerical values or text values.

referenced by:

• minAggregateFunction

max:

Group function to find the maximum value from a group of numerical values or text values.

referenced by:

• maxAggregateFunction

avg:

Group function to find the average value from a group of numerical values.

referenced by:

• avgAggregateFunction

first:

Group function to the first non-null value in an ordered result set.

referenced by:

• firstAggregateFunction

last:

Group function to the last non-null value in an ordered result set.

referenced by:

• lastAggregateFunction

stddev:

Group function to find the standard deviation from a group of numerical values.

referenced by:

• stdDevAggregateFunction

count:

Group function to find the number of values from a group of values.

referenced by:

• countAggregateFunction

listagg:

Group function which concatenates all individual values, separated by the separator when provided and comma plus space otherwise.

referenced by:

• listAggAggregateFunction

asc:

referenced by:

• sortDirection

desc:

referenced by:

• sortDirection

joinConditions:

A boolean expression which defines valid combinations of the join.

referenced by:

• joinStatement

selectList:

referenced by:

• uniqueSelectStatement

selectPart:

referenced by:

• selectList

aliased:

referenced by:

• dataSource
• pivotClause
• selectPart

labeled:

Defines the textual label of an expression. The label may contain resources in the format '{res:resource code}' such as 'My {res:itgen_description}'. Similar to the data type and alias of an expression, the label is maintained across selections. Application of a calculation or a SQL function resets the label to the empty value. User interfaces can choose to display the label when available instead of the column name to provide a more natural interface.

referenced by:

• selectPart
• sqlDataTypeExtended

part:

referenced by:

• countAggregateFunction
• selectPart

aggregateFunction:

referenced by:

• part
• pivotClause

sumAggregateFunction:

referenced by:

• aggregateFunction

productAggregateFunction:

referenced by:

• aggregateFunction

minAggregateFunction:

referenced by:

• aggregateFunction

maxAggregateFunction:

referenced by:

• aggregateFunction

firstAggregateFunction:

referenced by:

• aggregateFunction

lastAggregateFunction:

referenced by:

• aggregateFunction

avgAggregateFunction:

referenced by:

• aggregateFunction

stdDevAggregateFunction:

referenced by:

• aggregateFunction

listAggAggregateFunction:

referenced by:

• aggregateFunction

countAggregateFunction:

referenced by:

• aggregateFunction

zipAggregateFunction:

referenced by:

• aggregateFunction

allColumnsSpec:

Selects all columns from an SQL statement or from one or more data sources used. The except clause allows retrieval of all columns except the listed names. The except clause is typically used with wide result sets with possibly varying column lists from which a few pre-defined columns need to excluded.

referenced by:

• part

allColumnsSpecId:

referenced by:

• allColumnsSpec

allColumnsSpecColumnNamePrefix:

Defines the aliases of columns selected by the select all ('*') operator as their original alias, prefixed with the specified text. For instance, the select operator changes the alias of the column 'code' with the prefix 'pjt_' into 'pjt_code'.

referenced by:

• allColumnsSpec

allColumnsSpecColumnNamePostfix:

Defines the aliases of columns selected by the select all ('*') operator as their original alias, postfixed with the specified text. For instance, the select operator changes the alias of the column 'code' with the postfix '_from_table2' into 'code_from_table2'.

referenced by:

• allColumnsSpec

allColumnsSpecLabelPrefix:

Defines the label of columns selected by the select all ('*') operator as their original label, prefixed with the specified text. For instance, the select operator changes the label 'Name' of the column 'name' with the prefix 'Project ' into 'Project Name'.

referenced by:

• allColumnsSpec

allColumnsSpecLabelPostfix:

Defines the label of columns selected by the select all ('*') operator as their original label, postfixed with the specified text. For instance, the select operator changes the label 'Name' of the column 'name' with the postfix ' from Table2' into 'Name from Table2'.

referenced by:

• allColumnsSpec

ddlStatement:

All available Data Definition Language statements.

referenced by:

• sqlStatement

alterPersistentCacheStatement:

Besides an in-memory cache valid during the duration of a session, Invantive UniversalSQL offers an integrated cache storing data persistently using an on-premise or cloud relation database such as SQL Server or PostgreSQL. When configured, Invantive UniversalSQL first tries to find sufficiently fresh data in the cache. This reduces the number of data loads from slow data containers such as some cloud platforms. In general, the performance increase when the rows can be fully retrieved from a cache is between a factor 25 and 2.500.

Invantive UniversalSQL itself manages the table structure and table contents in the relation database used as a data cache. On initial use just provide an empty database. Platforms supported include SQL Server, Oracle, PostgreSQL and MySQL. Invantive UniversalSQL installs a repository consisting of a dozen tables. The repository tables have names starting with 'dc_'. Most repository tables are accompanied by a view named identically except for a trailing '_r'. The repository views join in all master tables for easier analysis from within the database storing the facts

For each table partition version, a so-called 'facts table' is created. A facts table contains a full copy of the rows retrieved from the data container. Facts tables have names starting with 'dcd_', followed by a unique hash signaling the table partition version. When necessary, additional database objects are maintained such as indexes to improve performance. As with facts table names, all column names are also hashed based upon an algorithm including the original column name. These facts tables are not intended for direct use using native SQL.

Each facts table has a unique state from the following state, with 'Ready' state signaling the now current version:

• Initializing ('I'): the facts table will be created.
• View creation ('V'): logical views will be created.
• Prepared ('P'): the facts table has been created, but contains yet no rows.
• Seeding ('S'): the facts table is being seeded with the contents of the previously current version.
• Loading ('L'): loading new facts from data container using water shed or another algorithm.
• Ready ('R'): the facts table is available and the current one to be used.
• Obsoleted ('O'): the facts table still exists, but the data has passed it's conservation period. Often a newer version is now current.
• Dropped ('D'): the facts table now longer exist, but the metadata is still present in the repository tables.

The persistent cache in the database can be used with native SQL when extended by Invantive Data Replicator. Invantive Data Replicator creates and maintains database view (a so-called 'partition view') for the now current version of table partition. Similarly, it can create an 'overall view', showing the rows across all partitions of the now current versions per partition. Invantive Data Replicator provides a high-performance high-volume operational data store with little effort.

The overall views are typically used for consolidation purposes, bringing together data across multiple companies or persons.

referenced by:

• ddlStatement

alterPersistentCachePurgeStatement:

The purge statement maintains existing table partition versions. It can purge table partition versions in several ways:

• ready: selects table partition versions in state 'Ready'. Table partition versions go to the state 'Obsolete'. Facts are not deleted.
• obsolete: selects table partition versions in state 'Obsolete'. Facts of these are deleted. Table partition versions go to the state 'Dropped'. The facts deleted might be used in queries that started running when they were in state 'Ready'. The facts can be deleted safely when the database platform supports reading data from rows already deleted, such as Oracle. Purging obsolete facts is often used as a background task separate from refresh to reduce refresh runtime, since the deletion of data has already been executed before refresh.
• droppable: selects table partition versions in state 'Dropped'. Facts are deleted when any facts are still present despite the dropped state. This will be useful only under exceptional conditions, since the dropped status indicates that the facts have already been deleted. Table partition versions remain in the state 'Dropped'.
• unknown: queries the database for tables that match the naming convention 'dcd...' or 'dcs...'. Any tables that is not registered in the repository is dropped from the database. This will be useful only under exceptional conditions, such as when the repository was unintentionally dropped or incorrectly changed.
• all: first performs the "unknown" variant and then the "droppable" variant. This will be useful only under exceptional conditions.

Specification of a data container limits the scope of table partition versions to the listed data container.

PSQL Packages

Invantive PSQL comes with a number of pre-defined packages. These can be queried using SystemPackages@DataDictionary and SystemPackageFunctions@DataDictionary.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheDownloadStatement:

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheConfigureWebhooksStatement:

Trickle loading is an advanced strategy to efficiently and fast update the replicate data. It requires all changes (insert, update and delete) on the replicated data to be registered as event messages, possibly with a delay.

The registration of changes can be activated on selected platforms using webhooks. Configuration of the webhooks can be done using this statement. When not specified, the license contract code of the current subscription will be used.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheRefreshStatement:

This statement triggers the refresh of replicated data of all connected or one specific data container alias. A refresh of the replicated data can also be triggered by executing a SQL statement specifying the use of replicated data of a specific age.

In default mode (without 'force'), a refresh is only executed on the specified data when the age of the replicated data exceeds the configured maximum age. An offset to the configured maximum age can be specified using the 'when obsolete within' clause. With a forced refresh, the replicated data is always replaced by a recent version of the source data.

The maximum number of parallel processes to replicate the data can be configured using the 'parallel' clause to increase throughput and decrease runtime of the replication process.

By default the approach as configured on each table partition is used to update the replica. However, a deviating approach can be specified. This is typically done after switching a table partition to trickle loading to run a one-time replication process with the copy approach, effectively ensuring that no changes have gone unnoticed.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheLoadStatement:

Loads all available tables across all connected data containers in the cache. Typically used for demonstrations.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheTableRefreshStatement:

Refresh all data of a specificied table. The options are explained at alterPersistentCacheRefreshStatement.

referenced by:

• alterPersistentCacheStatement

alterPersistentCachePartitionRefreshStatement:

Refresh all data of a specificied partition. The options are explained at alterPersistentCacheRefreshStatement.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheDropStatement:

Drops all facts stored in table partition versions from the database and the associated metadata from the cache repository for a range of table partitions. The tables and partitions can be specified using the table, partition and/or data container clause.

referenced by:

• alterPersistentCacheStatement

alterPersistentCacheSetStatement:

Change central maintained settings for the cache, including defaults for new tables or data containers.

• Fresh: number of seconds during which facts are considered fresh after completiong of loading.
• Retention: number of seconds trickle loading messages are kept available after processing.
• Metadata: number of seconds metadata are kept available after dropping the associated facts in the version.
• Metadata recyclebin: number of seconds dropped metadata are kept available.
• Version: reset data model version to another version to re-run upgrade scripts.
• Token: secret text used as token to retrieve trickle loading messages.
• Prefix/postfix: prefix and/or postfix for Invantive Data Replicator maintained database views.
• Maintain: indicator whether to maintain database views per table partition and/or overall table.
• My Messages: indicator whether to place a copy of incoming trickle loading messages in dc_my_incoming_messages for custom code purposes.
• Index Tables: whether to create non-unique indexes on the facts tables for speedier retrieval on database views.
• Auto Upgrade: whether to run a data model upgrade once.
• Data Container Prefix/postfix: prefix and/or postfix for Invantive Data Replicator maintained overall views.