How do I connect Invantive Data Replicator to NDJSON?

Execute the following steps to connect Invantive Data Replicator to NDJSON. You only need knowledge of Invantive Data Replicator and your own business.

NDJSON is a file format based upon JavaScript. A NDJSON file consists of one record in JSON-format per line. No array grouping or record separator is used.

No technical knowledge is needed nor knowledge on the NDJSON APIs to connect NDJSON to Invantive Data Replicator. This step-by-step plan will tell you exactly which steps you need to go through on Invantive Cloud to create your own Azure data warehouse with data from NDJSON.

The steps to connect Invantive Data Replicator to NDJSON are:

  1. Register an Invantive Cloud account.
  2. Create a NDJSON database.
  3. Make the NDJSON database available through the Microsoft OData connector.
  4. Connect Invantive Data Replicator to NDJSON through the connector.
  5. Load data from NDJSON into Invantive Data Replicator data warehouse.

Invantive Cloud offers a NDJSON connector to download data from NDJSON into Invantive Data Replicator, but there are over 75 other connectors available also for SQL, Power BI Desktop, Power BI Service, Power Query and/or Azure Data Factory.

Do you have a large environment? For effective downloads, you can of course also filter the data using the filter step or the transform button in Invantive Data Replicator.

When you are having questions, please check the forums for NDJSON. Every organisation is entitled to 1 hour of free training to help you get started. This free hour also applies to free plans.

Register Account on Invantive Cloud

Skip this step when you already have an account on Invantive Cloud. Otherwise execute the following steps one time to register an account on Invantive Cloud:

  1. Go to Invantive Cloud start page.

  2. Select the Log on-button.

  3. Select the Sign Up-button. Sign Up to Invantive Cloud.

  4. Enter your business email address. Enter business email address.

  5. Select the Next-button.

  6. Enter your personal data. Enter personal data.

  7. Select the Next-button.

  8. Enter your password and repeat the provided password.

  9. Select the Next-button. Enter two identical passwords.

  10. You will receive a six-digit verification code by email within 2 minutes.

  11. Enter the six-digit verification code. Six-digit verification code to validate email address.

  12. Select the Sign Up-button.

  13. Log on now. Sign Up.

  14. Make sure you have an authentication app installed on your phone. Select the Next-button. Prepare TOTP-configuration.

  15. Add the displayed QR code to the authentication app, enter the current verification code, and choose "Finish". Scan QR-code into TOTP-app and enter current code.

  16. Skip registration of the organization. Skip company registration.

  17. Complete your personal data and choose 'Save'. Complete personal data.

  18. Go to the dashboard. Navigate to dashboard.

  19. The Invantive Cloud dashboard will be shown.

You now have a login code on Invantive Cloud with which you can set up the connection with NDJSON and numerous other platforms. You will use the same Invantive login code and workflow for all other platforms.

Create NDJSON database

In this step, we set up a database with data from NDJSON. The database is "virtual" because it is not a traditional database, but is fed real-time from NDJSON. Invantive Cloud provides Invantive Data Replicator with a real-time link to NDJSON. The database will be used for all your NDJSON reporting with Invantive Data Replicator. So you only need to perform these steps once.

  1. Click the Add Database button. Add database.

  2. Select NDJSON from the list. Select NDJSON connector.

  3. Please fill out the form with login information for NDJSON.

  4. Select the OK-button.

Congratulations! You can now process data from NDJSON within the Invantive Cloud website. You can do this for example with the interactive SQL editor.

Grant connection from Invantive Data Replicator access to NDJSON

To retrieve the data from Invantive Data Replicator we create a link via Invantive Bridge Online. This creates a "bridge" between the cloud of Invantive and the standard OData connector that is available in every version of Invantive Data Replicator. You do not need to install anything locally: no connector, no ado.NET provider and no Invantive Data Replicator add-on neither.

Execute the following steps to safely use the data from NDJSON outside of Invantive Cloud:

  1. Next to the database you will find an orange text requesting you to allow access from your current location (IP address). Select the nearby orange button.

  2. You must completely disable IP address checking by entering an asterisk ("*") since Microsoft Invantive Data Replicator consists of hundreds of thousands of servers across the globe.

You are now ready to import the NDJSON data into Invantive Data Replicator.

Configure Invantive Data Replicator connector for NDJSON

You will now pull in Invantive Data Replicator data from NDJSON through the established link.

Please note again that Invantive Cloud under the hood handles all complexity such as setting up credentials such as a refresh token, acquiring an access token, optimizing and parallelizing access, accessing an API within the rate limits, scopes and security restrictions. There is absolutely no need for knowledge on complex technical topics such as OAuth access tokens or API. The Microsoft Azure Data Factory steps, activities and pipelines can be constructed using just the OData feed and basic authentication.

Execute the following steps:

  1. Start Microsoft Invantive Data Replicator and select 'Ingest'. Copy NDJSON data using Microsoft Azure Data Factory activity 'Ingest'

  2. Select 'Built-in copy task'. Built-in copy using NDJSON as source

  3. Select source type 'OData' and select 'New connection'. OData connection for NDJSON to Microsoft Azure Data Factory

  4. Enter in the window the Bridge Online URL of the Invantive Data Replicator database. This URL containing the data can be found in the Database-form of Invantive Cloud. Choose authentication type 'Basic authentication'. Enter the Invantive Cloud user name and password. OData URL for NDJSON with log on credentials

  5. The list of available Invantive Data Replicator tables appears. Select the desired tables and construct your pipeline in Microsoft Azure Data Factory. Select NDJSON tables and process them in Microsoft Azure Data Factory