Data Lake and Custom Connectors

Overview

Seeq provides a Software Development Kit (SDK) in Java and .NET that facilitates the creation of a connector that can access data sources such as Data Lakes and proprietary databases.  Common reasons to write a custom connector:

  • Your organization has invested in a "data lake" (for example, using Teradata, Hortonworks, Cloudera, DataStax, Redshift etc) that brings together multiple datasources in a non-standard schema.
  • You have data that is stranded in a legacy database or file format.
  • You have data in a SQL database that requires interface code that is beyond the capabilities of Seeq's built-in SQL Connectors.

Seeq's Connector SDK is designed to make the task of writing a new connector as straightforward as possible.

Getting Started

You will want to set up a development machine that can host a licensed, running copy of Seeq Server. The development machine should come close to the hardware specs of a normal Seeq Server installation. You can develop your connector on either Windows or Ubuntu Desktop.

Follow the instructions at Installing and Upgrading Seeq to download, install and license the most recent version of Seeq Server.

If you are new to Seeq, it is strongly recommended that you log in and go through the Interactive Tour, which will pop up automatically the first time you log in and is also accessible from the top-right "hamburger" menu. It is important that you understand what an asset, asset tree and signal are.

Once you feel comfortable with the Seeq product, download an appropriate version of the Connector SDK from the download page.

Extract the contents of the connector to a place on your hard drive.

On Windows, to avoid issues with long filenames, you should extract the Connector SDK to the root of a drive. For example, to C:\seeq-connector-sdk.

Once you have extracted the contents, open the README.md file that is in the root of the SDK package and it contains detailed steps that walk you through the process of setting up your development environment.

By the end of the README, you'll have a working "skeleton" connector that should be a functional starting point from which you can make changes to add your own custom connection code.