Topics: Coronavirus | AI | Data Science | Deep Learning | Machine Learning | Python | R | Statistics

KDnuggets Home » News » 2016 » Sep » Tutorials, Overviews » HPE Haven OnDemand: Powerful Data Connectors for the Cloud and Enterprise ( 16:n32 )

HPE Haven OnDemand: Powerful Data Connectors for the Cloud and Enterprise


HPE Haven OnDemand simplifies how you can interact with data, allowing it to be transformed into an asset anytime, anywhere. Find out how the Connector APIs can facilitate this interaction.




HPE Haven OnDemand

Start Connector API

The Start Connector API is used to start a previously created connector. The Start Connector API has only one required parameter, connector, which specifies the name of the connector to be started.

curl -X POST --form "connector=myconnector" 
  --form "ignore_previous_state=false" 
  --form "apikey=YOUR_API_KEY" 
  https://api.havenondemand.com/1/api/sync/startconnector/v1


Response:

{
  "connector": "myconnector",
  "token": "MTAuOC4xNi45OjcyMDA6RkVUQ0g6MTQ2NjgxMTExMTc5OC0xODE0OTYzNzY2"
}


The token it returns is a unique identifier of the currently started connector.

With a connector created and started, the real power of HPE Haven OnDemand APIs can now be harnessed. The data contained within a connector, once indexed, can be used by the myriad Haven OnDemand APIs of all types and be extracted, manipulated, and processed in meaningful ways.

While examples of these endless possibilities are not covered in this article, please refer to the following for further information:

Update Connector API

The Update Connector API is used to update the configuration of an existing connector. It functions similarly to the Create Connector API, with the obvious exception that the specified options are applied to an existing API. See the documentation for further information on this API.

Stop Connector API

The Stop Connector API requires one parameter, connector, used to specify the connector to stop.

curl -X POST --form "connector=myconnector" 
  --form "apikey=YOUR_API_KEY" 
  https://api.havenondemand.com/1/api/sync/stopconnector/v1


The response of the above POST method cURL call is shown below:

{
  "connector": "myconnector",
  "message": "Stopping connector... This process can take up 5 minutes to propagate."
}


And as easy as that, the connector has been stopped. In order to get rid of the connector, it must be deleted.

Delete Connector API

The Delete Connector is used to delete an existing connector. The following asynchronous POST method API call accomplishes the task of deleting myconnector:

curl -X POST --form "connector=myconnector" 
  --form "apikey=YOUR_API_KEY" 
  https://api.havenondemand.com/1/api/async/deleteconnector/v1


The response confirms success:

{
  "actions": [
    {
      "result": {
        "deleted": true,
        "connector": "myconnector"
      },
      "status": "finished",
      "action": "deleteconnector",
      "version": "v1"
    }
  ],
  "jobID": "w-eu_7842f6ad-be27-4494-a64d-b70854d313b2",
  "status": "finished"
}


Remember that delete means permanently, as in the connector cannot be reused at a later time.

A Dropbox Connector in Python

 
Now, let's change gears a little, and have a look at using the Haven OnDemand Python wrapper to create, start, and check the status of a dropbox connector.

Dropbox + HPE

The following Python code excerpt accomplishes the steps described above; check the comments for further information (note that a connector named myconnector already exists at this point as outlined in an earlier section).

# Import Haven OnDemand APIs
from havenondemand.hodclient import *

# Initiate Haven OnDemand client
client = HODClient('YOUR_API_KEY', version='v1')

### Create Connector API ###

# Dropbox-specific params
dropbox_config = {'full_dropbox_access': False}
dropbox_credentials = {'app_key': 'YOUR_APP_KEY', 'access_token': 'YOUR_ACCESS_TOKEN'}
dropbox_credentials_policy = {'notification_email': 'YOUR_EMAIL_ADDRESS'}

# Create connector params
create_params = {
    'flavor': 'dropbox_cloud', 
    'connector': 'my_connector', 
    'config': dropbox_config, 
    'credentials': dropbox_credentials,
    'credentials_policy': dropbox_credentials_policy,
    'destination': {'action': 'addtotextindex', 'index': 'myindex'}
}

# Create Connector API call
print 'Creating connector...'
response = client.post_request(create_params, HODApps.CREATE_CONNECTOR, async=False)
print 'Successfully created connector!'

### Start Connector API ###

# Start connector/check connector status params
params = {'connector': 'my_connector'}

# Start Connector API call
print 'Starting connector...'
response = client.post_request(params, HODApps.START_CONNECTOR, async=False)
print 'Successfully started connector!'

### Check Connector Status API ###

# Check Connector Status API call
print 'Checking connector status...'
response = client.post_request(params, HODApps.CONNECTOR_STATUS, async=False)

# Output status response
print response[0]

print 'Finished!'


As an extended example of the basic Dropbox use case outlined above, have a look at this much more thorough tutorial.

Conclusion

 
The HPE Haven OnDemand Connector APIs are a convenient way in which to move data onto the platform for further use with additional APIs. While one could, instead, specify particular files inside of API calls, in an API-dependent fashion, the benefit of using connectors is the persistence they afford the data, as well as their innate data indexing capabilities. When dealing repetitively with the same data, or when desiring to move large stores of data onto the Haven OnDemand platform for processing, the Connector APIs are definitely worth considering.

As always, be sure to keep to the HPE Haven OnDemand Developer Documentation handy when building with the Haven OnDemand APIs.

Related:


Sign Up

By subscribing you accept KDnuggets Privacy Policy