ETL Loader does not work in OrientDB Kubernetes cluster

11/7/2018

I am trying to use ETL loader for the data import in OrientDB. I have set up OrientDB (version 3.0.10) in a distributed mode in a Kubernetes cluster. When I try to use the import on one of the pods kubectl exec -it orientdbservice-0 -- /orientdb/bin/oetl.sh /orientdb/bin/data/import_json/venue.json it gets stuck on this message and then terminates:

OrientDB etl v.3.0.10 - Veloce (build eac0654847df662ca03b45a6a5efa5eadd229ca5, branch 3.0.x) https://www.orientdb.com

As the result, the database folder is created but the data is not fully imported. As there are no error messages regarding the file location I think that source and extractor parts work fine but I am not sure about the loader part in the json config file. What could be the cause of such behavior? And can I use some other ways to import the data?

Here is my venue.json file:

{
  "source": {"file": {
    "path" : "data/dataset_csv/venues.csv"
  }
  },
  "extractor": {
    "csv": {
      "columns": ["venue_id:string", "address_1:string", "city:string", "country:string", "distance:float", "lat:float", "localized_country_name:string", "lon:float", "venue_name:string", "rating:float", "rating_count:float", "state:string", "zip:integer", "normalised_rating:float"],
      "columnsOnFirstLine": true
    }
  },
  "transformers" : [
    { "vertex": { "class": "Venue" } }
  ],
  "loader": {
    "orientdb": {
      "dbURL": "plocal:/orientdb/databases/MeetupCluster",
      "dbAutoCreateProperties": true,
      "dbType": "graph",
      "wal": false,
      "batchCommit": 1000,
      "tx": true,
      "txUseLog": false,
      "useLightweightEdges" : true,
      "classes": [
        {"name": "Venue", "extends": "V"}
      ], "indexes": [
        {"class": "Venue", "fields":["venue_id:string"], "type":"UNIQUE" }
      ]
    }
  }
}

NOTE: ETL loader works fine on my local cluster created with docker-compose. And this file is one of the smallest in the database, so it is supposed to load fast.

UPDATE: I have set a url for the remote database in such a way:

  "dbURL": "remote:localhost/MeetupCluster"
  "serverUser": "root",
  "serverPassword": "pwd"
  "dbUser": "admin",
  "dbPassword": "admin"

And I get such errors:

Exception in thread "main" com.orientechnologies.orient.core.exception.OConfigurationException: Error on creating ETL processor
Caused by: com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database 'MeetupCluster'
Suppressed: com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database 'MeetupCluster'
Caused by: com.orientechnologies.orient.server.distributed.ODistributedException: No active nodes found to execute command: sql.select from OUser where name = ? limit 1
-- Cassie
data-import
kubernetes
orientdb

0 Answers