Register Lookups in Druid

HI,

I am trying to setup lookups in druid(version:0.15.0)

Added druid-lookups-cached-global to the loadList :

Submitting first lookup with curl -XPOST ‘http://localhost:<coordinator_IP>/druid/coordinator/v1/lookups/config’ -H ‘Content-Type:application/json’ -H ‘Accept:application/json’ -d @lkp_spec.json failed with error “error”:“Not initialized. If this is the first lookup, post an empty map to initialize”

As per documentation, tried submitting empty spec using curl -XPOST ‘http://localhost:<coordinator_IP>/druid/coordinator/v1/lookups/config’ -H ‘Content-Type:application/json’ -H ‘Accept:application/json’ -d {} but that fails with “error”:“Unknown error updating configuration”

Could anyone help in identifying what could have gone wrong?

Thanks

Soumya

Hi,

I’m assuming you have restarted all the nodes after adding the extension. Please check the below.

  1. What are the HTTP response status codes for the API invocations ?

  2. Check if the druid-lookups-cached-global extension has been loaded properly, looking at the Broker/Historical logs for any exceptions.

  3. Alternatively you could check “curl -X GET <broker_host>:8082/status” and/or “curl -X GET <historical_host>:8083/status” response.

You should see a module with artifact named druid-lookups-cached-global.

Thanks,

Sashi

Try creating a json file with {} and post the file.

Eric Graham

Solutions Engineer -** **Imply

**cell: **303-589-4581

email: eric.graham@imply.io

www.imply.io

Hi Sashi,

Yes, restarted services after adding extension. I get {“name”:“org.apache.druid.server.lookup.namespace.NamespaceExtractionModule”,“artifact”:“druid-lookups-cached-global”,“version”:“0.15.0-incubating”} while calling the get status api.

The APIs to register lookup doesn’t return any http response code, but the response are exactly as follows

{

“error”: “Unknown error updating configuration”

}

We are using PostgreSQL for metadata storage. In one of the google discussion, read about the PostgreSQL version support for UPSERT causing issues in this API. We have postgresql 8.3 version, but UPSERT is supported only from pg 9.5. Could this be an issue?

Hi Eric,

Tried with empty json file too, but same error (curl -XPOST ‘http://localhost:<coordinator_port>/druid/coordinator/v1/lookups/config’ -H ‘Content-Type:application/json’ -H ‘Accept:application/json’ -d @lkp_emtpy_spec.json)

Thank you

Soumya

I don’t have idea on the issue you have mentioned with Postgres. Checking all the Druid server logs for any exceptions might help.

Thanks,

Sashi

Can you try this?

curl -H “Content-Type: application/json” --data @empty.json http://localhost:8081/druid/coordinator/v1/lookups/config

Also try with a different version

curl -H “Content-Type: application/json” --data @empty.json http://localhost:8081/druid/coordinator/v2/lookups/config

Eric Graham

Solutions Engineer -** **Imply

**cell: **303-589-4581

email: eric.graham@imply.io

www.imply.io

Hi Eric,

It seems the problem was with the version of postgresql we were using. Finally we were able to register lookups using same curl commands by switching metastore to derby.

It looks like we need a postgresql version which support UPSERT operations.(which is 9.5 and above), but we have psql 8.3

ThankYou