Bulk Update Druid Lookups

I’m attempting to add multiple lookups at once using the “Bulk Update” guidelines found in the docs.

I can add individual lookups successfully by posting to /druid/coordinator/v1/lookups/. However, when I post multiple lookups to /druid/coordinator/v1/lookups/config I get a 405 Method Not Allowed, with the only allowed methods being HEAD,GET,OPTIONS.

Are you setting the content type to "application/json" in your bulk update
POST request? If not, can you try adding that? I wonder if that's
preventing the request from being routed to the proper handler.

Hi Jonathan. Thanks for the response. I am setting the content type. Here’s the full curl command.

curl -X POST \

http://myhost:8081/druid/coordinator/v1/lookups/config \

-H ‘Cache-Control: no-cache’ \

-H ‘Content-Type: application/json’ \

-H ‘Postman-Token: aa7e05e6-b2ed-ed49-74c2-cdb514e58764’ \

-d '{

“__default”: {

“fullname_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “full_name”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“email_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “email”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“title_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “title”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“user_type_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “user_type”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

}

},

“organization_name_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “organization_name_truth”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“entity_code_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “entity_code_truth”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“top_parent_organization_name_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “top_parent_name”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

},

“top_parent_entity_code_lookup”: {

“version”: “v2”,

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “jdbc”,

“connectorConfig”: {

“createTables”: true,

“connectURI”: “jdbc:postgresql://myjdbc:5432/mydb”,

“user”: “username”,

“password”: “password”

},

“table”: “users_master_with_hash”,

“keyColumn”: “userid_hash”,

“valueColumn”: “top_parent_entity_code”

},

“firstCacheTimeout”: 120000,

“injective”: false

}

}

}’

``

what version of Druid you’re using?

also, can you try following instead…

curl -vv -X 'POST' -H 'Content-Type:application/json' [http://myhost:8081/druid/coordinator/v1/lookups/config](http://myhost:8081/druid/coordinator/v1/lookups/config) -d@<file-containing-json>

and paste everything printed on terminal if it doesn't work.

I’m using version 0.10.1-iap4

Here’s the full output.
curl -vv -X ‘POST’ -H ‘Content-Type:application/json’ http://localhost:8081/druid/coordinator/v1/lookups/config -d@lookups.json

  • About to connect() to localhost port 8081

  • Trying 127.0.0.1… connected

  • Connected to localhost (127.0.0.1) port 8081

POST /druid/coordinator/v1/lookups/config HTTP/1.1

User-Agent: curl/7.15.3 (x86_64-unknown-linux-gnu) libcurl/7.15.3 OpenSSL/0.9.8w

Host: localhost:8081

Accept: /

Content-Type:application/json

Content-Length: 4838

Expect: 100-continue

< HTTP/1.1 100 Continue

HTTP/1.1 405 Method Not Allowed

< Date: Tue, 09 Jan 2018 15:31:12 GMT

< Allow: HEAD,GET,OPTIONS

< Content-Length: 0

< Server: Jetty(9.3.19.v20170502)

  • Connection #0 to host localhost left intact

  • Closing connection #0

``

In 0.10.1 version , that endpoint was “/druid/coordinator/v1/lookups” as noted in “http://druid.io/docs/0.10.1/querying/lookups.html” .

Please see release specific Druid documents as things change time to time :slight_smile: . We try to stay as much backward compatible as possible and provide updates in release notes but sometimes changes are necessary.

when you upgrade to 0.11.0 release or higher then you the new endpoint will be available.

Ahh… My bad. Before you asked, I thought I was on the latest version. Thank you for the help!