Lookup not loaded

Hi All,

I just started on writing queries in druid around couple of days back. Trying to create look up. I am using web console to create a look up and a csv which is in s3 as input. I was able to add the lookup through web console. While trying to access the lookup in the DRUID SQL am getting ‘LookUp’ not found null pointer exception.

So when i looked at the status of the lookup using ‘/druid/coordinator/v1/lookups/status?detailed=true’ it gives the below response.

{

“__default”: {

“lookupName”: {

“loaded”: false,

“pendingNodes”: [

“ipaddress:port”,

“ipaddress:port”,

“ipaddress:port”

]

}

}

}

I am using below JSON to create lookup.

{

“lookupExtractorFactory”: {

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “uri”,

“uri”: “s3bucketfilelocation.csv”,

“namespaceParseSpec”: {

“format”: “csv”,

“columns”: [

“demo_id”,

“demo_name”

]

},

“pollPeriod”: “PT30S”

},

“firstCacheTimeout”: 0

}

}

I have added ‘druid-lookups-cached-single’ to the ‘druid.extensions.loadList’ property. Are there any more properties changes to be done before creating the look ups? And what is the use of ‘druid.zk.paths.lookupTier’ property? Do we need to set anything for that and where to set that property?

Hi Varun,

Looking at the logs on those pendingNodes should help give some clues.

Thanks,

Sashi

Hi Varun,

Could you try Submitting this lookup configuration JSON to Druid coordinator endpoint which should load the lookups into Druid :

1] $ curl -H “Content-Type: application/json” --data @.json http://localhost:8081/druid/coordinator/v1/lookups/config

If this is the first time you’re submitting lookup to Druid coordinator, you may get an error like below :

{“error”:“Not initialized. If this is the first lookup, post an empty map to initialize”}

In this case , you need to submit an empty JSON to initialize the druid lookup :

2] $ curl -H “Content-Type: application/json” --data {} http://localhost:8081/druid/coordinator/v1/lookups/config

Once you submit the lookup spec , you can check if your lookup is loaded into druid broker or not by running below curl :

3] $ curl -X GET http://localhost:8082/druid/listen/v1/lookups
{“current”:{},“toLoad”:{},“toDrop”:}

If the lookup is not loaded wait for your poll interval to expire before the table is updated on the broker and run again the curl command.

You can also look into the druid service logs for the potential issues/errors.

Hope this helps.

Thanks and Regards,

Vaibhav

Hi Vaibhav,

After removing the “lookupExtractorFactory” from the spec the look up worked fine. Not sure what went wrong by including ‘lookupExtractorFactory’ and what is the use of that ‘lookupExtractorFractory’. After removing it the spec looked like below :

{

“type”: “cachedNamespace”,

“extractionNamespace”: {

“type”: “uri”,

“uri”: “s3bucketfilelocation.csv”,

“namespaceParseSpec”: {

“format”: “csv”,

“columns”: [

“demo_id”,

“demo_name”

]

},

“pollPeriod”: “PT30S”

},

“firstCacheTimeout”: 0

}

Hi Kumar,

May be you can use “druidpy” python module which helps to work on all the activities of lookup including posting, deleting, finding the False/True state lookups. Its simple to use!