Using Dynamic Configuration Lookups in Pivot

Hi,

I am trying to use Dynamic Configuration experimental Lookup feature. As documented here, I had posted the lookup mapping to druid coordinator. My mapping looks like:

{

"__default": {

    "account_uuid": {

      "type": "map",

      "map": {

        "gwew58ac-ea50-43a2-aaa0-54aca67412de": "example.com"

      }

    }

}

}

Using coordinator’s lookup API curl http://druid-master.blueshift.vpc:8081/druid/coordinator/v1/lookups/__default/account_uuid, I can verify that the mapping was successfully updated.

I am trying to make this mapping available in Pivot.

Pivot’s documentation(http://pivot.imply.io/app-settings) has a section on Lookups but I can’t make sense of the formula.

The example listed in the document is:

      - name: correctValue
        formula: $lookupKey.lookup('my_awesome_lookup')

What is ‘my_awesome_lookup’ in the above example?

Can I use lookup tier and id in Pivot? If yes, how? If I am missing something please let me know.

Thanks

Anuraj

FYI: Using imply 2.0.0 which bundles Druid 0.9.2 and Pivot 0.10.27.

In your case it should be “account_uuid

I have tried $account_uuid.lookup(‘account_uuid’) but didn’t work. Realized that lookup mapping is not propagated from coordinator to broker and historic due to following exceptions. Do I need to add some extension to druid.extensions.loadList?

I see the following exception in overlord log:

2016-12-17T23:14:37,695 ERROR [LookupCoordinatorManager–8] io.druid.server.lookup.cache.LookupCoordinatorManager - Error submitting to [http://ip-172-31-103-190.us-west-2.compute.internal:8100/druid/listen/v1/lookups]: {class=io.druid.server.lookup.cache.LookupCoordinatorManager, exceptionType=class java.io.IOException, exceptionMessage=Bad update request to [http://ip-172-31-103-190.us-west-2.compute.internal:8100/druid/listen/v1/lookups] : [500] : [Internal Server Error] Response: [:slight_smile:

��error�Could not resolve type id ‘account_uuid’ into a subtype of [simple type, class io.druid.query.lookup.LookupExtractorFactory]

at [Source: N/A; line: -1, column: -1]��]}

java.io.IOException: Bad update request to [http://ip-172-31-103-190.us-west-2.compute.internal:8100/druid/listen/v1/lookups] : [500] : [Internal Server Error] Response: [:slight_smile:

��error�Could not resolve type id ‘account_uuid’ into a subtype of [simple type, class io.druid.query.lookup.LookupExtractorFactory]

at [Source: N/A; line: -1, column: -1]��]

Historical and Broker node shows the same message:

2016-12-17T23:38:16,779 ERROR [qtp208437930-58] io.druid.server.listener.resource.ListenerResource - Exception in handling POSTAll request

java.lang.IllegalArgumentException: Could not resolve type id ‘account_uuid’ into a subtype of [simple type, class io.druid.query.lookup.LookupExtractorFactory]

at [Source: N/A; line: -1, column: -1]

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2774) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2716) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2707) ~[jackson-databind-2.4.6.jar:2.4.6]

at io.druid.server.listener.resource.AbstractListenerHandler$2.apply(AbstractListenerHandler.java:96) ~[druid-server-0.9.2.jar:0.9.2]

at com.google.common.collect.Maps$7.transformEntry(Maps.java:1810) ~[guava-16.0.1.jar:?]

at com.google.common.collect.Maps$10.getValue(Maps.java:1855) ~[guava-16.0.1.jar:?]

at com.google.common.collect.RegularImmutableMap.(RegularImmutableMap.java:87) ~[guava-16.0.1.jar:?]

at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:294) ~[guava-16.0.1.jar:?]

at io.druid.server.listener.resource.AbstractListenerHandler.handlePOSTAll(AbstractListenerHandler.java:89) ~[druid-server-0.9.2.jar:0.9.2]

at io.druid.server.listener.resource.ListenerResource.serviceAnnouncementPOSTAll(ListenerResource.java:92)

Resolved.

It’s because a while back I had POST’ed a bad mapping (see below). I was not able to delete it via /druid/coordinator/v1/lookups/ because you know id was “”. Nuked the entry from druid metadata store DELETE FROM druid.druid_config where name=‘lookups’. Uploaded the correct mapping and it worked.

{

"__default": {

    "": {

      "type": "account_uuid",

      "map": {

        "gwew58ac-ea50-43a2-aaa0-54aca67412de": "[example.com](http://example.com/)"

      }

    }

}

}