Druid 0.9.1.1: Enabling datasource level authorization in Druid

Hi guys,

As stated in Druid 0.9.1.1 release notes one new feature is listed among others: Enabling datasource level authorization in Druid, but couldn’t find any documentation regarding it and struggling in setting it up. Is it going to be available any time soon?

Asking because have to enable basic auth at Druid level in order to secure Broker queries from outside of the application.

Best regards,
Shinesun

Hello Shinesun,

Sorry, this feature is not documented mostly because it was experimental and we thought it is going to change. Anyways, I can briefly describe here how it works so that you can work with it.

**** This is what is already there and how it works ****

For the purpose of authorization, Druid HTTP endpoints will extract two information namely a Resource and an Action. As of now, there are three types of Resources in Druid - DATASOURCE, CONFIG and STATE. For any type of resource, possible actions are - READ or WRITE. Here are some examples of what it means in practice -
If a query is posted to Druid then that means the Resource is the ‘DATASOURCE’ being queried and the Action is a READ action
If an indexing task is submitted then the Resource is the DATASOURCE being indexed and the Action is WRITE
If a GET is done on coordinator dynamic config then the Resource is CONIFG and the Action is READ, vice-versa if a POST is done then Action is WRITE
If a GET is done on /status endpoint or /leader endpoint then the Resource is STATE and the Action is READ

Thus, POST or DELETE always corresponds to the WRITE action and GET to READ, the Resource type is pre-defined for each endpoint. DATASOURCE resource type contains more information about datasource name, CONFIG and STATE resource type does not carry any information about the exact configuration or state being accessed or modified.

For more implementation details read the master comment on PR - https://github.com/druid-io/druid/pull/2424

**** This is what you will have to implement ****

If the druid.auth.enabled is set to true then for each HTTP call Druid will extract the appropriate Resource and Action information depending on the endpoint. After this, Druid code will look for an implementation of AuthorizationInfo interface in the request attribute of the HTTP request with attribute name as AuthConfig.DRUID_AUTH_TOKEN (which is currently set to “Druid-Auth-Token”).

Therefore, you will need to implement a servlet Filter which will inject appropriate implementation of AuthorizationInfo interface as a request attribute (“Druid-Auth-Token”). After getting this object, Druid will then call isAuthorized(Resource resource, Action action) method of your implementation. Thus, you will write your authorization logic inside this method, you can use the Resource object to figure out what is being accessed. For example, for an indexing task POST, resource.getType() will return ResourceType.DATASOURCE and resource.getName() will return the datasource name. However, for all CONFIG and STATE ResourceType, return value of resource.getName() is same and does not matter.

Now the question is how to inject the servlet Filter in Druid. The answer is you will need to create a Druid extension. I am assuming you would already be knowing how to create and load custom extensions with Druid. So, I will just give some pointers here - http://druid.io/docs/latest/development/modules.html. Also, there are numerous extensions in the Druid repo here https://github.com/druid-io/druid/tree/master/extensions-core.

In your extension, you will need to implement ServletFilterHolder interface which will return your implementation of AuthorizationInfo. You can follow example of QosFilterHolder here - https://github.com/druid-io/druid/blob/master/server/src/main/java/io/druid/server/initialization/jetty/JettyBindings.java#L53 to write logic that can return your own implementation of AuthorizationInfo. Finally, bind your implementation of ServletFilterHolder in the DruidModule of your extension like this - https://github.com/druid-io/druid/blob/master/server/src/main/java/io/druid/server/initialization/jetty/JettyBindings.java#L48

CONCLUSION -

  • Implement AuthorizationInfo interface and write your authorization logic in the isAuthorized method
  • Inject this implementation in every request sent to Druid as a request attribute with name “Druid-Auth-Token” using servlet filter
  • Enable security by setting druid.auth.enabled=true

As noted this is an experimental feature and is not perfect. If you have any more questions, feel free to ask. Thanks

  • Parag

A minor correction - I said that “In your extension, you will need to implement ServletFilterHolder interface which will return your implementation of AuthorizationInfo.” but actually I meant “In your extension, you will need to implement ServletFilterHolder interface which will return your implementation of servlet Filter.”

Hi Parag,

I’m trying to implement what you’ve mentioned in your post, however, I don’t know where “druid.auth.enabled” is at?

Thanks.

Best

Shirley

Hi shirley,

AFAIK, druid.auth.enabled is the config you will need to set in druid runtime.props

the related code reference is https://github.com/nishantmonu51/druid/blob/master/server/src/main/java/io/druid/server/security/AuthConfig.java#L38-L38

Hi Nishant,

I have implemented authorization based on the instructions. It has the basic functionalities. However, I have one question, how to distinguish user input and druid itself’s communications. Currently the filter’s path is set to “/*”, and authorization is using basic authorization from request header.

This means it will check every request received. If Druid itself is sending request, since there is no user credentials, it will not pass the authorization. Is there a way to bypass this issue? Thanks!

Best

Shirley

In more detail, say if I have a path “/druid/v1/*” in the filter. Then whenever I want to query coordinator to get the metadata for druid, the filter will require user credentials. However, druid overlord also uses this path to do lots of operations, for example, getting task status in overlord console. I’m wondering if there is a way to allow druid perform those queries while restricting outside user for it?

Thanks.

Best

Shirley

UPDATED POST -
Hello Shirley,
This is how we have solved this problem - Druid uses HttpClient created here - https://github.com/druid-io/druid/blob/master/server/src/main/java/io/druid/guice/http/HttpClientModule.java#L103 for internal communication with the nodes. Therefore, one way to solve the problem you mentioned is to implement your own HttpClient by extending com.metamx.http.client.AbstractHttpClient. In your implementation of com.metamx.http.client.AbstractHttpClient, you will have to override the go method. In the go method, you get the Request object and thus you can set appropriate header in the request object using setHeader method such that your authorization module understands this header and puts appropriate implementation of AuthorizationInfo
in the request attribute that enables Druid nodes communicate with each
other. In our implementation, we set actual security tokens that authorize your Druid nodes for admin access to all actions on the resources but you can set it to whatever you want.
For gluing all these together you will need to extend HttpClientModule.HttpClientProvider and return an instance of your HttpClient in the get method of this class. So this class looks like this -

public class YourHttpClientProvider extends HttpClientModule.HttpClientProvider
{

  // class members...@Inject works here.

  public YourHttpClientProvider()
  {
  }

  public YourHttpClientProvider(Annotation annotation)
  {
    super(annotation);
  }

  public YourHttpClientProvider(Class<? extends Annotation> annotationClazz)
  {
    super(annotationClazz);
  }

  @Override
  public HttpClient get()
  {
    return new YourHttpClient(super.get(), ...other instantiating variables for this class);
  }
}

Finally you will need to bind this provider in the configure method of your DruidModule like this -

binder.bind(HttpClient.class)
      .annotatedWith(Global.class)
      .toProvider(new YourHttpClientProvider(Global.class))
      .in(LazySingleton.class);
binder.bind(HttpClient.class)
      .annotatedWith(Client.class)
      .toProvider(new YourHttpClientProvider(Client.class))
      .in(LazySingleton.class);

For reference YourHttpClient looks like this -

public class YourHttpClient extends AbstractHttpClient
{

  private final HttpClient delegate;
  // other class members

  public YourHttpClient(HttpClient delegate, ..other variables)
  {
    this.delegate = delegate;
    ...
  }

  @Override
  public <Intermediate, Final> ListenableFuture<Final> go(
      Request request,
      HttpResponseHandler<Intermediate, Final> handler,
      Duration readTimeout
  )
  {
    // some logic
    // manipulate request object..set headers...do something else so that this request has appropriate things
    // for your authorization module to identify that this an internal request and
    // put in appropriate instance of AuthorizationInfo in the request attribute
    return delegate.go(request, handler, readTimeout);
  }
}

All these implementations will go into your module. My bad, I should have included this information in the earlier posts. Let me know if you have any further questions.

Thanks,
Parag

Hi Parag

Thanks for all the info... we did implement everything as suggested by you but don't see our http client being called at all hence all internal requests are getting rejected..

Can you please help?

Are you sure that your authentication extension is being loaded by druid. If the module is loaded then you should see some log statements like -
INFO [main] io.druid.initialization.Initialization - Loading extension [] for class […]
OR
INFO [main] io.druid.initialization.Initialization - Adding classpath extension module [] for class […]
depending on how whether you load the extension using loadList or put the extension on classpath respectively. Please read “Loading community and third-party extensions (contrib extensions)” and “Loading extensions from classpath”, here - http://druid.io/docs/latest/operations/including-extensions.html
It would help to put some log statements in the configure method of your authentication module (that you are trying to load) to check if bindings are getting executed.