Druid S3 proxy connectivity issue

Hello

I’ve been having issues with reading file from s3 via proxy into druid. On further investigation, I was able to set the proxy details in jobProperties of tuningConfig.

However, I would like to set one more property:

clientConfiguration.setPreemptiveBasicProxyAuth(false);

in jobProperties.

Is there a way to set this particular client configuration in druid index json file? Thank you.

Hey Vikram,

Are you talking about doing this for a hadoop indexing job? If so, are you using s3n (based on jets3t) or s3a (based on aws-java-sdk)? The default is s3n, and since that uses jets3t, I believe you can get it to do preemptive auth by adding this to jets3t.properties and putting that on your classpath:

httpclient.authentication-preemptive=true

See https://jets3t.s3.amazonaws.com/toolkit/configuration.html for more details on jets3t configs.

Hey Gian,

I’m trying to read a file via S3n using a local Hadoop indexing job. I’m behind Proxy. I have configured http proxy information in jets3t.properties under _common directory, added S3 access and secretkey in jobProperties of the indexing job. However I m still facing socket exception. I’m not sure if any other additional configuration must be done (like setting access/secretkeys in overload/middle manager) etc.

Thank you

Vikram