Can't get Druid to ingest directly from Google Cloud Storage

I have made the necessary changes in the “environment” file to enable GCS on Druid’s console. But when I want to ingest I get this error:

Error: Cannot construct instance of, problem: Unable to provision, see the following errors: 1) Error in custom provider, Error reading credential file from environment variable GOOGLE_APPLICATION_CREDENTIALS, value ‘/home/user/druid/my_service_account_key.json’: File does not exist. at org.apache.druid.common.gcp.GcpModule.getHttpRequestInitializer(

I don’t know what information is necessary to be shared to diagnose the problem but I have set the path like this:


Is there a way to see if it actually can’t find the file or it’s a permission issue?

1 Like

Hands up I’ve not done GCS ingestion … but I note in the docs that there’s no “json” extension in the environment variable — should it have a json extension?

The google credentials file is in JSON format (at least the one I got following the GCS instructions). The Druid documentation is referring to a file which can’t be located. I didn’t try to remove the .json from the file but I got it working by mapping the VM folders to the docker folders (Volumes) in the compose YML file. So the file name might not be the issue. I guess my problem is the mapping volume issues.

So is this variable supposed to be a shell environment variable? or should I define this in my “environment” file? Can someone elaborate?

Hm let me see if I can find someone who knows to chip in…