Unable to build the docker image for the tutorial

I am trying to follow the tutorial on loading data from Hadoop cluster to Druid but when I try to build the docker image using the command

docker build -t druid-hadoop-demo:2.8.3 .

I get this error:

Step 13/54 : ENV JAVA_HOME /usr/lib/jvm/zulu-8

—> Using cache

—> 059fba6a0737

Step 14/54 : ENV PATH $PATH:$JAVA_HOME/bin

—> Using cache

—> 1dc88f3aa178

Step 15/54 : RUN curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/

—> Running in e0f92cf29ccd

gzip: stdin: unexpected end of file

tar: Child returned status 1

tar: Error is not recoverable: exiting now

The command ‘/bin/sh -c curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/’ returned a non-zero code: 2.

Any help is appreciated.

Can you run:
/bin/sh -c curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/’ and do you have permissions on that directory?

When I try to run just the command

curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/

``

I get the error message below so I don’t have permissions to create the folder, even with sudo, it does not help. Though when I try to run

the docker build -t druid-hadoop-demo:2.8.5 .

``

for druid-0.17, it works so I am not sure why I can create hadoop folder for one druid version but not the other.

hadoop-2.8.3/: Can’t create ‘hadoop-2.8.3’

hadoop-2.8.3/bin/: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/container-executor: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/hadoop: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/hadoop.cmd: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/hdfs: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/hdfs.cmd: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/mapred: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/mapred.cmd: Failed to create dir ‘hadoop-2.8.3’

hadoop-2.8.3/bin/rcc: Failed to create dir ‘hadoop-2.8.3’

``

This issue is fixed in 0.17.0. Please see https://github.com/apache/druid/issues/9268 for more details.

Thanks,

Chi

I am upgrading some existing services from druid 0.12 to druid 0.15 first. Is there a way I can change some files in druid 0.15 to make this work?

The PR linked from the issue I sent earlier shows the changes you need to make to the Dockerfile:
https://github.com/apache/druid/pull/8753/files

Thanks,

Chi

Hi Chi,
I tried building with the new Dockerfile listed in the PR, it fails at step 9 with this message:

Step 7/53 : RUN ssh-keygen -q -N “” -t rsa -f /root/.ssh/id_rsa

—> Using cache

—> 3d3670662a2d

Step 8/53 : RUN cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys

—> Using cache

—> 32a6a2b686e7

Step 9/53 : COPY setup-zulu-repo.sh /root/setup-zulu-repo.sh

COPY failed: stat /var/lib/docker/tmp/docker-builder091990083/setup-zulu-repo.sh: no such file or directory

``