Authorization causing not able to view running task list in coordinate console

Sudhanshu_Lenka · January 19, 2018, 10:01am

Hi All,

How to get rid of this exception.When i am submitting task from postman to druid/indexer/v1/supervisor , i am not able to see running task in coordinator console and their is other exception also. but able to see data source created.

Version Druid : 0.11 using imply 2.4.3

2018-01-19T09:51:15,121 ERROR [qtp1733142395-100] io.druid.server.security.PreResponseAuthorizationCheckFilter - Request did not have an authorization check performed.: {class=io.druid.server.security.PreResponseAuthorizationCheckFilter, uri=/, method=GET, remoteAddr=ip, remoteHost=ip}

2018-01-19T09:51:15,122 WARN [qtp1733142395-100] org.eclipse.jetty.server.HttpChannel - //ip:8081/

io.druid.java.util.common.ISE: Request did not have an authorization check performed.

at io.druid.server.security.PreResponseAuthorizationCheckFilter.handleAuthorizationCheckError(PreResponseAuthorizationCheckFilter.java:158) ~[druid-server-0.11.0-iap2.jar:0.11.0-iap2]

at io.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:91) ~[druid-server-0.11.0-iap2.jar:0.11.0-iap2]

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) ~[jetty-servlet-9.3.19.v20170502.jar:9.3.19.v20170502]

at io.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:85) ~[druid-server-0.11.0-iap2.jar:0.11.0-iap2]

at io.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:60) ~[druid-server-0.11.0-iap2.jar:0.11.0-iap2]

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) ~[jetty-servlet-9.3.19.v20170502.jar:9.3.19.v20170502]

at io.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:86) ~[druid-server-0.11.0-iap2.jar:0.11.0-iap2]

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) ~[jetty-servlet-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) ~[jetty-servlet-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) ~[jetty-servlet-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.Server.handle(Server.java:534) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) [jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) [jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) [jetty-io-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) [jetty-io-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [jetty-io-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]

at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

Regards,

Sudhanshu Lenka

Jonathan_Wei · January 19, 2018, 10:39pm

Hi Sudhanshu,

I did some investigation on this, I don’t think the PreResponseAuthorizationCheckFilter exception is causing the indexing tasks to not appear in the coordinator console (I tried with a build that had that PreResponseAuthorizationCheckFilter disabled and I still didn’t see the indexing tasks). I’ll look into why that exception is being thrown, but I suspect there’s a separate issue with the UI not displaying tasks.

If I look at the network traffic when loading the “indexing” section page, I see a GET being made to “http://localhost:8081/druid/indexer/v1/runningTasks”, in my local test setup I do see that call returning correct results, e.g.:


[{"id":"index_hadoop_wikiticker2_2018-01-19T22:32:21.187Z","type":"index_hadoop","createdTime":"2018-01-19T22:32:21.195Z","queueInsertionTime":"2018-01-19T22:32:21.197Z","state":null,"duration":null,"location":{"host":"localhost","port":8101,"tlsPort":-1},"metrics":null,"errorMsg":null},{"id":"index_kafka_wikiticker-kafka_be8acc5eae8e7a2_dalkhiin","type":"index_kafka","createdTime":"2018-01-19T22:10:57.225Z","queueInsertionTime":"2018-01-19T22:10:57.225Z","state":null,"duration":null,"location":{"host":"localhost","port":8100,"tlsPort":-1},"metrics":null,"errorMsg":null}]

Here I had a hadoop indexing task and a kafka indexing task running. I didn’t see anything in the UI page for indexing tasks though.

Jonathan_Wei · January 19, 2018, 10:42pm

I also see this error in the Chrome developer tools view when loading the coordinator console:


druid.js:138 TypeError: Cannot read property '3' of null

at Object.parseTaskId (druid.js:13739)

at Object.decorateTask (druid.js:13752)

at tasks.reduce.all (druid.js:13778)

at Array.reduce (<anonymous>)

at Object.processTasks (druid.js:13775)

at druid.js:13856

at druid.js:118

at J (druid.js:147)

at J (druid.js:147)

at druid.js:148

I’m not very familiar with this UI code, but it looks like there may be an issue with the UI parsing the task IDs it receives from Druid.

Sudhanshu_Lenka · January 22, 2018, 7:38pm

Hello Jon,

I went through details also. Not able to find root cause. When i hv started again an clean setup every thing works fine for me.

But noted pointed is that my kafak has some issue for recving data. At that particular time i get this error. After my kafka started working properly everything goes well again.

If i will able to reproduce this issue again. Sure i will note down factors for above issue.