Pivot not showing newly added columns of datasource

Hi All,

We had to add new columns to an existinf datasource on Druid. When segments were being created with new columns, they did not initially seem to get added automatically. So we dropped the datasource and started ingesting the segments again. This time the new columns were seen in the table definition using plyql or on hitting broker. However, the pivot UI doesn’t seem to be updated with the new columns. It still shows the previous schema of the datasource.

  1. Could someone please explain why is pivot not showing the new columns. Is there a bug associated with this or I am missing something?

  2. If an existing datasource has new columns , won’t they get automatically added to the druid datasource.?

Thanks in advance,

Regards,

Rajitha

Hello Rajitha,
I believe the way Pivot discovers the columns of a datasourre is via the metadata query of Druid.

You can verify if the metadata query is returning those new columns or not.

Also in all such cases where Pivot’s auto discovery of schema is not working as expected you can always fix such issues by adding the correct definition of the attributes in Pivot’s config.

Hi Rohit,

Thanks for your response.

I fired the metadata query and found the new columns show up in the query result.

Also in our case, there was no config which was passed while bringing up pivot. I tried to fetch the config using following command

pivot --druid your.druid.broker.host:8082 --print-config --with-comments > config.yaml

to fetch the default config file and also noticed the new columns appear in this config file. However when I tried to restart Pivot with this config file , I faced errors.

What worked for me was simply restarting Pivot without passing any config file. The new columns automatically showed up.

Regards,

Rajitha