Load TPC-H data into druid

Hi,

I have generated test data from db-gen tool.Do i have to convert .tbl files to .json files to post data to druid ?..if Yes,how do i do it ?

How do i post the generated sample data to druid ? and how do i write injection spec for it ?. I have attached a couple of sample test data generated from db-gen.

Thanks !

customer.tbl.1 (15.5 KB)

customer.tbl.99998 (15.6 KB)

supplier.tbl (2.7 MB)

Hi,

here is the indexTask spec I’m using for the lineitem table.

Best,

Jihoon

lineitem.json (2.28 KB)

Got it. Thank you !

Below is the sample spec that was working for me just in case someone needs it

{

“type”: “index”,

“spec”: {

“dataSchema”: {

“dataSource”: “tpch_lineitem_small”,

“parser”: {

“parseSpec”: {

“format”: “tsv”,

“delimiter”: “|”,

“columns”: [

“l_orderkey”,

“l_partkey”,

“l_suppkey”,

“l_linenumber”,

“l_quantity”,

“l_extendedprice”,

“l_discount”,

“l_tax”,

“l_returnflag”,

“l_linestatus”,

“l_shipdate”,

“l_commitdate”,

“l_receiptdate”,

“l_shipinstruct”,

“l_shipmode”,

“l_comment”

],

“timestampSpec”: {

“column”: “l_shipdate”,

“format”: “yyyy-MM-dd”

},

“dimensionsSpec”: {

“dimensions”: [

“l_orderkey”,

“l_partkey”,

“l_suppkey”,

“l_linenumber”,

“l_returnflag”,

“l_linestatus”,

“l_shipdate”,

“l_commitdate”,

“l_receiptdate”,

“l_shipinstruct”,

“l_shipmode”,

“l_comment”

]

}

}

},

“granularitySpec”: {

“type”: “arbitrary”,

“queryGranularity”: “DAY”,

“intervals”: [

“1980/2020”

]

},

“metricsSpec”: [

{

“type”: “count”,

“name”: “count”

},

{

“type”: “longSum”,

“fieldName”: “L_QUANTITY”,

“name”: “L_QUANTITY”

},

{

“type”: “doubleSum”,

“fieldName”: “L_EXTENDEDPRICE”,

“name”: “L_EXTENDEDPRICE”

},

{

“type”: “doubleSum”,

“fieldName”: “L_DISCOUNT”,

“name”: “L_DISCOUNT”

},

{

“type”: “doubleSum”,

“fieldName”: “L_TAX”,

“name”: “L_TAX”

},

{

“type”: “hyperUnique”,

“fieldName”: “L_SHIPMODE”,

“name”: “L_SHIPMODE”

}

]

},

“ioConfig”: {

“type”: “index”,

“firehose”: {

“type”: “local”,

“filter”: “lineitem.tbl.gz”,

“baseDir”: “/druid/current/indexfiles”

}

}

}

}