Why does spec document changing for overlord ingestion

Hi,

I am putting 137 dimension and while ingetiing I am seeing different logs.

I am seeing that some dimensions are going to the dimensionExclusion list I wonder why. I used ingestion with other document it was working fine.

  "dimensionsSpec" : {
            "dimensions" : [ "CM_JoiningDate", "CM_campaignId", "CM_companyName", "CM_dateAtWhichAppUninstalledFromAllDevices", "CM_designation", "CM_displayName", "CM_email", "CM_fb_ad_group_name", "CM_fb_ad_set_name", "CM_firstName", "CM_gender", "CM_hasUserRegisteredImmediatelyOnInstall", "CM_hasUserUninstalledAppFromAllDevices", "CM_isLeadSellerAnSP", "CM_lastName", "CM_mediaSource", "CM_name", "CM_numOfYearsOfExperience", "CM_personalEmail", "CM_phoneNumber", "CM_primaryCategory", "CM_primaryRegion", "CM_promoCodeUsedForRegisteringWithApp", "CM_promoCodeUsedForRegisteringWithAppDate", "CM_promoCodeUsedForRegistration", "CM_referralCodeForSharing", "CM_registeredUsingDevice", "CM_userLevelCode", "RF_JoiningDate", "RF_campaignId", "RF_companyName", "RF_dateAtWhichAppUninstalledFromAllDevices", "RF_designation", "RF_displayName", "RF_email", "RF_fb_ad_group_name", "RF_fb_ad_set_name", "RF_firstName", "RF_gender", "RF_hasUserRegisteredImmediatelyOnInstall", "RF_hasUserUninstalledAppFromAllDevices", "RF_isLeadSellerAnSP", "RF_lastName", "RF_mediaSource", "RF_numOfYearsOfExperience", "RF_personalEmail", "RF_phoneNumber", "RF_primaryCategory", "RF_primaryRegion", "RF_promoCodeUsedForRegisteringWithApp", "RF_promoCodeUsedForRegisteringWithAppDate", "RF_promoCodeUsedForRegistration", "RF_referralCodeForSharing", "RF_registeredUsingDevice", "RF_userIdReference", "RF_userLevelCode", "SP_JoiningDate", "SP_campaignId", "SP_companyName", "SP_dateAtWhichAppUninstalledFromAllDevices", "SP_designation", "SP_displayName", "SP_email", "SP_fb_ad_group_name", "SP_fb_ad_set_name", "SP_firstName", "SP_gender", "SP_hasUserRegisteredImmediatelyOnInstall", "SP_hasUserUninstalledAppFromAllDevices", "SP_isLeadSellerAnSP", "SP_lastName", "SP_mediaSource", "SP_numOfYearsOfExperience", "SP_personalEmail", "SP_phoneNumber", "SP_primaryCategory", "SP_primaryRegion", "SP_promoCodeUsedForRegisteringWithApp", "SP_promoCodeUsedForRegisteringWithAppDate", "SP_promoCodeUsedForRegistration", "SP_referralCodeForSharing", "SP_registeredUsingDevice", "SP_userCategoryType", "SP_userIdReference", "SP_userLevelCode", "action", "actionCode", "bothEndedTrx", "budgetCurrency", "buyByDate", "canAttendThefirstMeeting", "canDoPersonalIntro", "canProvideAdditionalHelpInfo", "canProvideEmail", "canProvidePhoneNumber", "canProvideTipsForConvertingTheSale", "canRevealSellersIdentity", "currLifecycleStateDescForBuyer", "currLifecycleStateDescForSeller", "currencyCode", "dateCreated", "deviceTypeUsed", "environment", "hasOfferBeenAccepted", "isActive", "isFirstLeadPost", "isPlaceHolderOfferSummary", "isReplaceOffer", "latestMessageDate", "leadCategory", "leadDescription", "leadExpiryDate", "leadIdReference", "leadNumber", "leadOwnersUserId", "leadPostingDate", "leadReferralType", "leadRegion", "leadSellersAvatarName", "leadSummary", "offerAmountForInfo", "offerMadeByUserDisplayText", "offerPaymentTypeCode", "offerPaymentTypeDesc", "productWanted", "promoCodeUsedForLeadPosting", "prospectiveBuyerType", "providedDesignationOfProspectiveBuyer", "providedEmailOfProspectiveBuyer", "providedPhoneNumberOfProspectiveBuyer", "referrerCanCode", "relationshipWithBuyer", "spinFees", "totalNumberOfActions", "userIdOfCM", "userIdReference", "viewTime", "whoWantsToBuy" ],
            "dimensionExclusions" : [ "numberOfOfferExpires", "numberOfOffersReceived", "timeStamp", "numberOfDaysSinceUserRegistered", "numberOfDaysFromPublishToExpiry", "totalNumberOfOutstandingTasksForBuyer", "offerAmount", "leadsMinimumOfferPrice", "lifeCycleStateCodeImmediatelyBeforeDispute", "maximumtotalNumberOfPymts", "minimumOfferPrice", "SP_age", "SP_ratingAsABuyer", "RF_numberOfDaysSinceUserRegistered", "SP_ratingAsASeller", "CM_age", "amountToCommit", "totalNumberOfAssists", "maxNumberOfBuyers", "numberOfOffersAccepted", "numberOfOffersWithdrawn", "SP_numberOfDaysSinceUserRegistered", "spiniServiceFeesPercentage", "totalNumberOfOutstandingTasksForSeller", "RF_ratingAsABuyer", "serviceTaxPercentageForServiceFees", "cmServiceFeesPercentage", "numberOfCommmitExpires", "RF_age", "leadPostCounts", "minimumtotalNumberOfPymts", "offerAmountForConversion", "budgeAmount", "numberOfActiveDisputes", "rating", "numberOfCommittedOffers", "RF_ratingAsASeller", "totalNumberOfPymts" ],
            "spatialDimensions" : [ ]
          }
        }
      },



Above are from logs.

In Spec document, I have only given all values in dimensions not in dimensonExclusions.
Also I can on success in ingestion, I am not getting timeBoundary query results for the given data Source.

Please let me know if I am missing anything.

Thanks in advance.

Thanks,
Aman

Hi ,

Also find the spec document attached.

It shows data ingested.

Task completed with status: {
  "id" : "index_spini_staging_g3_2015-11-29T15:56:50.984Z",
  "status" : "SUCCESS",
  "duration" : 27094
}

spini_master_spec.json (20.6 KB)

Hey Aman,

Your metrics and the timestamp were automatically added to dimensionExclusions. None of the dimensions you entered should be in there unless possibly if you have a dimension and a metric with the same name in which case you should use different names.

As for not the timeBoundary query not working, can you see your datasource in the coordinator console? (http://{COORDINATOR_IP}:{COORDINATOR_PORT})