I am looking for suggestions to solve this problem.
There are 1.313 distinct lanes (based on the below criteria):
Origin.Country&'|'& LT_set&'|'& Type&'|'& Origin.subregion&'|'& Origin.region&'|'& Destination.Country]&'|'& Destination.subregion&'|'& Destination.region&'|'& service&'|'& color
Objective is R2= correlation of determination > 0.6 for each lane (this means also the above aggregation can be changed and outliners data points can be excluded.
But the hard rules are:
Minimum of 20 shipments per lane
Minimum 6 Months data or 9 months data or 12 months data or 18 months data;
The Key lane needs to have at least: “Origin”, “ Destination” and LT_set, Type, service, color;
The R2 is calculated for each of the 1.313 lanes as following:
LinEst_R2(Cost,KG)
LinEst_B(Cost,KG)
LinEst_M (Cost,KG)
First Problem to be solved:
I need to Identify and Exclude the outliers shipments data points (with weight or costs) to make sure the R2 is < 0.6
Not sure if this can work or how to do this but I could calculate the distance between the data point and the R2 (in a chart) and Identify and Exclude the outliers shipments data points.
2. Second Problem to be solved: To increase the chance of R2>0.6 I need to:
reduce the 1.313 distinct lanes If the R2 does not reach >0.6
This means I could eliminate some fields to aggregate the Key Lanes having a higher number of cost and weight increases the chance for an R2> 0.6
Increase or decrease the date range (months) to reach an R2>0.6 with a preferred small number of months between 6 and 18 months.
Appreciate it if you have any suggestions to solve the above points 1 or 2 or both.
