Problem:

A supervisor at a Call Center with several telemarketers, gets lists of sales leads that need to follow up from a specific channel every day. These sales leads lists are arrived in several batches a day, contain different amount of sales leads. This supervisor will dispatch these leads to telemarketers to follow up. Some telemarketers have rich experiences so the supervisor will dispatch more leads to them, and some others like new comers will get less leads, but most telemarketers will get the same amount of the leads. So the supervisor gives different telemarketers different weights. For example, most telemarketers have weight 1; these rich experiences have weight 1.2; the new comers have weight 0.8. And these weights are changing from time to time as the new comers become more skillful, and the telemarketers just come and go as time goes by (Some quit their job, and some new hires).

This supervisor wants (a forced distribution dispatching mechanism such that):

- Every telemarketer gets a certain amount of sales leads after dispatching which is as close to his/her weight ratio as possible

- Moreover, from a long term’s view, the accumulated amount of the sales leads that had dispatched to a telemarketer, is as close to his/her weight ratio – varies over time – as possible

An example, this supervisor manages 3 telemarketers, and gets 100 sales leads at the 1st batch, 50 the 2nd. And the 2nd day, the supervisor gets 300 sales leads at the 3rd batch, and the went away a telemarketer, as well as a new comer joined in. Now s/he wants to dispatch these leads like this:

Analysis:

Take the above scenario for example, why is the dispatching scheme it shows really the best? Because from the supervisor’s 2 requirements, we can see the dispatching scheme needs to meet 2 “as close as possible”:

- The 1st is the distribution of the dispatching amount in every single batch is as close to the weight ratio distribution in that batch as possible;

- The 2nd is from the long term’s point of view, the distribution of the cummulative dispatching amount is as close to the varying weight ratio distribution as possible.

So what is the “as close as possible”? Suppose a telemarketer has a weight ratio 0.22, and the number of the sales leads in the pool is 10. If you assign 5 leads to that telemarketer, then the dispatching scheme does NOT meet the “as close as possible” requirements because 5/10 = 0.5, which is twice more than 0.22; if you assign 1 lead to the telemarketer, 1/10 = 0.1,  then this scheme is a bit close to 0.22; and if you assign 2 leads to the telemarketer, 2/10 = 0.2, then it is more close to the ideal ratio – 0.22!

Now that we can see the “as close as possible” means that the proportion of the amount of assigned leads in the leads pool is close to the weight ratio as possible. The best is the proportion of the assigned amount equals to the weight ratio. In the above example, if we assign 2.2 leads to that telemarketer, then it is the best. But this is an ideal scheme which is not permitted in the real world. So our goal is to make the error between the real scheme and the ideal scheme minimum. The error of the final scheme in the above example is 2/10 – 0.22 = 0.2 – 0.22 = –0.02.

Let’s change our point of view to another, to measure the error between the real dispatching scheme and the ideal scheme minimum, besides the ratio difference, we can also calculate the amount difference directly. For example, 2 – 10 * 0.22 = 2 – 2.2 = –0.2. In other words, the assigned amount in the real scheme, is 0.2 less than the ideal scheme (We should assign 2.2 leads to the telemarketer, but finally we assigned 2 instead due to the reality constraint). Since we can NOT avoid this error, so it is the minimum.

How to find the best assigned number 2? When we use the latter view of calculating the amount error, the method seems to reveal by itself: Firstly figure out the ideal assigned amount, then round the ideal value to the nearest integer.

Apply this method to every telemarketer, then the scheme will meet the 1st requirement: the actual assigned amount distribution must be the closest one to the ideal assigned amount distribution, so it also is the closest one to the weight ratio distribution in every single dispatching batch.

In some scenarios, the rounding process would bring some errors between the actual total assigned amount of leads and the total amount of total leads in the pool. For example, you should assign 100 leads, but because of the rounding, the scheme only assigned 99 leads or require 101 leads to assign. When this happened, we need to do some adjustment to the scheme in hand, and this adjustment still needs to make sure the final actual dispatching scheme is closest to the ideal dispatching scheme. The steps would be discussed in detail later (refer to the “Fine Tuning” step and its mathematical proof).

In summary, rounding the ideal dispatching amount to the nearest integer method can meet the 1st requirement. And for the 2nd requirement, if we want to make sure that the dispatching scheme is closest to the ideal scheme in every single batch, then we CAN NOT make sure that the accumulated actual scheme is still closest to the ideal accumulated scheme. For example:

The above schemes are the best in every single batch, but obviously it has problems from the long term’s point of view. Because all the sales leads go to A’s hand, and B would not get any leads in every dispatching batch. If we dispatch like this, then we can make sure the accumulated scheme is the best (closest to the ideal accumulated scheme), and what’s more, it is still very close to the ideal scheme in every single batch.

The batch #1 and #3 are still the best choice in the single batch’s point of view, only the #2 is not.

In fact we can introduce a concept of Cumulative Error, and try to make it minimum during the dispatching process, then this way we can make sure the accumulated dispatching scheme is the best, and at the same time, the scheme is the best or the 2nd best in every single batch (Detailed mathematical proof is in the last section of this article).

Solution: (4 steps in total)

1. Normalization.

Normalize the weight settings. Example:

2. Calculate the ideal dispatching scheme according to the Cumulative Error.

As what mentioned previously, we introduce the Cumulative Error to meet the supervisor’s 2nd requirement: from the long term’s perspective, the dispatching scheme should be the best (closest to the ideal dispatching scheme).

The ideal dispatching scheme to the telemarketer A is

The current total amount of sales leads in the pool times the current (normalized) weight ratio,

and then subtract the Cumulative Error before current dispatching (obviously it would be 0 if it is the 1st dispatching batch).

For example:

3. Rounding to the nearest integer.

Rounding the ideal dispatching amount to the nearest integer (If the rounding result is a negative integer, then we should use the 0 instead, because dispatching a negative number of sales leads makes no sense!), and record the current single batch error as well as the Cumulative Error after this dispatching (In the 1st time of dispatching, the current single batch error = the Cumulative Error after this dispatching; the subsequent dispatching, as discussed above, would need the Cumulative Error after the former dispatching that had recorded; And the subsequent Cumulative Error after dispatching = the Cumulative Error before dispatching (or the Cumulative Error after dispatching in the last batch)  + the Current Single batch error). For example:

We can see that the cumulative sum of squared errors decreased from 1.82 before dispatching to 0.22 after dispatching. So this way makes in the long term, the cumulative sum of squared errors tries to decrease, which meets the 2nd requirement. And at the same time the single batch error is also very close to the minimum (equal to the minimum or closest to it, proof later), so meets the 1st requirement too.

Please notice, the error = actual value – ideal value. The error can be positive or negative, but we treat the +0.3 as the same as –0.3, whose absolute value is the same. In other words, the error +0.3 differs from the ideal value in the same effect as the –0.3 does. I call the absolute value deviation. Using the plus-minus sign here is to let us know the difference is larger than or less than the ideal value. But we use the sum of squared errors in the subtotal column. The sum of squared errors is very convenient to measure the additive effect of all the errors attached to each telemarketer. The additive effect of errors can be also measured by the sum of the absolute value of errors, but it is more convenient to calculate if we use squared errors. For example, the error attached to A is +0.3, and attached to B is –0.3, then the additive effect of errors is 0.6, or the sum of squared errors 0.18.

4. Fine tuning if needed.

After the rounding step in the above example, the sum of dispatched amount of leads to all telemarketers = the total leads in the pool of the batch, so this step will be skipped.

But in some cases, due to the errors brought by the rounding can not be cancelled each other out, the sum of dispatched amount can be greater than or less than the total leads in the pool of the batch. For example:

In these cases we need to fine tune the rounded result, such that the sum of the dispatched amount = the total amount of leads in the pool.

Denote Δ = the sum of the dispatched amount – the total amount in the pool.

If Δ > 0 (Δ is a small integer), then we need to take Δ leads from the dispatched amount, and we need to do this one by one.

Among those telemarketers whose assigned amount is greater than 0 and whose weight setting is greater than 0, find the one whose Cumulative Error after dispatching is the maximum (the Cumulative Error can be positive or negative, we should only find the maximum from the positive ones – there must exist the positive errors, or else the Δ can’t be greater than 0 – to make sure the fine tuning doesn’t have a big impact to the whole errors). Now we subtract 1 from his/her assigned amount, and at the same time we update the single batch dispatching errors and the Cumulative Errors after dispatching.

Now recalculate the Δ (which is just Δ = Δ – 1), and then check if Δ is still greater than 0, if so then repeat the above step until the Δ = 0. Fine tuning over.

If Δ < 0 (Δ is a negative integer), then we need to add Δ leads to the dispatched amount, and we need to do this one by one.

Among those telemarketers whose weight setting is greater than 0, find the one whose Cumulative Error after dispatching is the minimum (the Cumulative Error can be positive or negative, we should only find the minimum from the negative ones – there must exist the negative errors, or else the Δ can’t be less than 0 – to make sure the fine tuning doesn’t have a big impact to the whole errors). Now we add 1 more leads to his/her assigned amount, and at the same time we update the single batch dispatching errors and the Cumulative Errors after dispatching.

Recalculate the Δ (which is just Δ = Δ + 1), and then check if Δ is still less than 0, if so then repeat the above step until Δ = 0. Find tuning over.

5. The dispatching scheme is made and this batch of dispatching complete successfully.

Relevant Question and Answers, and some mathematical proofs:

1. What is a ideal dispatching amount, and what is an actual/real dispatching amount?

Answer:

The ideal dispatching amount is a value that is calculated directly through simple mathematics, which doesn’t consider the reality constraints. This value can be a decimal that doesn’t make any sense.

The real/actual dispatching amount is rounding value of the ideal amount, this would cause an error between them. If lucky in some cases, the ideal dispatching amount is also an integer, then the ideal value = actual value, error is 0.

If the value is negative after rounding to the nearest integer, then we take 0 as the actual/real dispatching amount. Because the assigning a negative number of sales leads is as assigning a decimal number of sales leads, makes no sense.

2. The first part of the article mentioned that the weight settings would vary across the batches, but the steps in the solution didn’t discuss the weight settings vary scenarios. Why?

Answer:

As mentioned in the analysis part, we converted the goal of minimizing the difference between the ratio of the assigned amount and the total amount in the pool and the weight ratio to the goal of minimizing the difference between actual assigned amount and the ideal dispatching amount. The two goals are identical, and in the 2nd goal we had eliminated the impact of the weight factor, so we don’t need to consider its varying in calculating the best dispatching scheme. For the changed weight setting, we will still use the same method to calculate the ideal dispatching amounts (that is the amount in the pool times weight ratio and then subtract the Cumulative Error, no matter the weight ratio is changed or not, we re-calculate the ideal amounts in every dispatching batch. Because even if the weight settings stay the same, the amount of sales leads in the pool varies across batches, so we have to re-calculate the ideal dispatching amount every time before doing the dispatching).

3. The goal has been converted, are their effects identical?

Answer:

Yes.

[Proposition] For the following assigning scheme, when the error between the actual dispatching amount and the ideal dispatching amount achieved the minimum, then the error between the actual ratio of dispatching amount to the amount in the pool and the weight ratio also achieved the minimum. And vice versa.

Suppose the amount of leads in the pool is S，and the Cumulative Error is 0.

[Proof]: Here we only do the analysis to the TM1.

Denote the error between the actual dispatching amount and the ideal dispatching amount as ε:

ε1 = L1 - S*W1

Denote the error between the ratio of the actual dispatching amount to the amount in the pool as r：

r1 = L1 / S - W1

So we have r1 = ε1 / S. And obviously it is identical to discuss the minimum of r1 and of ε1.

[End of Proof]

3. The supervisor’s goal is to make sure the overall distribution of the dispatched amounts is as close to the overall distribution of the weight settings as possible. But the above solution and the proof just now is only against a single telemarketer, can it achieve the overall goal too?

Answer:

Yes, it can. First of all, we should find a way to measure the “as close as possible”. Against a single telemarketer, we used the error to measure it; to measure how close the overall distributions are, the Sum of Squared errors is a very good measurement. Of course, the more intuitive measurement might be the Sum of the Absolute value of errors, but it is harder to calculate since you need to consider 2 scenarios that the errors are positive or negative.

If for all telemarketers, their errors achieved the minimum, then the overall Sum of Squared errors must achieved the minimum too. Because the Sum of Squared errors are the sum of all non-negative numbers, so if the Sum didn’t achieve the minimum when all factors had achieve, then it means the sum can be even more smaller. But it is impossible because no factor can be smaller because all factors had achieve the minimum.

[Proposition] Denote the error as E. If E1, E2, …, En had achieved the minimum, then the sum E1² + E2² + ... + En² achieved the minimum too.

[The proof is skipped as it is so obvious]

[corollary] The fact that the error between the actual dispatching amount and the ideal dispatching amount against every telemarketer achieves the minimum is identical to the fact that the sum of squared errors of the overall dispatching scheme achieves the minimum.

3. Why is the result of the amount of leads in the pool times the telemarketer’s weight ratio, and then subtract the Cumulative Error against him/her considered the ideal dispatching amount?

Answer:

The ideal dispatching result is every telemarketer doesn’t get more or less sales leads than he should. Intuitively, the result of the amount in the pool times the weight ratio will make the single dispatching be the best. The effect of subtracting the Cumulative Error is dispatching a little less/more sales leads to those who had been assigned a little more/less in the previous dispatching batch.

[Proposition] A supervisor had dispatched n batches of sales leads, and in the i^th batch, the amount of sales leads in the pool is Si. The detailed dispatching scheme is as following (I = 1, 2, …, n).

So, if and only if the error between L1i (Actual) and S*W1i – D1i (Ideal) achieves the minimum, the absolute value of the Cumulative Error after dispatching achieves the minimum.

[Proof]

The Cumulative error after the i^th dispatching against TM1 = D1i + E1i = D1i + L1i - S*W1i = L1i - (S*W1i - D1i). Obviously, the more close that L1i to S*W1i – D1i, the less the Cumulative Error after dispatching is. And vice versa.

[End of Proof]

4. Do we have another way to do the fine tuning other than the way mentioned in the solution?

Answer:

Yes. Currently I can’t come up with another one to achieve the goal. If you can please share to me. But I am assure that this way can achieve the goal, in other words, this fine tuning method might not be the simplest, but it is correct. Why? Please think of it yourself, and pointing out the bug if any is welcome.