Mixed-integer programming methods for supply chain optimization

Transcription

Mixedinteger programming methods for supply chain optimization
C h r i s t o s T. M a r a v e l i a s
Chemical and Biological Engineering
University of Wisconsin, Madison, WI 53706, USA
July 19-29, 2011, Angra dos Reis, RJ, Brazil
Christos Maravelias, University of Wisconsin - Madison
Outline
1. Supply Chain Management
2. Linear Programming
3. Basic Mixedinteger Linear Programming
4. Advanced Mixedinteger Linear Programming
5. Software Overview
6. Combinatorial Optimization
7. Chemical Production Scheduling
8. Integration of Production Planning and Scheduling
The Chemical Supply Chain
Supply chain (SC):
network of facilities and distribution options for the procurement of materials; transformation of materials into intermediate and finished products; and distribution of these finished products to customers. Suppliers
Manufacturing
Facilities
Warehouses/
dist. centers
Retailers
Customers
Traditional PSE is concerned with the development of methods for the design, operation, and control of chemical systems. Chemical industry changes:
New products ⇒ multiproduct (and batch?) processes
Holistic view of operations
Tayur ‐ FOCAPO 2003:
Inventories in SCs: ~ $10 trillion (10% of US GDP)
Same customer satisfaction levels with 50% inventory
The goal in supply chain management is to coordinate materials, information and financial flows to fulfill customer demands while improving competitiveness of the supply chain as a whole; in coordinating these flows various decisions have to be made.
Supply Chain Planning Matrix (Meyr et al., 2002)
procurement
Short‐term
distribution
sales
Strategic Planning
Long‐term
Medium‐term
production
Master Planning
Purchasing
& Material
Production
Planning
Distribution
Planning
Requirements
Planning
Scheduling
Transport
Planning
Demand
Planning
Demand Fulfillment
Power Utilities
Integration across time scales and functions
Many new interesting problems
Power – industrial gases – steel SC Oil – industrial gases – chemicals SC
under power constraints
Refinery
Industrial
Gases
Chemicals
Customers
procurement
Short‐term
distribution
sales
Strategic Planning
Long‐term
Medium‐term
production
Master Planning
Purchasing
& Material
Production
Planning
Distribution
Planning
Requirements
Planning
Scheduling
Transport
Planning
Corn
Corn Grain
Demand
Planning
Demand Fulfillment
C4
Hydrolysis
Fermentation
Biomass to fuels and chemicals
Succinic acid
Fumaric acid
Sugarcane
Malic acid
Sugars
C5
Bagasse
C6
Corn Stover
Switch grass
Wood
Wood waste
Dehydration
Furfural
Dehydration
Levulinic
acid
FDCA
Fractionation
‐ Catalyst
‐ Steam
‐ Acid
‐ Enzymes Fast pyrolysis
Liquefaction
Bio‐oils
(Sugars, Acids)
& Lignin
Hydrolysis
Lignin
Demethoxylation
Hydrodeoxygenation
Zeolite upgrading
BTX
Phenols &BTX
Aromatics
procurement
Short‐term
distribution
sales
Strategic Planning
Long‐term
Medium‐term
production
Master Planning
Purchasing
& Material
Production
Planning
Distribution
Planning
Requirements
Planning
Scheduling
Transport
Planning
Demand
Planning
Demand Fulfillment
Integration of production planning and scheduling Exxon Mobil:
2% reduction in operating costs, 20% inventory reduction (Shobrys & White, 2002)
DuPont: Capital tied up in inventory reduced from $165 to $90 million (Shobrys & White, 2002)
Outline
Linear Programming
Standard optimization problem:
max f(x)
s.t. g(x) = 0
h(x) ≤ 0
x∈X
n - number of variables
m - number of equality constraints
l – number of inequalities
m<n
If functions f, g, h are linear and variables x are continuous, we have a Linear Programming (LP) Model.
Notes:
1. Free z∈ℜ variables can be replaced by two nonnegative variables x, y ≥ 0:
z=x–y
2. Inequalities can be transformed to equalities with slack variables
x1 + 2x2 ≤ 10 ⇒ x1 + 2x2 + s1 = 10, s1 ≥ 0
3. Minimization problems can be expressed as maximization problems:
min f(x) ⇔ max –f(x)
Thus, we can use the standard LP form:
(LP) max z = cTx
s.t. Ax = b
x≥0
where x: (n+l)-vector; A:(m+l)×(n+l) matrix; c: (n+l)-vector; b: (m+l)-vector
LP Geometry
A process is used for the production of products P1 and P2 from raw materials A and B according to:
2A + B → 3P1
(1)
A + 2B → 3P2
(2)
If the maximum availability of raw materials A and B is 10 kg/hr, the price of both chemicals is $100/kg,
and the demand for products P1 and P2 is 12 and 13 kg/hr, respectively, what is the optimal production mix?
The problem can be represented as the following network (the labels refer to the streams, not species):
A
B
SA
SB
SA1
SA2
SB1
SB2
Reaction 1
Reaction 2
S1
P1
S2
P2
FP2
If FP1 and FP2 are the amounts of products P1 and P2,
respectively, the optimization problem (P1) can be
stated as follows:
max Z = 100 FP1 + 100 FP 2
0 ≤ FP1 ≤ 12, 0 ≤ FP 2 ≤ 13
2FP1 + FP2 ≤ 30
20
st. 2 FP1 + FP 2 ≤ 30
FP1 + 2 FP 2 ≤ 30
30
(P1)
FP2 ≤ 13
FP1 ≤ 12
10
FP1 + 2FP2 ≤ 30
10
20
30
FP1
LP Geometry
max 10x1+10x2
s.t 2x1 + x2 ≤ 30
x1+2x2 ≤ 30
x1
≤ 12
x2 ≤ 13
x1 , x2 ≥ 0
Definition:
Definition:
Definition:
max 10x1+10x2
s.t 2x1 + x2 + s1
= 30
x1+2x2
+ s2
= 30
x1
+ s3
= 12
x2
+ s4 = 13
x1, x2, s1, s2, s3, s4 ≥ 0
30
x2
20
A basic solution to an (LP) is determined by fixing (n-m)
variables to 0, and solving the (m×m) system to obtain the remaining.
• Variables fixed at zero are non-basic variables.
• Variables obtained by solving the equalities are basic.
A basic feasible solution to an (LP) is a basic solution that satisfies
nonnegativity constraints.
x2 + s4 =13
10
x∈F is an extreme point iff x = α x1 + (1-α) x2 and α∈(0,1), x1, x2∈F
implies x1 = x2.
i.e. x can not be obtained by linear combination of x , x ∈ F: x ≠ x .
⇒ The basic feasible points of an LP are the extreme point of its feasible region.
1
2
2x1 + x2 + s1 = 30
1
2
x1 + s3 = 12
x1 + 2x2 + s2 = 30
x1
10
20
30
Basic solution
Basic‐feasible solution (extreme point)
⎡x ⎤
Let A = [B | N], x = ⎢ B ⎥ where B: m×m full-rank, N: m×(n-m), xB: m-vector and xN: (n-m)-vector
⎣ xN ⎦
Ax = BxB + NxN = b
If xN = 0 ⇒ xB = B-1b - B-1NxN
z = cBxB +
(-cBT B-1N
T
+ cN )xN
Basic point
Objective function
Theorem 1: A point x in F is an extreme point iff xB = B1b ≥ 0, xN = 0
Theorem 2: The optimal solution x* of (LP) lies at an extreme point of F.
Theorem 3: Point x* is optimum if Reduced Cost RC = ‐cBT B1N + cNT ≤ 0
Simplex Method
Main idea of Simplex: If current basis non‐optimal, swap a non‐basic variable for a basic one until (‐cBTB‐1N + cNT) ≤ 0T
we start at x1=x2=0
⎡2
⎢1
⎢
⎢1
⎢
⎣0
N
1M
2M
0M
1M
1
0
0
0
0
1
0
0
0
0
1
0
0⎤
0⎥⎥
0⎥
⎥
1⎦
xN = [s2, s1] = [0, 0]
xB = [x2, s3, x1, s4] = [10, 2, 10, 3]
z = 200 – (10/3)s2 – (10/3)s1
x2
B
‐1
xB = B b
z = cBxB + (‐cBTB‐1N + cNT)xN
(0,13)
(4,13)
(10,10)
xN = [s3, s1] = [0, 0]
xB = [x2, s2, x1, s4] = [6, 6, 12, 7]
z = 180 + 10s3 – 10s1
(6,12)
xN = [x1, x2] = [0, 0]
xB = [s1, s2, s3, s4] = [30, 30, 12, 13]
z = 0 + 10x1 + 10x2
max 10x1+10x2
s.t 2x1 + x2 + s1
= 30
+ s2
= 30
x1+2x2
+ s3
= 12
x1
+ s4 = 13
x2
x1, x2, s1, s2, s3, s4 ≥ 0
(0,12)
x1
xN = [s3, x2] = [0, 0]
xB = [s1, s2, x1, s4] = [6, 18, 12, 13]
z = 120 + 10x2 – 10s3

Simplex algorithm developed by G. Dantzig (1947)
Very popular due to wide range of applications formulated using LP
Considered as one of the most important algorithms in the 20th century
Not polynomial but outperforms polynomial algorithms (Ellipsoid, Khachiyan Khachiyan, 1972)
Simplex Method Remarks
Degeneracy
A basic solution is degenerate if a basic variable is equal to zero.
In this case, the Simplex method may go through many iterations without increasing the value of the objective
function, and the same basis is encountered again (Cycling).
To avoid cycling, several rules have been developed; e.g. Bland’s rule.
Initialization
It is not always easy to find an initial basis.
For instance, if after adding the slack variable we have some components of b that are negative.
Infeasible Æ
In this case we apply Simplex in two phases.
Phase I:
To each equality
∑a
j
ij x j
= bi with bi<0, subtract an artificial variable ui, add constraint ui ≥ 0 and solve:
min Σui
s.t. Ax = b -u
u≥0
The original LP has a feasible solution iff the auxiliary LP has an optimal solution with objective value equal to zero.
Phase II: Use the last basis of phase I as a starting basis for Phase II, after you delete artificial variables ui and change
objective function.
Termination
It is not guaranteed that a feasible LP has an optimal solution.
The problem can be unbounded. The simplex method detects such programs and returns a “proof” of unboundedness.
⇒ When checking to determine the leaving variable, no constraint becomes binding.
Duality I
Given a primal LP problem:
min cT x
s.t. Ax = b
x≥0
(LP-P)
the corresponding dual LP problem is formulated as:
(LP-D)
min bT y
T
s.t. A y ≤ c
How do we construct (LP-D) from (LP-P)?
For each constraint of (LP-P), there is a variable in (LP-D)
For each variable of (LP-P), there is a constraint in (LP-D)
If (LP-P) is min problem, the dual is max.
If the constraints of (LP-P) are equalities (inequalities) the variables of (LP-D) are unconstrained (constrained)
If the variables of (LP-P) are free (non-negative), the constraints of (LP-D) are equalities (inequalities)
Non-negative variables result in ≤ inequalities in max problems
Non-negative variables result in ≥ inequalities in min problems
The dual of an LP can be used to better study and even solve faster an LP
Duality II
Let FP and FD be the feasible region of (LP-P) and (LP-D), respectively, we introduce the following convention:

T
If FP = ∅ infP c x = +∞

T
If FD = ∅ supD b y = −∞
x∈F
y∈F
c T x ≥ sup b T y
Theorem 1: For any (LP): xinf
D
∈F P
[Proof]
y∈F
c T x = sup b T y
Theorem 2: If either FP≠ 0 or FD≠ 0, then xinf
P
D
∈F
y∈F
[Proof]
c T x = maxD b T y
⇒ if one of these problems is solvable, then so is the other and xmin
P
∈F
y∈F
Primal
Optimal
Infeasible
Unbounded
Optimal
√
-
Dual
Infeasible
√
√
Unbounded
√
-
Duality III
Thus, we can solve (LP-D) instead of (LP-P) if the first is easier.
Computationally, the number of constraints is more important than the no of variables
If the (m×n) primal (LP-P) has m >> n, then it may be easier to solve the dual (LP-D)
Furthermore, the dual solution provides us with sensitivity information
The dual variable corresponding to a primal constraint, indicates whether this constraint is active;
and by how much
Dual values = shadow prices = marginal values
Sign of dual variables
Why equality constraints, have unrestricted dual variables?
What does the sign say about the RHS?
Nonnegative variables in (LP-P) result in ≤ inequalities in max problems
≥ inequalities in min problems
Duality IV
Re-optimization
Let assume that we have solved to optimality (LP-P) to obtain z* = cT x* = bT y*
What happens if we add a new constraint am+1 x = bm+1 to (LP-P)?
⇒ (LP-P) may become infeasible.
What happens to (LP-D)?
We add a variable; if we set ym+1= 0, we still have a feasible solution
⇒ We can use the existing basis to continue.
Dual Simplex Method
In general, a feasible non-optimal point of (LP-P) corresponds to an infeasible point of (LP-D) [Why???].
In the simplex algorithm, we try to reduce infeasibility of the dual problem.
Dual simplex method
Commercial solvers use there three powerful LP algorithms:
Primal Simplex
Dual Simplex
Barrier method
Different methods perform better in different problems
Commercial solvers detect certain problem structures & automatically choose appropriate method
If we know the structure we can also choose the method
Outline
MIP in Chemical Engineering
Many decisions are discrete in nature:
Process Design:
9 Selection of equipment units (e.g. distillation vs. extraction) ‐ Superstructure Optimization
9 Selection of equipment sizes
9 Matching in heat exchanger networks (hot ↔ cold pairs)
Process operations:
9 Decision to expand/retrofit a process
9 Split of customer orders to batches
9 Assignment of batches to equipment units
9 Sequencing of batches in the same unit
Process Control:
9 Selection of linearized model for MPC
9 On/off control action
Applications: Systems Biology
9 Gene addition and/or knock‐out Mixed-integer programming methods for supply chain optimization
Discrete Decisions in Process Design
Example: What is the best way to produce Vinyl Chloride Monomer?
Turkay & Grossmann, 1998
Chlorine
Direct
Chlorination
Low P
Purge
Ethylene
Oxychlorination
Flash
High P
Air
Water
Hydrogen
Chloride
Oxygen
Hydrogen
Chloride
Ethylene
Dichloride
Reactions:
Direct chlorination
C2H4 + Cl2→ C2H4Cl2
Oxychlorination
C2H4 + 2HCl + 1/2O2 → C2H4Cl2 + H2O
Pyrolisis
C2H4Cl2 → C2H3Cl + HCl
Separation: HCl/C2H3Cl/C2H4Cl2 mixture
1. Separate HCl/C2H3Cl – C2H4Cl2
2. Separate HCl – C2H3Cl
Vinyl Chloride
1. Separate HCl – C2H3Cl/C2H4Cl2
2. Separate C2H3Cl – C2H4Cl2
Ethylene Dichloride
Optimization Decisions:
Distillation sequence (discrete), sizing (discrete or continuous), operating conditions (continuous)
Discrete variables are also used to:
Represent logic conditions
Enforce set of constraints (often including only continuous variables)
Discrete Decisions in Process Operations
Example: Scheduling of a multi-product, multi-stage plant
Products
(Batches)
1st stage
Reaction
2nd stage
Separation1
3rd stage
Separation2
Optimization Decisions:
Number of batches
Assignment of batches to units
Sequencing of batches in a unit
Starting time (continuous)
Basic Integer Programming Problems
Knapsack Problem

We are given n items to be packed in a knapsack
Each item i∈{1, 2, …n} has a value pi and a weight wi
The capacity of the knapsack is W
The goal is to choose the items that fit in the knapsack with the max value
We define:
Xi = 1 if item i is chosen
Objective:
Items
1
Knapsack
2
.
.
.
n
max z = ∑ pi X i
(K.1)
i, j
∑w X
i
i
(K.2)
≤W
Capacity constraint
i
X i ∈ {0,1} ∀i
If we had multiple resources j∈{1, 2, ..m} (e.g. weight, volume, etc.) with capacities Wj
we would replace (K.2) with (K.3):
∑w X
ij
i
≤ Wj
∀j
(K.3)
i
Assignment Problem

We are given n jobs and n machines (classes/instructors, routes/buses)
Each job i∈{1, 2, …n} has to be assigned to a machine j∈{1, 2, … n}
The cost of assignment is cij
The objective is to find the assignment with the minimum total cost
We define:
Xij = 1 if job i is assigned to machine j
Objective:
min z = ∑ cij X ij
i, j
Jobs
Machines
1
1
2
2
.
.
.
n
.
.
.
n
(A.1)
∑X
ij
= 1 ∀i
(A.2)
Every job is assigned
∑X
ij
= 1 ∀j
(A.3)
Exactly one job per machine
j
i
X ij ∈ {0,1} ∀i, j

If solved as LP, there is always an optimal solution with integral values
Studied by Carl Gustav Jacobi (1890)
Solved effectively by the Hungarian method (Kuhn, 1955)
Improved by Munkres (1957)
Traveling Salesman Problem
There are n cities to be visited exactly once by a traveling salesman
The distance/cost from city i to city j is cij
The objective is to find the sequence of cities that yield the minimum
total distance/cost
Cities
We define:
Xij = 1 if we visit city j after city i
Objective:
min z = ∑ cij X ij
i, j
(TSP.1)
∑X
ij
= 1 ∀j
(TSP.2) There is an incoming arc for every city j
∑X
ij
= 1 ∀i
(TSP.3) There is an outgoing arc for every city i
i
j
X ij ∈ {0,1} ∀i, j
Exercise
Are constraints (TSP.2) & (TSP.3) enough?
What type of solution can we get?
How can we improve our model?
Set Covering Problem

We are given n tasks to be covered by m<n machines (flights/crew)
Each machine j∈{1, …m} can carry out a subset of tasks Ij⊂I = {1, 2, …n}
The compatibility is represented via parameter: αij = 1 if i∈Ij
The cost of a machine is cj
The objective is to find the set of machines that covers all tasks and has
the minimum cost
Jobs
Machines
1
1
2
2
.
.
.
n
.
.
.
m
We define:
Xj = 1 if machine j is chosen
Objective:
min z = ∑ c j X j
(SC.1)
j
∑a
ij
X j ≥ 1 ∀i
(SC.1)
Every task is covered
j
X j ∈ {0,1} ∀j
Basic Mixedinteger Programming Problems
Facility Location Problem – I
Facilities
There are n (existing + new) facilities (plants) and m customers
We want to decide how to satisfy customer demand
What new facilities to build, how much to produce and ship
The capacity (cost of setting up) of facility i∈{1, 2, …n} is αi (fi)
The cost of shipping material from i to j is cij
The demand of customer j∈{1, 2, …m} is dj
cij
dj
1
1
2
2
.
.
.
.
.
.
n
We define:
Xi = 1 if we set up (install) facility i
Yij = amount shipped from i to j
Objective:
ai, fi
Customers
min z = ∑ f i X i + ∑ cijYij
m
(FLI.1)
Minimize set-up and shipping costs
(FLI.2)
Customer demand satisfaction
(FLI.3)
Facility capacity constraint
i, j
i
∑Y
≥ dj
∑Y
≤ ai X i
ij
∀j
i
ij
∀i
j
X i ∈ {0,1} ∀i,
Yij ≥ 0 ∀i, j
More realistic problem:
9 There is also production cost pi, transportation set-up cost tij, and transportation capacity sij
9 The problem has to be solved over multiple planning periods
9 The network involves multiple products
Basic Mixedinteger Programming Problems
Facilities
Facility Location Problem – II
The capacity (cost of setting up) of facility i∈{1, 2, …n} is αi (fi)
The production cost in facility i is pi
The setup (variable) cost of shipping material from i to j is tij (cij)
The capacity of transportation link (i→j) is sij
The demand of customer j∈{1, 2, …m} is dj
We define:
Xi = 1 if we set up (install) facility i
Yij = amount shipped from i to j
Zij = 1 if transportation link (i→j) is set-up
Objective: min z =
ai, fi
Customers
cij
dj
1
1
2
2
.
.
.
.
.
.
n
sij, tij
m
∑ f X + ∑ p ∑Y + ∑ t Z + ∑ c Y
(FLII.1)
Minimize total cost
∑Y
≥ dj
(FLII.2)
Customer demand satisfaction
∑Y
≤ ai X i
∀i
(FLII.3)
Facility capacity constraint
∀i, j
(FLII.4)
Transportation capacity and set-up
i
i
i
i
i
ij
ij
j
ij
i, j
ij
ij ij
i, j
∀j
i
ij
j
Yij ≤ sij Z ij
X i , Z ij ∈ {0,1} ∀i, j
Yij ≥ 0 ∀i, j
Modeling with 01 Variables
1. Select at least one item from a set j = {1, 2, …, m}
OR operator: y1 ∨ y2 ∨…ym
∑y
→
2. Select exactly one item
Exclusive OR: y1 ∨ y2 ∨ … ym
3. If select k then select l
Implication: k ⇒ l
→
→
yl ≥ yk
4. If and only if
Equivalence: k ⇔ l →
yk = yl
5. Disjunctions
j
≥1
j
∑y
j
=1
j
f1(x) ≤ 0 OR f2(x) ≤ 0
y1 + y2 = 1
f1(x) ≤ M1(1 - y1)
f2(x) ≤ M2(1 - y2)
where M1, M2 are sufficiently large numbers
6. Discontinuous Functions/ Domains
⎧α + bx
⎩0
Cost = ⎨
L ≤ x ≤U
x=0
New variable y: If y = 0 ⇒ x = 0
If y = 1 ⇒ L ≤ x ≤ U
Formulation:
c = αy + bx
Ly ≤ x ≤ Uy y∈{0,1}
Cost
α
L
U
x
Modeling with Disjunctions
We often want to enforce different conditions in different cases
Grossmann & co‐workers, 1998
Examples: Selection of reactor type
Selection of an additional flash tank
CO, CO2, H2
3-A)
4-A)
Flash
Drum1
Methanol Synthesis
Isothermal Reactor
3-B)
xj
Variables of streams feeding unit k
k
xOUT
Variables of streams leaving unit k
wk
Operational parameters of unit k
x
Flash
Drum2
Methanol Synthesis
Adiabatic Reactor
Process stream variables of stream j
k
IN
4-B)
Mathematical model of unit k :
Methanol
(
(
)
)
k
k
⎡ h k wk , xIN
, xOUT
= 0⎤
⎢ k k k k
⎥
⎢ g w , xIN , xOUT ≤ 0⎥
⎢ C k = f k (S k ) ⎥
⎣
⎦
Distillation
column
Water
To find the optimal design we want to be able to handle the following logic constraints:
⎡
⎤ ⎡
yk
⎤
¬y k
⎢ k k k k
⎥ ⎢ k
⎥
k
⎢ h w , xIN , xOUT = 0 ⎥ ⎢ w = 0 , C = 0 ⎥
⎢ g k w k , x k , x k ≤ 0⎥ ∨ ⎢ k
k
⎥
IN
OUT
⎢
⎥ ⎢ xIN = 0 , xOUT = 0⎥
⎢⎣ C k = f k ( S k ) ⎥⎦ ⎢⎣
⎥⎦
(
(
)
)
We have to convert the disjunction into MIP constraints, handled by MIP solvers
Reformulations
BigM reformulation
⎤
⎡
⎤ ⎡
y1
y2
⎥
⎢
⎥ ⎢
⎢ 1 ≤ x1 ≤ 3 ⎥ ∨ ⎢7 ≤ x1 ≤ 10⎥
⎢ 2 ≤ x2 ≤ 4 ⎥ ⎢ 6 ≤ x2 ≤ 8 ⎥
⎦
⎣
⎦ ⎣
(
(
)
)
(
(
)
)
(
(
)
)
(
(
)⎤
) ⎥⎥⎦
(
(
)
)
(
(
)⎤
) ⎥⎥⎦
⎡ − M ⋅ 1 − y1 + 1 ≤ x1 ≤ 3 + M ⋅ 1 − y1 ⎤ ⎡− M ⋅ 1 − y 2 + 7 ≤ x1 ≤ 10 + M ⋅ 1 − y 2
⎢
⎥ ⎢
1
1
2
2
⎢⎣− M ⋅ 1 − y + 2 ≤ x2 ≤ 4 + M ⋅ 1 − y ⎥⎦ ⎢⎣ − M ⋅ 1 − y + 6 ≤ x2 ≤ 8 + M ⋅ 1 − y
y1 + y 2 = 1
x2
⎡ − M ⋅ y 2 + 1 ≤ x1 ≤ 3 + M ⋅ y 2 ⎤
⎢
⎥
2
2
⎢⎣− M ⋅ y + 2 ≤ x2 ≤ 4 + M ⋅ y ⎥⎦
⎡− M ⋅ 1 − y 2 + 7 ≤ x1 ≤ 10 + M ⋅ 1 − y 2
⎢
2
2
⎢⎣ − M ⋅ 1 − y + 6 ≤ x2 ≤ 8 + M ⋅ 1 − y
Selecting : M = 7
x1
⎡ − 7 ⋅ y 2 + 1 ≤ x1 ≤ 3 + 7 ⋅ y 2 ⎤
⎢
⎥
2
2
⎢⎣− 7 ⋅ y + 2 ≤ x2 ≤ 4 + 7 ⋅ y ⎥⎦
(
(
)
)
(
(
)⎤
) ⎥⎥⎦
⎡− 7 ⋅ 1 − y 2 + 7 ≤ x1 ≤ 10 + 7 ⋅ 1 − y 2
⎢
2
2
⎢⎣ − 7 ⋅ 1 − y + 6 ≤ x2 ≤ 8 + 7 ⋅ 1 − y
Reformulations
Convexhull reformulation
x2
x1 = x11 + x12
⎡
⎤ ⎡
⎤
y1
y2
⎢
⎥ ⎢
⎥
⎢ 1 ≤ x1 ≤ 3 ⎥ ∨ ⎢7 ≤ x1 ≤ 10⎥
⎢ 2 ≤ x2 ≤ 4 ⎥ ⎢ 6 ≤ x2 ≤ 8 ⎥
⎣
⎦ ⎣
⎦
x2 = x12 + x22
y1 + y 2 = 1
,
x2 = x12 + x22
(
(
x12 ≤ M ⋅ y 2
x12 ≤ M ⋅ y1
x22 ≤ M ⋅ y 2
⎡ 1 ⋅ y1 ≤ x11 ≤ 3 ⋅ y1 ⎤ ⎡7 ⋅ y 2 ≤ x12 ≤ 10 ⋅ y 2 ⎤
, ⎢
⎢
1
1
1⎥
2
2
2 ⎥
⎣⎢2 ⋅ y ≤ x2 ≤ 4 ⋅ y ⎦⎥ ⎣⎢ 6 ⋅ y ≤ x2 ≤ 8 ⋅ y ⎦⎥
x1 = x11 + x12
x1
x11 ≤ M ⋅ y1
)
)
,
x11 ≤ M ⋅ y1
x12 ≤ M ⋅ y 2
x12 ≤ M ⋅ y1
x22 ≤ M ⋅ y 2
(
(
)
)
⎡ 1 ⋅ 1 − y 2 ≤ x11 ≤ 3 ⋅ 1 − y 2 ⎤ ⎡7 ⋅ y 2 ≤ x12 ≤ 10 ⋅ y 2 ⎤
⎢
⎢
2
1
2 ⎥
2
2
2 ⎥
⎣⎢2 ⋅ 1 − y ≤ x2 ≤ 4 ⋅ 1 − y ⎥⎦ ⎢⎣ 6 ⋅ y ≤ x2 ≤ 8 ⋅ y ⎦⎥
General Mixedinteger Programming Model
min cT x + dT y
s.t. Ax + By ≤ b
x ≥ 0, y integer
max cT x
s.t. Ax ≤ b
xj integer, j∈NI = {1...p}
or
If Ni = ∅ (i.e. there are no y variables) ⇒ Linear Programming (LP) Model
If Ni = N (i.e. there are no x variables) ⇒ (Pure) Integer Programming (IP) Model
If ∅ ≠ Ni ⊂ N (i.e. we have both x and y) ⇒ Mixed-Integer Programming (MIP or MILP) Model
A general integer Y∈{0, 1, 2, 3, …N} can be modeled via k binary variables Yk:
Y=
∑ kY
k = 0 ,... N
k
So, the general MIP problem is:
or
min cT x + dT y
s.t. Ax + By ≤ b
x ≥ 0, y∈{0, 1}
min cT x + dT y
s.t. (x, y)∈ X
LPrelaxation of a MIP Model General MIP problem (P):
min cT x + dT y
s.t. Ax + By ≤ b
x ≥ 0, y∈{0,1}
(P)
If we replace the integrality requirement y∈{0,1} with y∈[0,1] we obtain a relaxation (RP) of (P):
min cT x + dT y
s.t. Ax + By ≤ b
x ≥ 0, y∈[0,1]
(RP)
In general, a relaxation of a problem is obtained by:
Either removing constraints (i.e. expanding the feasible region)
Decreasing the objective function (e.g. removing a nonegative term)
min c1T x + c2T x
s.t. A1 x ≤ b1
A2 x ≤ b 2
(P)
x∈X
where x, c1, c2 ≥ 0
min c1T x + c2T x
s.t. A1 x ≤ b1
(RP1)
x∈X
min c1T x
s.t. A1 x ≤ b1
A2 x ≤ b 2
x∈X
Larger
feasible
region
z(RP2) ≤ z(P)
(RP2)
Better obj.
value at the
same point
BranchandBound: Basic Ideas
LPbased method:
Lang & Doig, 1960
Integrality constraints, y∈{0,1}, are replaced by y∈[0, 1]
(P)
max cT x + dT y
(LRP)
max
cT x + dT y
s.t.
Ax + By = b
⇒
s.t. Ax + By = b
p
x ≥ 0, y∈{0, 1}
x ≥ 0, y∈[0, 1]p
LP problem P(i) is solved at node i, with feasible region Fi and objective Zi
We know how to solve (LRP) effectively using Simplex (vs. other relaxations)
Key observations:

If a node is infeasible, then all its descendants will also be infeasible
A child node will always have a lower (worse) objective function (bounding)
Any feasible solution Zi in node i provides a lower bound on the optimal solution Z*
Easy to add rows (constraints) to (LRP) (why?)
(x1,x2) = (1.75, 1)
ZLP = 9.25
max 3x1 + 4x2
4x1+3x2 ≤ 10
x1∈{0,1,2,3}, x2∈{0,1}
x2
x1 ≤ 1
x1 ≥ 2
(2, 0.66)
ZLP=8.66
(1, 1)-INT
ZLP = 7.0
1
(2.5, 0)-INT
ZLP = 7.5
0
0
1
2
3
x1
x1 ≤ 2
x2 ≤ 0
x2 ≥ 1
x1 ≥ 3
INFEASIBLE
(2,1) ZLP=6 INFEASIBLE
Branchandbound: Basic Ingredients
1.
2.
3.
4.
5.
The list L of problems to be solved
Relaxation (RP): upper (& lower?) bounding procedure
Branching Rule: replace problem P(i) in node i by problems P(i1), P(i2), …, P(iq): FR(i1)∪…∪FR(iq) = FR(i)
Node (subproblem) selection rule: how to select a problem from the current subproblem list L
Pre-processing and variable fixing procedures
(x1,x2) = (0.75, 0.3)
ZLP = 9.25
(0, 0.8)
ZLP = 8.5
x2 ≤ 0
(0, 0)-INT
ZLP = 7

x1 ≤ 0
x2 ≥ 1
(0, 1) -INT
ZLP = 8
x1 ≥ 1
On which binary/constraint to branch?
(1, 0.7)
ZLP=8.75
x2 ≤ 0
x2 ≥ 1
(1, 0) -INT
ZLP =7.5
Which active node to evaluate next?
INFEASIBLE
Method first proposed by Lang and Doig (1960)
Around the same time an alternative method was developed (cutting planes)
Brute force branch‐and‐bound remained ineffective for many years
Cutting planes were not used for many years
Branchandbound Algorithm
1. Initialization
L = {P} ZU := +∞
2. Termination
If L = ∅ then
If ZU = +∞, then X = ∅ (infeasible problem); STOP
If ZU < +∞, then the solution (x, y)∈X with ZU = cx + dy is optimal; STOP
3. Node Selection and Solution
Using SSR select subproblem (node) P(i)∈L and let L := L\{P(i)}
Compute the optimal LP-value Zi and LP-solution (xi, yi) for P(i).
4. Pruning
If Zi ≥ ZU, then GOTO 2.
(P(i) is either infeasible or dominated by the best solution (upper bound) found so far)
If Zi < ZU, then
If yi is integral (i.e. yji is integral ∀j∈NI), then (a better solution is found)
Update upper bound by setting ZU:= Zi
Update list L by removing dominated subproblems:
for each i’∈L: If Zi’ ≥ ZU, then L := L\{P(i’)}
GOTO 2.
5. Branching*
Select j∈NI such that yji is fractional; yj is the branching variable
Update list L by adding programs with fixed yj values
Set L:= L ∪ {P(i0), P(i1)} where
P(i0) = P(i) ∩ {(x,y)∈ ℜn+ × ℜl+ | yj ≤ ⎣yji⎦}
P(i1) = P(i) ∩ {(x, y)∈ ℜ n+ × ℜl+ | yj ≥ ⎡yji⎤}
GOTO 2.
min
s.t.
cT x + dT y
(x, y)∈ X
* Occurs only when P(i) has not been pruned, i.e. when Zi < ZU and yji is fractional for some j ∈ NI
Branchandbound Tree
0
1
2
3
7
4
8
9
5
10
11
6
12
13
14
Level
0
Nodes
1 = 20
1
2 = 21
2
4 = 22
3
8 = 23
4
16 = 24
15 16 17 18 …
Notes:
Level denotes the number of fixed variables
Total number of nodes = 1 + 2 + … + 2N = 2N+1-1, where N = no of binary variables [Why?]
The goal is to explore much fewer nodes via bounding
How do we carry out branching?
Branching
(x1,x2) = (0.75, 0.3)
ZLP = 9.25
(0, 0.8)
ZLP = 8.5
x2 ≥ 1
x2 ≤ 0
(0, 0)-INT
ZLP = 7
x1 ≤ 0
(0, 1) -INT
ZLP = 8
x1 ≥ 1
On which binary/constraint to branch?
(1, 0.7)
ZLP=8.75
x2 ≤ 0
x2 ≥ 1
(1, 0) -INT
ZLP =7.5
Which active node to evaluate next?
INFEASIBLE
Variable Selection Rules
1. Try to find a good solution fast: Obtain a good lower bound ⇒ Prune many nodes fast
2. Choose a variable that results in bad solutions: Prune as many nodes as possible ⇒ Keep the b&b tree small
Node selection rules
Try to find good feasible solutions as early as possible
In general:
The number of active nodes increases if the active node is chosen high-up
It takes a long time to improve the upper bound if the active node is chosen from down low
We want balanced branching and tight bounds: choose j∈NI that yields:
{
max min{Z
j
x j =0
,Z
x j =1
}
}
Example
INT:
Integer feasible
INF: Infeasible
FRAC: Fractional
FRAC
ZU=16
FRAC FRAC
ZU=18 ZU=14.5
INT
ZU=15 INF
INF INT
ZU=16
FRAC
ZU=14
Current best (ZU)
Closed (infeas or Zi > ZU)
Open (Zi < ZU)
Exercise
{
max min{Z
j
x j =0
,Z
x j =1
}
}
Example
Current best (ZU)
Open (Zi < ZU)
ZU=14
(0,…,0.1, 0.4, 0.95,...1)
ZU=15
ZU=14.5
{
max min{Z
j
x j =0
,Z
x j =1
}
}
Example
Current best (ZU)
Open (Zi < ZU)
ZU=14
(0,…,0.1, 0.4, 0.95,...1)
y=0
ZU=15
ZU=14.5
y=1
ZU=14.1 ZU=16
{
max min{Z
j
x j =0
,Z
x j =1
}
}
Example
Current best (ZU)
Open (Zi < ZU)
ZU=14
(0,…,0.1, 0.4, 0.95,...1)
ZU=15
ZU=14.5
y=0
y=1
ZU=15
ZU=15.2
Node Selection Rules
Depth-first Search:
Always choose one of the new subproblems just generated; typically the one with the best lower bound
Backtrack when node is discarded
0
1
2
12.5
12.8
3
4
12.85
5
z=12
13.3
6
13.2
12.95
7
8
13.4
9
Advantages:
13.5
10
Can potentially find a (good) solution fast
Re-optimization involves the addition of a
single constraint (???)
Requires modest storage
Disadvantages:
“Hidden” good solutions difficult to find
Breadth-first Search:
Expand all nodes at each level before go to the next level
0
z=12
1
2
12.5
12.8
3
4
12.85
5
13.3
7
8
13.2
12.95
9
13.4
13.2
10 11
13.3
6
13.4
13
12
13
13.5
13.5
14
13.1
Disadvantages:
Examine many non-promising nodes
Feasible solutions are found late
Create very large tree (memory requirements)
Not used in practice
Best-first Search:
Always choose the subproblem with the best lower bound
0
z =12
1
2
12.5
12.8
3
4
7
5
13.3
12.85
8
12.95
13.2
6
13.2
13
11
12
13.5
9
13.4
13.1
10
13.5
Advantages:
Examines potentially good nodes
Uniformly tighten bounds
Disadvantages:
Re-optimization is more expensive
Tree tends to be larger
Modern commercial solvers use a
combination of depth-first and best-first.
Special Branching Schemes
Special Ordered Sets 1 (SOS1):
AND x1 + x2 + x3 + x4 + x5 = 1 AND only one non-zero,
xi ≥ 0
(0, 0.5, 0.5, 0, 0)
INFEASIBLE
x 1 + x2 = 1
x 3 + x4 + x5 = 1
or
(0, 0.5, 0.5, 0, 0)
INFEASIBLE
x1 = 1
x2 = 1
x3 = 1
x4 = 1
x5 = 1
Special Ordered Sets 2 (SOS2):
xi ≥ 0 AND x1 + x2 + x3 + x4 + x5 = 1 AND at most two non-zeros AND consecutive
(0, 0.5, 0, 0.5, 0)
INFEASIBLE
x1=0, x2=0
x3=0, x4=0
Preprocessing
Preprocessing applies simple logic to reformulate and tighten the LP relaxation
For MIP problems is better to spend extra time initially to reduce the possibility of long runs
Probing
Temporarily set a 0-1 variable to 0 or 1 and then redo the logical testing.
5x1 + 3x2 ≥ 4 becomes infeasible when x1=0
⇒ x1=1 in every feasible solution
5x1 + 4x2 + x3 ≤ 8
If x1=1 then x2=0 and vice versa
⇒ x1 + x2 ≤ 1
(0.8, 0)
(1, 0.75, 0)
2x1 + x2 + x3 ≥ 1 becomes strictly redundant when x1=1.
⇒ Can be replaced by x1 + x2 + x3 ≥ 1
(0.5, 0, 0)
Generate Bounds
e.g., If the upper bound of a 0-1 variable is less than 1, then it can be fixed to 0.
Detect Infeasibilities and Redundant Constraints
If Li ≤ Axi ≤ Ui (for l ≤ x ≤ u) then Axi ≤ bi is redundant if Ui ≤ bi and is infeasible for Li < bi.
Preprocessing
In summary preprocessing:

Improves bounds
Identifies infeasibility and redundant constraints
Fixes variables
Generates new valid inequalities
Tightens existing inequalities (coefficient improvement)
Not possible to perform all operations and save all information
Computational Effort for preprocessing:
1. Techniques applied to one row ⇒ Always worth doing
2. Saving and preprocessing implication and clique tables
3. Full probing on the matrix
Common strategy:
1. Generate clique and implication inequalities
2. Store them in separate tables
3. Add them to the linear program only when they are violated by the current LP solution
Heuristics
Solution Time proportional to tree size
Tree is large when there are many active nodes; i.e. nodes with zi < zbest = ZU
How do we keep the size of the tree small?
Try to reduce zi as fast as possible ⇒ Variable Selection Rule
Try to find large zbest as fast as possible ⇒ Heuristics
Also for large scale instances: “good feasible solutions may be all we can hope for”
If we know the structure of the problem: Use any known heuristic to provide an initial zbest
e.g. use Lin-Kernigham heuristic to get a usually very good solution for the TSP
If we do not know a heuristic: Use LP-based heuristics:
Enumerate 0-1 vectors in the neighborhood of the current LP solution and check feasibility
OCTANE heuristic (Balas et al., 2002)
Successive rounding heuristics
Fractional variables sequentially rounded until an integral solution or infeasibility is detected
Equivalent to “diving” in the b&b tree: branching on a variable and examine only one child
Motivation for BranchandCut
LP Relaxation Feasible Region
IP Feasible Region X
x2
max 3x1 + 4x2
(M)
4x1+3x2 ≤ 10
x1∈{0,1,2,3}, x2∈{0,1}
max 3x1 + 4x2
(M)
4x1+3x2 ≤ 10
x1∈{0,1,2,3}, x2∈{0,1}
x2
1
1
0
0
1
2
3
0
x1
0
1
2
3
x1
Convex Hull of IP Feasible Region
(M)
max 3x1 + 4x2
4x1+3x2 ≤ 10
x1∈{0,1,2,3}, x2∈{0,1}
x2
1
0
0
1
2
3
x1
Valid Inequalities and Cutting Planes
LP Relaxation Feasible Region
If we have CH(X) we solve just one LP
x2
We need facet defining inequalities
Has been done for some problems
1
Valid Inequality
0
0
1
2
3
Facet
In general very difficult
Try to approximate instead
Find cutting planes
x1
Cutting Plane
4x1 + 3x2 ≤ 10
⇒ x1 ≤ 2
x2
1
0
0
1
2
3
x1
GomoryChvatal Inequalities
Original inequality (ith row):
Σj aij xj ≤ bi
(1)
If all the variables in (1) are non-negative integers then the GC inequality is given by:
Σj ⎣aij⎦ xj ≤ ⎣bi⎦
(2)
A GC inequality can be derived by the LP optimal simplex table from a row with a fractional variable
Pure IP’s can be solved by successively adding GC inequalities to the optimal LP solution
Examples:
x1 + x2
≤ 1.5
x1 + 0.75x2 ≤ 2.5
1.5x1 + x2
≤2
⇒ x1 + x2 ≤ 1
Cuts-off (0.75, 0.75)
⇒ x1 ≤ 2.5
Cuts-off (2.5, 0)
⇒ x1 + x2 ≤ 2
Not tighter
Cover Inequalities
Original knapsack inequality:
Σj∈J aj xj ≤ b,
xj∈{0,1} (1)
Let C be a subset of J such that Σj∈C aj > b.
Then a valid cover inequality is given by:
Σj∈C xj ≤ |C| - 1
xj∈{0,1} (2)
We have to separate cover inequalities
Cover inequalities can be made tighter by lifting; i.e. including coefficients for j∉C.
Example:
5x1 + 5x2 + 4x3 ≤ 8
⇒ x1 + x2 ≤ 1
⇒ x2 + x3 ≤ 1
⇒ x1 + x2 + x3 ≤ 1
Cuts-off (1, 0.5, 0), (0.5, 1, 0)
Cuts-off (0, 1, 0.5), (0, 0.5, 1)
Lifted tighter inequality: Gives the Convex Hull
Example Revisited
x2
max 3x1 + 4x2
4x1+3x2 ≤ 10
x1∈{0,1,2,3}, x2∈{0,1}
1
0
0
1
2
3
4x1+3x2 ≤ 10
0 ≤ x1 ≤ 3
0 ≤ x2 ≤ 1
⇒
x1
Gomory-Chvatal Inequality:
Cover Inequalities:
4x1 + 3x2 ≤ 10
x1 + 0.75x2 ≤ 2.5
x1 + ⎣0.75⎦ x2 ≤ ⎣2.5⎦
4x1 + 3x2 ≤ 10
x1 ≤ 2
x1 + x2 ≤ 2
Three units of x1 is a cover:
Lifted cover inequality:
⇒ x1 ≤ 2
x2
x2
1
1
0
0
1
2
3
x1
0
0
1
2
3
x1
BranchandCut Algorithm
zLP(2)
zLP(1)
Root Node
zLP(3)
(1)
(2)
CP(2)
CP(1)
(3)
CP(3)
BranchandCut Algorithm
Root Node
(1)
zLP(4)
(2)
(3)
(5)
(4)
x1≥2
(6)
(7)
(8)
x1≤1
zLP(5)
zLP(6)
zLP(7)
zLP(8)
MIP Computational History: 1950 1998

1954: Dantzig, Fulkerson, S. Johnson: 42 city TSP
Solved to optimality using cutting planes and LP

1957: Gomory
Cutting plane algorithm: A complete solution

1960: Land, Doig; 1965 Dakin
Branch-and-bound (B&B)

1971: MPSX/370, Benichou et al.

1972: UMPIRE, Forrest, Hirst, Tomlin

1972 – 1998: Good B&B remained the state-of-the-art in commercial codes, in spite of
• 1973 Padberg
• 1974 Balas (disjunctive programming)
• 1983 Crowder, Johnson, Padberg: PIPX, pure 0/1 MIP
• 1987 Van Roy and Wolsey: MPSARX, mixed 0/1 MIP
• Grötschel, Padberg, Rinaldi …TSP (120, 666, 2392 city models solved)
MIP Computational History: 1998 Date
Linear programming
• Stable, robust dual simplex
Variable/node selection
• Influenced by traveling salesman problem
Primal heuristics
• 8 different tried at root node; retried based upon success
• Construction & improvement heuristics
• Relaxation-induced neighbor search
Node pre-solve
• Fast, incremental bound strengthening (very similar to Constraint Programming)
Presolve – numerous small ideas
• Probing in constraints:
∑ xj ≤ (∑ uj) y, y = 0/1
Î xj ≤ ujy (for all j)
Cutting planes
• Cliques, disjunctive, flow covers, Gomory, GUB covers, knapsack covers, implied
bounds, mix-integer rounding, path cuts, etc.
• Aggregation
• …
Computational Progress
• Algorithms
Primal simplex in 1988 versus best(primal,dual,barrier) today 2,360x
• Machines
800x
• Net: Algorithm * Machine ~ 1,900,000x
*
Bixby, Fenelon, Gu, Rothberg, Wunderling. A New Generation of MIP Codes,
IP Conference in Honor of Egon Balas, June 3-5 2002, Carnegie Mellon University, Pittsburgh, PA
California Unit Commitment – 7 Day Model
UNITCAL_7: 48939 cons, 25755 vars (2856 binary)
Previous attempts (by model formulator): 2 Day model: 8 hours, no progress
7 Day model: 1 hour to solve initial LP
CPLEX 8.0 on 2.0 GHz Pentium 4 Running defaults ...
Reduced MIP has 38872 rows, 19989 columns, and 105612 nonzeros.
MIP emphasis: balance optimality and feasibility
Root relaxation solution time =
6.70 sec.
Node
0
*
*
*
*
*
*
Nodes
Left
0
0+
150+
150+
850+
930+
1550+
0
43
43
294
143
278
Objective
1.9396e+07
1.9472e+07
1.9529e+07
1.9559e+07
1.9566e+07
1.9570e+07
IInf
781
354
358
345
399
363
0
0
0
0
0
0
Best Integer
2.0669e+07
1.9648e+07
1.9646e+07
1.9645e+07
1.9638e+07
1.9636e+07
Cuts/
Best Node
1.9396e+07
Cuts: 1220
Cuts: 1260
Cuts: 555
Cuts: 313
Cuts: 266
1.9570e+07
1.9591e+07
1.9591e+07
1.9618e+07
1.9618e+07
1.9627e+07
ItCnt
19505
21719
25122
27311
28751
30151
30151
47897
47897
98212
102700
127050
Gap
5.32%
0.29%
0.28%
0.14%
0.10%
0.04%
GUB cover cuts applied: 2
Clique cuts applied: 27
Cover cuts applied: 4
Implied bound cuts applied: 1038
Flow cuts applied: 1019
Flow path cuts applied: 5
Gomory fractional cuts applied: 152
Integer optimal, tolerance (0.0001/1e-06) : Objective =
1.9635572051e+07
Current MIP best bound =
1.9633624784e+07 (gap = 1947.27, 0.01%)
Solution time = 724.41 sec. Iterations = 174362 Nodes = 3575 (1104)
A customer model:
44 cons, 51 vars, 167 nzs, maximization
51 general integer variables (and no bounds)
Branch-and-cut:
Initial integer solution -2186.0
Initial upper bound
-1379.4
…after 1.4 days, 32,000,000 B&C nodes, 5.5 Gig tree
Integer solution and bound: UNCHANGED
What’s wrong?
Free vars chase each other off to infinity.
Maximize
x + y + z
Subject To
2 x + 2 y ≤ 1
z = 0
x free y free
x,y integer
This problem can be solved in several ways
Euclidean reduction on the constraint [Presolve]
Removing z=0, objective is integral [Presolve]
Locally valid cuts at the nodes
However: Branch-and-bound cannot solve!!!
Outline
How Can We Solve Hard Problems?
Traditional Approaches
Better models
Better solvers
Customized methods
• Decomposition algorithms
• Problem‐specific algorithms
New and emerging trend
Exploiting computational power – parallel computing
Better Models I
The “Pigeon Hole” problem:
There are n+1 pigeons and n holes.
We want to place all pigeons in the holes but in such way that no pair of pigeons is in the same hole.
Infeasible, but how is the branch and bound tree ?
Let xij=1 if pigeon i is in hole j.
Assignment of every pigeon in a hole:
No pair of pigeons in the same hole:
Σj xij = 1
xij + xkj ≤ 1
∀i
∀j, ∀i,k | i<k
Σj xij = 1
Σi xij ≤ 1
∀i
∀j
• Feasible LP relaxation: xij=1/n
• LP rel. feasible after branching
• Total enumeration is needed
Alternative Formulation
Assignment of every pigeon in a hole:
At most one pigeon at each hole:
• Replace n3 constraints (2 var’s) with n constraints (n+1 var’s)
⇒ INFEASIBLE LP relaxation
Better Models II
The discrete time, finite horizon, economic lot‐sizing problem
There is a given demand dt for each period t which can be satisfied by production in any period t’ ≤ t.
For each period there is a production cost, a holding cost and a setup cost (if production occurs).
yt: production in period t
st: inventory in period t
xt = 1 if production occurs at t
yt ≤ Dt xt = (Σt’=t..T dt’ )xt
st = st-1 + yt – dt
∀t
∀t
Optimal LP relaxation: fractional: xt = yt /Dt
ytt’: production in period t for demand at t’
xt = 1 if production occurs at t
Σt=1..t’ ytt’ = dt’
ytt’ ≤ dt’ xt
∀t’
∀t’, ∀t≤t’
Optimal LP relaxation: ⇒ LP solution is also MIP solution
Stronger Formulations Using Cutting Planes Ideas
x + y ≥ 3.5,
x, y ≥ 0, y integral
y
4.0
3.0
(0, 3.5)
2.0
1.0
1.0
2.0
3.0
4.0
x
Stronger Formulations Using Cutting Planes Ideas
x + y ≥ 3.5,
x, y ≥ 0, y integral
y
y ≥ ⎡3.5⎤
4.0
Mixed-integer
Rounding cut:
2x + y ≥ 4
3.0
y ≤ ⎣3.5⎦
2.0
1.0
1.0
2.0
3.0
4.0
x
Better models
Better solvers
Customized methods
• Problem‐specific algorithms
How Can We Guide Search?
Guess correct branching binary
Heuristic: Choose fractional variable closest to 0.5
Does it always work?
z
1
0
x2
×
Branching on x1
1
x1
Branching on x2
No, try to find the sensitivities for x1 and x2.
How???
Branching Using Pseudocosts
z
d1
u1
0
x1*
1
x1
z
d2
u2
0
x2*
1
Choose x2 although
x1 is closer to 0.5
x2
We want to choose the variable with the maximum min {dj, uj}
Degradations dj and uj can be estimated from dual simplex method: = (reduced cost)×change
Choose the variable with the maximum estimated degradation
Strong Branching
But the reduced cost (dual value) is valid only for small perturbations
Can we do better than this, i.e. can we get a better estimate of the impact of adding xj = 0 and xj = 1?
[0.1, 0, 0.5, 0.6, 0.8]
10
x3
12.5
[0.1, 1, 0, 0.5, 0.2]

=
0
x
3
=
1
[0.3, 0, 1, 0.4, 0.6]
13.5
Choose a subset of fractional binaries
For each chosen binary fix to xj = 0 and xj = 1 and start solving resulting LP’s.
Stop after a fixed number of Simplex iterations
Choose the binary that led to poor subproblems
[0.1, 0, 0.6, 0.4, 0.3]
[0.1, 0, 0.6, 0.4, 0.3]
10
10
x1
11
=
0
[0.1, 0, 0.6, 0.4, 0.3]
x
1
=
1
x3
14
12
=
0
[0.1, 0, 0.6, 0.4, 0.3]
10
x
3
=
1
x4
13
12.5
=
0
10
x
4
=
1
x5
13
14
=
0
x
5
=
1
11.5
OK, … What Can We Do Realistically?
Pseudo-costs, strong branching, all variants of node selection rules (depth-first, best-first,
etc.), heuristics and other exotic methods have been implemented in existing software
Even if we knew how to implement all of them ourselves, it would be difficult to do it more
efficiently than their implementation within CPLEX, Xpress, etc.
But …, knowing what these methods do allows us to use them, or know how/when to use
them
Main point of this subsection:
Before you start developing a new method make sure that you are not re-inventing the wheel
Understand the problem and which of the existing methods can help
Find corresponding solver options
Experiment …
Better models
Better solvers
Customized methods
• Problem-specific algorithms
Decomposition
Basic Decomposition Idea
Algebra
x1+2x2+2x3
2x1+ x2 + x3
3x2+4x3
x3+3x4+ x5
x4+2x5
Optimization
=6
=6
=7
=8
=8
x1=2
x2=1
x3=1
x4=2
x5=3
Underdetermined
⇒ Many solutions
⇒ Solve (M1) & (M2) many times
M1
M2
Solution time: 2N
(M)
→ 210
(Μ1) & (Μ2)
→ 26+26
= 1024 sec
= 128 sec
Key Issues
• How do we decompose?
• Which constraints do we keep/remove?
• How do we iterate?
• What type of information from (M2) can we use to guide the solution of (M1)?
• How do we pass this information?
• How do we solve models M1 and M2?
• Submodels (M1) and (M2) may be different (e.g. MIP vs. LP)
• Do we have to use the same algorithm?
• If not, how do we make them compatible?
Lagrangean Relaxation
Consider the problem:
min z = cTx
s.t. Ax ≤ b
Dx ≤ d
x∈X
(P)
(1)
(2)
where constraints in eq. (1) are “nice”, but constraints in eq. (2) are “hard”.
Our goal is to develop an easy subproblem that does not violate eq. (2) substantially.
min z = cTx + u(Dx-d)
s.t. Ax ≤ b
(1)
x∈X, u ≥ 0
(LR)
For every u ≥ 0, (LR) is a relaxation of (P) because
a) the feasible region of (LR) is greater, and
b) the objective value of (P) is always greater than the one of (LR) for all feasible solutions.
Key Idea: Try to find the correct values for penalties u.
What do the multipliers u look like?
If a constraint in Dx ≤ d is violated, we would like to increase u.
If a constraint in Dx ≤ d is violated by a solution of (LR), this constraint would be active in (P).
⇒ u is the dual value of a violated constraint in (P).
Lagrangean Relaxation Implementation
s.t. Ax ≤ b
(1)
x∈X, u ≥ 0
(LR)
Select K, ε, mk
Set UB = +∞, LB= - ∞
Start with u0 = 0
k = k+1
Solve (LR)
Obtain LB
Optional
Return UB
& Solution
YES
(LR) feasible?
Modify (LR) solution to
obtain a feasible solution & UB
NO
YES
|UB-LB|<ε
or k=K?
NO
Update uk:
uk+1 = max {uk - mk (Dx-d),0}*
mk is the step length
* Subgradient optimization
Lagrangean Relaxation – Choice of Dual
min z = cTx
s.t.
A1x ≤ b1
A2x ≤ b2
A3x ≤ b3
x∈X
x2
(P)
(1)
(2)
(3)
x2
Which constraints should be dualized?
x2
CHLR1
CH
CHLR2
PI
P II
PI
P II
P II
P
P LR 2
P LR1
P III
(a) LP-relaxation of (M3): P=PI∩PII∩PIII
x1
x1
(b) Dualization of AIIIx = eIII: PLR1 = PI∩PII
: feasible regions defined by subsets of constraints; : integer points;
: convex hull of feasible integer points;
: intersection of constraint subsets.
P III
x1
(c) Dualization of AIx = eI: PLR1 = PII∩PIII
: feasible integer points;
Key issues:
The strength of the resulting Lagrangian dual bound LB
The ease of solution of the Lagrangian problem (LR)
Lagrangean Relaxation Remarks
min z = cTx
s.t. Ax ≤ b
Dx ≤ d
x∈X
min z = cTx
s.t. Ax ≤ b
Dx = d
x∈X
(P)
Hard constraints
are inequalities
s.t. Ax ≤ b
(1)
x∈X, u ≥ 0
(LR)
(P)
Hard constraints
are equalities
s.t. Ax ≤ b
(1)
x∈X
(LR)
(1)
(2)
(1)
(2)
?
Lagrangean Relaxation Example
Structure of original model:
Independent blocks with linking constraints
Independent unit models + flow constraints
Multi-period models with inventory constraints
Assignment + sequencing constraints
min z = cTx
s.t. Ax ≤ b
Dtx ≤ dt t=1, …T
x∈X
(1)
(2)
Dualize linking
constraints
min z = cTx + u(Ax-b)
s.t.
Dtx ≤ dt t=1, …T
x∈X, u ≥ 0
(2)
Lagrangian relaxation (LR) easy to solve
Bender’s Decomposition
Structure of original model:
Independent blocks with linking variables
Stochastic programming: here-and-now & wait-and-see decisions
Hierarchical decisions
How can we decompose the problem?
max c0 x + Σk ck xx
A0x
= b0
(1)
Akx + Bkyk = bk, k=1,…,K (2)
Benders Decomposition
1.
2.
3.
4.
Solve problem master problem (MP): A0x = b0
Fix linking variables x
Solve K decomposed problems Bkyk = bk – Akx
Generate cuts and add them to (MP)
Cuts are generated from dual information
5. Check convergence; if not go to 1
“Bender’s decomposition lends itself well to parallelization” …
Hybrid Methods
x
cTx
min z =
s.t. Ax
≤e
Bx + Dy≤ f
x∈X, y∈Y
(P)
(1)
(2)
y
1
2
3
4
where constraints in eq. (2) are “hard” for mathematical programming but “easy” if we use a different solution method.
Remove
Constraints
Hybrid Model
min cx
s.t. Ax ≤ e
g(x,y) ≤ 0
x∈X, y∈Y
Relaxed Model (RM)
min cx
Lower
s.t. Ax ≤ e
Bound
x∈X
min cx
(RM)
s.t. Ax ≤ e
x∈X
x = xk
Add
Constraints
Subproblem (SM)
min cx
s.t. Ax ≤ e
x = xk
g(x,y) ≤ 0
x∈X, y∈Y
x∉X
g(xk, y) ≤ 0 (SM)
y∈Y
Upper
Bound
(Feasible )
Better models
Better solvers
Customized methods
• Problem-specific algorithms
What Is Parallel Computing?
Parallel
Processing
Multiple machines
distributed globally
Grid
Computing
Parallel
Computing
Distributed
Computing
Multiple CPUs on a
machine
Cluster
Computing
Multiple machines
located close by
Multiple machines
Parallel Integer Programming
LP-based branch-and-bound has inherent structure for parallelization
• Subproblems associated with each node are independent
• Evaluate nodes simultaneously
Fundamental issue for parallelizing branch-and-cut: Management of cuts
• Minimize cut generation time: do not develop same cuts several times
• Maximize useful sharing: pass relevant cuts
• Avoid flooding the communication system with cuts
Nodes solved at the master computer
Information flow: master → slave
• Subproblem formulation
• Variable bounds
×
Information flow: slave → master
• New feasible solutions
• Current lower bound
• Global cutting planes
Condor Project @ UW Madison

Started in early 80s at CS Dept.
http://www.cs.wisc.edu/condor/
Manages 1000 workstations and delivers 650 CPU days on a day
Gained popularity from around the world
• Industry, government, academia
Matchmaker
Problem
Solver
Agent
Resource
Shadow
Sandbox
Implementation Challenges*

Theoretical foundations well-known and well documented
Many algorithmic and implementation issues need to be resolved
Example: Cut Management
The addition of cuts results in smaller trees BUT
Extra time spent in generating cuts (regardless of whether all cuts are used)
Extra time spent on solving LP’s at each node
Questions:
When to generate cuts and how many?
Which cuts (if any) to add to the active linear problem?
When to delete previously generated cuts?
Basic ideas:
Generate cuts only every k1 nodes
Perform only k2 rounds of cut generation
Have only limited number of active cuts; keep inactive cuts in the “cut pool”
Basic Management Scheme:
1. Solve the active linear program
2. Identify inactive cuts; delete them from the active LP; add them to the cut pool
3. Search the cut pool for violated cuts; add a subset to the active LP and go to 1
4. Generate violated cuts. Add them to the cut pool and go to 1
* Johnson, E.L. Progress in Linear Programming-Based Algorithms for Integer Programming: An Exposition.
INFORMS Journal on Computing, Vol. 12, No 1, Winter 2000
Promising Research Areas*
Applications
Robust models that integrate planning and operations
Massively large, multi-period problems
Multi-scenario stochastic problems; long solution times may be acceptable
Recourse models that correct in real-time planning solutions
Have to be solved in minutes
Methods
Parallel Integer Programming
• LP-based branch-and-bound is a natural candidate for parallelization
Subproblems associated with each node are independent; evaluate nodes simultaneously
• Fundamental issue for parallelizing branch-and-cut: Management of cuts
Integration between heuristic and cutting planes
Integration with local search and constraint programming
* Johnson, E.L. Progress in Linear Programming-Based Algorithms for Integer Programming: An Exposition.
INFORMS Journal on Computing, Vol. 12, No 1, Winter 2000
Outline
Software Overview
Papers usually focus on How_to_Solve_a_Problem
How_to_solve_a_problem is only a single step of the overall process
Modeling life-cycle
1.
2.
3.
4.
5.
Develop the model
Generate instance(s) of the model
Solve the instance(s)
Verify the results/model
Modify the model
Ideally, software has to nurture the entire modeling life cycle
Two components of this “ideal” environment have received the most attention:
• The modeling module:
GAMS, AIMMS, Xpress-IVE, OPL Studio
• The solver module:
CPLEX, XPRESS-MP, OSL
The two modules have to communicate with each other ⇒ MPS format
• All modeling modules are able to produce an MPS input file and to process an MPS output file
• All solver modules are able to process an MPS input file and to produce an MPS output file.
Modeling Systems
Modeling systems provide an interface with a variety of different solvers (algorithms).
The user supplies a model (i.e. parameters, variables and constraints) using an easy language.
The user-generated input file is compiled to generate a standard optimization file (e.g. .lp, .mps)
This file is passed to the appropriate solver
The solution found by the solver is communicated back to the modeling system, and reported via an
output file.
A simple diagram for GAMS is shown below.
Input file
XXX.gms
Output file
XXX.lst
GAMS
User
Solvers
LP Solver 1
NLP Solver 1
MIP Solver 1
LP Solver 2
NLP Solver 2
MIP Solver 2
MINLP Solver
MIP Solver 3
Solvers
There are many different solvers for each problem (LP, NLP, MINLP, etc.)
Different solvers are good for different classes of problems; e.g. MINOS vs. CONOPT for NLPs
Modeling Systems
GAMS:
Old language; limited flow control & visualization aids
Compatible with many solvers
AIMMS:
New, user-friendly language; fancy interface
Compatible with many solvers (not as many as GAMS)
Xpress-IVE: Product of Dash Optimization → Solvers of Dash Optimization
Provides tools for algorithm customization
OPL Studio: Product of ILOG → Solvers of ILOG
Provides tools for algorithm customization
Enhancements
Advanced users often want to specify branching rules, customize cut generation routines, etc.
To support this functionality software developers provide callable library version of the software
Modeling Levels
* Irv Lustig, Embedding CPLEX Algorithms Overview
Modeling Levels
Modeling Levels
Outline
Graph Theory
A graph G is a pair (V, E) of a set of vertices V (points) and set of edges E (lines)
Undirected graph: undirected edges, (i, j) = (j, i)
a
u
Directed graph: directed edges, (i, j) ≠ (j, i)
d
Network: directed graph with capacitated edges
If i, j∈V, then (i, j)∈E is an edge; i is the tail, j is the head. z
v
b
e
c
w
Graph G1 = (V1, E1)
V1 = {u, v, w, z}
E1 = {(u, v), (u, w), (v, w), (v, z), (w, z)}
E1 = {a, b, c, d, e}
Graph Theory
A graph G is a pair (V, E) of a set of vertices V (points) and set of edges E (lines)
Undirected graph: undirected edges, (i, j) = (j, i)
Directed graph: directed edges, (i, j) ≠ (j, i)
1
2
4
3
Network: directed graph with capacitated edges
If i, j∈V, then (i, j)∈E is an edge; i is the tail, j is the head. Digraph G2 = (V2, E2)
V1 = {1, 2, 3, 4}
E1 = {(1, 2), (2, 1), (2, 3), (3, 4),
(4, 1), (4, 2)}
Representation of Graphs and Networks
A vertex i is incident to an edge (i, j)
Incidence matrix:
Nodes in Rows, edges in columns – 1 if a node is incident to an edge
In directed graphs, incident matrix has a ‐1 for the tail and +1 for the head
a
u
d
z
v
b
e
1
2
4
3
c
w
(u, v) (u, w) (v, w) (v, z ) ( w, z )
u ⎡ 1
1
⎤
⎢
⎥
v
1
1
1
⎢
⎥
w ⎢
1
1
1 ⎥
⎢
⎥
z ⎣
1
1 ⎦
1
2
3
4
(1,2) (2,1) (2,3) (3,4) (4,1) (4,2)
+1
⎡ −1 +1
⎤
⎢+ 1 − 1 − 1
+ 1⎥
⎢
⎥
+1 − 1
⎢
⎥
⎢
⎥
+ 1 − 1 − 1⎦
⎣
Familiar Combinatorial Optimization Problems
Traveling Salesman Problem
There are n cities to be visited exactly once by a traveling salesman;
the distance/cost from city i to city j is cij.
Find the Hamiltonian of minimum total distance/cost
Problem appears in 1830s; treated mathematically in 1930s (Menger)
One of the most intensively studied problem in optimization
No effective algorithm exists
It was shown it is NP-hard (Karp, 1972)
xij = 1 if we visit city j after city i
Integer programming formulation
Subtour elimination: for every V1,V2: V = V1∪V2, V1∩V2 = ∅
ensure there is an edge between V1 and V2.
Exponential number of subtour elimination constraints
IP formulation very hard
min ∑ cij xij
i, j
∑x
ji
= 1 ∀i
There is an incoming edge
ij
= 1 ∀i
There is an outgoing edge
j
∑x
j
Subtour elimination constraints
xij ∈ {0,1} ∀i, j
Matchings
Given a graph G = (V, E), a matching M in G is a set of pairwise
non-adjacent edges; i.e., no two edges share a common vertex.
Find a matching that contains the maximum number of edges.
A perfect matching is a matching which matches all vertices
Matchings in bipartite graphs are related to the assignment problem
Finding a maximum matching in bipartite graphs is easy
Finding a maximum matching in general graphs is harder
Polynomial algorithm for the general case by Edmonds (1965)
Formulation for (complete) bipartite graphs solved as LP
Equivalent to assignment problem, which was solved
effectively before IP formulations
Maximum matchings
Jobs
Machines
Perfect matching in bipartite graph: solution to assignment problem
xij = 1 if edge (i,j) is selected
max ∑ cij xij
i, j
∑x
ij
= 1 ∀i ∈ V1
∑x
ij
= 1 ∀j ∈ V2
j∈V2
i∈V1
xij ∈ {0,1} ∀i, j
Matchings
Given a graph G = (V, E), a matching M in G is a set of pairwise
non-adjacent edges; i.e., no two edges share a common vertex.
Find a matching that contains the maximum number of edges.
A perfect matching is a matching which matches all vertices
Matchings in bipartite graphs are related to the assignment problem
Finding a maximum matching in bipartite graphs is easy
Finding a maximum matching in general graphs is harder
Polynomial algorithm for the general case by Edmonds (1965)
Formulation for (complete) bipartite graphs solved as LP
Equivalent to assignment problem, which was solved
effectively before IP formulations
Odd-set constraints: for vertex subset U⊂V: |U| > 2
Problem solved as LP if all odd-set constraints are added
Maximum matchings
Jobs
Machines
Perfect matching in bipartite graph: solution to assignment problem
xe = 1 if edge e is selected
max ∑ ce xe
e
∑x
e∈δ ( i )
e
≤ 1 ∀i
| U |⎥
≤ ⎢⎢
∀U ⊂ V
⎥
2
⎣
⎦
e∈E (U )
xe ∈ {0,1} ∀e
∑x
e
Odd‐set
constraints
Integer Programming and Combinatorial Optimization
Hard combinatorial optimization problems have also hard IP formulations
Algorithmically well‐solved combinatorial problems have good IP formulations
Network problems are easy also as LP/IP models
Have other types of problems similar structures?
Can we formulate other problems as network problems?
Can we generalize what we learnt from networks to general MIP models?
Why network problems are so easy?
Polytopes and Integrality
Definition 1. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax ≤ b}. Definition 2. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u ∈ Rn such that
l ≤ x ≤ u; i.e. P = {x: Ax ≤ b, l ≤ x ≤ u for l, u ∈ Rn}. Definition 3. A nonempty polyhedron is integral if its vertices are integral.
Polyhedron
Polytope
P = {x: Ax = b}
Integral Polytope
Remarks 1) A set of inequalities can be converted into a set of equalities by introducing slack variables, and a set of equalities Ax = b can be converted into a set of inequalities ([A, ‐A]T x ≤ [b –b]T). 2) A polyhedron can be described by a set of linear equalities, P = {x: Ax = b }, and a polytope can be described by a set of linear equalities and bounding inequalities, P = {x: Ax = b, l ≤ x ≤ u}. 3) A popytope can also be described as the convex hull of a finite number of points. Conversely, the convex hull of a finite number of points is a polytope.
4) A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear
combination of points in P\{v}. A polytope P is the convex hull of its vertices.
Polytopes and Integrality
Theorem 1. The polyhedron P(A, b) = {x: Ax ≤ b, x ≥ 0} is integral for all integral vectors b∈Zm if and only if A is totally unimodular (Hoffman & Kruskal, 1956). Proposition 1. If A is TU and vectors b, b’, d and d’ are integral, then P(b, b’, d, d’) = {x∈: b’ ≤ Ax ≤ b, d’≤ x ≤ d} is integral if it is not empty. Definition 4. An integral matrix A is totally unimodular (TU) if the determinant of each square submatrix of A is equal to 0, 1, or ‐1.
Proposition 2. For every non‐singular submatrix R of a TU matrix, R‐1 is integral. Proposition 3. A {0, 1} matrix is an interval matrix (or has the consecutive 1’s property) if for all rows with alm = al’m = 1, l < l’, implies al”m = 1 for l < l” < l’. Interval matrices are TU (Fulkerson & Gross, 1965). Proposition 4. If a {0, ±1} matrix has no more than two non zero coefficients in each column, and Σl alm = 0 if column m has two coefficients, then A is TU (from Ghouila‐Houri, 1962). P = {x: Ax = b}
A totally unimodular
Proposition 3
Proposition 4
. . . Integral Polytope
Networks and Integrality
Proposition 4. If a {0, ±1} matrix has no more than two non zero coefficients in each column, and Σl alm = 0 if column m has two coefficients, then A is TU (from Ghouila‐Houri, 1962). Incidence matrix of networks satisfies this condition
Matrix A is used to express flow balance constraints in shortest‐path, max‐flow problems
1
2
4
3
(1,2) (2,1) (2,3) (3,4) (4,1) (4,2)
1 −1
1
1
⎤
⎡
2 ⎢ 1 −1
1 ⎥
−1
A=
⎥
⎢
1 −1
3 ⎢
⎥
1
1
1
−
−
⎦
⎣
4
Very few problems can be formulated using constraints that lead to TU matrices
Side constraints destroy the structure
Can we develop tighter formulations from combinatorial optimization insight?
If yes, can we add these constraints in our formulation immediately?
Outline
procurement
Short‐term
distribution
sales
Strategic Planning
Long‐term
Medium‐term
production
Master Planning
Purchasing
& Material
Production
Planning
Distribution
Planning
Requirements
Planning
Scheduling
Transport
Planning
Demand
Planning
Demand Fulfillment
As chemical engineers, we know the specifics of chemical production
Study all problems (time‐scales) of manufacturing function
Manufacturing Function
Strategic Planning
•
•
•
•
Network of plants/warehouses/markets
What new plants/warehouses to build?
What are the interconnections?
Investment/retrofit costs
Production Planning
•
•
•
•
•
Network of existing production facilities
Where/when is a product produced?
Production and inventory profile of products
Transportation/holding costs
Aggregate plant capacities
Pacific
Northwest
Rockies
Midwest
Northeast
Southwest
South
Manufacturing sites
Warehouses
Markets
Southeast
Production Profiles
Production
(tn/month) 1
8
6
4
2
0
0
2
4
6
Pr. A
8
Pr. B
10
12
Pr. C
Time
(months)
Scheduling
•
•
•
•
Single site/plant
How do we optimally use our resources?
Batch-sizes, storage policies, sequencing
Utility/changeover costs
Reactor 1
Reactor 2
Column
Heating
Reaction 2
Reaction 3
Reaction 1 Reaction 1
Separation
Time (days)
Scheduling
Allocation of limited resources to tasks over time
- Michael Pinedo
Scheduling has been practiced in manufacturing sectors since early 20th century
First scheduling publications in the early 1950
– Naval Research Logistics Quarterly
Closely related to the growth of computers and algorithms
– Research in 1970s – Complexity theory
Scheduling has widespread applications
–
–
–
–
–
–
–
Airlines industry
Shipping industry
Railways
Manufacturing/production/any Industry
Government / non-government Organizations
Educational/Research Institutions
Sports / Recreation
Purely multi-disciplinary
– Computer Scientists, Operations researchers, mathematicians, business science, etc.
Very challenging problem
– 8 unsolved problem instances in MIPLIB 2003
– Instance timtab2 (railway timetabling problem) has seen no optimal solution yet
10% gap even after several days, 113 Binaries, 381 Continuous
Preliminaries
Single-Stage Scheduling
N jobs to be processed in M machines
9 Assignment of jobs to machines
9 Sequencing of jobs in the same machine
pij , cij , sii
M1
task1
task3
task4
M2
Resources
(e.g. machines)
task2
task5
M3
task2 task4
M4
task1
task2
task4
M5
0
10
task5
task2
20
task3
30
40
Time (min/hr/day)
Classes of Scheduling Problems
α/β/γ
Processing characteristics
• Preemption
• Release/due times
• Setup times
Machine environment
• Single machine
• Parallel machines
• Machines in series
Objective
• Makespan
• Tardiness
• Cost
jobs
dA
A
B
M
B
D
A
C
C
D
Single machine
tardiness
Sequence-dependent
Setup time
jobs
A
B
C
M1
M1
M2
M2
A
B
C
D
D
Parallel machines
makespan
Chemical Production Scheduling
• Jobs ⇒ Batches (sometimes orders)
• Operations ⇒ Stages or Tasks
• Machines ⇒ Units
• Modeling of storage is important (solid, liquid and gas phases)
⇒ Storage vessels & states
• Utilities are also important (steam, water, electricity)
⇒ Utilities (or resources)
• No Preemption
• Variable Processing Times
• Changeover times
Classification of Chemical Production Scheduling Problems
Multi-stage processes
Orders
(batches)
1st stage
J1
2nd stage
J2
...
...
Sequential structure:
Kth stage
JK
...
Finished
Orders
...
Orders := batches i∈I
Stages k∈K = {1,2, …K}
Units j∈J = J1∪J2∪ …JK
Special Cases:
Single-stage:
Flow-shop:
K=1
|J1|= …=|JK|=1
Network (multi-purpose) processes
Batch splitting/mixing; recycle streams
Utility requirements
F1
S1
Int1
S3
ADD
5%
F2
95%
P2
S4
S5
P1
10%
Int2
90%
50%
S6
50%
P3
⇒ Links between seemingly independent tasks
⇒ Need to account for variable no of batches via common time representation
Batchbased Approaches
Goal:
Exploit sequential structure of multi‐stage processes
Approach: Predefine number of batches (lot‐sizing, batching, task selection)
Solve fixed scheduling problem
Scheduling
Batching
Demand (orders)
A
Batches
A
B
B
C
C
. . .
Batches i∈I
Stages k∈K
Units j∈J
Scheduling Formulations
Slotbased
Precedencebased
Pinto & Grossmann, 1995; Lamba & Karimi, 2002
Xij = 1 if task i is assigned to unit j
Yii’j = 1 if batch i precedes batch i’ in unit j
Tik = Finish time of batch i in stage kv
Xij
Tasks (i)
Hui & Gupta, 2000; Mendez et al. 2001
Yijm = 1 if batch i is assigned to slot m of j
Tik = Finish time of slot m of unit j
Ttik = Transfer time k → k+1
Yii’j
Units (j)
Xij → Yii’j
Tasks (i)
Networkbased Approaches
Goal:
Model multi-purpose batch processes
Account for batch splitting/mixing; recycle streams; utility requirements
Approach: Use a common time coordinate
Express assignment and balance constraints for every time point/period/event
Discrete-time Formulations
Kondili et al. (1993); Pantelides (1994)
RM
Int
T1
T2
A
T3
B
Time points/periods t∈T
2.0 hr
Δt = 0.5 hr
1.5 hr
3.0 hr
Wijt = 1 if task i starts in unit j at time point t
Bijt = Batch-size of task i in unit j at time point t
Sst = Amount of state s at time point t
Uut = Consumption of utility u at time t
T1
T2
T3
0
1
2
Assignment:
3
4
∑ ∑
i∈I j
5
t − pi +1
tˆ =t
6
7
8
Wijtˆ ≤ 1 ∀j , t
t (hr)
WijtVijMIN ≤ Bijt ≤ Wijt VijMAX
∀i,t , j ∈ K i
Mass Balance: STst = STst −1 + ∑i∈T ρ is ∑ j∈K Bij ,t − pis − ∑i∈T ρ is ∑ j∈K Bijt ≤ S sMAX
Utility:
U ut = ∑t ∑ j∈K
s
∑θ
i
pi −1
i
=0
Tasks i∈I
Units j∈J
States s∈S
Utilities u∈U
s
(α uiθ Wijt −θ + β uiθ Bijt −θ ) ≤ U uMAX
i
∀u, t
∀s, t
Variables defined ∀t∈T
Constraints expressed ∀t∈T
Large MIP models
Constant processing times
Modeling Problems 1
1. Singe-unit problem
Find the schedule that yields the minimum total completion time (makespan)
A
Batches i∈I = {1, 2, …, N}
Processing time: τi
Release/due times: ri/di
B
M
C
D
1.1. Single-unit with release/due times
Same as 1, but batches have release/due dates.
The objective is the minimization of makespan
1.2. Single-unit and changeover times
Same as 1 but there are changeover times between batches
The objective is the minimization of makespan
1.3. Single-unit and changeover costs
Same as 1.2 but there are changeover costs between batches
The objective is the minimization of cost
Modeling Problems 2
2. Singe-stage (multi-unit) problem
N batches have to be processed in a single stage that consists of M parallel units.
A
Batches i∈I = {1, 2, …, N}
Units j∈J = {1, 2, … , M}
Processing time:
τij
Processing cost:
cij
B
C
M1
M2
D
2.1. Single-stage problem with release/due times
Same as 2, but batches have release/due dates
The objective is the minimization of lateness/tardiness
2.2. Single-stage problem with changeover costs and times
Same as 2.1, but there are changeover costs γii’j and changeover times δii’j between batches
The objective is the minimization of cost (subject to meeting the release/due times)
Modeling Problems 3
3. Multi-stage problem
N batches have to be processed in K stages; each stage has parallel units.
Batches i∈I = {1, 2, …, N}
Units j∈J = {1, 2, … , M}
Stages k∈K; Jk is the set of units in stage k
Processing time:
τij
Processing cost:
cij
jobs
A
B
M1
M2
M3
C
D
3.1. Multi-stage problem with release/due times and changeover times/costs
Same as 3, but batches have release/due dates and changeovers
Model the minimization of makespan, tardiness, cost and earliness
Modeling Problems 4
S1
1h
Reaction1
1h
Heat
30%
2h
90%
Separation
40%
3h
70%
Reaction2
S2
10%
S3
60%
S5
S6
2h
Reaction 3
States
S4
Tasks
S7
Units
Reaction 2
Heating
Reactor 1
Reaction 3
Reactor 2
Reaction 1
Column
Separation
Reaction 1
Time (h)
Inventory
S2
S3
0
1
2
3
4
5
6
Time (h)
Scheduling: Modeling
Discrete‐time Formulations
T1
T2
T3
0
1
2
3
4
5
6
7
8 t (hr)
Fixed time points ‐ Constant processing times
Continuous‐time Formulations
Global grid: Pantelides et al.; Reklaitis et al.; Lee et al.; Barbosa‐Póvoa et al.; Grossmann et al.; Karimi et al.
Unit‐specific grid (events): Floudas et al.; Ierapetritou et al.; Giannelos & Georgiadis
T1
T2
T3
1
2
3
4
5
6
7
(periods)
Variable time points ‐ Variable processing times
Mixed‐time Formulation
Maravelias (2006)
T1
T2
T3
0
1
2
3
4
5
6
7
8 t (hr)
Fixed time points ‐ Variable processing times
Scheduling: Modeling
Discrete‐time Formulations
T1
T2
T3
0
1
2
3
4
5
6
7
8 t (hr)
Fixed time points ‐ Constant processing times
Continuous‐time Formulations
Global grid: Pantelides et al.; Reklaitis et al.; Lee et al.; Barbosa‐Póvoa et al.; Grossmann et al.; Karimi et al.
Unit‐specific grid (events): Floudas et al.; Ierapetritou et al.; Giannelos & Georgiadis
T1
T2
T3
1
2
3
4
5
6
7
(periods)
Variable time points ‐ Variable processing times
Mixed‐time Formulation
Maravelias (2006)
T1
T2
T3
0
1
2
3
4
5
6
7
8 t (hr)
Fixed time points ‐ Variable processing times
Are Smaller Formulations Better?
RMC
RMAB
I1
9
9
9
9
9
I2
ABI1
CI1
ABI2
INTABI
CI2
INTCI
II1
BII1
CII1
INTBII
Continuous processing
Five products - three stages
Time Horizon: 2 weeks
Intermediate due dates, shipment dates
Backlogged demand
INTCII
Demand
Shipments
Backlogs
III2
III1
AIII1
BIII1
A
CIII1
B1
BIII2
C1
CIII2
B2
C2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Equipment Unit Gantt Chart
I1
CI1
ABI1
I2
ABI2
II1
CII1
III1
0
2
BII1
AIII1
BIII2
1
ABI2
CII1
BIII1
III2 CIII2
AB
CI2
BII1
AIII1
CI1
3
4
5
CII1
BIII1
CIII2
6
8
B
CIII1
BIII2
7
A
AIII1
C
CIII2
9
10
11
12
13
Idle
1 4 t (d)
Effect of Strong Valid Inequalities
(M0)
Basic STN
Formulation
(M0)
(C1)
(C2)
(C3)
(C4)
(C5)
(C6)
# Const.
LP-relax
Objective
CPU-sec
LP-gap (%)
Nodes
1,907
2,182.8
1,766.7
62.2
23.6
5,689
(M1)
+C1
2,043
2,054.4
1,766.7
12.5
16.3
622
(M2)
+C2
2,043
2,166.7
1,766.7
12.1
22.6
1,402
(M3)
+C3
1,907
2,173.1
1,766.7
15.3
23.0
1,183
(M4)
+C4
1,907
2,130.5
1,766.7
10.0
20.6
758
(M5)
+C5
1,975
2,176.6
1,766.7
46.2
23.2
3,210
(M6)
+C6
2,043
2,060.3
1,766.7
9.5
16.6
1,297
Inequalities (C1), (C2), (C3), (C4) and (C6) appear to be the most effective
Combinations of these inequalities are studied further
# Const.
LP-relax
Objective
CPU-sec
LP-gap (%)
Nodes
(M10)
(M11)
(M12)
+ C1,
+ C1,
+ C1,
C2, C6 C2, C3 C2, C4
2,315
2,179
2,179
2,037.3 1,964.0 2,003.7
1,766.7 1,766.7 1,766.7
3.8
4.9
2.9
15.3
11.2
13.4
275
277
176
(M13)
+C1-4
2,247
1964.0
1,766.7
2.6
11.2
88
(M14)
+ C3,
C4, C6
2,111
2,046.6
1,766.7
8.1
15.8
646
(M16)
+C1-4,
C6
2,383
1,964.0
1,760.0
3.8
11.2
150
Models (M11), (M12), (M13) & (M16) are studied further using a set of six
problems
Model (M16) appears to be the most effective
BranchandCut Algorithm: Basic Idea
Additional improvements needed for practical applications
How can we use remaining valid inequalities?
⇒ Branch‐and‐cut algorithm
Advantages:
1. Do not load unnecessary constraints
⇒ Smaller incidence matrix
⇒ Each node is solved faster 2. Use a larger pool of constraints
Tighter formulation after a few rounds of cut generation
Combines effective (LP) solution and tight formulation
Disadvantage:
Very difficult to compete with CPLEX, XPRESS
Do not exploit built‐in cut generation routines
Have to write new algorithm to add user cuts Branch on one fractional variable
Add violated cuts ⇒ Find new (fractional) solution
Pruned node
Open node
Branchandcut Algorithm: Implementation

Implementation in XPRESSIVE using XPRESSMosel
Do not remove/change pre‐solve and cut‐generation routines
Add few cuts (C2, C3 & C5) at the root node to facilitate heuristics
Add violated high priority cuts (C1, C4, C6 & C7) at the root node Allow addition of high priority cuts at nodes of intermediate depth
Add low priority cuts (C8‐C13) later
Add all C2, C3 & C5 constraints in the formulation
Add violated C1, C4, C6 and C7 constraints in k1 rounds at the root node
Add violated constraints C1, C4, C6 and C7
in nodes of depth k| k2≤k≤k3
Add violated constraints C1, C4, C6, C7 and C8C11
in nodes of depth k| k3≤k≤k4
Branchandcut Algorithm: General Remarks
Remarks:
1. Parameters k1, k2, k3 and k4 depend on the size of the problem
Typical values:
5‐10 iterations at the root node Start adding high priority cuts after 200‐1000 nodes, at depth 20‐200
Start low priority cuts at depth 100‐1,000
2. Algorithm requires fewer number of nodes, but is expensive at each node
To reduce overhead: Add cuts every 5‐10 nodes
Add first 2‐4 violated cuts
3. On the fly tuning necessary
Customized Methods
Precedencebased
Pinto & Grossmann, 1995; Lamba & Karimi, 2002
Yii’j = 1 if batch i precedes batch i’ in unit j
Tik = Finish time of batch i in stage kv
Xij
Tasks (i)
Yii’j
Units (j)
What is the problem?
Sequence-based:
Ti 'k ≥ Tik +
∑τ
ij
j∈JA ( i , k )
X ij − M (1 − Yii 'k ) ∀(i, i ') ∈ IL, k ∈ K
Slot-based:
Tti ,k −1 ≤ Ts jm + M (1 − U ijm )
∀i, k > 1, j ∈ J k , m ≤ M j
Ttik ≥ Ts jm + τ ijU ijm − M (1 − U ijm ) ∀i, k < K , j ∈ J k , m ≤ M j
Sequencing is achieved via big-M constraints
⇒ Resulting MILP formulations are not strong
Compared to manufacturing problems:
1. Unknown type and number of batches (tasks); unknown assignments of tasks to units
2. Mixing of intermediates; variable batch-size and processing time
There are good algorithms for problems with fixed type and number of tasks and fixed assignments
Can we combine these algorithms with MILP methods?
Decompose problem in two subproblems
1. Determine type and number of tasks and assignments of units to tasks
2. Solve reduced problem with an efficient, problem-specific algorithm
Constraint Programming Preliminaries
New Modeling and Solution Paradigm
Developed in early 90’s in AI
Very effective for classes of optimization problems
• Highly constrained (feasibility) problems
• Some scheduling problems
Special “constructs” and constraints for classes of problems
Constructs: activity X, unary resource Y
Constraints: X requires Y (GLOBAL)
A → B, A ∨ B (LOGIC)
⇒ Highly Expressive
⇒ Effective local search Search is based on constraint propagation
Constraint Programming – Basic Idea
Problem
Find the order at which jobs A, B and C are sequenced on machine M
Job
A
B
C
Processing Time
2
4
3
Release
0
1
2
Due
5
9
7
Problem
Job
A
B
C
Processing Time
2
4
3
A∈{1,2,3}
B∈{1,2,3}
C∈{1,2,3}
B→ 1
A→ 1
B→ 2
C→ 1
C→ 3
C→ 2
Release
0
1
2
C→ 3
A→B→C
B→ 3
Due
5
9
7
A→ 2
B→ 1
B→ 2
A→ 3
B→ 3
B→ 1
B→ 2
B→ 3
C→ 2
A→C→B
Problem
Job
A
B
C
A∈{1,2,3}
B∈{1,2,3}
C∈{1,2,3}
B→ 2
Processing Time
2
4
3
Release
0
1
2
Due
5
9
7
A must be 1st
C must be 2nd
A→ 1
B→ 3
C→ 3 C→ 2
A→B→C A→C→B
Mathematical vs. Constraint Programming
Mathematical Programming
Intelligent search strategy but computationally expensive for large problems
Computationally effective for optimization problems with many feasible solutions
Not effective for feasibility problems and machine sequencing problems
Constraint Programming
Fast algorithms for special problems
Computationally effective for highly constrained, feasibility and machine sequencing problems
Not effective for optimization problems with complex structure and many feasible solutions
MAIN IDEA
Decompose problem into two parts
9 Use MP for high‐level optimization decisions
9 Use CP for low‐level sequencing decisions
Decomposition Strategy
• Express problem in an aggregated MP form
Solve MIP Master Problem
• Use MP to identify potentially good solutions
max production
s.t. RELAXATION
• Fix no/type of tasks, assignment of tasks to units
Obtain UB
Fix no/type of tasks,
assignment to units
Add integer cuts
• Fix no/type of tasks and assignment decisions
Solve CP Subproblem
• Problem is highly constrained: suitable for CP
max production
s.t. ALL CONSTRAINTS
w/ fixed no/type of tasks
• If feasible, obtain lower bound
• Add integer cut and continue until bounds converge
Obtain LB
Production
Z*
×
×
0
2
4
×
×
Upper bound
× Feasible solution
Iterations
6
8
10
Example
Food (5)
Flakes2
T10
Flour
Coater (2)
T20
T30
Packer (3)
Flakes1
Loops
T21
T31
Coated Flakes
Coated Loops
Corn
Flakes
T12
T22
T32
Frosted
Flakes
Unlimited Storage
Froot
Loops
Zero Wait
Finite Storage
Maximize Production
Horizon
15
20
25
n
10
12
15
0-1
180
216
270
MILP STN Model
Cont. Constr. LP Relax Obj
1,163 2,473
14
12
1395
2965
18
16*
1743
3703
24
22
Hybrid MILP/CP
CPU s # iters Obj CPU s
58.5
3
12
0.32
16,749.2
3
16
0.37
523.6
3
22
1.05
Minimize Makespan
Demand
4/5/6
5/6/8
5 / 8 / 10
n
12
13
15
0-1
216
234
270
MILP STN Model
Cont. Constr. LP Relax Obj
1393
2938
12
19
1509
3220
15
1741
3712
18
-
Hybrid MILP/CP
CPU s # iters Obj CPU s
32,077
1
19
0.23
>36,000
1
23
21.02
>36,000
1
27
0.64
General Hybrid Algorithm
∑ ∑D
Multipurpose Batch Plant
10%
S3
Separation
Reaction1
Heat
S1
i∈I ( j ) c
90%
Reaction2
70%
30%
60%
S5
Reaction 3
F2
F3
T10
T20
T30
S10
S20
S30
T11
T21
T31
S11
S21
S31
S7
BiMIN Z ic ≤ Bic ≤ BiMAX Z ic
S s = S 0 + ∑ ∑ ρ is Bic − ∑ ∑ ρ is Bic
T12
T22
T32
P1
P2
P3
I
c
i
Ss ≥ ds
∀s ∈ FP
S s ≤ Cs
∀s ∈ INT
∑ ∑D
i∈I ( j ) c
ic
c
∀s ∈ FP
i
∀j
∀i, ∀c
S s = S 0 + ∑∑1⋅ Bic − ∑∑1⋅ Bic
i
∀s
c
Z ic ≤ MS − ST j − EST j
BiMIN Z ic ≤ Bic ≤ BiMAX Z ic
∀j
∀i, ∀c
O
Ss ≥ ds
Single-stage
¾ Drop stage index k
Z ic ≤ MS − ST j − EST j
i
S6
Multi-stage: no utilities
unlimited storage
F1
S4
40%
S2
ic
∀s
c
X oj ≤ max o∈O {d o }− mino∈O {ro } ∀j
∑Z
= 1 ∀o
oj
oj
O
∑ ∑ Bics ≥ d s
i
j
Preprocessing: Better Cover Cuts
∀s ∈ FP
c
Task[i,c] requires Unit[j]
∀j,∀i∈I(j),∀c
Task[i,c] requires Ric Utility[r] ∀i,∀c
Task[i,c] consumes ρcisBis State[s] ∀i,∀c,∀s
Task[i,c] produces ρpisBics State[s] ∀i,∀c,∀s
Task[i,c].end ≤ MS
∀i,∀c
BiMIN ≤ Bic ≤ BiMAX
∑ ∑B
i∈O ( s ) c
∑D
o∈O ( j )
BiMIN ≤ Bic ≤ BiMAX ∀i, ∀c
Ric = α i + β i Bic ∀i, ∀c
is
≥ ds
∀i, ∀c
∀s ∈ FP
Task[i,c] requires Unit[j]
∀j,∀i∈I(j),∀c
Task[i,c] precedes Task[i’,c] ∀(i,i’)∈A,∀c
∀i,∀c
Task[i,c].end ≤ MS
Task[o,j].start ≥ ro ∀o,∀j
Task[o,,j].end ≤ do ∀o,∀j
Task[o,j] requires Unit[j] ∀o,∀k,∀j|Zok=1
∀o,∀k,∀j|Zok=1
Task[o,k,j].end ≤ MS
Integration with Problemspecific Algorithms
Example: Minimization of cost of multi-stage problem for orders with release and due times
N orders have to be processed sequentially in K stages, where each stage consists of Mk units.
Each order i has release ri and due di time, and a processing cost cij and processing time τij.
The objective is to minimize the sum of processing costs subject to meeting the release and due times.
Master Problem: Assignment
Subproblem:
Sequencing
Subproblem is a traditional OR problem (job-shop problem) ⇒ There are efficient algorithms
Use Shifting Bottleneck Procedure (Adams and Balas, 1988) to solve the subproblem
Computational Results
Obj
P2D1
P2D2
P3D1
P3D2
P4D1
P4D2
P5D1
P5D2
153
188
56
1113
149
946
111
704
MIP
CPU s
4.4
0.8
446.6
0.4
27.3
1.4
4041.7
2767.8
CP
CPU s
0.1
0.3
4.2
1375.0
447.2
359.0
293.0
712.6
MIP/CP: CUT H
Iter’s/Cuts
CPU s
16/19
0.7
4/3
0.1
29/45
8.7
5/5
0.4
17/22
28.8
20/26
11.0
43/56
318.4
2/2
13.7
MIP/SBP
Iter’s/Cuts
CPU s
4/4
1.3
1/0
0.2
26/32
16.9
2/2
1.1
12/15
7.8
6/14
9.8
19/25
17.2
2/1
0.4
Exploit Parallel Computing

Computer manufacturers focus on multi‐core architectures
Computational resources become cheap
Algorithms that utilize new architectures or exploit parallelization necessary
Modeling languages provide tools for parallel algorithm development
Parallel
Processing
Multiple machines
Grid
Computing
Parallel
Computing
Distributed
Computing
Multiple CPUs on a
machine
Cluster
Computing
Multiple machines
located close by
Multiple machines
Parallel Computing & Integer Programming
LP‐based branch‐and‐bound has inherent structure for parallelization
Subproblems associated with each node are independent
Evaluate nodes simultaneously; share cuts and bounds
Nodes solved at the master computer
Information flow: master → slave
• Subproblem formulation
• Variable bounds
Information flow: slave → master
• New feasible solutions
• Current lower bound
• Global cutting planes
×
Fundamental issues for parallelizing branch‐and‐cut
Generate good subproblems: not too easy or too hard
Minimize subproblem (task) and cut generation time
Maximize useful sharing: pass bound information relevant cuts
How Can We Use Grid Computing?
Master-worker Paradigm
Master
Master generates and spawns all tasks (subproblems)
tasks can be user-defined or generated automatically
Workers execute tasks and return results
Master collects results
Subproblems
1. Independent (e.g. scenarios in stochastic programming)
Workers
Submission Loop:
modelname.solvelink=3;
loop(scenario,
parameter1 = spara1(scenario); parameter2 = spara2(scenario);
solve modelname using mip minimizing objective;
h(scenario) = modelname.handle;
);
2. Solver generated
Subtrees resulting from strong branching
3. User generated
Manually fix binary variables
(Bussieck et al., 2007)
Optimization on the Grid
1. Master creates a separate directory for each subproblem
Unique handle for each subproblem
“finished” file when task completed
“trigger” file when new incumbent
2. Condor ships the subproblems to workers; executed on a Sandbox
3. Master collects results using handlecollect
Repeatedly checks for completion; collection time can be adjusted
4. Update on incumbent solution using utility condor_chirp
If new global incumbent is found, trigger file is created
It also puts the new incumbent to the subproblem’s directory in Master computer.
Master
“finished” file upon
completion of subproblem
…
“trigger file” is created if
new incumbent is found
C
on
do
r
Separate directory
for each subproblem
Worker 1
Worker 2
Worker N
Sandbox
Sandbox
Sandbox
condor_chirp utility
Fetch: copies trigger file
Remove: removes trigger file after copying
Put: places new incumbent in directory of master
Dynamic Decomposition
1. Run branch-and-bound on master to generate 400 open nodes
2. Spawn open nodes to servers
3. Communicate bounds and incumbents every 5 min; wait for 1 hr
Integer solution
Pruned node
Fractional (open) node
Dynamic Decomposition
1.
2.
3.
4.
Run branch-and-bound on master to generate 200 open nodes
Spawn open nodes to servers
Communicate bounds and incumbents; wait for 1 hr
Decompose hard problems dynamically again using strong branching
Integer solution
Pruned node
Fractional (open) node
Results 1
Example
10 orders, 3 stages, 2 units/stage; dissimilar units, forbidden assignments and paths Two instances parameterized via demand
Instance 1: ~20 batches
Strong Branching to generate 400 subproblems
13 hr of CPU time, 2hr of wall clock time
2,905,742 nodes
Strong Branching generated too many subproblems
Filled the disk of the submitting machine!
Failed in solving this hard problem
A better scheme is needed to generate subproblems
Exploit Problem Structure
Hierarchy of decisions:
∑X
Zil = 1 if batch l is selected towards order i
j∈JA ( i , k )
X(i,l)j = 1 if batch (i,l) is assigned to unit j
Y(i,l)(i’,l’)k = 1 if batch (i,l) preecedes batch (i’,l’) in stage k
ilj
= Z il
∀i ∈ I , l ≤ limax , k ∈ K
X ilj + X i 'lj − 1 ≤ Yili 'l 'k + Yi 'l 'ilk
∀(i, l , i ' l ') ∈ IL, k , j
Batch
Selection
Zil
Batch assignment
Xilj
Batch sequencing
Yili′l′k
Jobshop
problem
Traditional
Multi-stage
scheduling
Domainbased Dynamic Decomposition
Master computer
Decomposition generates subproblems
P
Fix Zil
Subproblems solved in workers
Collect and updates are used
P1
P2
P3
PM1
P4
Π1 = {P1, P2,…. PM}
Master collects unsolved problems;
2nd-level decomposition by fixing Xilj
in one stage
P12
P22
PM2 2(2)
P32
Π2 = U Π 2m , m = 1,2,….M
P14
m
Fix Xilj
Promising
where Π 2m = {P1m , P2m , P3m ,.....}
Non-promising
Process repeated as necessary
4
P1,1
4
P1,3
4
P1,2
4
P1,M3(4)
Π3 = U Π 3sm , m = 1,2,….M
m,s
Promising
Non-promising
m
m
m
where, Π3sm = {Ps,1
, Ps,2
, Ps,3
,.....}
Few 1st‐level subproblems are promising (and hard); reasonable batching
2nd‐level decomposition can potentially generate too many subproblems
Process can be combined with strong branching
Results 2
Example
10 orders, 3 stages, 2 units/stage; dissimilar units, forbidden assignments and paths Two instances parameterized via demand
Decomposition with strong branching
13 hr of CPU time, 2hr of wall clock time, 2,905,742 nodes
1st‐level decomposition followed by strong branching
7.5 min of wall clock time; 9,601 nodes
Decomposition with strong branching
Failed to solve the problem
1st‐, 2nd‐, 3rd‐ level decomposition (Zil + 2 stages) followed by strong branching
6 2nd‐level unsolved problems → 56,628 3rd‐level subproblems
26 3rd‐level unsolved subproblems → 31,761 subproblems
17.7 days of CPU time; 9 hr of wall clock time; 222,065,793 nodes
Outline
procurement
Short‐term
distribution
sales
Strategic Planning
Long‐term
Medium‐term
production
Master Planning
Purchasing
& Material
Production
Planning
Distribution
Planning
Requirements
Planning
Scheduling
Transport
Planning
Demand
Planning
Demand Fulfillment
Integration of production planning and scheduling Exxon Mobil: 2% reduction in operating costs, 20% inventory reduction (Shobrys & White, 2002)
DuPont: Capital tied up in inventory reduced from $165 to $90 million (Shobrys & White, 2002)
Problem Statement
Given:
Planning horizon divided into T uniform or non‐uniform time periods t∈{1, 2, … T}.
Products (items), i∈I = {A, B, … } with demand δit and holding cost θi.
Resource constraints: unit and utility capacities, raw material availability, etc.
Production costs:
these may include variable and fixed costs.
Determine
Production Pit and inventory levels Iit to minimize cost
Manufacturing Facility
I i ,0
Pi ,1
δi ,1
P1
P1
P1
P2
P2
P2
I i ,1
Pi , 2
I i,2
Pit
I i ,t −1
δi , 2
I it
δit
Warehouse
Distribution Center
Production planning provides production targets (set points) for each period
t=0
P1 P2
P1 P2
1
2
P1 P2
t-1
P1 P2
t
Why Integration with Scheduling?
150 ≤ P2 ≤ 200 (kg/d)
RM
200 ≤ P1 ≤ 300 (kg/d)
U1
INT
(ZW)
U2
P3 = 150 (kg/d)
U3
PB
PB
2,000
2,000
AG: PA+PB ≤ 2,100
A
U1: PA+PB ≤ 2,100
B
PB
2,000
U3: PB ≤ 1,050
1,000
1,000
1,000
2,000 PA
(a) Bottleneck constraint
U2: PA ≤ 1,400
1,000
1,000
2,000 PA
(b) Unit capacity constraints 1,000
2,000 PA
(c) True feasible region
(via scheduling)
Bottleneck and unit capacity (aggregate) constraints insufficient
Chemical processes: batch splitting/mixing, recycle streams, utilities, storage constraints
Scheduling models needed to provide feasibility and cost information
4
General Production Planning Formulation
z = min ∑t (Cpt + Cht )
P1
I it = I i ,t −1 + Pit − δit , ∀i, t
P2
Cht = ∑i θi I it , ∀t
f (Pit , i ∈ I ) ≤ 0, ∀t
Cpt = g (Pit , i ∈ I ), ∀t
How much can we produce
and at what cost?
Satisfy demand?
I it
At what (inventory) cost?
Pit
I i ,t −1
Hidden
scheduling problem
δit
Pit , I it ≥ 0
Scheduling
Given:
A production facility (processing units, storage vessels, utilities) & processing cost (feedstocks, utilities, etc)
Production recipes (stoichiometric coefficients, processing times, utility requirements)
Production targets Determine the optimal:
Number and type of processing tasks (batches and lots)
Allocation of resources to processing tasks
Sequencing of tasks on processing units
Batching
Demand (orders)
A
B
C
D
E
Batch‐unit Assignment
Batches
A1 A2 A3
A3
B1 B2
B1
C1
B2
D1 D2
C1
E1
A2
A1
Sequencing & Timing
U1
A1
A2
A3
C1
U2
D1
D2
E1
D1
D2
B1
B2
E1
PlanningScheduling Integration
z = min ∑t (Cpt + Cht )
I it = I i ,t −1 + Pit − δit , ∀i, t
Linking
Pkt = S kn , ∀k ∈ K FP , ∀t , n = N
n '≤ n
∑ ∑W
k∈K j n '> n − τ i
kn '
Integrated
Formulation
≤ 1 ∀j , ∀n
βkMINWkn ≤ Bkn ≤ βkMAX Wkn
∀k , ∀n
S in = S i , n −1 + ∑ ρik Bk ,n − τi + ∑ ρik Bkn ≤ ζ iMAX
k∈Pi
(
∀k , ∀n
Scheduling
k∈Ci
Cpt = ∑∑ γkWkn + εk Bkn
k
Optimization
Planning
)
n
Pkt , I kt ≥ 0, Win ∈ {0,1}, Bin , S kt ≥ 0
Planning time grid
(big-bucket Ht)
Weeks-months
Scheduling time grid
(small-bucket hn)
Hours-days
t=0
1
H1
n=0
1
2
…
hn
2
Pi ,1
Pi , 2
Si , N
Si , 2 N
N N+1
…
2N
Planning
Scheduling
Challenges in Chemical Production Planning
Modeling
Different production environments Wide range of processing constraints
Integration with multiple planning functions
⇒ Develop general modeling framework
Theory
Discrete decisions lead to mixed‐integer programming models
Existing hardware and software insufficient
⇒ Develop fundamental optimization theory
⇒ Use underlying theory to develop better solution methods
Solution Methods
Applications: optimize over long horizon
optimize multiple systems simultaneously
Intractable large‐scale models
⇒ Develop general & computationally effective methods
Projectionbased strategy
The Modeling Challenge
Batch processing
No batch mixing
Cleaning‐in‐place
between batches
Cooling utilities
Yeast requirements
Release time for media
Continuous
processing
Semi‐continuous
processing
Litro
500 BPM
FERMENTACIÓN
¼ STD
700 BPM
D. W.
DILUTION
TANQUES
DE
GOBIERNO I
½ STD
600 BPM
½ PREMIER
600 BPM
TANQUES
DE
GOBIERNO II
½ T.A., ¼ BOH.
750 BPM
Products grouped in families
Changeover times & costs between families
Orders at due dates
Inventory constraints
Unit‐product compatibility
Storage constraints
Litro
350 BPM
REPOSO
LATA
1200 LPM
TANQUES DE GOBIERNO
½ T.A EXP
750 BPM
TANQUE REPOSO
Fermentation
Filtering
Storage
Bottling
Modeling Schools of Thought
Sequential processing
Orders
(batches)
1st stage
J1
2nd stage
J2
...
...
Kth stage
JK
...
Operations follow a defined sequence
The same batch/lot processed in all stages
Finished
Orders
...
No batch mixing (blending) or splitting allowed
Common in biotech, specialty chemical, and pharmaceutical industries
Batchbased methods
Network processing
40%
RM1
40%
A
60%
Int3
ImB
60%
Int2
10%
Batch splitting/mixing allowed
Recycle streams, utility requirements
80%
90%
RM2
B
20%
Common in bulk chemicals & oil industries
Materialbased methods
Int1
RM3
Implications of Modeling Diversity
Network
R1
M
Feed1
Feed 3
Material‐based
60%
T1
40%
Separator
U6/U7
S3
70%
30%
T2
P3
(5 orders)
S2
P4
Intermediate 2
Separator
U6/U7
(7 orders)
Batch‐based
S4
S5
U4
U6
S6
U5
U7
Stage 1
Stage 2
{U2, U3}
{U2, U3}
S2
T3
S1
Intermediate 1
Reactor
U4/U5
Reactor U3
{U1}
Reactor
U4/U5
P2
Feed 2
S1
S1
R4
R2
40%
Mixer U1
Traditional
process
representation
P1
Reactor U2
intermediate
Sequential
70%
30%
60%
R3
{U1}
60% T1
S4
S3
30%
70%
S5
T2
{U4,U5}
T4
S7
{U6,U7}
S8
Network task
Network state
Hybrid state
S10
Sequential task
Sequential state
T5
{U2, U3}
40%
S2
{U2, U3}
T3
Sundaramoorthy & Maravelias, 2011
S6
{U4,U5}
T6
S9
{U6,U7}
T7
11
Industrial Application
Processing Stages: Fermentation → Filtering → Storage → Bottling
Production Type: Batch, continuous, and semi‐continuous
Planning Horizon: 6 weeks
Production Environment: Fermentation
Bottling
Filtering
Storage
Litro
500 BPM
FERMENTACIÓN
¼ STD
700 BPM
D. W.
DILUTION
TANQUES
DE
GOBIERNO I
½ STD
600 BPM
½ PREMIER
600 BPM
TANQUES
DE
GOBIERNO II
½ T.A., ¼ BOH.
750 BPM
Litro
350 BPM
REPOSO
8 processing units (lines)
22 products families
25 product subfamilies
162 products
LATA
1200 LPM
TANQUES DE GOBIERNO
½ T.A EXP
750 BPM
TANQUE REPOSO
Product families: Products (for bottling) are grouped into families
Changeover costs/times between families; setup costs/times between products
Products belong to subfamilies
major changeover
Product family
Product (item)
Product family
minor changeover
(sequence independent)
Kopanos et al.,, 2011
Industrial Application: Executed Schedule
Industrial Application: Executed Schedule
Industrial Application: Integrated Appoach
No “memory” across planning periods
Industrial Application: Integrated Approach
No “memory” across planning periods
Minimize setup time & costs
Industrial Application: Results
Executed Schedule
Schedule Found Using Integrated Framework
Comparison with Implemented Solution
Modeling
Litro
500 BPM
FERMENTACIÓN
¼ STD
700 BPM
D. W.
DILUTION
Different production environments TANQUES
DE
GOBIERNO I
½ STD
600 BPM
½ PREMIER
600 BPM
Wide range of processing constraints
TANQUES
DE
GOBIERNO II
½ T.A., ¼ BOH.
750 BPM
Litro
350 BPM
REPOSO
LATA
1200 LPM
TANQUES DE GOBIERNO
½ T.A EXP
TANQUE REPOSO
750 BPM
Theory
Solution Methods
Polyhedral Background Summary
Definition 1. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax ≤ b}. Definition 2. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u ∈ Rn: l ≤ x ≤ u; i.e. P = {x: Ax ≤ b, l ≤ x ≤ u for l, u ∈ Rn}. Remark 1: A set of inequalities Ax ≤ b can be converted into a set of equalities Ax + Is = b, s ≥ 0. A polytope can be described by as P = {x: Ax = b, l ≤ x ≤ u}. Remark 2: A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear combination of points in P\{v}. A polytope P is the convex hull of its vertices.
Definition 3. A nonempty polyhedron is integral if its vertices are integral.
Definition 4. An integral matrix A is totally unimodular (TU) if the determinant of each square submatrix of A is equal to 0, 1, or ‐1.
Theorem 1. If A is TU and vectors b, b’, d and d’ are integral, then P(b, b’, d, d’) = {x∈: b’ ≤ Ax ≤ b, d’≤ x ≤ d} is integral if it is not empty (Hoffman & Kruskal, 1956). Theorem 2. If a {0, ±1} matrix has no more than two non zero coefficients in each column, and Σl alm = 0 if column m has two coefficients, then A is TU (Ghouila‐Houri, 1962). Theorem 3. A {0, 1} matrix is an interval matrix (or has the consecutive 1’s property) if for all rows with alm = al’m = 1, l < l’, implies al”m = 1 for l < l” < l’. Interval matrices are TU (Fulkerson & Gross, 1965). Theorem 1
A totally unimodular
Polyhedron
P = {x: Ax = b}
Polytope
P = {x: Ax = b, l ≤ x ≤ u}
Integral Polytope
P = {x: Ax = b, l ≤ x ≤ u}
Theorem 2
Theorem 3
. . .
MIP Formulations
Polyhedral Background Summary
Definition 1. A polyhedron P is the solution set of a finite system of linear inequalities, i.e. P = {x: Ax ≤ b}. Definition 2. A polyhedron P is a polytope (i.e. a bounded polyhedron) if there exist l, u ∈ Rn: l ≤ x ≤ u; i.e. P = {x: Ax ≤ b, l ≤ x ≤ u for l, u ∈ Rn}. Remark 1: A set of inequalities Ax ≤ b can be converted into a set of equalities Ax + Is = b, s ≥ 0. A polytope can be described by as P = {x: Ax = b, l ≤ x ≤ u}. Remark 2: A vector v of a polyhedron is a vertex (extreme point) if and only if it cannot be written as a linear combination of points in P\{v}. A polytope P is the convex hull of its vertices.
Definition 3. A nonempty polyhedron is integral if its vertices are integral.
Definition 5. A rational matrix is κ‐regular if for all its non‐singular square submatrices R, κR‐1 is integral. Theorem 4. If A is an m×n integral matrix, then the polyhedron P(A, b’, b, d’, d) = {x: b’ ≤ Ax ≤ b, d’ ≤ x ≤ d} is integral for all a,b∈κZm and l, u ∈ κZn, iff A is κ‐regular (Appa and Kotnyek, 2004). Proposition 1.Let A be a rational matrix and σ and κ positive integers. Then σ ‐1A is κ‐regular iff A is σκ‐regular.
Proposition 2.Let A be κ‐regular. Then the following matrices are also κ‐regular: i) the transpose of A, ii) any submatrix of A, iii) the matrix obtained by multiplying a row or column of A by ‐1, iv) the matrix obtained by interchanging two rows or columns of A, v) the matrix obtained by duplicating a row or column of A, vi) the matrix obtained by dividing a row or column of A by a non‐zero integer.
Theorem 4
Polyhedron
P = {x: Ax = b}
Polytope
P = {x: Ax = b, l ≤ x ≤ u}
Integral Polytope
P = {x: Ax = b, l ≤ x ≤ u}
k‐regular
Proposition 1
Proposition 2
. . .
MIP Formulations
Discretetime MIP Scheduling Model
Given are a set of tasks i∈I, processing units j∈J, chemicals (states) k∈K, and a scheduling horizon η.
• A processing unit j can be used to carry out tasks i∈Ij.
• The fixed production rate of task i is ρi.
• A chemical can be consumed (produced) by multiple tasks i∈Ik- (i∈Ik+).
• Each task consumes and produces one chemical.
• Chemical k is stored in a dedicated tank with capacity ζk.
The time horizon is divided into periods n∈N={1, 2, …, N} of uniform length Δt = η/N,
• Win∈{0, 1}; it is equal to 1 if task i is processed during period n.
• Bin ≥ 0; extend (amount processed) of task i during period n.
• Ssn ≥ 0; inventory of chemical k at time point n.
∑W
i∈I j
in
= 1,
S kn = S k,n −1 +
∀j , n
∑B
in
i∈I k +
βi = ρi Δ t ,
S kn = S k ,n −1 +
−
∑B
in
+ γkn ≤ ζ k ,
∀k , n
i∈I k −
∀i Bin = βi Win ,
∀i , n
∑ βiWin − ∑ βi Bin + γkn ,
∀k , n
i∈I k +
{
}
P A = w ∈ [0,1]I ⋅ N : A A w = 1
{
}
P MB = x = ( s, w ) ∈ [0, ξ ] × R+K ⋅ N : A MB x = 1
i∈I k −
Win ∈ {0,1}, ∀i, n; S kn ∈ [0, ζ k ], ∀k , n
w ∈ {0,1} , s ∈ [0, ξ ]
i ⋅n
Polyhedral Results
P
P = P A I P MB
PA
{
}
= {x = ( s, w ) ∈ [0, ξ ] × R
P MB
P A = w ∈ [0,1] I ⋅ N : A A w = 1
P MB
K ⋅N
+
: A MB x = γ
}
Proposition 3. Matrix AA is an interval matrix; thus, it is totally unimodular.
Corollary 1. Polytope PA is integral.
Let lcmβ be the least common multiple of all batchsizes, lcmβ = lcm(βi, i∈I) = lcm(β);
Let gcfε be the greatest common factor of deliveries γkn and capacities ζk, gcfε = gcf(γkn, k∈K, n∈N; ζk, k∈K) = gcf(ε).
If κ* is the greatest common factor of lcmβ and gcfε, κ* = gcf(lcmβ, gcfε), then lcmβ = κ* ⋅ κβ and gcfε = κ* ⋅ κε
Proposition 5. Matrix Α is (κβ κ* κε)‐regular.
Proposition 6. Polyhedron PMB is integral if Δt = gcfε/lcmρ, where lcmρ is the least common multiple of production rates ρi, lcmβ = lcm(βi, i∈I) = lcm(β).
Proposition 7. If λ is the least common multiple of the denominators of rational parameters γ, ζ and ρ, then the vertices of PMB are w‐integral if Δt = Δt* = gcf(λε)/lcm(λρ).
Geometric Interpretation
4
ζ = 4,
RM
S0
4
WT2
WT2
=4
3
U1
T1
ρ=2
3
2
P
INT
ζ = 2,
S0
=0
A
P
1
A
0
0
1
P
WT1
2
ζ = 2, S0 = 0
2
4
0
1
P
2
3
4
b) Δt = 0.75 hr
2
WT2
gcf(λε) = 2, lcm(λρ) = 2
Δt* = 1
WT1
0
3
a) Δt = 0.5 hr
FP
MB
1
P
ρ=1
P
P
U2
T2
2
MB
WT2
A
P
1
A
P
1
P
MB
MB
P
P
WT1
0
0
1
c) Δt = 1.0 hr
0
2
0
1
WT1
2
d) Δt = 1.5 hr
Example 1
U2
U1
T11
In1: 2‐week horizon; demands in 0.5 increments
Proposition 7 ⇒ Δt* = 6 hr
T12
INT1
T21
T41
A
INT2
T32
INT4
INT3
U4
T22
U3
T31
Model and solution statistics for instances Ex1/In-1/Δt.
Δt (hr)
12
8
6
Periods (N)
28
42
56
Constraints
449
673
897
Binary var’s
448
672
896
Continuous var’s
309
463
617
LP-relaxation
198.275 202.161
204.0
Optimal solution
203.675 221.461
204.0
Integrality gap (%)* 2.65
8.71
0.00
CPU time (sec)
56.9
770.0
0.2
Nodes
103,631 874,116
5
Optimality gap (%) *
LP
*
* Integrality gap: IG = (Z - Z )/Z
ZLP = LP-relaxation; Z* = Optimal solution
RM2
RM1
3 stages; 5 units; 11 tasks; 11 chemicals
Three instances
T42
B1
U5
T43
C1
T51
B2
T52
C2
4
84
1,345
1,344
925
205.95
207.708
0.85
104.3
82,968
-
Example 1
RM2
RM1
Three instances
U2
U1
T11
Proposition 7 ⇒ Δt* = 6 hr
Proposition 7 ⇒ Δt* = 2.4 hr
T12
INT1
T21
T32
INT4
INT3
T41
A
INT2
U3
T31
U4
T22
T42
B1
U5
T43
C1
T51
B2
T52
C2
Δt (hr)
6
4
3
2.4
2
Periods (N)
56
84
112
140
168
Constraints
897
1,345
1,793
2,241 2,689
Binary var’s
896
1,344
1,792
2,240 2,688
Continuous var’s
617
925
1,233
1,541 1,849
LP-relaxation
197.2725
199.143
200.081 200.64 201.028
†
†
Best/optimal solution 215.7550
209.739
209.619† 200.64 206.473†
Integrality gap* (%) 8.0-8.6
3.2-5.1
3.8-4.6
0
1.3-2.6
CPU time (sec)
1,200
1,200
1,200
0.34
1,200
Nodes
985,501
717,501
699,601 0
520,501
Optimality gap (%)
0.65
1.96
0.77
1.36
† Instance not solved to optimality.
* If not solved to optimality we calculate a range [IGMIN, IGMAX].
IGMIN = [Z*(1-OG) – ZLP]/ Z*(1-OG); IGMAX = (Z* - ZLP)/Z*.
where ZLP = LP-relaxation; Z* = Best solution; OG = Optimality gap.
More Polyhedral Results
Production planning with backlogged demand
{
S kn
}
k=A
P A = w ∈ [0,1]I ⋅ N : A A w = 1
{
P MB2 = ( s, u, w ) ∈ [0, ξ ] × R +K ⋅ N × R +I ⋅ N : A s s + A u u + A w w = γ
}
k = RM
k = INT
Proposition 8.
i) Matrix Α2 is (κβ κ*)-regular.
ii) Polyhedron PMB2 is integral for Δt+ = gcfε/lcmρ.
iii) The vertices of PMB2 are w-integral for Δt* = gcf(λε)/lcm(λρ).
k = FP
n
1
2
3
1
2
3
1
2
3
k=B
U kn
k=C
k=A
k=B
W in
k=C
i = T1
i = T2
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
1 2 3 1 2 3
1
-1
β T1
-1 1
1 -1
β T1
-1 1
1 -1
β T1
1
-1
-β T1
β T2
-1 1
1 -1
-β T1
β T2
-1 1
1 -1
-β T1
β T2
1
-1
-β T2
-1 1
1 -1
-β T2
-1 1
-1 1
-β T2
As
Au
Aw
Production planning with backlogged demand and shipments
{
}
P A = w ∈ [0,1]I ⋅ N : A A w = 1
{
S kn
P MB3 = x = ( s, w, v , u) ∈ [0, ξ ] × R +I ⋅ N × R +K ⋅ N × R +K ⋅ N : A 3 x = [0, γ ]T
0⎤
⎡A s A w I
where A 3 x = ⎢
0 − I A u ⎥⎦
⎣0
}
k=A
k = RM
k = INT
k = FP
Proposition 9.
i) Matrix Α3 is (κβ κ*)-regular.
ii) Polyhedron PMB3 is integral if Δt+ = gcfε/lcmρ.
iii) The vertices of PMB3 are w-integral if Δt* = gcf(λε)/lcm(λρ).
k = RM
k = INT
k = FP
n
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
k=B
W in
k=C
i = T1
V kn
i = T2
k=A
k=B
U kn
k=C
k=A
k=B
k=C
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
1
1
β T1
1
-1 1
β T1
-1 1
β T1
1
1
-β T1
β T2
1
-1 1
-β T1
β T2
1
β T2
1
-1 1
-β T1
1
-β T2
1
-1 1
-β T2
1
-1 1
-β T2
1
-1
-1
-1
1 -1
-1
1 -1
-1
-1
-1
1 -1
-1
1 -1
-1
-1
-1
1 -1
-1
1 -1
Example 2
RM2
RM1
η = 8 weeks (56 days); U11
T111
T112
T121
U13
T122
IN1A
In‐1: Demand must be satisfied on time
Proposition 11 ⇒ Δt* = (1/6) d = 4 hr
RM3
U12
T131
T123
T132
IN1CD
IN1B
U22
U21
T211
T212
T213
T211
T214
IN2A
T212
U33
U32
T321
T312
IN3A1
IN3B1
T322
IN3A2
T214
IN2D
IN2B
U31
T311
T213
IN2C
T331
IN3B2
T332
IN3C
IN3D
U43
U41
T434
T411
T412
U42
T413
T414
A11 B11 A21 B21
T432
T421 T422 T423 T424
A12
U44
T433
B12
T441 T442 T443 T444
T431
A22
B22
C1
C2
D1
D2
Model and solution statistics for instances of Example 2: In1/Δt.
Δt (hr)
12
8
6
4
3
Periods (N)
112
168
224
336
448
Constraints
4,481
6,721
8,961
13,441
17,921
Binary var’s
5,488
8,232
10,976
16,464
21,952
Continuous var’s
3,137
4,705
6,273
9,409
12,545
LP-relaxation
1,576.5 1,588.9
1,595.0 1,601.1 1,604.2
Best/optimal solution 1,601.1 *
Integrality gap (%)
n/a
n/a
n/a
n/a
0
CPU time (sec)
1,200
1,200
1,200
1,200
24.1
Nodes
294,900 177,000 112,800 114
33,400
*
Optimality gap (%)
n/a
n/a
n/a
0
n/a
* If no integer feasible solution is found, optimality and integrality gaps cannot be
calculated.
Example 2
RM2
RM1
η = 8 weeks (56 days); U11
T111
T112
T121
U13
T122
IN1A
In‐1: Demand must be satisfied on time
Proposition 11 ⇒ Δt* = (1/6) d = 4 hr
In‐2: Shipments/backlogs are introduced
Proposition 11 ⇒ Δt* = (1/6) d = 4hr
RM3
U12
IN1CD
IN1B
U22
T211
T212
T213
T211
T214
IN2A
T321
T312
IN3B1
T322
IN3A2
T214
IN2D
IN2C
T331
IN3B2
T332
IN3C
IN3D
U43
T434
T411
T412
T414
A11 B11 A21 B21
T432
T421 T422 T423 T424
A12
U44
T433
U42
T413
4
360
18,709
17,640
18,709
3,636.7
3,637.1
0.01
234.2
1,350
0
T213
U33
U32
T311
Δt (hr)
12
8
6
Periods (N)
120
180
240
Constraints
6,229
9,349
12,469
Binary var’s
5,880
8,820
11,760
Continuous var’s
6229
9,349
12,469
LP-relaxation
3,652.3
3,644.5
3,640.6
†
†
Best/optimal solution 3,848.7
3,858.5
3,801.3†
0.04-5.10 0.41-5.55 0.02-4.23
Integrality gap* (%)
CPU time (sec)
1,200
1,200
1,200
Nodes
155,762
95,981
67,501
Optimality gap (%)
5.06
5.16
4.21
T212
IN2B
U31
U41
*
T132
U21
IN3A1
†
T131
T123
B12
T441 T442 T443 T444
T431
A22
B22
C1
C2
D1
D2
3
480
24,949
23,520
24,949
3,634.7
3,651.5†
0.01-0.46
1,200
11,251
0.45
Instance not solved to optimality
If not solved to optimality we calculate a range [IGMIN, IGMAX].
IGMIN = [Z*(1-OG) – ZLP]/ Z*(1-OG); IGMAX = (Z* - ZLP)/Z*.
where ZLP = LP-relaxation; Z* = Best solution; OG = Optimality gap.
Remarks
Integrality of PA and PMB is interesting theoretically
It cannot be directly used (e.g. in decomposition methods)
Can be proved only for subset of problems
Our results are useful in practice:
• Starting point for extended results and special cases
• Computational performance of commercial solvers is improved
Proof of results suggests network structure
Study relationship between production facilities and graphtheoretic networks
Problem Representation
MIP Formulation
Network Structure of Scheduling MIP Model Theorem 2. If a {0, ±1} matrix has no more than
two non zero coefficients in each column, and Σl
alm = 0 if column m has two coefficients, then A
is TU
P = {x: Ax = b}
A: totally unimodular
(TU)
Integral Polytope
RM1
RM2
U1
U2
T11
T12
T21
RM1
T11
INT1
U3
INT4
U5
T41
A
T42 T43
T51
T52
C1
B2
C2
B1
INT3
Production Facility
INT4
T43
T41
T42
A
B1
Processing restrictions correspond to known T32
T32
U4
3
T22
INT2
T31
INT3
4
(1,2) (2,1) (2,3) (3,4) (4,1) (4,2)
1
1
⎤
2 ⎢ 1 −1
−1
1⎥
⎥
⎢
1 −1
3
⎥
⎢
1 − 1 − 1⎦
4⎣
1 ⎡− 1
T21
INT2
T31
2
RM2
T12
T22
INT1
1
T52
problems in graph theory
T51
C1
B2
Digraph
C2
29
Modeling
Litro
500 BPM
FERMENTACIÓN
¼ STD
700 BPM
D. W.
DILUTION
Different production environments TANQUES
DE
GOBIERNO I
½ STD
600 BPM
½ PREMIER
600 BPM
Wide range of processing constraints
½ T.A., ¼ BOH.
TANQUES
DE
GOBIERNO II
750 BPM
Litro
350 BPM
REPOSO
LATA
1200 LPM
TANQUES DE GOBIERNO
½ T.A EXP
750 BPM
TANQUE REPOSO
Theory
P
PA
P MB
Solution Methods
Example Revisited
150 ≤ P2 ≤ 200 (kg/d)
200 ≤ P1 ≤ 300 (kg/d)
RM
U1
INT
(ZW)
A
U2
P3 = 150 (kg/d)
B
U3
PB
2,000
∑w P
l
k
k∈K
kt
≤Π
l
Cp = f(PA,PB)
l ∈ LF
FP
PA ≤ PB
∑w P
Cpt ≥ γ l +
l
k
k∈K
1,000
1.5PA +0.5PB ≤ 2,100
l ∈ LO
PB
PB ≥ 0
1,000
kt
FP
PA
2,000 PA
Surrogate cost model
Process Attainable Region
All necessary production feasibility information in: All necessary production cost information in:
Can we systematically develop these constraints?
∑w P
l
k
k∈K
kt
≤ Πl
l ∈ LF
FP
Cpt ≥ γ l +
∑w P
l
k
k∈K
kt
l ∈ LO
FP
⇒ We have to analyze the underlying scheduling model
Projectionbased Method
Projectionbased method:

Develop feasibility/optimality constraints that involve only planning (x) variables
Convergence between overestimation (OE) and underestimation (UE) of true convex hull
1.
2.
3.
4.
Choose initial search vectors w = [w1, w2, …wK]T Solve scheduling model (M1) for each vector to obtain a UE vertex and an OE inequality Run Quickhull to convert UE vertices into inequalities Iterate until MPD stops improving: Find maximum perpendicular distance (MPD) and a new search vector Solve (M2) for new search vector; obtain new vertex and new inequality Run Quickhull to update UE 5. Set PAR equal to UECH Mixed-integer programming methods for supply chain optimization
Rollinghorizon Approach
P1
(PP) z = min ∑t (Cpt + Cht )
P2
I it = I i ,t −1 + Pit − δit , ∀i, t
∑w P
l
k
k∈K
kt
≤Π
l ∈ LF
l
Pit
I i ,t −1
How much can we produce
and at what cost?
I it
Satisfy demand?
At what (inventory) cost?
FP
Cpt ≥ γ l +
∑w P
l
k
k∈K
FP
kt
l ∈ LO
δit
Pit , I it ≥ 0

Model (PP) yields a complete production planning solution: Pit and Iit
If detailed schedules are required, model (PP) can be used in a rolling horizon algorithm (S‐PP) to provide production targets for (weekly) scheduling models
0) 1) 2) 3) 4) Set θ = 1 Solve (PP) for planning periods t∈{θ, θ +1, …, T} to obtain targets Pkθ, Pkθ+1, …, PkT Solve scheduling model for period t = θ to meet target Pkθ If production target is met exactly (i.e. SkN = Pkθ, ∀k∈KFP), fix Pkθ, , and return to 2 otherwise, fix Pkθ , and re‐solve (PP) to update targets Pkθ+1, Pkθ+2, …, PkT , Set θ =θ+1, and return to 2 Mixed-integer programming methods for supply chain optimization
Example 1
40%
HT
RI
RII
RM1
H
40%
H
Int2
60%
60%
R1, R2, R3
Int1
RM2
Sep
R2
Kondili et al., 1993
A
10%
ImP2
S
80%
90%
R3
R1
S
B
20%
RM3
R216
PB (kg)
R120: Ht = 120 hr
R168: Ht = 168 hr
4500
R216: Ht = 216 hr
R168
3000
R120
1500
0
PA
0
1000
2000
3000 (kg)
Example 1
40%
HT
RI
RM1
H
40%
H
Int2
60%
ImP2
60%
R1, R2, R3
Int1
RII
Sep
R2
A
RM2
S
80%
S
90%
R3
R1
10%
B
20%
RM3
PB (kg)
R500: SIn2MAX = 500 Kg
R1000: SIn2MAX = 1,000 Kg
R1500: SIn2MAX = 1,500 Kg
3000
2000
R500
1000
0
R1000
R1500 P
A
0
600
1200
1800
(kg)
Example 2
Production Planning
48 1‐week periods
F1
U1
U2
U3
U4
U3
U4
T1, T4
T2
T3
T5, T6
T7, T9
T8, T10
S1
T1
5%
F2
95%
T7
S2
T2
S5
10%
T8
Int1
T3
S4
T4
S3
50%
Int2
90%
50%
T9
S6
Papageorgiou and Pantelides, 1996
T5
P1
T6
P2
T10
P3
Projection of scheduling feasible region onto Pi
1000
PC
500
0
0
500
500
PB
1000
1500
1500
1000 P
A
Example 2: Results
3000
(kg)
Inventory
Production
2000
1000
0
0
12
24
36
48
Inventory (Ikt) and production (Pkt) levels for product A
2500
(kg)
1500
500
-500
1
48
Demand
Shipments
-1500
Backlog
Demand (Demkt), shipment (Dkt) and backlog (Ukt) profiles for product A
Example 2: Results
Solutions of (PP) and (S-PP)
Holding cost ($)
Backlog cost ($)
Production cost ($)
Total cost ($)
CPU secs
(kg)
Production Targets
2500
Model (PP)
574,756
183,933
408,405
1,167,094
0.162
Algorithm (S-PP)
568,345
210,221
411,910
1,190,475
5406.092
Deviation from Production Targets
(kg)
75
2000
50
1500
1000
25
500
0
0
1
Absolute deviation between production targets predicted by (PP) and
production amounts obtained by detailed scheduling (SPP)
Average total production (PA+PA+PC): Average total absolute deviation (PP)‐(S‐PP):
1,820.0 kg
4.5 kg → 0.25%
Conclusions
General Modeling Framework
Litro
500 BPM
FERMENTACIÓN
¼ STD
700 BPM
D. W.
No decoupling into subproblems ⇒ better solutions
DILUTION
TANQUES
DE
GOBIERNO I
½ STD
600 BPM
½ PREMIER
600 BPM
Unique problem representation ⇒ better IT systems
Computational advances for wide range of problems
½ T.A., ¼ BOH.
TANQUES
DE
GOBIERNO II
750 BPM
Litro
350 BPM
REPOSO
LATA
1200 LPM
TANQUES DE GOBIERNO
½ T.A EXP
750 BPM
TANQUE REPOSO
Polyhedral & Graphtheoretic Results
P
PA
P MB
Rigorous treatment of MIP tightness
Provides insights into formulation of large‐scale problems
Integration of combinatorial optimization algorithms
Projectionbased Method
PB
Compact description of production targets and costs
Can be combined with any scheduling formulation
2,000
∑w P
l
k
kt
l ∈ LF
k∈K FP
PA ≤ PB
1.5PA +0.5PB ≤ 2,100
1,000
Generated once off‐line; improved on‐line if necessary
Effective production planning approach
≤ Πl
PB ≥ 0
1,000
2,000 PA
Questions?