Generate simplified solution reports in CSV format

This commit is contained in:
2020-09-18 11:12:15 -05:00
parent ec0fd7aed6
commit ff23004022
26 changed files with 8023 additions and 23 deletions

View File

@@ -7,4 +7,5 @@ module RELOG
include("instance.jl")
include("graph.jl")
include("model.jl")
include("reports.jl")
end

View File

@@ -1,8 +1,12 @@
# Input Data Format
# Input and Output Data Formats
The first step when using RELOG is to describe the reverse logistics pipeline and the relevant data. RELOG accepts as input a JSON file with three sections: `parameters`, `products` and `plants`. Below, we describe each section in more detail.
In this page, we describe the input and output JSON formats used by RELOG. In addition to these, RELOG can also produce [simplified reports](reports.md) in tabular data format.
## Parameters
## Input Data Format (JSON)
RELOG accepts as input a JSON file with three sections: `parameters`, `products` and `plants`. Below, we describe each section in more detail.
### Parameters
The **parameters** section describes details about the simulation itself.
@@ -12,7 +16,7 @@ The **parameters** section describes details about the simulation itself.
|`building period (years)` | List of years in which we are allowed to open new plants. For example, if this parameter is set to `[1,2,3]`, we can only open plants during the first three years. By default, this equals `[1]`; that is, plants can only be opened during the first year. |
### Example
#### Example
```json
{
"parameters": {
@@ -22,7 +26,7 @@ The **parameters** section describes details about the simulation itself.
}
```
## Products
### Products
The **products** section describes all products and subproducts in the simulation. The field `instance["Products"]` is a dictionary mapping the name of the product to a dictionary which describes its characteristics. Each product description contains the following keys:
@@ -41,7 +45,7 @@ Each product may have some amount available at the beginning of each time period
| `longitude (deg)` | The longitude of the location.
| `amount (tonne)` | The amount of the product initially available at the location. Must be a timeseries.
### Example
#### Example
```json
{
@@ -84,7 +88,7 @@ Each product may have some amount available at the beginning of each time period
}
```
## Processing Plants
### Processing plants
The **plants** section describes the available types of reverse manufacturing plants, their potential locations and associated costs, as well as their inputs and outputs. The field `instance["Plants"]` is a dictionary mapping the name of the plant to a dictionary with the following keys:
@@ -121,7 +125,7 @@ The keys in the `capacities (tonne)` dictionary should be the amounts (in tonnes
| `fixed operating cost ($)` | The cost to keep the plant open, even if the plant doesn't process anything. Must be a timeseries.
| `variable operating cost ($/tonne)` | The cost that the plant incurs to process each tonne of input. Must be a timeseries.
### Example
#### Example
```json
{
@@ -166,9 +170,14 @@ The keys in the `capacities (tonne)` dictionary should be the amounts (in tonnes
}
```
## Current limitations
### Current limitations
* Each plant can only be opened exactly once. After open, the plant remains open until the end of the simulation.
* Plants can be expanded at any time, even long after they are open.
* All material available at the beginning of a time period must be entirely processed by the end of that time period. It is not possible to store unprocessed materials from one time period to the next.
* Up to two plant sizes are currently supported. Variable operating costs must be the same for all plant sizes.
* Up to two plant sizes are currently supported. Variable operating costs must be the same for all plant sizes.
## Output Data Format (JSON)
To be documented.

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 586 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

View File

@@ -3,10 +3,14 @@
**RELOG** is an open-source supply chain optimization package focusing on reverse logistics and reverse manufacturing. The package uses Mixed-Integer Linear Programming to determine where to build recycling plants, what size should these plants have and which customers should be served by which plants. The package supports custom reverse logistics pipelines, with multiple types of plants, multiple types of product and multiple time periods.
<img src="images/ex_transportation.png" width="1000px"/>
### Table of Contents
* [Usage](usage.md)
* [Data Format](format.md)
* [Input and Output Data Formats](format.md)
* [Simplified Solution Reports](reports.md)
* [Optimization model](model.md)
### Source Code

269
src/docs/reports.md Normal file
View File

@@ -0,0 +1,269 @@
# Simplified Solution Reports
In addition to the full output format described in [data formats](format.md), RELOG can also generate a number of simplified reports in tabular data format (CSV), which can be more easily processed by spreadsheet software (such as Microsoft Excel), by data analysis libraries (such as Pandas) or by relational databases (such as SQLite).
In this page, we also illustrate what types of charts and visualizations can be produced from these tabular data files. The sample charts have been produced using Python, matplotlib, seaborn and geopandas.
## Plants report
Report showing plant costs, capacities, energy expenditure and utilization factors.
Generated by `RELOG.write_plants_report(solution, filename)`. For a concrete example, see [nimh_plants.csv](https://github.com/ANL-CEEESA/RELOG/blob/master/test/fixtures/nimh_plants.csv).
| Column | Description
|:--------------------------------------|---------------|
| `plant type` | Plant type.
| `location name` | Location name.
| `year` | What year this row corresponds to. This reports includes one row for each year in the simulation.
| `latitude (deg)` | Latitude of the plant.
| `longitude (deg)` | Longitude of the plant.
| `capacity (tonne)` | Capacity of the plant at this point in time.
| `amount processed (tonne)` | Amount of input material received by the plant this year.
| `utilization factor (%)` | Amount processed by the plant this year divided by current plant capacity.
| `energy (GJ)` | Amount of energy expended by the plant this year.
| `opening cost ($)` | Amount spent opening the plant. This value is only positive if the plant became operational this year.
| `expansion cost ($)` | Amount spent this year expanding the plant capacity.
| `fixed operating cost ($)` | Amount spent for keeping the plant operational this year.
| `variable operating cost ($)` | Amount spent for processing the input material this year.
| `total cost ($)` | Sum of all previous plant costs.
### Sample charts
* Bar plot with total plant costs per year, grouped by plant type (in Python):
```python
import pandas as pd
import seaborn as sns; sns.set()
data = pd.read_csv("plants_report.csv")
sns.barplot(x="year",
y="total cost ($)",
hue="plant type",
data=data.groupby(["plant type", "year"])
.sum()
.reset_index());
```
<img src="../images/ex_plant_cost_per_year.png" width="500px"/>
* Map showing plant locations (in Python):
```python
import pandas as pd
import geopandas as gp
# Plot base map
world = gp.read_file(gp.datasets.get_path('naturalearth_lowres'))
world = world[world.continent == 'North America']
ax = world.plot(color='white', edgecolor='50', figsize=(15,15))
# Plot plant locations
data = pd.read_csv("plants_report.csv")
points = gp.points_from_xy(data["longitude (deg)"],
data["latitude (deg)"])
gp.GeoDataFrame(data, geometry=points).plot(ax=ax);
```
<img src="../images/ex_plant_locations.png" width="1000px"/>
## Plant outputs report
Report showing amount of products produced, sent and disposed of by each plant, as well as disposal costs.
Generated by `RELOG.write_plant_outputs_report(solution, filename)`. For a concrete example, see [nimh_plant_outputs.csv](https://github.com/ANL-CEEESA/RELOG/blob/master/test/fixtures/nimh_plant_outputs.csv).
| Column | Description
|:--------------------------------------|---------------|
| `plant type` | Plant type.
| `location name` | Location name.
| `year` | What year this row corresponds to. This reports includes one row for each year in the simulation.
| `product` | Product being produced.
| `amount produced (tonne)` | Amount of product produced this year.
| `amount sent (tonne)` | Amount of product produced by this plant and sent to another plant for further processing this year.
| `amount disposed (tonne)` | Amount produced produced by this plant and immediately disposed of locally this year.
| `disposal cost ($)` | Disposal cost for this year.
### Sample charts
* Bar plot showing total amount produced for each product, grouped by year (in Python):
```python
import pandas as pd
import seaborn as sns; sns.set()
data = pd.read_csv("plant_outputs_report.csv")
sns.barplot(x="amount produced (tonne)",
y="product name",
hue="year",
data=data.groupby(["product name", "year"])
.sum()
.reset_index());
```
<img src="../images/ex_amount_produced.png" width="500px"/>
## Plant emissions report
Report showing amount of emissions produced by each plant.
Generated by `RELOG.write_plant_emissions_report(solution, filename)`. For a concrete example, see [nimh_plant_emissions.csv](https://github.com/ANL-CEEESA/RELOG/blob/master/test/fixtures/nimh_plant_emissions.csv).
| Column | Description
|:--------------------------------------|---------------|
| `plant type` | Plant type.
| `location name` | Location name.
| `year` | Year.
| `emission type` | Type of emission.
| `amount (tonne)` | Amount of emission produced by the plant this year.
### Sample charts
* Bar plot showing total emission by plant type, grouped type of emissions (in Python):
```python
import pandas as pd
import seaborn as sns; sns.set()
data = pd.read_csv("plant_emissions_report.csv")
sns.barplot(x="plant type",
y="emission amount (tonne)",
hue="emission type",
data=data.groupby(["plant type", "emission type"])
.sum()
.reset_index());
```
<img src="../images/ex_emissions.png" width="500px"/>
## Transportation report
Report showing amount of product sent from initial locations to plants, and from one plant to another. Includes the distance between each pair of locations, amount-distance shipped, transportation costs and energy expenditure.
Generated by `RELOG.write_transportation_report(solution, filename)`. For a concrete example, see [nimh_transportation.csv](https://github.com/ANL-CEEESA/RELOG/blob/master/test/fixtures/nimh_transportation.csv).
| Column | Description
|:--------------------------------------|---------------|
| `source type` | If product is being shipped from an initial location, equals `Origin`. If product is being shipped from a plant, equals plant type.
| `source location name` | Name of the location where the product is being shipped from.
| `source latitude (deg)` | Latitude of the source location.
| `source longitude (deg)` | Longitude of the source location.
| `destination type`| Type of plant the product is being shipped to.
| `destination location name`| Name of the location where the product is being shipped to.
| `destination latitude (deg)` | Latitude of the destination location.
| `destination longitude (deg)` | Longitude of the destination location.
| `product`| Product being shipped.
| `year`| Year.
| `distance (km)`| Distance between source and destination.
| `amount (tonne)`| Total amount of product being shipped between the two locations this year.
| `amount-distance (tonne-km)`| Total amount being shipped this year times distance.
| `transportation cost ($)`| Cost to transport this amount of product between the two locations for this year.
| `transportation energy (GJ)`| Energy expended transporting this amount of product between the two locations.
### Sample charts
* Bar plot showing total amount-distance for each product type, grouped by year (in Python):
```python
import pandas as pd
import seaborn as sns; sns.set()
data = pd.read_csv("transportation_report.csv")
sns.barplot(x="product",
y="amount-distance (tonne-km)",
hue="year",
data=data.groupby(["product", "year"])
.sum()
.reset_index());
```
<img src="../images/ex_transportation_amount_distance.png" width="500px"/>
* Map of transportation lines (in Python):
```python
import pandas as pd
import geopandas as gp
from shapely.geometry import Point, LineString
import matplotlib.pyplot as plt
from matplotlib import collections
# Plot base map
world = gp.read_file(gp.datasets.get_path('naturalearth_lowres'))
ax = world.plot(color='white', edgecolor='50', figsize=(14,7))
ax.set_ylim([23, 50])
ax.set_xlim([-128, -65])
# Draw transportation lines
data = pd.read_csv("transportation_report.csv")
lines = [[(row["source longitude (deg)"], row["source latitude (deg)"]),
(row["destination longitude (deg)"], row["destination latitude (deg)"])
] for (index, row) in data.iterrows()]
ax.add_collection(collections.LineCollection(lines,
linewidths=0.25,
zorder=1,
alpha=0.5,
color="50"))
# Draw source points
points = gp.points_from_xy(data["source longitude (deg)"],
data["source latitude (deg)"])
gp.GeoDataFrame(data, geometry=points).plot(ax=ax,
color="0.5",
markersize=1);
# Draw destination points
points = gp.points_from_xy(data["destination longitude (deg)"],
data["destination latitude (deg)"])
gp.GeoDataFrame(data, geometry=points).plot(ax=ax,
color="red",
markersize=50);
```
<img src="../images/ex_transportation.png" width="1000px"/>
## Transportation emissions report
Report showing emissions for each trip between initial locations and plants, and between pairs of plants.
Generated by `RELOG.write_transportation_emissions_report(solution, filename)`. For a concrete example, see [nimh_transportation_emissions.csv](https://github.com/ANL-CEEESA/RELOG/blob/master/test/fixtures/nimh_transportation_emissions.csv).
| Column | Description
|:--------------------------------------|---------------|
| `source type` | If product is being shipped from an initial location, equals `Origin`. If product is being shipped from a plant, equals plant type.
| `source location name` | Name of the location where the product is being shipped from.
| `source latitude (deg)` | Latitude of the source location.
| `source longitude (deg)` | Longitude of the source location.
| `destination type`| Type of plant the product is being shipped to.
| `destination location name`| Name of the location where the product is being shipped to.
| `destination latitude (deg)` | Latitude of the destination location.
| `destination longitude (deg)` | Longitude of the destination location.
| `product`| Product being shipped.
| `year`| Year.
| `distance (km)`| Distance between source and destination.
| `shipped amount (tonne)`| Total amount of product being shipped between the two locations this year.
| `shipped amount-distance (tonne-km)`| Total amount being shipped this year times distance.
| `emission type` | Type of emission.
| `emission amount (tonne)` | Amount of emission produced by transportation segment this year.
### Sample charts
* Bar plot showing total emission amount by emission type, grouped by type of product being transported (in Python):
```python
import pandas as pd
import seaborn as sns; sns.set()
data = pd.read_csv("transportation_emissions_report.csv")
sns.barplot(x="emission type",
y="emission amount (tonne)",
hue="product",
data=data.groupby(["product", "emission type"])
.sum()
.reset_index());
```
<img src="../images/ex_transportation_emissions.png" width="500px"/>

View File

@@ -46,12 +46,21 @@ All user parameters specified above must be provided to RELOG as a JSON file, wh
After creating a JSON file describing the reverse manufacturing process and the input data, the following example illustrates how to use the package to find the optimal set of decisions:
```julia
# Import package
using RELOG
RELOG.solve("/home/user/instance.json",
output="/home/user/solution.json")
# Solve optimization problem
solution = RELOG.solve("/home/user/instance.json")
# Write full solution in JSON format
RELOG.write(solution, "solution.json")
# Write simplified reports in CSV format
RELOG.write_plants_report(solution, "plants.csv")
RELOG.write_transportation_report(solution, "transportation.csv")
```
The optimal logistics plan will be stored in the output file specified. See [data format page](format.md) for a description of this output file.
For a complete description of the file formats above, and for a complete list of available reports, see the [data format page](format.md).
## 4. Advanced options

View File

@@ -312,6 +312,7 @@ function get_solution(model::ManufacturingModel)
"Send" => OrderedDict(),
"Dispose" => OrderedDict(),
),
"Input product" => plant.input.name,
"Total input (tonne)" => [0.0 for t in 1:T],
"Total output" => OrderedDict(),
"Latitude (deg)" => plant.latitude,

268
src/reports.jl Normal file
View File

@@ -0,0 +1,268 @@
# RELOG: Reverse Logistics Optimization
# Copyright (C) 2020, UChicago Argonne, LLC. All rights reserved.
# Released under the modified BSD license. See COPYING.md for more details.
using DataFrames
using CSV
function plants_report(solution::Dict)::DataFrame
df = DataFrame()
df."plant type" = String[]
df."location name" = String[]
df."year" = Int[]
df."latitude (deg)" = Float64[]
df."longitude (deg)" = Float64[]
df."capacity (tonne)" = Float64[]
df."amount processed (tonne)" = Float64[]
df."utilization factor (%)" = Float64[]
df."energy (GJ)" = Float64[]
df."opening cost (\$)" = Float64[]
df."expansion cost (\$)" = Float64[]
df."fixed operating cost (\$)" = Float64[]
df."variable operating cost (\$)" = Float64[]
df."total cost (\$)" = Float64[]
T = length(solution["Energy"]["Plants (GJ)"])
for (plant_name, plant_dict) in solution["Plants"]
for (location_name, location_dict) in plant_dict
var_cost = zeros(T)
for (src_plant_name, src_plant_dict) in location_dict["Input"]
for (src_location_name, src_location_dict) in src_plant_dict
var_cost += src_location_dict["Variable operating cost (\$)"]
end
end
var_cost = round.(var_cost, digits=2)
for year in 1:T
opening_cost = round(location_dict["Opening cost (\$)"][year], digits=2)
expansion_cost = round(location_dict["Expansion cost (\$)"][year], digits=2)
fixed_cost = round(location_dict["Fixed operating cost (\$)"][year], digits=2)
total_cost = round(var_cost[year] + opening_cost + expansion_cost + fixed_cost, digits=2)
capacity = round(location_dict["Capacity (tonne)"][year], digits=2)
processed = round(location_dict["Total input (tonne)"][year], digits=2)
utilization_factor = round(processed / capacity * 100.0, digits=2)
energy = round(location_dict["Energy (GJ)"][year], digits=2)
latitude = round(location_dict["Latitude (deg)"], digits=6)
longitude = round(location_dict["Longitude (deg)"], digits=6)
push!(df, [
plant_name,
location_name,
year,
latitude,
longitude,
capacity,
processed,
utilization_factor,
energy,
opening_cost,
expansion_cost,
fixed_cost,
var_cost[year],
total_cost,
])
end
end
end
return df
end
function plant_outputs_report(solution::Dict)::DataFrame
df = DataFrame()
df."plant type" = String[]
df."location name" = String[]
df."year" = Int[]
df."product name" = String[]
df."amount produced (tonne)" = Float64[]
df."amount sent (tonne)" = Float64[]
df."amount disposed (tonne)" = Float64[]
df."disposal cost (\$)" = Float64[]
T = length(solution["Energy"]["Plants (GJ)"])
for (plant_name, plant_dict) in solution["Plants"]
for (location_name, location_dict) in plant_dict
for (product_name, amount_produced) in location_dict["Total output"]
send_dict = location_dict["Output"]["Send"]
disposal_dict = location_dict["Output"]["Dispose"]
sent = zeros(T)
if product_name in keys(send_dict)
for (dst_plant_name, dst_plant_dict) in send_dict[product_name]
for (dst_location_name, dst_location_dict) in dst_plant_dict
sent += dst_location_dict["Amount (tonne)"]
end
end
end
sent = round.(sent, digits=2)
disposal_amount = zeros(T)
disposal_cost = zeros(T)
if product_name in keys(disposal_dict)
disposal_amount += disposal_dict[product_name]["Amount (tonne)"]
disposal_cost += disposal_dict[product_name]["Cost (\$)"]
end
disposal_amount = round.(disposal_amount, digits=2)
disposal_cost = round.(disposal_cost, digits=2)
for year in 1:T
push!(df, [
plant_name,
location_name,
year,
product_name,
round(amount_produced[year], digits=2),
sent[year],
disposal_amount[year],
disposal_cost[year],
])
end
end
end
end
return df
end
function plant_emissions_report(solution::Dict)::DataFrame
df = DataFrame()
df."plant type" = String[]
df."location name" = String[]
df."year" = Int[]
df."emission type" = String[]
df."emission amount (tonne)" = Float64[]
T = length(solution["Energy"]["Plants (GJ)"])
for (plant_name, plant_dict) in solution["Plants"]
for (location_name, location_dict) in plant_dict
for (emission_name, emission_amount) in location_dict["Emissions (tonne)"]
for year in 1:T
push!(df, [
plant_name,
location_name,
year,
emission_name,
round(emission_amount[year], digits=2),
])
end
end
end
end
return df
end
function transportation_report(solution::Dict)::DataFrame
df = DataFrame()
df."source type" = String[]
df."source location name" = String[]
df."source latitude (deg)" = Float64[]
df."source longitude (deg)" = Float64[]
df."destination type" = String[]
df."destination location name" = String[]
df."destination latitude (deg)" = Float64[]
df."destination longitude (deg)" = Float64[]
df."product" = String[]
df."year" = Int[]
df."distance (km)" = Float64[]
df."amount (tonne)" = Float64[]
df."amount-distance (tonne-km)" = Float64[]
df."transportation cost (\$)" = Float64[]
df."transportation energy (GJ)" = Float64[]
T = length(solution["Energy"]["Plants (GJ)"])
for (dst_plant_name, dst_plant_dict) in solution["Plants"]
for (dst_location_name, dst_location_dict) in dst_plant_dict
for (src_plant_name, src_plant_dict) in dst_location_dict["Input"]
for (src_location_name, src_location_dict) in src_plant_dict
for year in 1:T
push!(df, [
src_plant_name,
src_location_name,
round(src_location_dict["Latitude (deg)"], digits=6),
round(src_location_dict["Longitude (deg)"], digits=6),
dst_plant_name,
dst_location_name,
round(dst_location_dict["Latitude (deg)"], digits=6),
round(dst_location_dict["Longitude (deg)"], digits=6),
dst_location_dict["Input product"],
year,
round(src_location_dict["Distance (km)"][year], digits=2),
round(src_location_dict["Amount (tonne)"][year], digits=2),
round(src_location_dict["Amount (tonne)"][year] *
src_location_dict["Distance (km)"][year],
digits=2),
round(src_location_dict["Transportation cost (\$)"][year], digits=2),
round(src_location_dict["Transportation energy (J)"][year] / 1e9, digits=2),
])
end
end
end
end
end
return df
end
function transportation_emissions_report(solution::Dict)::DataFrame
df = DataFrame()
df."source type" = String[]
df."source location name" = String[]
df."source latitude (deg)" = Float64[]
df."source longitude (deg)" = Float64[]
df."destination type" = String[]
df."destination location name" = String[]
df."destination latitude (deg)" = Float64[]
df."destination longitude (deg)" = Float64[]
df."product" = String[]
df."year" = Int[]
df."distance (km)" = Float64[]
df."shipped amount (tonne)" = Float64[]
df."shipped amount-distance (tonne-km)" = Float64[]
df."emission type" = String[]
df."emission amount (tonne)" = Float64[]
T = length(solution["Energy"]["Plants (GJ)"])
for (dst_plant_name, dst_plant_dict) in solution["Plants"]
for (dst_location_name, dst_location_dict) in dst_plant_dict
for (src_plant_name, src_plant_dict) in dst_location_dict["Input"]
for (src_location_name, src_location_dict) in src_plant_dict
for (emission_name, emission_amount) in src_location_dict["Emissions (tonne)"]
for year in 1:T
push!(df, [
src_plant_name,
src_location_name,
round(src_location_dict["Latitude (deg)"], digits=6),
round(src_location_dict["Longitude (deg)"], digits=6),
dst_plant_name,
dst_location_name,
round(dst_location_dict["Latitude (deg)"], digits=6),
round(dst_location_dict["Longitude (deg)"], digits=6),
dst_location_dict["Input product"],
year,
round(src_location_dict["Distance (km)"][year], digits=2),
round(src_location_dict["Amount (tonne)"][year], digits=2),
round(src_location_dict["Amount (tonne)"][year] *
src_location_dict["Distance (km)"][year],
digits=2),
emission_name,
round(emission_amount[year], digits=2),
])
end
end
end
end
end
end
return df
end
write_plants_report(solution, filename) =
CSV.write(filename, plants_report(solution))
write_plant_outputs_report(solution, filename) =
CSV.write(filename, plant_outputs_report(solution))
write_plant_emissions_report(solution, filename) =
CSV.write(filename, plant_emissions_report(solution))
write_transportation_report(solution, filename) =
CSV.write(filename, transportation_report(solution))
write_transportation_emissions_report(solution, filename) =
CSV.write(filename, transportation_emissions_report(solution))