Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect pieces for SSP 2024/ScenarioMIP #235

Draft
wants to merge 183 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
183 commits
Select commit Hold shift + click to select a range
50aebf6
Add ScenarioMIP doc stub; link to #234
khaeru Sep 26, 2024
b1888b2
Create `/project/ssp/script` directory and add `__init__.py`
measrainsey Oct 2, 2024
d84f244
Create `/project/ssp/script/util` directory and add `__init__.py`
measrainsey Oct 2, 2024
a671698
Add functions
measrainsey Oct 2, 2024
d09b435
Add script to modify Low scenarios
measrainsey Oct 2, 2024
eba5fe2
Add script to modify Very Low scenarios
measrainsey Oct 2, 2024
96c20b6
Add UE shares files
measrainsey Oct 2, 2024
f05495c
Change to using `package_data_path()`
measrainsey Oct 2, 2024
17a80f0
Remove manual setting of path
measrainsey Oct 2, 2024
39c02d7
Clean up scripts and remove commented sections
measrainsey Oct 2, 2024
e24999a
Add new materials and industry reporting
macflo8 Nov 5, 2024
e695332
Correct syntax for materials reporting to change units
OFR-IIASA Nov 7, 2024
ffbba1a
Extend and rename variable mappings
macflo8 Nov 7, 2024
f573c5b
Update docs
macflo8 Aug 27, 2024
6245251
Revise materials documentation
macflo8 Aug 29, 2024
4f77a15
Update cement furnaces efficiency
macflo8 May 26, 2024
6a24102
Update clinker CCS diffusion constraint
macflo8 May 26, 2024
0ede740
Update cement and aluminum demand SSP5
macflo8 May 27, 2024
5de91a0
Add SSP specific input steel/cement data
macflo8 May 27, 2024
faac2ad
Add SSP changes to data_steel
macflo8 May 27, 2024
2c4a69a
Remove unused low temp cement furnaces
macflo8 May 28, 2024
16a3d13
Update base year steel and cement demands
macflo8 May 28, 2024
cfd15fd
Update data read functions to new SSP argument
macflo8 May 28, 2024
6744e19
Keep solar and biomass base year industry bounds
macflo8 Jun 3, 2024
419e020
Adjust base year cement demand of CHN region
macflo8 Jun 3, 2024
0957610
Fix low temp cement furnace ACT to 0
macflo8 Jun 3, 2024
0424da7
Update non-metallic minerals base year calibration
macflo8 Jun 3, 2024
6de5c3a
Add cement furnace base year bounds
macflo8 Jun 3, 2024
d590fa2
Silence pandas FutureWarnings in reporting
macflo8 Jun 3, 2024
f3d56bc
Improve materials reporting
macflo8 Jun 3, 2024
9745aab
Add non-CO2 emissions factors to industry furnaces
macflo8 Jun 5, 2024
829cd6a
Remove deprecated Materials gas heat pumps
macflo8 Jun 5, 2024
51ee70c
Remove duplicated technology in set and format
macflo8 Jun 5, 2024
ae24143
Fix non-CO2 furnace build bugs
macflo8 Jun 5, 2024
be5b7b1
Fix bug in refactored materials reporting function
macflo8 Jun 6, 2024
7ed3fb6
Add furnaces to existing GAINS link relation
macflo8 Jun 6, 2024
6278243
Fix missing years error caused by data_generic.py
macflo8 Jun 18, 2024
9ee6ba8
Exclude chemicals flow from i_spec calibration
macflo8 Jul 29, 2024
073a7ee
Reorganize materials commands 2
macflo8 Jul 29, 2024
3f766a1
Add updated cement calibration stats
macflo8 Jul 30, 2024
5c2c00c
Run TE-tool with "gdp" method by default
macflo8 Aug 1, 2024
8ebd0bc
Add cost updates from costs/new_inputs branch
macflo8 Aug 1, 2024
e53dded
Clean up new custom SSP build helper
macflo8 Aug 1, 2024
92ba15f
Add filter for relations in methanol build
macflo8 Aug 1, 2024
d154594
Add technologies to remove section to set.yaml
macflo8 Aug 1, 2024
b78a12b
Add extra MACRO calibration CLI command
macflo8 Aug 1, 2024
3cd0604
Add updated aluminum demands
macflo8 Aug 1, 2024
de34240
Revert to SSP2 for all SSPs for steel/cement
macflo8 Aug 1, 2024
adbf8ff
Add industry non-CO2 emission data file
macflo8 Aug 6, 2024
dbcb9ab
Add share constraint generator
macflo8 Aug 14, 2024
c35bca9
Move "other industry" utilities to module
macflo8 Aug 14, 2024
3616f3d
Extend docstrings and type hints
macflo8 Aug 16, 2024
e72fa6a
Split calibration function
macflo8 Aug 16, 2024
8fa05ea
Move CCS parametrization to input files
macflo8 Aug 16, 2024
c812672
Move set modification to set.yaml
macflo8 Aug 16, 2024
cedb76d
Add filename to iea_data_path cli option
macflo8 Aug 19, 2024
a9d1ce2
Add new residual industry calibration
macflo8 Aug 19, 2024
4bf45e2
Generalize coal_i share constraint helper
macflo8 Aug 19, 2024
43ca07e
Remove deprecated CCS function from build
macflo8 Aug 19, 2024
90e5f1e
Fix new "other industry" calibration workflow
macflo8 Aug 21, 2024
eafbbe5
Update data_generic to sdmx Code set
macflo8 Sep 2, 2024
6606656
Extend type hints and docstrings to data_util
macflo8 Sep 2, 2024
2dc95c7
Add water sets missing in SSP_dev
macflo8 Sep 2, 2024
3f6a4aa
Add MTO_petro 0 bound_activity_up for 2025
macflo8 Sep 2, 2024
baaaad7
Move co2_trans_disp parametrization to file
macflo8 Sep 2, 2024
d98f55b
Reorganize parametrization scripts
macflo8 Sep 2, 2024
d1617d5
Add new demand and activity data to other industry
macflo8 Sep 2, 2024
de85ec5
Add low temp heat share constraint modifier
macflo8 Sep 2, 2024
eca86e2
Integrate parametrization updates in build
macflo8 Sep 2, 2024
c525723
Add 0 bounds for low temp furnace modes
macflo8 Sep 2, 2024
6034f9d
Fix fix_cost timeseries break for i_therm tecs
macflo8 Sep 3, 2024
262df7d
Update SSP MACRO calibration workflow
macflo8 Sep 3, 2024
d4f0bef
Extend SSP 2020 activity calibration procedure
macflo8 Sep 3, 2024
8abe89c
Remove unused option from material build command
macflo8 Sep 5, 2024
e18d32f
Rename share constraint generator function
macflo8 Sep 5, 2024
c20fdb2
Remove unused context from materials reporting
macflo8 Sep 5, 2024
c5ff035
Fix in/output unit of furnaces input file
macflo8 Oct 5, 2024
1ce240d
Add new plastics CO2_Emission calculator
macflo8 Oct 12, 2024
8296b5d
Extend methanol fs downstream supply chain
macflo8 Oct 15, 2024
842339f
Add missing SSP industry input files
macflo8 Oct 17, 2024
b70518e
Fix material balance of MTO_petro technology
macflo8 Oct 22, 2024
6bfae19
Add new CO2_ind calculator for chemicals
macflo8 Oct 23, 2024
52a71c3
Add embodied emissions for ethanol to ethylene
macflo8 Oct 27, 2024
f0fc3a9
Remove "World" meth_ind_fs parameter regions
macflo8 Oct 28, 2024
b731bab
Fix biomass_nh3_ccs CO2_Emission coefficient
macflo8 Oct 28, 2024
3405088
Add ethanol_to_ethylene CO2_ind emi factor
macflo8 Oct 28, 2024
990d7a7
Replace get_nodes with tested utility
macflo8 Nov 1, 2024
8809251
Add CO2_cc coefficient for cokeoven
macflo8 Nov 5, 2024
9b60d34
Fix merge duplication
macflo8 Nov 11, 2024
acf5d0d
Migrate ssp cost updates
macflo8 Nov 11, 2024
5ce042d
Delete unused tab in cement input xlsx
macflo8 Nov 13, 2024
41fcba8
Update High scenario cost reductions
GamzeUnlu95 Nov 13, 2024
431445e
Add DAC to cost projection tool
ywpratama Nov 6, 2024
428abb7
Add add_dac tool and related data
ywpratama Nov 6, 2024
e2c3092
Add draft of CCS and DAC implementation script
ywpratama Nov 6, 2024
73876a1
Update DAC parameters
ywpratama Nov 7, 2024
9d43199
Fix bug and improve year_act parameter cut
ywpratama Nov 7, 2024
23cdaa2
Add low overshoot scenarios script
ywpratama Nov 11, 2024
e067a68
Add CO2_cc for leakages
ywpratama Nov 15, 2024
fb0165e
Add add_ccs_setup function
ywpratama Nov 15, 2024
2782674
Update data reading in add_tech function
ywpratama Nov 18, 2024
ed32f47
Add macro calibration files for all SSPs
macflo8 Nov 19, 2024
c2aa2af
Fix methanol final energy allocation bug
macflo8 Nov 21, 2024
613dd77
Add macro calibration files for SSPs
OFR-IIASA Dec 2, 2024
629e769
Change default region config in model/config from R14 to R12
OFR-IIASA Dec 2, 2024
85ed8a9
Change data_methanol syntax for reading file
OFR-IIASA Dec 2, 2024
a4b41e8
Add scenario transact statement to DAC setup workflow
OFR-IIASA Dec 2, 2024
7171131
Add diffusion constraint changes
GamzeUnlu95 Nov 27, 2024
5abbf17
Differentitate growth_new_capacity_up for SSPs
GamzeUnlu95 Dec 2, 2024
33927de
Differentiate the scalar used for initial_capacity_new for SSPs
GamzeUnlu95 Dec 2, 2024
98f849a
Update steel diffusion constraints for steel in baselines
GamzeUnlu95 Dec 4, 2024
bf186d9
Add custom cost differentiation for materials
GamzeUnlu95 Dec 4, 2024
7b9e851
Add steel recycling changes
GamzeUnlu95 Dec 5, 2024
84f5aaf
Fix 2025 material demands for all SSPs
macflo8 Nov 25, 2024
9b6ef76
Add updated aluminum capacity calibration
macflo8 Nov 25, 2024
52ea046
Add updated aluminum activity calibration
macflo8 Nov 25, 2024
c744fba
Remove aluminum timeseries growth constraints
macflo8 Nov 27, 2024
17ff064
Update aluminum sector furnace efficiencies
macflo8 Nov 28, 2024
90f89c6
Add typing to ssp functions to resolve warning
macflo8 Dec 2, 2024
9b49ab2
Remove unneeded rows from parametrization dfs
macflo8 Dec 2, 2024
6be0acb
Remove power sector module from build
macflo8 Dec 2, 2024
5bc5ba6
Add aluminum updates
macflo8 Dec 2, 2024
2c5cb3a
Set fuel constraints for aluminum refining
macflo8 Dec 2, 2024
41e9dc6
Limit soderberg aluminum new capacity
macflo8 Dec 2, 2024
f6cba10
Add regionally differentiated alumina heat inputs
macflo8 Dec 4, 2024
173610a
Fix 2025 industry energy demands for all SSPs
macflo8 Dec 5, 2024
a6872b5
Remove whitespaces in materials cost csv
macflo8 Dec 6, 2024
7592acf
Differentiate aluminum recycling
GamzeUnlu95 Dec 6, 2024
e8dceb3
Remove operation reserve for DAC
ywpratama Dec 6, 2024
91722a3
Update CO2 injection rates using literature values
ywpratama Dec 6, 2024
2b97fa0
Remove CO2 outputs at useful level
ywpratama Dec 6, 2024
452d787
Update clinker CCS heat demand
macflo8 Dec 6, 2024
0778826
Add exception for LED methanol reporting
macflo8 Dec 9, 2024
4ab6d11
Fix nuc_lc set config
adrivinca Dec 10, 2024
998a81c
Pre-filter remove.set to speed up the build process
adrivinca Dec 10, 2024
8630edc
Exclude pre-firstmodelyear years from material rep
macflo8 Dec 12, 2024
31de47c
Add dac module to function type hints
measrainsey Dec 17, 2024
40add14
Move `dac` cost reduction assumptions to module folder
measrainsey Dec 17, 2024
0ef4523
Add reduction_year as parameter in Config
measrainsey Jan 27, 2025
c9e3a8c
Switch from hardcoding reduction year to using Config.reduction_year
measrainsey Jan 27, 2025
cfd2bf8
Add attributes for Config.final_projection_year and Config.final_mode…
measrainsey Jan 27, 2025
351a728
Update projections code to use new year parameters
measrainsey Jan 27, 2025
8c14d81
Update docstrings and comments
measrainsey Jan 27, 2025
aa4b2f7
Edit missing column name
measrainsey Jan 27, 2025
d718385
Edit config comments
measrainsey Jan 27, 2025
094efbe
Add functions to modify pre-base year costs values
measrainsey Jan 27, 2025
50374f8
Add example of how to use update_scenario_costs()
measrainsey Jan 27, 2025
b064b99
Start changelog for SSP development
macflo8 Dec 20, 2024
75eac89
Refine formatting of SSP changelog
macflo8 Jan 7, 2025
6c726d7
Fix missing MEA alumina refining input
macflo8 Jan 7, 2025
7de93cf
Fix materials reporting bugs
macflo8 Jan 19, 2025
3e64b73
Add cost to alumina trade
macflo8 Jan 24, 2025
2231035
Delete deprecated input data
macflo8 Jan 25, 2025
978b280
Refactor aluminum input data
macflo8 Jan 26, 2025
be2cebe
Refactor cement input data
macflo8 Jan 26, 2025
2dab70a
Refactor steel input data
macflo8 Jan 27, 2025
4b8748f
TEMPORARY fix cooling function to enable integration with SSP workflow
OFR-IIASA Dec 6, 2024
750a5d7
Add argument for cost-tool function call horizon
OFR-IIASA Jan 29, 2025
ec6bd7c
Fix input level for furnace_biomass_refining tec
macflo8 Jan 29, 2025
5cdaabe
Fix 2025 methanol demand
macflo8 Jan 29, 2025
6260075
Add correction for accounting of CF4 emissions from industry
OFR-IIASA Jan 30, 2025
7601b6d
Fix materials timeseries read-in
macflo8 Jan 30, 2025
8249703
Remove 2020 entries from steel relation
macflo8 Jan 30, 2025
531ae51
Remove `final_year` attribute from Config
measrainsey Jan 27, 2025
5b8fc4c
Remove use of .costs.Config.final_year in 3 places
khaeru Feb 6, 2025
233571a
Satisfy ruff, mypy in 3 files
khaeru Feb 6, 2025
7cb9af2
Adjust dac_htg co2_cc and co2 utilisation setup
ywpratama Feb 13, 2025
7bdba24
Fix bugs on meth_h2 co2 input
ywpratama Feb 14, 2025
7ae9b5a
Fix Other Sector string manipulation bug
macflo8 Feb 17, 2025
82375ae
Fix aggregation of Final Energy Methanol
macflo8 Feb 17, 2025
8f36dc3
Fix typo in chemicals biomass aggregation
macflo8 Feb 17, 2025
14996b9
Set Liquids|Other to 0 after re-allocation
macflo8 Feb 17, 2025
cc14113
Fix Industry Electricity components
macflo8 Feb 17, 2025
d260b70
Update reporting to new meth_h2 modes
macflo8 Feb 17, 2025
f6aa214
Fix type hints using `or`
glatterf42 Feb 19, 2025
f4cac2d
Implement meth_h2 update in DAC-CCS workflow
macflo8 Feb 19, 2025
af3f637
Remove transact within transact statement
OFR-IIASA Feb 19, 2025
5900272
Adjust Materials reporter computation
macflo8 Feb 19, 2025
cf9e3d2
Fix industry reporting issues
macflo8 Mar 4, 2025
43ec038
Add missing electricity in ammonia aggregate
macflo8 Mar 4, 2025
cbd8c5e
Fix double counting of high-value chemicals
macflo8 Mar 4, 2025
c3874ec
Add renewable hist_2025 technologies
OFR-IIASA Feb 20, 2025
8112c0e
Add (but disabled) temporary fix for handling incosistent entries fro…
OFR-IIASA Feb 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ Commonly used classes may be imported directly from :mod:`message_ix_models`.
project/nextgen-carbon
project/ngfs
project/prisma
project/scenariomip
project/shape
project/sparccle
project/ssp
Expand Down
138 changes: 138 additions & 0 deletions doc/material/whatsnew.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
What's new
**********

Next release
============

SSP 3.0 development
---------------

By topic:

.. contents::
:local:
:backlinks: none


Non-model changes
^^^^^^^^^^^^^^^^^

- the ``--datafile`` option for the the material-ix build command was removed since it is not used anymore.
- a ``material-ix calibrate`` command has been added that can be used to run the MACRO calibration step in isolation.
- values from model parameter data that have a year_act and year_vtg columns greater than the technology lifetime are dropped since they are inflating the model size unnecessarily.
- a module :mod:`.model.material.share_constraints` for commodity share constraints was created, but is not fully used in the build yet.
.. todo:: consider differentiating the constraints by SSP

Demand
""""""

- the demand commodity ``level`` of methanol has been changed from ``final_material`` to ``demand``
- an additional demand projection setting for aluminum, steel and cement has been introduced called "highest" which SSP5 scenarios are mapped to
- the 2025 demands are fixed to the projected SSP2 values for each SSP
- Aluminum base year demand has been updated based on IAI MFA output for 2020
- Cement base year demand has been updated based on GlobBulk Consulting data for 2020
- Steel base year demand has been updated based on worldsteel association data for 2020


Post processing
"""""""""""""""

- the new industry reporter has been implemented :mod:`.model.material.report`, with mapping files in ``data/material/reporting``

Model development
^^^^^^^^^^^^^^^^^

.. _general-changes:

General
"""""""

- the power sector module has been deactivated for the SSP builds
- coal_i ``technology`` is share constrained in each region based on 2020 IEA statistics
- the low temperature share constraint for other industry is updated to reflect explicit modelling of heavy industry

.. todo:: compile low temperature literature and set up references

- the .tools.cost module has been updated to run in "gdp" mode when called by the materials build
- the .tools.cost module is called twice if ``--update_costs`` option is :any:`True` in ``material-ix build`` command to be able to get the correct cost projections for the non-MESSAGEix-Materials industry technologies (e.g ``coal_i``, ``sp_el_I`` etc.)
- the following utility functions were added/updated in utils.py
- for mapping country names with `ISO 3166-1 alpha-3 <https://en.wikipedia.org/wiki/ISO_3166-1_alpha-3>`_ codes with `pycountry <https://pypi.org/project/pycountry/>`_.

.. todo:: integrate in :mod:`.util.pycountry`

- adding a ``R12`` column with region values mapped based on a iso column of the same dataframe.
- the MACRO calibration excel input file updater is now also updating the ``demand_ref`` tab based on the given scenario ``demand`` parameter values.
- the MACRO calibration excel input file updater can be run in "extrapolation" mode or just take values from the scenario directly.


Aluminum
""""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the aluminum build reads SSP differentiated input data by reading from :file:`data/materials/aluminum/<SSPX>/aluminum_techno_economic.xlsx`
- the historical capacity of smelters is calibrated until 2020 using the genisim dataset.
- the historical activity of smelters is calibrated until 2020 using British Geological Survey data (Soderberg activity is calibrated assuming capacity shares computed from genisim dataset).
- the historical activity of smelters is calibrated until 2020 using British Geological Survey data.
- the historical activity of alumina refining is calibrated until 2020 using British Geological Survey data.
- a trade model for alumina is introduced to be able to calibrate refining and smelting activity.
- the process heat fuel consumption is calibrated using IAI data.
- the regional alumina refining process heat intensities are calibrated using IAI data.

Ammonia
"""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- a unit issue in the emission factor of ``biomass_NH3_ccs`` ``technology`` was fixed.

Cement
""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the cement build reads SSP differentiated input data by reading from :file:`data/materials/cement/<SSPX>/Global_cement_MESSAGE.xlsx`.
- selected regional ``CEMENT`` values in :file:`residual_industry_2019.csv` were updated since they seemed wrong when comparing the data to the IEA cement report.
.. todo:: TODO: paste link to IEA cement report here

- the heat input for clinker CCS addons was lowered a lot.
.. todo:: insert US NREL publication reference here

Other industry
""""""""""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the demands for the other industry are generated based on IEA historical data of the non-explicitly modelled sectors

.. note:: the new demand model still sits in a private repository

- the sector furnaces were missing non CO2 emission factors of their original MESSAGE counterpart (e.g ``coal_i``). Thus, they were copied from the originals and scaled with the input coefficients.
- the furnaces are now writing into the ``IndThermDemLink`` relation, that is required for the MESSAGE-GAINS linkage.

.. _methanol-changes:

Methanol
""""""""

- the model structure was slightly updated to simplify and correct the carbon emission balance accounting

- the negative emission coefficients that represent the carbon stored in long lived products, were moved from ``meth_t_d`` technology to a new technology ``meth_ind_fs``.

- the carbon balance of ``MTO_petro`` was not correct. The process emissions and the ``input``/``output`` parameters were updated based on new literature.

.. todo:: compile literature list and summarize in a paragraph

Petrochemicals
""""""""""""""

- together with the changes in :ref:`methanol-changes` the carbon accounting was improved:

- the negative emission coefficients that represent the carbon stored in long lived products, were moved from ``steam_cracker_petro`` ``technology`` to the ``production_HVC`` ``technology``.

- since carbon capture in plastics was moved upstream, ``ethanol_to_ethylene_petro`` ``technology`` needed a positive emission factor to represent the combusted chemicals part produced with ethylene feedstock from ethanol.

Steel
"""""

- the steel build reads SSP differentiated input data by reading from :file:`data/materials/steel/<SSPX>/Global_steel_MESSAGE.xlsx`.
- the maximum recycling relation was updated.
- the minimum recycling relation was changed for SSP1.
- the ``cokeoven_steel`` ``technology`` got its own bottom up CO2 emission accounting coefficient, to separate that from industrial emissions and move it to transformation emissions.
- the cost changes done in the last PR (insert steel hydrogen PR link) were also transferred to the :mod:`tools.cost` module and custom reduction rates were assigned.
12 changes: 12 additions & 0 deletions doc/project/scenariomip.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
ScenarioMIP
***********

There is currently no dedicated code in :mod:`message_ix_models` associated with this project.
This page is a place-holder.

See also :doc:`/project/ssp`.

Project information
===================

- Website: https://wcrp-cmip.org/mips/scenariomip/
9 changes: 9 additions & 0 deletions doc/project/ssp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ For the 2023–2025 update process:

- Project lead, lead modeler: :gh-user:`OFR-IIASA`

.. warning::

The 2024 SSP update is **under development**.
For details, see:

- Tracking issue :issue:`234`.
- `Issues and PRs labeled 'p:SSP-2024' <https://github.com/iiasa/message-ix-models/issues?q=label%3Ap:SSP-2024>`_ on the :mod:`.message_ix_models` GitHub repository.

See also :doc:`/project/scenariomip`.

Structure
=========
Expand Down
Loading
Loading