Transport improvements from 2025-W09 #300

khaeru · 2025-02-24T13:16:51Z

Adjust efficiencies of LDV PHEV technologies using duty cycle data (carried from Transport improvements from 2025-W05 #289 and Transport improvements from 2024-W49 #259).
Use/test LDV parametrization for non-demand parameters.
- Load factor (output)
- Confirm all mentioned at iiasa/message_data#580.
Debug possible double-counting via summation in IAMC-structured reporting of final energy (discussed 2025-03-06).
Add $y$ dimension to pdt-elasticity.csv, expand current data (discussed 2025-03-06).
Allow to distinguish LED-SSP1 from LED-SSP2 in pdt-cap.csv input data (discussed 2025-03-11).

How to review

Read the diff and note that the CI checks all pass.
More TBA.

PR checklist

Continuous integration checks all ✅
Add or expand tests; coverage checks both ✅
Add, expand, or update documentation.
- Input and output data flows for MESSAGEix-Transport are now completely documented.
Update doc/whatsnew.

codecov · 2025-02-24T13:31:05Z

Codecov Report

Attention: Patch coverage is 97.52066% with 12 lines in your changes missing coverage. Please review.

Project coverage is 77.4%. Comparing base (a45fce4) to head (6de5497).

Files with missing lines	Patch %	Lines
message_ix_models/model/transport/data.py	97.7%	4 Missing ⚠️
message_ix_models/util/sdmx.py	97.9%	3 Missing ⚠️
message_ix_models/model/transport/ldv.py	94.5%	2 Missing ⚠️
...sage_ix_models/tests/model/transport/test_build.py	88.2%	2 Missing ⚠️
message_ix_models/report/operator.py	94.1%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #300     +/-   ##
=======================================
- Coverage   78.3%   77.4%   -0.9%     
=======================================
  Files        217     215      -2     
  Lines      16851   16922     +71     
=======================================
- Hits       13196   13114     -82     
- Misses      3655    3808    +153

Files with missing lines	Coverage Δ
message_ix_models/model/transport/build.py	`95.0% <100.0%> (+<0.1%)`	⬆️
message_ix_models/model/transport/demand.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/disutility.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/groups.py	`97.0% <100.0%> (-0.1%)`	⬇️
message_ix_models/model/transport/key.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/non_ldv.py	`98.1% <100.0%> (-0.1%)`	⬇️
message_ix_models/model/transport/operator.py	`86.2% <100.0%> (-0.1%)`	⬇️
message_ix_models/model/transport/report.py	`60.0% <100.0%> (-0.7%)`	⬇️
message_ix_models/project/ssp/transport.py	`97.8% <100.0%> (ø)`
...ssage_ix_models/tests/model/transport/test_data.py	`86.4% <100.0%> (+0.8%)`	⬆️
... and 10 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Move to .report.operator from .transport.operator.

- Add data/transport/input-share.csv, .transport.files.input_share. - Incorporate via .ldv.prepare_tech_econ(). - Simplify key handling in prepare_tech_econ()

- Allow use of scenario="LED" in some cases and scenario="SSP1" in others. - Select LDV load factor data using "LED" label where necessary.

Fix LDV speed in SSP 1 scenario. For narrative consistency, ldv speed in SSP 1 should stay constant (or maybe even decrease) rather than increasing.

Based on discussion on 06-03-2025, experimenting with rather dramatic change in pdt-elasticity values by different regions for SSP1.

- Interpolate on this dimension in .demand. - Duplicate existing data for y in {2020, 2110}.

iiasa/message_data#599 (comment) Update R12_SAS value given the reasons in the thread above

See description. For now it is just a block comment. But these values are to be used to calculate LED-SSP1 variant

for current experimentation

- Move and adjust tests. - Sort entries in .transport.files. - Add .transport.files.iter_files(). - Add .transport.key.exo, populated using iter_files() - Adjust references.

…by merging functionality with Dataflow.__init__(). - Add Dataflow(module, …, i_o) arguments, Dataflow.FLAGS, Dataflow.intent. - Use shorthand functions in .transport.{data,files}. - Add .util.sdmx.{collect_structures,get}().

- Drop .transport.files submodule, moving contents to .data. - Update, expand, sort, and copyedit input data docs for consistency. - Add output data flows to documentation.

Avoid TypeError in broadcast_wildcard() with mixed str and sdmx.model.common.Code.

different short-term and long-term values

R12_WEU & NAM for NAM

RAIL Speed changed to 70 for SSP 1

revision of LED SSP 1 speeds

khaeru added enh New features or functionality transport MESSAGEix-Transport variant or transport in the base model labels Feb 24, 2025

khaeru self-assigned this Feb 24, 2025

khaeru requested a review from glatterf42 as a code owner February 24, 2025 13:16

khaeru marked this pull request as draft February 24, 2025 13:23

khaeru force-pushed the transport/2025-w09 branch 2 times, most recently from 93819ec to 15d2a67 Compare February 28, 2025 15:26

khaeru added 5 commits March 10, 2025 13:26

TEMPORARY Enable "transport" CI workflow for PR

733bced

Add check of LDV energy input

d7c2738

Use genno.Keys in .transport.ldv

a85ccc2

Handle >1 dimensions in broadcast_wildcard()

35df38d

- Move to .report.operator from .transport.operator.

Fix handling multiple dims in MappingAdapter

a68d431

khaeru force-pushed the transport/2025-w09 branch from cd7b261 to 1dcd054 Compare March 10, 2025 12:27

khaeru and others added 7 commits March 11, 2025 10:36

Handle input share of LDV PHEV technology

63253a4

- Add data/transport/input-share.csv, .transport.files.input_share. - Incorporate via .ldv.prepare_tech_econ(). - Simplify key handling in prepare_tech_econ()

Add .transport.operator.indexer_scenario()

c137ea5

- Allow use of scenario="LED" in some cases and scenario="SSP1" in others. - Select LDV load factor data using "LED" label where necessary.

Improve docstring, comments in test_debug()

90617a7

Update SSP 1 ldv speed.csv

d9daa89

Fix LDV speed in SSP 1 scenario. For narrative consistency, ldv speed in SSP 1 should stay constant (or maybe even decrease) rather than increasing.

Experimental pdt-elasticity values for SSP1.csv

6b9eca9

Based on discussion on 06-03-2025, experimenting with rather dramatic change in pdt-elasticity values by different regions for SSP1.

Add 'y' dimension to .transport.exo.pdt_elasticity

bea700d

- Interpolate on this dimension in .demand. - Duplicate existing data for y in {2020, 2110}.

Update R12_SAS pdt-cap

bee2848

iiasa/message_data#599 (comment) Update R12_SAS value given the reasons in the thread above

khaeru force-pushed the transport/2025-w09 branch from 88851b1 to bee2848 Compare March 11, 2025 09:38

r-aneeque added 3 commits March 11, 2025 12:58

LED-SSP1 mode speeds

abae0ea

See description. For now it is just a block comment. But these values are to be used to calculate LED-SSP1 variant

TEMPORARY limit runs to SSP 1 & SSP 2

a6c4c0e

for current experimentation

remove white spaces

da8f2c6

khaeru force-pushed the transport/2025-w09 branch from 2df36dc to ef606e7 Compare March 12, 2025 09:49

khaeru added 4 commits March 12, 2025 12:15

Move .transport…ExogenousDataFile to .util.sdmx.Dataflow

4462840

- Move and adjust tests. - Sort entries in .transport.files. - Add .transport.files.iter_files(). - Add .transport.key.exo, populated using iter_files() - Adjust references.

Remove .util.sdmx.make_dataflow()

4b9b764

…by merging functionality with Dataflow.__init__(). - Add Dataflow(module, …, i_o) arguments, Dataflow.FLAGS, Dataflow.intent. - Use shorthand functions in .transport.{data,files}. - Add .util.sdmx.{collect_structures,get}().

Add Dataflow._update_doc() to generate docstrings

c102885

Consolidate data flows in .transport.data

43c182f

- Drop .transport.files submodule, moving contents to .data. - Update, expand, sort, and copyedit input data docs for consistency. - Add output data flows to documentation.

khaeru added 2 commits March 12, 2025 12:15

Ensure "c::transport+base" items are str

2c54781

Avoid TypeError in broadcast_wildcard() with mixed str and sdmx.model.common.Code.

Add get_version(with_dev=...) argument

be40603

khaeru force-pushed the transport/2025-w09 branch from ef606e7 to be40603 Compare March 12, 2025 11:20

khaeru and others added 27 commits March 12, 2025 13:09

Adjust for ixmp/message_ix 3.7.0; Python 3.9

6c6201f

Speed setting for LED SSP 1

99a3755

Update pdt-elasticity values for SSP 1 & SSP 2

ca3b6fb

TEMPORARY Run SSP 1 only

12d0a96

REVERT SSP 1 speeds

9cf1085

remove R12_NAM share weight convergence

af4d1ef

Experimental swc for R12_NAM

dfc5169

Update pdt-elasticity values for SSP 1

079f6a4

different short-term and long-term values

Experimental swc for R12_NAM

371752b

R12_WEU & NAM for NAM

Update speed for SSP1 RAIL

d5b20c6

RAIL Speed changed to 70 for SSP 1

Experimental pdt-elasticity values for SSP1 GN (except NAM)

7cbb455

Experimental swc for R12_NAM blank

6700827

Revert back last changes (blank, WEU) to R12_NAM swc

0d99ef0

Experimental: AIR speed mode in certain regions in SSP 1

e5858fe

LED SSP 1 speeds

797df23

revision of LED SSP 1 speeds

correct LED SSP1 speeds

cb72c4d

Revert Speeds back to SSP1

04552a1

Update pdt-elasticity for SSP2

b07905f

Update transport to run SSP 2 only

069d3b8

correction for pdt-elasticity SSP2

e39f025

Update pdt-elasticity for SSP 3

f34c0a6

TEMPORARY Run SSP3 only

25b4869

fix the SSP3 value for R12_RCPA

4e3db92

Update pdt-elasticity for SSP 4

201b613

Update load-factor-ldv for SSP differtiation

9a048ea

Update pdt-cap for LED SSP1

7b9e381

run all SSPs baseline version

6de5497

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transport improvements from 2025-W09 #300

Transport improvements from 2025-W09 #300

khaeru commented Feb 24, 2025 •

edited

Loading

codecov bot commented Feb 24, 2025 •

edited

Loading

Transport improvements from 2025-W09 #300

Are you sure you want to change the base?

Transport improvements from 2025-W09 #300

Conversation

khaeru commented Feb 24, 2025 • edited Loading

How to review

PR checklist

codecov bot commented Feb 24, 2025 • edited Loading

Codecov Report

khaeru commented Feb 24, 2025 •

edited

Loading

codecov bot commented Feb 24, 2025 •

edited

Loading