Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect pieces for SSP 2024/ScenarioMIP #235

Draft
wants to merge 183 commits into
base: main
Choose a base branch
from
Draft

Collect pieces for SSP 2024/ScenarioMIP #235

wants to merge 183 commits into from

Conversation

khaeru
Copy link
Member

@khaeru khaeru commented Sep 26, 2024

This PR is to track the working branch ssp-dev, which exists to help with work on the SSP 2024/ScenarioMIP project by collecting or pointing to all the code that is actually used to execute the workflow.

The goal is to be complete and up-to-date as possible for that purpose; other aspects (code quality, cleanliness, or readiness for main) are less important.

How to use

  • Per Collect pieces for SSP 2024/ScenarioMIP workflows #234, it is not necessary to use this branch. If you can make an atomic PR for individual, small changes, do that first. This branch will be regularly rebased on latest main, so will include those updates.
  • Update this branch, e.g. by cherry-picking commits or copying code from various other working branches used for this project.
  • Make a PR that targets into this branch, and then request that someone review it. Continue work on this branch, rather than the PR source branch.
  • DO NOT use the button “Update branch” → "Update with merge commit” to keep this branch up to date with main.
    • If necessary, use “Update branch” → “Update with rebase”, or ask in Slack for the branch to be rebased.

How to review

TBD: this PR may not eventually be merged.

PR checklist

  • Continuous integration checks all ✅ TBD if this will be required.
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

@khaeru khaeru added the p:SSP-2024 2024 SSP updates and ScenarioMIP label Sep 26, 2024
Copy link

codecov bot commented Sep 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.0%. Comparing base (78f361c) to head (c0b0872).

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #235     +/-   ##
=======================================
- Coverage   77.9%   77.0%   -0.9%     
=======================================
  Files        213     213             
  Lines      16512   16512             
=======================================
- Hits       12872   12724    -148     
- Misses      3640    3788    +148     
Files with missing lines Coverage Δ
message_ix_models/model/config.py 100.0% <ø> (ø)
message_ix_models/model/material/build.py 67.8% <ø> (ø)
message_ix_models/model/material/cli.py 24.2% <ø> (ø)
message_ix_models/model/material/data_aluminum.py 91.6% <ø> (ø)
...ssage_ix_models/model/material/data_ammonia_new.py 94.1% <ø> (ø)
message_ix_models/model/material/data_cement.py 74.2% <ø> (ø)
message_ix_models/model/material/data_generic.py 98.5% <ø> (ø)
message_ix_models/model/material/data_methanol.py 92.0% <ø> (ø)
message_ix_models/model/material/data_petro.py 78.7% <ø> (ø)
message_ix_models/model/material/data_steel.py 87.0% <ø> (ø)
... and 11 more

... and 7 files with indirect coverage changes

@khaeru
Copy link
Member Author

khaeru commented Nov 14, 2024

@GamzeUnlu95 @macflo8 —there now appears to be a conflict in a particular materials data file. I think this is the same one modified in #247 (FYI also @glatterf42 who reviewed there).

Can you please say whether these two changes to the file are perhaps the same, or different? Does one supersede the other, or are they parallel/non-overlapping?

This will allow to rebase and update the branch, which we need to do (urgently) in order for it to be usable in the ScenarioMIP process.

@khaeru
Copy link
Member Author

khaeru commented Nov 14, 2024

Can you please say whether these two changes to the file are perhaps the same, or different? Does one supersede the other, or are they parallel/non-overlapping?

The reply (in Slack) was that the changes in #247 supersede the ones that were on this branch.
Accordingly, I did:

  • git switch ssp-dev && git pull --rebase update my local copy of the branch.
  • git rebase -i main initiate interactive rebase.
  • Chose edit for the commit with subject "Move co2_trans_disp parametrization to file" —the only commit on this branch that modified the file.
  • Removed the changes to that file from the commit:
    git restore --staged message_ix_models/data/material/steel/Global_steel_MESSAGE.xlsx 
    git restore message_ix_models/data/material/steel/Global_steel_MESSAGE.xlsx 
    
  • git rebase --continue
  • git push --force

wind_res2,Renewable,0.15,0.3,0.53,0.53,0.65,0.75
wind_res3,Renewable,0.15,0.3,0.53,0.53,0.65,0.75
wind_res4,Renewable,0.15,0.3,0.53,0.53,0.65,0.75
dac_lt,CCS,0.1,0.3,0.5,0.7,0.9
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@measrainsey @ywpratama I think there is probably an issue to be resolved here.

  • In the earlier-merged commits on this branch, an additional column low_medium was added to this table, leading to 6 total columns with numeric values.
  • In parallel, the 3 rows for dac_lt etc. were added with only 5 numeric values.
  • Thus for instance the value 0.5 appears to be in the low_medium column, when the DAC-related commit added them to a medium column, and so on for the rest of the row.

I was able to deconflict the branch history, but the latter rows probably also need values (or NaNs) for low_medium inserted so that the entered values appear in the right place.

Please let me know if that's unclear or if you need help making the adjustment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha! @ywpratama could you add low_medium values for the DAC technologies? since i wouldn't know the assumptions to make here

@khaeru
Copy link
Member Author

khaeru commented Nov 26, 2024

I see the branch is showing conflicts, so I will rebase manually and force-push.

@khaeru
Copy link
Member Author

khaeru commented Nov 26, 2024

I see the branch is showing conflicts, so I will rebase manually and force-push.

Now done.

@macflo8
Copy link
Contributor

macflo8 commented Nov 29, 2024

I will drop my 3 most recent commits and force-push because they break the model build . The commits will be added later again when the industry revisions are fully functional.

@khaeru
Copy link
Member Author

khaeru commented Dec 3, 2024

Rebased after merge of #255.

@khaeru khaeru force-pushed the ssp-dev branch 2 times, most recently from d00bdab to 1e47d3c Compare December 4, 2024 15:32
@khaeru
Copy link
Member Author

khaeru commented Dec 4, 2024

@khaeru
Copy link
Member Author

khaeru commented Dec 4, 2024

Rebased after the merge of #258.

In doing so, there was a conflict in message_ix_models/data/costs/materials/tech_map.csv. I resolved this in favour of the version on main per that other PR, which is more recent.

FYI @measrainsey if you are making a PR into main and happen to notice that it touches files also modified on this branch, please feel free to cherry-pick particular commits from this branch into a smaller PR branch. That way, they can be reviewed and merged, and then they will vanish from this branch when it is later rebased. In the long run this can shrink the work we must eventually do to incorporate the contents of this branch in main.

@macflo8
Copy link
Contributor

macflo8 commented Dec 6, 2024

After clarifying with @ywpratama, we figured out that the last two commits were pushed accidentally by him. I will drop them now and force-push.

macflo8 and others added 29 commits March 10, 2025 17:32
- Move SSP differentiated parameters from input data to functions
- Remove duplicated input files for SSPs
- Change input data format from xlsx to csv
- Move SSP differentiated parameters from input data to functions
- Remove duplicated input files for SSPs
- Change input data format from xlsx to csv
Add missing scaling of 2025 values with residual demand factor
Cast datatypes of year columns to integers
Exclude 2020 from year list for max_regional_recycling_steel relation
parametrization
In all 3 cases, removed value 2110 is the same as the default value for the
renamed field 'final_model_year'. Remove the argument to allow the default to be
used.
- Apply `ruff format`.
- Address format check errors or add exclusions.
- Address or add mypy exclusions.
Only replace Liquids|Other and not Other Sector strings
- Move electricity mapping
- Rename function variables for readability
- Register new modes
- Transfer original meth_h2 mode parametrization to new modes
- Delete old meth_h2 mode parametrization
Drop "mode" column in genno Reporter

Needs to be dropped to allow for aggregation of methanol production with
new meth_h2 modes
- Add Solar to Other Sector total
- Add other chemicals to aggregates
- Add missing FE HVC total
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p:SSP-2024 2024 SSP updates and ScenarioMIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants