Skip to content

Commit bbd8ca5

Browse files
khaerukhaeru
committed
Restructure top-level documentation
- Invoke sphinx-autogen separately to generate auto summaries. - Provide explicit files for each module; include the autosummaries and provide a place for additional content.
1 parent f99368c commit bbd8ca5

19 files changed

+143
-57
lines changed

‎.gitignore

+1-2
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,7 @@ instance/
7171

7272
# Sphinx documentation
7373
doc/_build/
74-
doc/api
75-
doc/data
74+
doc/_api
7675

7776
# PyBuilder
7877
target/

‎doc/Makefile

+6-2
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,13 @@ BUILDDIR = _build
1212
help:
1313
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
1414

15-
.PHONY: help Makefile
15+
.PHONY: help Makefile autogen
16+
17+
# Invoke sphinx-autogen using api.in, which is not within the doc sources
18+
autogen:
19+
sphinx-autogen -o _api -t _template api.in
1620

1721
# Catch-all target: route all unknown targets to Sphinx using the new
1822
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19-
%: Makefile
23+
%: Makefile autogen
2024
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎doc/_template/autosummary/module.rst

+6-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11
.. From https://stackoverflow.com/a/62613202/2362198
22
3-
{{ ("``." + fullname.split(".", maxsplit=1)[1] + "`` " + objtype) | underline(line="*") }}
3+
.. {{ (":mod:`." + fullname.split(".", maxsplit=1)[1] + "` " + objtype) | underline(line="*") }}
4+
5+
Code reference
6+
==============
7+
8+
.. currentmodule:: {{ fullname }}
49

510
.. automodule:: {{ fullname }}
611

‎doc/adb.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Asian Development Bank
2+
**********************
3+
4+
.. include:: _api/transport_data.adb.rst

‎doc/api.in

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
.. currentmodule:: transport_data
2+
3+
.. autosummary::
4+
:toctree: _api
5+
:recurse:
6+
7+
adb
8+
cli
9+
config
10+
estat
11+
iamc
12+
jrc
13+
org
14+
proto
15+
registry
16+
testing
17+
tests
18+
util

‎doc/cli.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Command-line interface
2+
**********************
3+
4+
.. include:: _api/transport_data.cli.rst

‎doc/config.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Configuration
2+
*************
3+
4+
.. include:: _api/transport_data.config.rst

‎doc/estat.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Eurostat
2+
********
3+
4+
.. include:: _api/transport_data.estat.rst

‎doc/iamc.rst

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
IAM Consortium
2+
**************
3+
4+
The "IAMC format" or "IAMC template" refers to a variety of similar data formats with a similar structure, developed by the **Integrated Assessment Modeling Consortium (IAMC)** and
5+
others, for data that is output from or input to integrated assessment models.
6+
Similar DSDs are commonly used in related research.
7+
8+
These data structures are characterized by:
9+
10+
- Common dimensions, including:
11+
12+
- "Model", "Scenario" —identifying a model and particular configuration of that model.
13+
- "Region" —geography.
14+
- "Year" —if in ‘long’ format; or also commonly a ‘wide’ format with one column per distinct year.
15+
- "Unit" —in some cases this is effectively an attribute; in other cases, it may be a dimension.
16+
See below.
17+
- "Variable" —see below.
18+
19+
- Combination of several data flows in the same file:
20+
21+
- Codes appearing in the "Variable" dimension are strings with a varying number of parts separated by the pipe ("|") character.
22+
- The first (or only) part indicates the measure concept, e.g. "Population".
23+
- Subsequent parts are codes for additional dimensions.
24+
For example in "Population|Female|50—59Y", "Female" may be a code for a gender dimension, and "50–59Y" may be a code for an age dimension. The data message usually does not identify these dimensions, and separate structural information may be provided only as text and not in machine-readable formats.
25+
26+
- Specification of "templates" in the form of files in the same format as the data, with no observation values.
27+
These provide code lists for the "Variable" and sometimes other dimensions.
28+
These thus, more or less explicitly, specify the measures, dimensions, codes, etc. for the various data flows included.
29+
30+
.. include:: _api/transport_data.iamc.rst

‎doc/index.rst

+38-17
Original file line numberDiff line numberDiff line change
@@ -37,31 +37,52 @@ General
3737
Data and metadata
3838
=================
3939

40-
.. autosummary::
41-
:toctree: data
40+
The following modules contain code tailored to *individual data providers*.
41+
They handle tasks including:
42+
43+
- Parse (meta)data or structure information from providers' idiosyncratic data formats.
44+
- Provide metadata and structure information that is implicit; not directly available from the provider itself.
45+
46+
.. toctree::
4247
:caption: Data and metadata
43-
:recursive:
48+
:hidden:
4449

45-
~transport_data.adb
46-
~transport_data.estat
47-
~transport_data.iamc
48-
~transport_data.jrc
49-
~transport_data.org
50-
~transport_data.proto
50+
adb
51+
estat
52+
iamc
53+
jrc
54+
org
55+
proto
56+
57+
- :mod:`.adb`: :doc:`adb`
58+
- :mod:`.estat`: :doc:`estat`
59+
- :mod:`.iamc`: :doc:`iamc`
60+
- :mod:`.jrc`: :doc:`jrc`
61+
- :mod:`.org`: :doc:`org`
62+
- :mod:`.proto`: :doc:`proto`
5163

5264
Common code and utilities
5365
=========================
5466

55-
.. autosummary::
56-
:toctree: api
67+
The following modules contain *generic* code and utilities usable with (meta)data from multiple providers or sources or created by TDCI.
68+
69+
.. toctree::
5770
:caption: Common code and utilities
58-
:recursive:
71+
:hidden:
5972

60-
~transport_data.cli
61-
~transport_data.config
62-
~transport_data.registry
63-
~transport_data.tests
64-
~transport_data.util
73+
cli
74+
config
75+
registry
76+
testing
77+
tests
78+
util
79+
80+
- :mod:`.cli`: :doc:`cli`
81+
- :mod:`.config`: :doc:`config`
82+
- :mod:`.registry`: :doc:`registry`
83+
- :mod:`.testing`: :doc:`testing`
84+
- :mod:`.tests`: :doc:`tests`
85+
- :mod:`.util`: :doc:`util`
6586

6687
Indices and tables
6788
==================

‎doc/jrc.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
JRC of the European Commission
2+
******************************
3+
4+
.. include:: _api/transport_data.jrc.rst

‎doc/org.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
TDCI itself
2+
***********
3+
4+
.. include:: _api/transport_data.org.rst

‎doc/proto.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
2023 TDC prototype
2+
******************
3+
4+
.. include:: _api/transport_data.proto.rst

‎doc/registry.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
(Meta)data registry
2+
*******************
3+
4+
.. include:: _api/transport_data.registry.rst

‎doc/testing.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Utilities for testing
2+
*********************
3+
4+
.. include:: _api/transport_data.testing.rst

‎doc/tests.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Test suite
2+
**********
3+
4+
.. include:: _api/transport_data.tests.rst

‎doc/util.rst

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Utilities
2+
*********
3+
4+
.. include:: _api/transport_data.util.rst

‎transport_data/config.py

-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
"""Configuration."""
2-
31
import json
42
from dataclasses import asdict, dataclass, field, fields
53
from pathlib import Path

‎transport_data/iamc/__init__.py

-33
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,5 @@
11
"""Handle data and structure for IAMC-like formats.
22
3-
The "IAMC format" or "IAMC template" refers to a variety of similar data formats with a
4-
similar structure, developed by the Integrated Assessment Modeling Consortium (IAMC) and
5-
others, for data that is output from or input to integrated assessment models. Similar
6-
DSDs are commonly used in related research.
7-
8-
These data structures are characterized by:
9-
10-
- Common dimensions, including:
11-
12-
- "Model", "Scenario" —identifying a model and particular configuration of that model.
13-
- "Region" —geography.
14-
- "Year" —if in ‘long’ format; or also commonly a ‘wide’ format with one column per
15-
distinct year.
16-
- "Unit" —in some cases this is effectively an attribute; in other cases, it may be a
17-
dimension. See below.
18-
- "Variable" —see below.
19-
20-
- Combination of several data flows in the same file:
21-
22-
- Codes appearing in the "Variable" dimension are strings with a varying number of
23-
parts separated by the pipe ("|") character.
24-
- The first (or only) part indicates the measure concept, e.g. "Population".
25-
- Subsequent parts are codes for additional dimensions. For example in
26-
"Population|Female|50—59Y", "Female" may be a code for a gender dimension, and
27-
"50–59Y" may be a code for an age dimension. The data message usually does not
28-
identify these dimensions, and separate structural information may be provided only
29-
as text and not in machine-readable formats.
30-
31-
- Specification of "templates" in the form of files in the same format as the data, with
32-
no observation values. These provide code lists for the "Variable" and sometimes other
33-
dimensions. These thus, more or less explicitly, specify the measures, dimensions,
34-
codes, etc. for the various data flows included.
35-
363
.. todo:: Add a function to generate distinct DSDs for each data flow in a data set.
374
.. todo:: Add function(s) to reshape IAMC-like data.
385
"""

0 commit comments

Comments
 (0)