{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Advanced functionality - pandas groupby with pymrio satellite accounts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook examplifies how to directly apply [Pandas](https://pandas.pydata.org/) core functions (in this case [groupby and aggregation](https://pandas.pydata.org/pandas-docs/stable/groupby.html)) to the pymrio system." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## WIOD material extension aggregation - stressor w/o compartment info" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we use the WIOD MRIO system (see the notebook [\"Automatic downloading of MRIO databases\"](autodownload.ipynb#WIOD-download) for how to automatically retrieve this database) and will aggregate the WIOD material stressor for used and unused materials. We assume, that the WIOD system is available at" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "wiod_folder = \"/tmp/mrios/WIOD2013\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get started we import pymrio" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import pymrio" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the example here, we use the data from 2009:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "wiod09 = pymrio.parse_wiod(path=wiod_folder, year=2009)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "WIOD includes multiple material accounts, specified for the \"Used\" and \"Unused\" category, as well as information on the total. We will use the latter to confirm our calculations:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
regionAUS...RoW
sectorAtBC15t1617t18192021t22232425...6364J7071t74LMNOP
stressor
Biomass_animals_Used238.4871900.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_feed_Used314501.7757750.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_food_Used78736.3484300.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_forestry_Used21443.7129520.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_other_Used647.0385630.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_coal_Used0.0000004.084490e+050.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_gas_Used0.0000003.671908e+040.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_oil_Used0.0000002.191849e+040.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_other_Used0.0000000.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_construction_Used0.0000001.098489e+050.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_industrial_Used0.0000002.444270e+040.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_metals_Used0.0000007.019911e+050.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_animals_Unused38.0940640.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_feed_Unused194.5976670.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_food_Unused17925.8413580.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_forestry_Unused3216.5569430.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Biomass_other_Unused128.6102530.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_coal_Unused0.0000006.430405e+060.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_gas_Unused0.0000004.759046e+030.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_oil_Unused0.0000004.822068e+030.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Fossil_other_Unused0.0000000.000000e+000.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_construction_Unused0.0000003.015773e+030.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_industrial_Unused0.0000003.389710e+040.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Minerals_metals_Unused0.0000006.919846e+050.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Total437071.0631968.472253e+060.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
\n", "

25 rows × 1435 columns

\n", "
" ], "text/plain": [ "region AUS \\\n", "sector AtB C 15t16 17t18 19 \n", "stressor \n", "Biomass_animals_Used 238.487190 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_feed_Used 314501.775775 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_food_Used 78736.348430 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_forestry_Used 21443.712952 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_other_Used 647.038563 0.000000e+00 0.0 0.0 0.0 \n", "Fossil_coal_Used 0.000000 4.084490e+05 0.0 0.0 0.0 \n", "Fossil_gas_Used 0.000000 3.671908e+04 0.0 0.0 0.0 \n", "Fossil_oil_Used 0.000000 2.191849e+04 0.0 0.0 0.0 \n", "Fossil_other_Used 0.000000 0.000000e+00 0.0 0.0 0.0 \n", "Minerals_construction_Used 0.000000 1.098489e+05 0.0 0.0 0.0 \n", "Minerals_industrial_Used 0.000000 2.444270e+04 0.0 0.0 0.0 \n", "Minerals_metals_Used 0.000000 7.019911e+05 0.0 0.0 0.0 \n", "Biomass_animals_Unused 38.094064 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_feed_Unused 194.597667 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_food_Unused 17925.841358 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_forestry_Unused 3216.556943 0.000000e+00 0.0 0.0 0.0 \n", "Biomass_other_Unused 128.610253 0.000000e+00 0.0 0.0 0.0 \n", "Fossil_coal_Unused 0.000000 6.430405e+06 0.0 0.0 0.0 \n", "Fossil_gas_Unused 0.000000 4.759046e+03 0.0 0.0 0.0 \n", "Fossil_oil_Unused 0.000000 4.822068e+03 0.0 0.0 0.0 \n", "Fossil_other_Unused 0.000000 0.000000e+00 0.0 0.0 0.0 \n", "Minerals_construction_Unused 0.000000 3.015773e+03 0.0 0.0 0.0 \n", "Minerals_industrial_Unused 0.000000 3.389710e+04 0.0 0.0 0.0 \n", "Minerals_metals_Unused 0.000000 6.919846e+05 0.0 0.0 0.0 \n", "Total 437071.063196 8.472253e+06 0.0 0.0 0.0 \n", "\n", "region ... RoW \\\n", "sector 20 21t22 23 24 25 ... 63 64 J \n", "stressor ... \n", "Biomass_animals_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_feed_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_food_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_forestry_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_other_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_coal_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_gas_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_oil_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_other_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_construction_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_industrial_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_metals_Used 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_animals_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_feed_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_food_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_forestry_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Biomass_other_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_coal_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_gas_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_oil_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Fossil_other_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_construction_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_industrial_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Minerals_metals_Unused 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "Total 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 \n", "\n", "region \n", "sector 70 71t74 L M N O P \n", "stressor \n", "Biomass_animals_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_feed_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_food_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_forestry_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_other_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_coal_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_gas_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_oil_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_other_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_construction_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_industrial_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_metals_Used 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_animals_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_feed_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_food_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_forestry_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Biomass_other_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_coal_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_gas_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_oil_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Fossil_other_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_construction_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_industrial_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Minerals_metals_Unused 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Total 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "\n", "[25 rows x 1435 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "wiod09.mat.F" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To aggregate these with the Pandas groupby function, we need to specify the groups which should be grouped by Pandas.\n", "Pymrio contains a helper function which builds such a matching dictionary.\n", "The matching can also include regular expressions to simplify the build:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'Biomass_animals_Used': 'Material Used',\n", " 'Biomass_feed_Used': 'Material Used',\n", " 'Biomass_food_Used': 'Material Used',\n", " 'Biomass_forestry_Used': 'Material Used',\n", " 'Biomass_other_Used': 'Material Used',\n", " 'Fossil_coal_Used': 'Material Used',\n", " 'Fossil_gas_Used': 'Material Used',\n", " 'Fossil_oil_Used': 'Material Used',\n", " 'Fossil_other_Used': 'Material Used',\n", " 'Minerals_construction_Used': 'Material Used',\n", " 'Minerals_industrial_Used': 'Material Used',\n", " 'Minerals_metals_Used': 'Material Used',\n", " 'Biomass_animals_Unused': 'Material Unused',\n", " 'Biomass_feed_Unused': 'Material Unused',\n", " 'Biomass_food_Unused': 'Material Unused',\n", " 'Biomass_forestry_Unused': 'Material Unused',\n", " 'Biomass_other_Unused': 'Material Unused',\n", " 'Fossil_coal_Unused': 'Material Unused',\n", " 'Fossil_gas_Unused': 'Material Unused',\n", " 'Fossil_oil_Unused': 'Material Unused',\n", " 'Fossil_other_Unused': 'Material Unused',\n", " 'Minerals_construction_Unused': 'Material Unused',\n", " 'Minerals_industrial_Unused': 'Material Unused',\n", " 'Minerals_metals_Unused': 'Material Unused',\n", " 'Total': 'Total'}" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "groups = wiod09.mat.get_index(\n", " as_dict=True,\n", " grouping_pattern={\".*_Used\": \"Material Used\", \".*_Unused\": \"Material Unused\"},\n", ")\n", "groups" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note, that the grouping contains the rows which do not match any of the specified groups. \n", "This allows to easily aggregates only parts of a specific stressor set. To actually omit these groups\n", "include them in the matching pattern and provide None as value.\n", "\n", "To have the aggregated data alongside the original data, we first copy the detailed satellite account:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "wiod09.mat_agg = wiod09.mat.copy(new_name=\"Aggregated matrial accounts\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, we use the pymrio get_DataFrame iterator together with the pandas groupby and sum functions to aggregate the stressors.\n", "For the dataframe containing the unit information, we pass a custom function which concatenate non-unique unit strings." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "for df_name, df in zip(\n", " wiod09.mat_agg.get_DataFrame(data=False, with_unit=True, with_population=False),\n", " wiod09.mat_agg.get_DataFrame(data=True, with_unit=True, with_population=False),\n", "):\n", " if df_name == \"unit\":\n", " wiod09.mat_agg.__dict__[df_name] = df.groupby(groups).apply(\n", " lambda x: \" & \".join(x.unit.unique())\n", " )\n", " else:\n", " wiod09.mat_agg.__dict__[df_name] = df.groupby(groups).sum()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
regionAUS...RoW
sectorAtBC15t1617t18192021t22232425...6364J7071t74LMNOP
Material Unused21503.7002857.168884e+060.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Material Used415567.3629101.303369e+060.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
Total437071.0631968.472253e+060.00.00.00.00.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
\n", "

3 rows × 1435 columns

\n", "
" ], "text/plain": [ "region AUS \\\n", "sector AtB C 15t16 17t18 19 20 21t22 23 \n", "Material Unused 21503.700285 7.168884e+06 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Material Used 415567.362910 1.303369e+06 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Total 437071.063196 8.472253e+06 0.0 0.0 0.0 0.0 0.0 0.0 \n", "\n", "region ... RoW \\\n", "sector 24 25 ... 63 64 J 70 71t74 L M N O \n", "Material Unused 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Material Used 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "Total 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "\n", "region \n", "sector P \n", "Material Unused 0.0 \n", "Material Used 0.0 \n", "Total 0.0 \n", "\n", "[3 rows x 1435 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "wiod09.mat_agg.F" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Material Unused 1000 tonnes\n", "Material Used 1000 tonnes\n", "Total 1000 tonnes\n", "dtype: object" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "wiod09.mat_agg.unit" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use with stressors including compartment information:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The same regular expression grouping can be used to aggregate stressor data which is given per compartment.\n", "To do so, the matching dict needs to consist of tuples corresponding to a valid index value in the DataFrames. \n", "Each position in the tuple is interprested as a regular expression. \n", "Using the get_index method gives a good indication how a valid grouping dict should look like:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{('emission_type1', 'air'): ('emission_type1', 'air'),\n", " ('emission_type2', 'water'): ('emission_type2', 'water')}" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tt = pymrio.load_test()\n", "tt.emissions.get_index(as_dict=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With that information, we can now build our own grouping dict, e.g.:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "agg_groups = {(\"emis.*\", \".*\"): \"all emissions\"}" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{('emission_type1', 'air'): 'all emissions',\n", " ('emission_type2', 'water'): 'all emissions'}" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "group_dict = tt.emissions.get_index(as_dict=True, grouping_pattern=agg_groups)\n", "group_dict" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Which can then be used to aggregate the satellite account:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "for df_name, df in zip(\n", " tt.emissions.get_DataFrame(data=False, with_unit=True, with_population=False),\n", " tt.emissions.get_DataFrame(data=True, with_unit=True, with_population=False),\n", "):\n", " if df_name == \"unit\":\n", " tt.emissions.__dict__[df_name] = df.groupby(group_dict).apply(\n", " lambda x: \" & \".join(x.unit.unique())\n", " )\n", " else:\n", " tt.emissions.__dict__[df_name] = df.groupby(group_dict).sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this case we loose the information on the compartment. To reset the index do:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "\n", "tt.emissions.set_index(pd.Index(tt.emissions.get_index(), name=\"stressor\"))" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
regionreg1reg2...reg5reg6
sectorfoodminingmanufactoringelectricityconstructiontradetransportotherfoodmining...transportotherfoodminingmanufactoringelectricityconstructiontradetransportother
stressor
all emissions1987315.271008791.38524377356.1828413081.552901538.315387134.122779986.110291268.61902772.74376842.094...4649916017964832.320604104.18286580.6125872643.056775747.77561126.83208793455812326.338415421
\n", "

1 rows × 48 columns

\n", "
" ], "text/plain": [ "region reg1 \\\n", "sector food mining manufactoring electricity \n", "stressor \n", "all emissions 1987315.27 1008791.385 24377356.18 28413081.55 \n", "\n", "region reg2 \\\n", "sector construction trade transport other food \n", "stressor \n", "all emissions 2901538.31 5387134.1 22779986.1 10291268.6 1902772.74 \n", "\n", "region ... reg5 reg6 \\\n", "sector mining ... transport other food mining \n", "stressor ... \n", "all emissions 376842.094 ... 46499160 17964832.3 20604104.1 8286580.6 \n", "\n", "region \\\n", "sector manufactoring electricity construction trade transport \n", "stressor \n", "all emissions 125872643.0 56775747.7 7561126.8 32087934 55812326.3 \n", "\n", "region \n", "sector other \n", "stressor \n", "all emissions 38415421 \n", "\n", "[1 rows x 48 columns]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tt.emissions.F" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.20" } }, "nbformat": 4, "nbformat_minor": 4 }