{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pymrio Tutorial\n",
    "\n",
    "A complete tutorial covering all top-level functions of pymrio, using the test MRIO system.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "## Setup and Installation\n",
    "\n",
    "Before starting this tutorial, make sure you've got pymrio installed. You can grab it from conda-forge or PyPi.\n",
    "Use pip, mamba, conda, or whatever package manager you prefer to get it sorted. For example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "## Getting Started with Test MRIO Data\n",
    "\n",
    "We begin by importing pymrio and loading the test MRIO system. This small test system contains six regions and eight sectors, making it ideal for learning purposes.\n",
    "\n",
    "Note that any other MRIO database can be used with the same functions demonstrated here.\n",
    "The test system serves only as a representative example for larger, real-world datasets. See the other notebooks on MRIO downloading and handling (for example for [EXIOBASE](working_with_exiobase.ipynb)) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IO System with parameters: Z, Y, unit, population, meta, factor_inputs, emissions\n",
      "Type of object: <class 'pymrio.core.mriosystem.IOSystem'>\n",
      "Available extensions: ['Factor Inputs', 'Emissions']\n",
      "Regions: Index(['reg1', 'reg2', 'reg3', 'reg4', 'reg5', 'reg6'], dtype='object', name='region')\n",
      "Sectors: Index(['food', 'mining', 'manufactoring', 'electricity', 'construction',\n",
      "       'trade', 'transport', 'other'],\n",
      "      dtype='object', name='sector')\n",
      "Final demand categories: Index(['Final consumption expenditure by households',\n",
      "       'Final consumption expenditure by non-profit organisations serving households (NPISH)',\n",
      "       'Final consumption expenditure by government',\n",
      "       'Gross fixed capital formation', 'Changes in inventories',\n",
      "       'Changes in valuables', 'Export'],\n",
      "      dtype='object', name='category')\n",
      "Extensions: ['Factor Inputs', 'Emissions']\n",
      "Rows in emissions extension: MultiIndex([('emission_type1',   'air'),\n",
      "            ('emission_type2', 'water')],\n",
      "           names=['stressor', 'compartment'])\n"
     ]
    }
   ],
   "source": [
    "import pymrio\n",
    "\n",
    "# Load the test MRIO system\n",
    "test_mrio = pymrio.load_test()\n",
    "\n",
    "# Display basic information about the system\n",
    "print(test_mrio)\n",
    "print(\"Type of object:\", type(test_mrio))\n",
    "print(\"Available extensions:\", test_mrio.extensions)\n",
    "\n",
    "# Get regions and sectors\n",
    "print(\"Regions:\", test_mrio.regions)\n",
    "print(\"Sectors:\", test_mrio.sectors)\n",
    "print(\"Final demand categories:\", test_mrio.Y_categories)\n",
    "print(\"Extensions:\", test_mrio.extensions)\n",
    "print(\"Rows in emissions extension:\", test_mrio.emissions.rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Search Functionality\n",
    "\n",
    "Pymrio offers comprehensive search capabilities to find specific accounts, regions, sectors, stressors, and impacts:\n",
    "The terms are the same as the pandas regex method names and work in the same way.\n",
    "For more details, check out the [explore notebook](explore.ipynb) and the [pandas regex documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/text.html#working-with-regular-expressions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Search results for 'food': {'index': MultiIndex([('reg1', 'food'),\n",
      "            ('reg2', 'food'),\n",
      "            ('reg3', 'food'),\n",
      "            ('reg4', 'food'),\n",
      "            ('reg5', 'food'),\n",
      "            ('reg6', 'food')],\n",
      "           names=['region', 'sector']), 'sectors': Index(['food'], dtype='object', name='sector')}\n"
     ]
    }
   ],
   "source": [
    "# Search for specific terms across the system\n",
    "search_results = test_mrio.find(\"food\")\n",
    "print(\"Search results for 'food':\", search_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Contains 'electricity': MultiIndex([('reg1', 'electricity'),\n",
      "            ('reg2', 'electricity'),\n",
      "            ('reg3', 'electricity'),\n",
      "            ('reg4', 'electricity'),\n",
      "            ('reg5', 'electricity'),\n",
      "            ('reg6', 'electricity')],\n",
      "           names=['region', 'sector'])\n"
     ]
    }
   ],
   "source": [
    "# More specific search methods\n",
    "contains_results = test_mrio.contains(\"electricity\")\n",
    "print(\"Contains 'electricity':\", contains_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extension search for 'emission': {'Factor Inputs': Index([], dtype='object', name='inputtype'), 'Emissions': MultiIndex([('emission_type1',   'air'),\n",
      "            ('emission_type2', 'water')],\n",
      "           names=['stressor', 'compartment'])}\n",
      "Full match for 'reg1': MultiIndex([('reg1',          'food'),\n",
      "            ('reg1',        'mining'),\n",
      "            ('reg1', 'manufactoring'),\n",
      "            ('reg1',   'electricity'),\n",
      "            ('reg1',  'construction'),\n",
      "            ('reg1',         'trade'),\n",
      "            ('reg1',     'transport'),\n",
      "            ('reg1',         'other')],\n",
      "           names=['region', 'sector'])\n"
     ]
    }
   ],
   "source": [
    "# Search within extensions\n",
    "extension_search = test_mrio.extension_contains(\"emission\")\n",
    "print(\"Extension search for 'emission':\", extension_search)\n",
    "\n",
    "# Full match search\n",
    "match_results = test_mrio.match(\"reg1\")\n",
    "print(\"Full match for 'reg1':\", match_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip**: Use the find method to get a quick overview where you find a specific term, in particular for mrio systems with multiple extensions.\n",
    "For example, the following finds \"air\" in the compartment information of one extension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Search for occurance of >air< in the whole system: {'emissions_index': MultiIndex([('emission_type1', 'air')],\n",
      "           names=['stressor', 'compartment'])}\n"
     ]
    }
   ],
   "source": [
    "print(\"Search for occurance of >air< in the whole system:\", test_mrio.find(\"air\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "## Core Calculations with calc_all\n",
    "\n",
    "The `calc_all` method is fundamental to pymrio analysis. It automatically identifies missing tables and calculates all necessary accounts:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before the calculation, we have the following accounts available in the test MRIO system:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before calc_all:\n",
      "['Z', 'Y', 'unit', 'population']\n",
      "['F', 'F_Y', 'unit']\n"
     ]
    }
   ],
   "source": [
    "print(\"Before calc_all:\")\n",
    "print(test_mrio.DataFrames)\n",
    "print(test_mrio.emissions.DataFrames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate all missing parts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymrio.core.mriosystem.IOSystem at 0x7fe341d63d00>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_mrio.calc_all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After calculation, these accounts are available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After calc_all:\n",
      "['Z', 'Y', 'x', 'A', 'L', 'unit', 'population']\n"
     ]
    }
   ],
   "source": [
    "print(\"After calc_all:\")\n",
    "print(test_mrio.DataFrames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we now also have several classical EE-MRIO results available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Emissions accounts:\n",
      "['F', 'F_Y', 'S', 'S_Y', 'M', 'D_cba', 'D_pba', 'D_imp', 'D_exp', 'unit', 'D_cba_reg', 'D_pba_reg', 'D_imp_reg', 'D_exp_reg', 'D_cba_cap', 'D_pba_cap', 'D_imp_cap', 'D_exp_cap']\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nEmissions accounts:\")\n",
    "print(test_mrio.emissions.DataFrames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "D_cba (consumption-based): region                              reg1                               \\\n",
      "sector                              food         mining manufactoring   \n",
      "stressor       compartment                                              \n",
      "emission_type1 air          2.056183e+06  179423.535893  9.749300e+07   \n",
      "emission_type2 water        2.423103e+05   25278.192086  1.671240e+07   \n",
      "\n",
      "region                                                                \\\n",
      "sector                       electricity  construction         trade   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.188759e+07  3.342906e+06  3.885884e+06   \n",
      "emission_type2 water        1.371303e+05  3.468292e+05  7.766205e+05   \n",
      "\n",
      "region                                                          reg2  \\\n",
      "sector                         transport         other          food   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.075027e+07  1.582152e+07  1.793338e+06   \n",
      "emission_type2 water        4.999628e+05  8.480505e+06  2.136528e+05   \n",
      "\n",
      "region                                    ...          reg5                \\\n",
      "sector                            mining  ...     transport         other   \n",
      "stressor       compartment                ...                               \n",
      "emission_type1 air          19145.604911  ...  4.209505e+07  1.138661e+07   \n",
      "emission_type2 water         3733.601474  ...  4.243738e+06  7.307208e+06   \n",
      "\n",
      "region                              reg6                              \\\n",
      "sector                              food        mining manufactoring   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.517235e+07  1.345318e+06  7.145075e+07   \n",
      "emission_type2 water        4.420574e+06  5.372216e+05  1.068144e+07   \n",
      "\n",
      "region                                                                \\\n",
      "sector                       electricity  construction         trade   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          3.683167e+07  1.836696e+06  4.241568e+07   \n",
      "emission_type2 water        5.728136e+05  9.069515e+05  5.449044e+07   \n",
      "\n",
      "region                                                  \n",
      "sector                         transport         other  \n",
      "stressor       compartment                              \n",
      "emission_type1 air          4.805409e+07  3.602298e+07  \n",
      "emission_type2 water        8.836484e+06  4.634899e+07  \n",
      "\n",
      "[2 rows x 48 columns]\n"
     ]
    }
   ],
   "source": [
    "print(\"D_cba (consumption-based):\", test_mrio.emissions.D_cba)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ghosh Calculations in calc_all\n",
    "\n",
    "When `calc_all` is executed, it can optionally calculate Ghosh inverse matrices for downstream analysis:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IO System with parameters: Z, Y, x, A, B, L, G, unit, population, meta, factor_inputs, emissions\n"
     ]
    }
   ],
   "source": [
    "test_mrio.calc_all(include_ghosh=True)\n",
    "print(test_mrio)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This also calculates downstream multipliers M_down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg1</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg2</th>\n",
       "      <th>...</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg5</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>sector</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>...</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>stressor</th>\n",
       "      <th>compartment</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>emission_type1</th>\n",
       "      <th>air</th>\n",
       "      <td>1.045907</td>\n",
       "      <td>26.607878</td>\n",
       "      <td>0.015271</td>\n",
       "      <td>22.163630</td>\n",
       "      <td>0.034868</td>\n",
       "      <td>0.00822</td>\n",
       "      <td>0.030033</td>\n",
       "      <td>0.047107</td>\n",
       "      <td>0.001025</td>\n",
       "      <td>23.995315</td>\n",
       "      <td>...</td>\n",
       "      <td>0.076524</td>\n",
       "      <td>0.020483</td>\n",
       "      <td>0.010557</td>\n",
       "      <td>17.377205</td>\n",
       "      <td>0.025397</td>\n",
       "      <td>15.093929</td>\n",
       "      <td>0.210979</td>\n",
       "      <td>0.048656</td>\n",
       "      <td>0.029646</td>\n",
       "      <td>0.029898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>emission_type2</th>\n",
       "      <th>water</th>\n",
       "      <td>0.074245</td>\n",
       "      <td>0.657819</td>\n",
       "      <td>0.000959</td>\n",
       "      <td>0.238178</td>\n",
       "      <td>0.000609</td>\n",
       "      <td>0.00032</td>\n",
       "      <td>0.001693</td>\n",
       "      <td>0.001882</td>\n",
       "      <td>0.000068</td>\n",
       "      <td>1.061891</td>\n",
       "      <td>...</td>\n",
       "      <td>0.011976</td>\n",
       "      <td>0.003882</td>\n",
       "      <td>0.001871</td>\n",
       "      <td>0.667438</td>\n",
       "      <td>0.002734</td>\n",
       "      <td>0.333868</td>\n",
       "      <td>0.023223</td>\n",
       "      <td>0.005806</td>\n",
       "      <td>0.003279</td>\n",
       "      <td>0.003230</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "region                          reg1                                       \\\n",
       "sector                          food     mining manufactoring electricity   \n",
       "stressor       compartment                                                  \n",
       "emission_type1 air          1.045907  26.607878      0.015271   22.163630   \n",
       "emission_type2 water        0.074245   0.657819      0.000959    0.238178   \n",
       "\n",
       "region                                                                \\\n",
       "sector                     construction    trade transport     other   \n",
       "stressor       compartment                                             \n",
       "emission_type1 air             0.034868  0.00822  0.030033  0.047107   \n",
       "emission_type2 water           0.000609  0.00032  0.001693  0.001882   \n",
       "\n",
       "region                          reg2             ...      reg5            \\\n",
       "sector                          food     mining  ... transport     other   \n",
       "stressor       compartment                       ...                       \n",
       "emission_type1 air          0.001025  23.995315  ...  0.076524  0.020483   \n",
       "emission_type2 water        0.000068   1.061891  ...  0.011976  0.003882   \n",
       "\n",
       "region                          reg6                                       \\\n",
       "sector                          food     mining manufactoring electricity   \n",
       "stressor       compartment                                                  \n",
       "emission_type1 air          0.010557  17.377205      0.025397   15.093929   \n",
       "emission_type2 water        0.001871   0.667438      0.002734    0.333868   \n",
       "\n",
       "region                                                                 \n",
       "sector                     construction     trade transport     other  \n",
       "stressor       compartment                                             \n",
       "emission_type1 air             0.210979  0.048656  0.029646  0.029898  \n",
       "emission_type2 water           0.023223  0.005806  0.003279  0.003230  \n",
       "\n",
       "[2 rows x 48 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_mrio.emissions.M_down"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "See the [math section](../math.rst) of the documentation for further details on the Ghosh calculations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Search and Extract Functionality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extracting Specific Accounts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also extract consumption-based accounts for a specific stressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CBA emissions by region for emission_type1:\n",
      "region                              reg1                               \\\n",
      "sector                              food         mining manufactoring   \n",
      "stressor       compartment                                              \n",
      "emission_type1 air          2.056183e+06  179423.535893  9.749300e+07   \n",
      "\n",
      "region                                                                \\\n",
      "sector                       electricity  construction         trade   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.188759e+07  3.342906e+06  3.885884e+06   \n",
      "\n",
      "region                                                          reg2  \\\n",
      "sector                         transport         other          food   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.075027e+07  1.582152e+07  1.793338e+06   \n",
      "\n",
      "region                                    ...          reg5                \\\n",
      "sector                            mining  ...     transport         other   \n",
      "stressor       compartment                ...                               \n",
      "emission_type1 air          19145.604911  ...  4.209505e+07  1.138661e+07   \n",
      "\n",
      "region                              reg6                              \\\n",
      "sector                              food        mining manufactoring   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          1.517235e+07  1.345318e+06  7.145075e+07   \n",
      "\n",
      "region                                                                \\\n",
      "sector                       electricity  construction         trade   \n",
      "stressor       compartment                                             \n",
      "emission_type1 air          3.683167e+07  1.836696e+06  4.241568e+07   \n",
      "\n",
      "region                                                  \n",
      "sector                         transport         other  \n",
      "stressor       compartment                              \n",
      "emission_type1 air          4.805409e+07  3.602298e+07  \n",
      "\n",
      "[1 rows x 48 columns]\n"
     ]
    }
   ],
   "source": [
    "cba_emission1 = test_mrio.emissions.D_cba.loc[[\"emission_type1\"]]\n",
    "print(\"CBA emissions by region for emission_type1:\")\n",
    "print(cba_emission1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And extract data for specific regions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Total CBA emissions for the selected regions:\n",
      "region                              reg1          reg3\n",
      "stressor       compartment                            \n",
      "emission_type1 air          2.077521e+08  3.457988e+08\n",
      "emission_type2 water        8.642744e+07  3.753335e+08\n"
     ]
    }
   ],
   "source": [
    "reg1_data = test_mrio.emissions.D_cba_reg[[\"reg1\", \"reg3\"]]\n",
    "print(\"\\nTotal CBA emissions for the selected regions:\")\n",
    "print(reg1_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides the direct access to the DataFrames explained above, one can also\n",
    "extract data into dictionaries for alternative access.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "emission_type1_data = test_mrio.emissions.get_row_data(\"emission_type1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This extracts all data available for >emission_type1<"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['F', 'F_Y', 'S', 'S_Y', 'M', 'M_down', 'D_cba', 'D_pba', 'D_imp', 'D_exp', 'unit', 'D_cba_reg', 'D_pba_reg', 'D_imp_reg', 'D_exp_reg', 'D_cba_cap', 'D_pba_cap', 'D_imp_cap', 'D_exp_cap'])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "emission_type1_data.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced Search Patterns\n",
    "\n",
    "Use regular expressions for more complex searches:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Emission... occurances: {'emissions_index': MultiIndex([('emission_type1',   'air'),\n",
      "            ('emission_type2', 'water')],\n",
      "           names=['stressor', 'compartment'])}\n",
      "Extensions containing 'type': {'Factor Inputs': Index([], dtype='object', name='inputtype'), 'Emissions': MultiIndex([('emission_type1',   'air'),\n",
      "            ('emission_type2', 'water')],\n",
      "           names=['stressor', 'compartment'])}\n"
     ]
    }
   ],
   "source": [
    "emis_search = test_mrio.find(\"emission.*\")\n",
    "print(\"Emission... occurances:\", emis_search)\n",
    "\n",
    "all_extension_search = test_mrio.extension_contains(\"typ+\")\n",
    "print(\"Extensions containing 'type':\", all_extension_search)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Functions from iomath\n",
    "\n",
    "Pymrio's `iomath` module provides low-level functions for specific calculations:\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Manual A matrix calculation matches: True\n",
      "Manual L matrix calculation matches: True\n",
      "Manual multiplier calculation matches: True\n"
     ]
    }
   ],
   "source": [
    "from pymrio.tools import iomath\n",
    "\n",
    "# Calculate specific matrices manually\n",
    "A_manual = iomath.calc_A(test_mrio.Z, test_mrio.x)\n",
    "print(\"Manual A matrix calculation matches:\", np.allclose(A_manual, test_mrio.A))\n",
    "\n",
    "# Calculate Leontief matrix\n",
    "L_manual = iomath.calc_L(test_mrio.A)\n",
    "print(\"Manual L matrix calculation matches:\", np.allclose(L_manual, test_mrio.L))\n",
    "\n",
    "# Calculate multipliers\n",
    "S = test_mrio.emissions.S\n",
    "M_manual = iomath.calc_M(S, test_mrio.L)\n",
    "print(\n",
    "    \"Manual multiplier calculation matches:\",\n",
    "    np.allclose(M_manual, test_mrio.emissions.M),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gross Trade Analysis\n",
    "\n",
    "The `calc_gross_trade` function provides insights into bilateral trade flows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "gross_trade = test_mrio.get_gross_trade()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This give the total trade flows from one region/sectors to other regions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>reg1</th>\n",
       "      <th>reg2</th>\n",
       "      <th>reg3</th>\n",
       "      <th>reg4</th>\n",
       "      <th>reg5</th>\n",
       "      <th>reg6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th>sector</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">reg1</th>\n",
       "      <th>food</th>\n",
       "      <td>0.0</td>\n",
       "      <td>9.874311e+03</td>\n",
       "      <td>3.772336e+03</td>\n",
       "      <td>2.327343e+02</td>\n",
       "      <td>1.231784e+03</td>\n",
       "      <td>4.615724e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mining</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2.905520e+03</td>\n",
       "      <td>3.657874e+03</td>\n",
       "      <td>4.020829e+02</td>\n",
       "      <td>6.660429e+02</td>\n",
       "      <td>9.281742e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manufactoring</th>\n",
       "      <td>0.0</td>\n",
       "      <td>6.027532e+07</td>\n",
       "      <td>5.111218e+07</td>\n",
       "      <td>2.709138e+07</td>\n",
       "      <td>3.349291e+07</td>\n",
       "      <td>3.814142e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>electricity</th>\n",
       "      <td>0.0</td>\n",
       "      <td>3.775794e+03</td>\n",
       "      <td>3.629075e+02</td>\n",
       "      <td>2.492309e+00</td>\n",
       "      <td>2.222702e+03</td>\n",
       "      <td>9.412412e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>construction</th>\n",
       "      <td>0.0</td>\n",
       "      <td>6.629450e+02</td>\n",
       "      <td>2.530807e+02</td>\n",
       "      <td>2.995250e+02</td>\n",
       "      <td>1.537160e+03</td>\n",
       "      <td>1.401676e+02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "region                reg1          reg2          reg3          reg4  \\\n",
       "region sector                                                          \n",
       "reg1   food            0.0  9.874311e+03  3.772336e+03  2.327343e+02   \n",
       "       mining          0.0  2.905520e+03  3.657874e+03  4.020829e+02   \n",
       "       manufactoring   0.0  6.027532e+07  5.111218e+07  2.709138e+07   \n",
       "       electricity     0.0  3.775794e+03  3.629075e+02  2.492309e+00   \n",
       "       construction    0.0  6.629450e+02  2.530807e+02  2.995250e+02   \n",
       "\n",
       "region                        reg5          reg6  \n",
       "region sector                                     \n",
       "reg1   food           1.231784e+03  4.615724e+03  \n",
       "       mining         6.660429e+02  9.281742e+02  \n",
       "       manufactoring  3.349291e+07  3.814142e+07  \n",
       "       electricity    2.222702e+03  9.412412e+02  \n",
       "       construction   1.537160e+03  1.401676e+02  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gross_trade.bilat_flows.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As well as the totals for each region"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>exports</th>\n",
       "      <th>imports</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th>sector</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">reg1</th>\n",
       "      <th>food</th>\n",
       "      <td>1.972689e+04</td>\n",
       "      <td>1.504225e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mining</th>\n",
       "      <td>8.559694e+03</td>\n",
       "      <td>1.418970e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manufactoring</th>\n",
       "      <td>2.101132e+08</td>\n",
       "      <td>3.888102e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>electricity</th>\n",
       "      <td>7.305137e+03</td>\n",
       "      <td>7.365582e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>construction</th>\n",
       "      <td>2.892878e+03</td>\n",
       "      <td>5.157738e+03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           exports       imports\n",
       "region sector                                   \n",
       "reg1   food           1.972689e+04  1.504225e+05\n",
       "       mining         8.559694e+03  1.418970e+05\n",
       "       manufactoring  2.101132e+08  3.888102e+08\n",
       "       electricity    7.305137e+03  7.365582e+03\n",
       "       construction   2.892878e+03  5.157738e+03"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gross_trade.totals.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extension Methods: Concatenate, Convert, and Characterize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extension Concatenation\n",
    "\n",
    "The `extension_concate` method allows combining multiple extensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('emission_type1',   'air'),\n",
       "            ('emission_type2', 'water'),\n",
       "            ('emission_type1',   'air'),\n",
       "            ('emission_type2', 'water')],\n",
       "           names=['stressor', 'compartment'])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a copy for demonstration\n",
    "ext_emis2 = test_mrio.emissions.copy()\n",
    "# Combine two extensions with same index structure\n",
    "new_ext = pymrio.extension_concate(\n",
    "    test_mrio.emissions, ext_emis2, new_extension_name=\"emissions_combined\"\n",
    ")\n",
    "new_ext.rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combining extensions with different indicies results in a new index called >indicator<. Any indicies not avaialable in one of the extensions is set to NaN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extension All with parameters: name, F, F_Y, S, S_Y, M, M_down, D_cba, D_pba, D_imp, D_exp, unit, D_cba_cap, D_imp_reg, D_pba_reg, D_cba_reg, D_exp_reg, D_exp_cap, D_pba_cap, D_imp_cap\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "MultiIndex([('emission_type1',   'air'),\n",
       "            ('emission_type2', 'water'),\n",
       "            (   'Value Added',     nan)],\n",
       "           names=['indicator', 'compartment'])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_ext = pymrio.extension_concate(\n",
    "    test_mrio.emissions, test_mrio.factor_inputs, new_extension_name=\"All\"\n",
    ")\n",
    "print(all_ext)\n",
    "all_ext.rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In any case, the extension can be attached to the mrio object and used alongside the others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Factor Inputs', 'Emissions', 'All']\n",
      "['factor_inputs', 'emissions', 'all_ext']\n"
     ]
    }
   ],
   "source": [
    "test_mrio.all_ext = all_ext\n",
    "print(test_mrio.extensions)\n",
    "print(test_mrio.extensions_instance_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extension Conversion\n",
    "\n",
    "The `convert` and and `extension_convert` methods transforms extensions based on mapping functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "conversion_factors = pd.DataFrame(\n",
    "    columns=[\n",
    "        \"stressor\",\n",
    "        \"compartment\",\n",
    "        \"total__stressor\",\n",
    "        \"factor\",\n",
    "        \"unit_orig\",\n",
    "        \"unit_new\",\n",
    "    ],\n",
    "    data=[\n",
    "        [\"emis.*\", \"air|water\", \"total_sum_tonnes\", 1e-3, \"kg\", \"t\"],\n",
    "        [\"emission_type[1|2]\", \".*\", \"total_sum\", 1, \"kg\", \"kg\"],\n",
    "        [\"emission_type1\", \".*\", \"air_emissions\", 1e-3, \"kg\", \"t\"],\n",
    "        [\"emission_type2\", \".*\", \"water_emissions\", 1000, \"kg\", \"g\"],\n",
    "        [\"emission_type1\", \".*\", \"char_emissions\", 2, \"kg\", \"kg_eq\"],\n",
    "        [\"emission_type2\", \".*\", \"char_emissions\", 10, \"kg\", \"kg_eq\"],\n",
    "    ],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importantly, the columns names >stressor< and >compartment< match the index names of the extension to be converted.\n",
    "Bridge columns are columns with '__' in the name. They defined a new name >impact< and how it is based on a previous column name.\n",
    "The column >Factor< is the conversion factor, and the last 2 columns define new colums and check the orignal ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg1</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg2</th>\n",
       "      <th>...</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg5</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sector</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>...</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>air_emissions</th>\n",
       "      <td>1.848065e+03</td>\n",
       "      <td>9.864481e+02</td>\n",
       "      <td>2.361379e+04</td>\n",
       "      <td>2.813910e+04</td>\n",
       "      <td>2.584142e+03</td>\n",
       "      <td>4.132656e+03</td>\n",
       "      <td>2.176699e+04</td>\n",
       "      <td>7.842091e+03</td>\n",
       "      <td>1.697937e+03</td>\n",
       "      <td>3.473782e+02</td>\n",
       "      <td>...</td>\n",
       "      <td>4.229932e+04</td>\n",
       "      <td>1.077383e+04</td>\n",
       "      <td>1.577800e+04</td>\n",
       "      <td>6.420956e+03</td>\n",
       "      <td>1.131724e+05</td>\n",
       "      <td>5.602253e+04</td>\n",
       "      <td>4.861838e+03</td>\n",
       "      <td>1.819562e+04</td>\n",
       "      <td>4.704654e+04</td>\n",
       "      <td>2.163287e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>char_emissions</th>\n",
       "      <td>5.088634e+06</td>\n",
       "      <td>2.196329e+06</td>\n",
       "      <td>5.486327e+07</td>\n",
       "      <td>5.901802e+07</td>\n",
       "      <td>8.342249e+06</td>\n",
       "      <td>2.081009e+07</td>\n",
       "      <td>5.366396e+07</td>\n",
       "      <td>4.017596e+07</td>\n",
       "      <td>5.444229e+06</td>\n",
       "      <td>9.893957e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>1.265970e+08</td>\n",
       "      <td>9.345772e+07</td>\n",
       "      <td>7.981707e+07</td>\n",
       "      <td>3.149816e+07</td>\n",
       "      <td>3.533468e+08</td>\n",
       "      <td>1.195772e+08</td>\n",
       "      <td>3.671656e+07</td>\n",
       "      <td>1.753144e+08</td>\n",
       "      <td>1.817509e+08</td>\n",
       "      <td>2.110913e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_sum</th>\n",
       "      <td>1.987315e+06</td>\n",
       "      <td>1.008791e+06</td>\n",
       "      <td>2.437736e+07</td>\n",
       "      <td>2.841308e+07</td>\n",
       "      <td>2.901538e+06</td>\n",
       "      <td>5.387134e+06</td>\n",
       "      <td>2.277999e+07</td>\n",
       "      <td>1.029127e+07</td>\n",
       "      <td>1.902773e+06</td>\n",
       "      <td>3.768421e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>4.649916e+07</td>\n",
       "      <td>1.796483e+07</td>\n",
       "      <td>2.060410e+07</td>\n",
       "      <td>8.286581e+06</td>\n",
       "      <td>1.258726e+08</td>\n",
       "      <td>5.677575e+07</td>\n",
       "      <td>7.561127e+06</td>\n",
       "      <td>3.208793e+07</td>\n",
       "      <td>5.581233e+07</td>\n",
       "      <td>3.841542e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_sum_tonnes</th>\n",
       "      <td>1.987315e+03</td>\n",
       "      <td>1.008791e+03</td>\n",
       "      <td>2.437736e+04</td>\n",
       "      <td>2.841308e+04</td>\n",
       "      <td>2.901538e+03</td>\n",
       "      <td>5.387134e+03</td>\n",
       "      <td>2.277999e+04</td>\n",
       "      <td>1.029127e+04</td>\n",
       "      <td>1.902773e+03</td>\n",
       "      <td>3.768421e+02</td>\n",
       "      <td>...</td>\n",
       "      <td>4.649916e+04</td>\n",
       "      <td>1.796483e+04</td>\n",
       "      <td>2.060410e+04</td>\n",
       "      <td>8.286581e+03</td>\n",
       "      <td>1.258726e+05</td>\n",
       "      <td>5.677575e+04</td>\n",
       "      <td>7.561127e+03</td>\n",
       "      <td>3.208793e+04</td>\n",
       "      <td>5.581233e+04</td>\n",
       "      <td>3.841542e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>water_emissions</th>\n",
       "      <td>1.392505e+08</td>\n",
       "      <td>2.234330e+07</td>\n",
       "      <td>7.635692e+08</td>\n",
       "      <td>2.739816e+08</td>\n",
       "      <td>3.173965e+08</td>\n",
       "      <td>1.254478e+09</td>\n",
       "      <td>1.012999e+09</td>\n",
       "      <td>2.449178e+09</td>\n",
       "      <td>2.048354e+08</td>\n",
       "      <td>2.946394e+07</td>\n",
       "      <td>...</td>\n",
       "      <td>4.199841e+09</td>\n",
       "      <td>7.191006e+09</td>\n",
       "      <td>4.826108e+09</td>\n",
       "      <td>1.865625e+09</td>\n",
       "      <td>1.270019e+10</td>\n",
       "      <td>7.532137e+08</td>\n",
       "      <td>2.699288e+09</td>\n",
       "      <td>1.389231e+10</td>\n",
       "      <td>8.765784e+09</td>\n",
       "      <td>1.678255e+10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "region                    reg1                                            \\\n",
       "sector                    food        mining manufactoring   electricity   \n",
       "total                                                                      \n",
       "air_emissions     1.848065e+03  9.864481e+02  2.361379e+04  2.813910e+04   \n",
       "char_emissions    5.088634e+06  2.196329e+06  5.486327e+07  5.901802e+07   \n",
       "total_sum         1.987315e+06  1.008791e+06  2.437736e+07  2.841308e+07   \n",
       "total_sum_tonnes  1.987315e+03  1.008791e+03  2.437736e+04  2.841308e+04   \n",
       "water_emissions   1.392505e+08  2.234330e+07  7.635692e+08  2.739816e+08   \n",
       "\n",
       "region                                                                    \\\n",
       "sector            construction         trade     transport         other   \n",
       "total                                                                      \n",
       "air_emissions     2.584142e+03  4.132656e+03  2.176699e+04  7.842091e+03   \n",
       "char_emissions    8.342249e+06  2.081009e+07  5.366396e+07  4.017596e+07   \n",
       "total_sum         2.901538e+06  5.387134e+06  2.277999e+07  1.029127e+07   \n",
       "total_sum_tonnes  2.901538e+03  5.387134e+03  2.277999e+04  1.029127e+04   \n",
       "water_emissions   3.173965e+08  1.254478e+09  1.012999e+09  2.449178e+09   \n",
       "\n",
       "region                    reg2                ...          reg5                \\\n",
       "sector                    food        mining  ...     transport         other   \n",
       "total                                         ...                               \n",
       "air_emissions     1.697937e+03  3.473782e+02  ...  4.229932e+04  1.077383e+04   \n",
       "char_emissions    5.444229e+06  9.893957e+05  ...  1.265970e+08  9.345772e+07   \n",
       "total_sum         1.902773e+06  3.768421e+05  ...  4.649916e+07  1.796483e+07   \n",
       "total_sum_tonnes  1.902773e+03  3.768421e+02  ...  4.649916e+04  1.796483e+04   \n",
       "water_emissions   2.048354e+08  2.946394e+07  ...  4.199841e+09  7.191006e+09   \n",
       "\n",
       "region                    reg6                                            \\\n",
       "sector                    food        mining manufactoring   electricity   \n",
       "total                                                                      \n",
       "air_emissions     1.577800e+04  6.420956e+03  1.131724e+05  5.602253e+04   \n",
       "char_emissions    7.981707e+07  3.149816e+07  3.533468e+08  1.195772e+08   \n",
       "total_sum         2.060410e+07  8.286581e+06  1.258726e+08  5.677575e+07   \n",
       "total_sum_tonnes  2.060410e+04  8.286581e+03  1.258726e+05  5.677575e+04   \n",
       "water_emissions   4.826108e+09  1.865625e+09  1.270019e+10  7.532137e+08   \n",
       "\n",
       "region                                                                    \n",
       "sector            construction         trade     transport         other  \n",
       "total                                                                     \n",
       "air_emissions     4.861838e+03  1.819562e+04  4.704654e+04  2.163287e+04  \n",
       "char_emissions    3.671656e+07  1.753144e+08  1.817509e+08  2.110913e+08  \n",
       "total_sum         7.561127e+06  3.208793e+07  5.581233e+07  3.841542e+07  \n",
       "total_sum_tonnes  7.561127e+03  3.208793e+04  5.581233e+04  3.841542e+04  \n",
       "water_emissions   2.699288e+09  1.389231e+10  8.765784e+09  1.678255e+10  \n",
       "\n",
       "[5 rows x 48 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_emis = test_mrio.emissions.convert(\n",
    "    conversion_factors, new_extension_name=\"converted_emissions\"\n",
    ")\n",
    "new_emis.F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip**: Due to the regular expression capabilities this function\n",
    "is quite powerful but also rather slow.\n",
    "Use is before doing the full analysis, and use the characterization function\n",
    "for \"standard\" characterization tasks (see below).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Characterization of stressors\n",
    "\n",
    "Pymrio uses an innovative string-matching approach to characterize stressors.\n",
    "This method matches stressors in the characterization table (in long format)\n",
    "with those in the MRIO system, ensuring consistent stressor mapping,\n",
    "automatic unit verification, and flexibility regardless of entry order. It\n",
    "also handles characterization factors for stressors not present in the\n",
    "satellite account, efficiently manages region- and sector-specific factors,\n",
    "and supports characterization across different extensions.\n",
    "\n",
    "Unlike traditional matrix multiplication methods, which require strict 1:1\n",
    "correspondence and precise ordering, this approach is more flexible.\n",
    "Characterization can be performed using either an extension object method or\n",
    "a top-level function that accepts MRIO objects or extension collections.\n",
    "\n",
    "To start, we need to first define a characterization factors table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "char_factors = pd.DataFrame(\n",
    "    {\n",
    "        \"stressor\": [\"emission_type1\", \"emission_type2\", \"emission_type3\"],\n",
    "        \"compartment\": [\"air\", \"water\", \"land\"],\n",
    "        \"impact\": [\"climate_change\", \"acidification\", \"eutrophication\"],\n",
    "        \"factor\": [25.0, 1.5, 0.8],  # kg CO2-eq, SO2-eq, PO4-eq\n",
    "        \"impact_unit\": [\"kg CO2-eq\", \"kg SO2-eq\", \"kg PO4-eq\"],\n",
    "        \"stressor_unit\": [\"kg\", \"kg\", \"kg\"],\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can be used to characterize the emissions extension of the test MRIO system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "characterization_result = test_mrio.emissions.characterize(\n",
    "    factors=char_factors,\n",
    "    characterized_name_column=\"impact\",\n",
    "    characterization_factors_column=\"factor\",\n",
    "    characterized_unit_column=\"impact_unit\",\n",
    "    orig_unit_column=\"stressor_unit\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result contains a validation table, informing about the missing stressor.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>stressor</th>\n",
       "      <th>compartment</th>\n",
       "      <th>impact</th>\n",
       "      <th>factor</th>\n",
       "      <th>impact_unit</th>\n",
       "      <th>stressor_unit</th>\n",
       "      <th>error_unit_impact</th>\n",
       "      <th>error_unit_stressor</th>\n",
       "      <th>error_missing_stressor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>emission_type1</td>\n",
       "      <td>air</td>\n",
       "      <td>climate_change</td>\n",
       "      <td>25.0</td>\n",
       "      <td>kg CO2-eq</td>\n",
       "      <td>kg</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>emission_type2</td>\n",
       "      <td>water</td>\n",
       "      <td>acidification</td>\n",
       "      <td>1.5</td>\n",
       "      <td>kg SO2-eq</td>\n",
       "      <td>kg</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>emission_type3</td>\n",
       "      <td>land</td>\n",
       "      <td>eutrophication</td>\n",
       "      <td>0.8</td>\n",
       "      <td>kg PO4-eq</td>\n",
       "      <td>kg</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         stressor compartment          impact  factor impact_unit  \\\n",
       "0  emission_type1         air  climate_change    25.0   kg CO2-eq   \n",
       "1  emission_type2       water   acidification     1.5   kg SO2-eq   \n",
       "2  emission_type3        land  eutrophication     0.8   kg PO4-eq   \n",
       "\n",
       "  stressor_unit  error_unit_impact  error_unit_stressor  \\\n",
       "0            kg              False                False   \n",
       "1            kg              False                False   \n",
       "2            kg              False                False   \n",
       "\n",
       "   error_missing_stressor  \n",
       "0                   False  \n",
       "1                   False  \n",
       "2                    True  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "characterization_result.validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TIP**: Alway verify and check via the validation table.\n",
    "It is also returned in cases when the characterization can not be performed (e.g. due to unit errors)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The characterized is available as the second attribute of the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymrio.core.mriosystem.Extension at 0x7fe339b9d9d0>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "characterization_result.extension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg1</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg2</th>\n",
       "      <th>...</th>\n",
       "      <th colspan=\"2\" halign=\"left\">reg5</th>\n",
       "      <th colspan=\"8\" halign=\"left\">reg6</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sector</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>...</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "      <th>food</th>\n",
       "      <th>mining</th>\n",
       "      <th>manufactoring</th>\n",
       "      <th>electricity</th>\n",
       "      <th>construction</th>\n",
       "      <th>trade</th>\n",
       "      <th>transport</th>\n",
       "      <th>other</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>impact</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>acidification</th>\n",
       "      <td>2.088757e+05</td>\n",
       "      <td>3.351494e+04</td>\n",
       "      <td>1.145354e+06</td>\n",
       "      <td>4.109723e+05</td>\n",
       "      <td>4.760948e+05</td>\n",
       "      <td>1881716.7</td>\n",
       "      <td>1.519499e+06</td>\n",
       "      <td>3673767.0</td>\n",
       "      <td>307253.16</td>\n",
       "      <td>44195.916</td>\n",
       "      <td>...</td>\n",
       "      <td>6.299762e+06</td>\n",
       "      <td>1.078651e+07</td>\n",
       "      <td>7.239162e+06</td>\n",
       "      <td>2.798438e+06</td>\n",
       "      <td>1.905029e+07</td>\n",
       "      <td>1.129821e+06</td>\n",
       "      <td>4.048932e+06</td>\n",
       "      <td>20838469.5</td>\n",
       "      <td>1.314868e+07</td>\n",
       "      <td>25173829.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>climate_change</th>\n",
       "      <td>4.620162e+07</td>\n",
       "      <td>2.466120e+07</td>\n",
       "      <td>5.903447e+08</td>\n",
       "      <td>7.034775e+08</td>\n",
       "      <td>6.460354e+07</td>\n",
       "      <td>103316407.5</td>\n",
       "      <td>5.441747e+08</td>\n",
       "      <td>196052265.0</td>\n",
       "      <td>42448432.50</td>\n",
       "      <td>8684453.750</td>\n",
       "      <td>...</td>\n",
       "      <td>1.057483e+09</td>\n",
       "      <td>2.693456e+08</td>\n",
       "      <td>3.944499e+08</td>\n",
       "      <td>1.605239e+08</td>\n",
       "      <td>2.829311e+09</td>\n",
       "      <td>1.400563e+09</td>\n",
       "      <td>1.215460e+08</td>\n",
       "      <td>454890525.0</td>\n",
       "      <td>1.176164e+09</td>\n",
       "      <td>540821700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eutrophication</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 48 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "region                  reg1                                            \\\n",
       "sector                  food        mining manufactoring   electricity   \n",
       "impact                                                                   \n",
       "acidification   2.088757e+05  3.351494e+04  1.145354e+06  4.109723e+05   \n",
       "climate_change  4.620162e+07  2.466120e+07  5.903447e+08  7.034775e+08   \n",
       "eutrophication  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "\n",
       "region                                                                \\\n",
       "sector          construction        trade     transport        other   \n",
       "impact                                                                 \n",
       "acidification   4.760948e+05    1881716.7  1.519499e+06    3673767.0   \n",
       "climate_change  6.460354e+07  103316407.5  5.441747e+08  196052265.0   \n",
       "eutrophication  0.000000e+00          0.0  0.000000e+00          0.0   \n",
       "\n",
       "region                 reg2               ...          reg5                \\\n",
       "sector                 food       mining  ...     transport         other   \n",
       "impact                                    ...                               \n",
       "acidification     307253.16    44195.916  ...  6.299762e+06  1.078651e+07   \n",
       "climate_change  42448432.50  8684453.750  ...  1.057483e+09  2.693456e+08   \n",
       "eutrophication         0.00        0.000  ...  0.000000e+00  0.000000e+00   \n",
       "\n",
       "region                  reg6                                            \\\n",
       "sector                  food        mining manufactoring   electricity   \n",
       "impact                                                                   \n",
       "acidification   7.239162e+06  2.798438e+06  1.905029e+07  1.129821e+06   \n",
       "climate_change  3.944499e+08  1.605239e+08  2.829311e+09  1.400563e+09   \n",
       "eutrophication  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "\n",
       "region                                                                \n",
       "sector          construction        trade     transport        other  \n",
       "impact                                                                \n",
       "acidification   4.048932e+06   20838469.5  1.314868e+07   25173829.5  \n",
       "climate_change  1.215460e+08  454890525.0  1.176164e+09  540821700.0  \n",
       "eutrophication  0.000000e+00          0.0  0.000000e+00          0.0  \n",
       "\n",
       "[3 rows x 48 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "characterization_result.extension.F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more details on region-specific characterization and characterization\n",
    "across multiple extensions, see the notebook stressor_characterization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "## Parsing, saving and loading MRIOs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "### Parsing MRIOs\n",
    "\n",
    "Pymrio supports any symmetric MRIO table and provides automatic downloading and parsing for several common datasets.\n",
    "For details, see the sections \"Automatic MRIO download\" and \"Handling MRIO data\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving processed MRIOs\n",
    "You can save your MRIO after you've parsed and analysed it. Pymrio lets you\n",
    "save in text, pickle or parquet formats. Parquet works well if your dataset is on the\n",
    "larger side.\n",
    "\n",
    "\n",
    "\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "# Create temporary directory for demonstration\n",
    "temp_dir = Path(tempfile.mkdtemp())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference for into the supported formats it given by the argument to the >save_all< method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/A.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/unit.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/B.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/x.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/L.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/metadata.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/Y.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/G.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/population.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/Z.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/F.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/S.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/unit.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_cba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/M_down.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_exp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/M.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_imp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_cba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_imp_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_pba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_imp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_pba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/F_Y.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_pba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_exp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_cba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/S_Y.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/all_ext/D_exp_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/F.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/S.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/unit.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_cba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/M_down.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_exp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/M.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_imp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_cba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_imp_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_pba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_imp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_pba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/F_Y.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_pba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_exp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_cba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/S_Y.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/emissions/D_exp_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/F.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/S.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/unit.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_cba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/M_down.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_exp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/M.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_imp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_cba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_imp_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_pba.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_imp_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_pba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_pba_cap.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_exp.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_cba_reg.txt'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_txt/factor_inputs/D_exp_cap.txt')]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save to text format\n",
    "txt_path = temp_dir / \"test_mrio_txt\"\n",
    "test_mrio.save_all(txt_path, table_format=\"txt\")\n",
    "list(txt_path.glob(\"**/*\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/x.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/Z.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/metadata.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/population.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/L.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/Y.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/unit.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/G.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/A.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/B.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/F.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_imp_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_exp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_cba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/F_Y.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/M.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_imp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/S_Y.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/S.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_pba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_cba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_imp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_cba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_exp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_pba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/unit.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/M_down.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_pba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/all_ext/D_exp_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/F.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_imp_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_exp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_cba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/F_Y.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/M.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_imp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/S_Y.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/S.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_pba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_cba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_imp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_cba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_exp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_pba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/unit.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/M_down.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_pba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/emissions/D_exp_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/F.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_imp_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_exp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_cba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/M.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/file_parameters.json'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_imp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/S.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_pba_cap.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_cba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_imp.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_cba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_exp_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_pba_reg.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/unit.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/M_down.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_pba.parquet'),\n",
       " PosixPath('/tmp/tmph4w3mja7/test_mrio_parquet/factor_inputs/D_exp_cap.parquet')]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save to parquet format\n",
    "parquet_path = temp_dir / \"test_mrio_parquet\"\n",
    "test_mrio.save_all(parquet_path, table_format=\"parquet\")\n",
    "list(parquet_path.glob(\"**/*\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In both cases, each DataFrame (account) is stored as separate file,\n",
    "with satellite accounts a subfolders. The file >file_paramters.json<\n",
    "stores the definition of index/columns such that files can be read\n",
    "back in correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IO System with parameters: Z, Y, x, A, B, L, G, unit, population, meta, factor_inputs, emissions, all_ext\n",
      "IO System with parameters: Z, Y, x, A, B, L, G, unit, population, meta, all_ext, emissions, factor_inputs\n",
      "IO System with parameters: Z, Y, x, A, B, L, G, unit, population, meta, all_ext, emissions, factor_inputs\n"
     ]
    }
   ],
   "source": [
    "mrio_reload_txt = pymrio.load_all(txt_path)\n",
    "mrio_reload_parquet = pymrio.load_all(parquet_path)\n",
    "print(test_mrio)\n",
    "print(mrio_reload_txt)\n",
    "print(mrio_reload_parquet)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up temporary directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Temporary files cleaned up\n"
     ]
    }
   ],
   "source": [
    "import shutil\n",
    "\n",
    "shutil.rmtree(temp_dir)\n",
    "print(\"Temporary files cleaned up\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "\n",
    "We covered the main functionality of pymrio in this tutorial.\n",
    "\n",
    "Pymrio has many more features, including aggregation, renaming and restructuring, and analysing the source of stressors. You can explore these topics in more detail in the following example notebooks:\n",
    "\n",
    "- [Aggregation Examples](aggregation_examples.ipynb)\n",
    "- [Adjusting, Renaming and Restructuring](adjusting.ipynb)\n",
    "- [Analysing the Source of Stressors (Flow Matrix)](buildflowmatrix.ipynb)\n",
    "\n",
    "For working with specific MRIO databases, see for example:\n",
    "\n",
    "- [EXIOBASE](working_with_exiobase.ipynb)\n",
    "- [WIOD](working_with_wiod.ipynb)\n",
    "- [GLORIA](working_with_gloria.ipynb)\n",
    "- [OECD-ICIO](working_with_oecd_icio.ipynb)\n",
    "\n",
    "You can also check the [API Reference](../api_references.rst) for a full overview of available functions and classes.\n",
    "\n",
    "If you have questions or need help, please open an issue on our [GitHub page](https://github.com/IndEcol/pymrio). We're happy to help!\n",
    "\n",
    "Thank you for following the tutorial, and good luck with your MRIO analyses!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.20"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}