Metadata and change recording

DEPRECATED: this will be removed and changed to loguru soon

Each pymrio core system object contains a field ‘meta’ which stores meta data as well as changes to the MRIO system. This data is stored as json file in the root of a saved MRIO data and accessible through the attribute ‘.meta’:

[1]:
import pymrio

io = pymrio.load_test()
[2]:
io.meta
[2]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20210224 10:41:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio
20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json
[3]:
io.meta("Loaded the pymrio test sytem")
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20210224 10:41:58 - NOTE -  Loaded the pymrio test sytem
20210224 10:41:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio
20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json

We can now do several steps to modify the system, for example:

[4]:
io.calc_all()
io.aggregate(region_agg="global")
[4]:
<pymrio.core.mriosystem.IOSystem at 0x7f4ed6d73d30>
[5]:
io.meta
[5]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate population vector
20210224 10:41:58 - MODIFICATION -  Aggregate industry output x
20210224 10:41:58 - MODIFICATION -  Aggregate transaction matrix Z
20210224 10:41:58 - MODIFICATION -  Aggregate final demand y
20210224 10:41:58 - MODIFICATION -  Reset to absolute flows
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions
 ... (more lines in history)

Notes can added at any time:

[6]:
io.meta.note("First round of calculations finished")
[7]:
io.meta
[7]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio/metadata.json
History:
20210224 10:41:58 - NOTE -  First round of calculations finished
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate population vector
20210224 10:41:58 - MODIFICATION -  Aggregate industry output x
20210224 10:41:58 - MODIFICATION -  Aggregate transaction matrix Z
20210224 10:41:58 - MODIFICATION -  Aggregate final demand y
20210224 10:41:58 - MODIFICATION -  Reset to absolute flows
 ... (more lines in history)

In addition, all file io operations are recorde in the meta data:

[8]:
io.save_all("/tmp/foo")
[8]:
<pymrio.core.mriosystem.IOSystem at 0x7f4ed6d73d30>
[9]:
io_new = pymrio.load_all("/tmp/foo")
[10]:
io_new.meta
[10]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v1
File: /tmp/foo/metadata.json
History:
20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs
20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/emissions
20210224 10:41:58 - FILEIO -  Loaded IO system from /tmp/foo
20210224 10:41:58 - FILEIO -  Saved testmrio to /tmp/foo
20210224 10:41:58 - NOTE -  First round of calculations finished
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate population vector
 ... (more lines in history)

The top level meta data can be changed as well. These changes will also be recorded in the history:

[11]:
io_new.meta.change_meta("Version", "v2")
[12]:
io_new.meta
[12]:
Description: test mrio for pymrio
MRIO Name: testmrio
System: pxp
Version: v2
File: /tmp/foo/metadata.json
History:
20210224 10:41:58 - METADATA_CHANGE -  Changed parameter "version" from "v1" to "v2"
20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs
20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/emissions
20210224 10:41:58 - FILEIO -  Loaded IO system from /tmp/foo
20210224 10:41:58 - FILEIO -  Saved testmrio to /tmp/foo
20210224 10:41:58 - NOTE -  First round of calculations finished
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions
20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
20210224 10:41:58 - MODIFICATION -  Aggregate extensions...
 ... (more lines in history)

To get the full history list, use:

[13]:
io_new.meta.history
[13]:
['20210224 10:41:58 - METADATA_CHANGE -  Changed parameter "version" from "v1" to "v2"',
 '20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/factor_inputs',
 '20210224 10:41:58 - FILEIO -  Added satellite account from /tmp/foo/emissions',
 '20210224 10:41:58 - FILEIO -  Loaded IO system from /tmp/foo',
 '20210224 10:41:58 - FILEIO -  Saved testmrio to /tmp/foo',
 '20210224 10:41:58 - NOTE -  First round of calculations finished',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20210224 10:41:58 - MODIFICATION -  Aggregate extensions...',
 '20210224 10:41:58 - MODIFICATION -  Aggregate extensions...',
 '20210224 10:41:58 - MODIFICATION -  Aggregate population vector',
 '20210224 10:41:58 - MODIFICATION -  Aggregate industry output x',
 '20210224 10:41:58 - MODIFICATION -  Aggregate transaction matrix Z',
 '20210224 10:41:58 - MODIFICATION -  Aggregate final demand y',
 '20210224 10:41:58 - MODIFICATION -  Reset to absolute flows',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20210224 10:41:58 - MODIFICATION -  Leontief matrix L calculated',
 '20210224 10:41:58 - MODIFICATION -  Coefficient matrix A calculated',
 '20210224 10:41:58 - MODIFICATION -  Industry output x calculated',
 '20210224 10:41:58 - NOTE -  Loaded the pymrio test sytem',
 '20210224 10:41:58 - FILEIO -  Load test_mrio from /home/konstans/proj/pymrio/pymrio/mrio_models/test_mrio',
 '20171024 12:11:47 - FILEIO -  Created metadata file ../test_mrio/metadata.json']

This can be restricted to one of the history types by:

[14]:
io_new.meta.modification_history
[14]:
['20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20210224 10:41:58 - MODIFICATION -  Aggregate extensions...',
 '20210224 10:41:58 - MODIFICATION -  Aggregate extensions...',
 '20210224 10:41:58 - MODIFICATION -  Aggregate population vector',
 '20210224 10:41:58 - MODIFICATION -  Aggregate industry output x',
 '20210224 10:41:58 - MODIFICATION -  Aggregate transaction matrix Z',
 '20210224 10:41:58 - MODIFICATION -  Aggregate final demand y',
 '20210224 10:41:58 - MODIFICATION -  Reset to absolute flows',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension emissions',
 '20210224 10:41:58 - MODIFICATION -  Calculating accounts for extension factor_inputs',
 '20210224 10:41:58 - MODIFICATION -  Leontief matrix L calculated',
 '20210224 10:41:58 - MODIFICATION -  Coefficient matrix A calculated',
 '20210224 10:41:58 - MODIFICATION -  Industry output x calculated']

or

[15]:
io_new.meta.note_history
[15]:
['20210224 10:41:58 - NOTE -  First round of calculations finished',
 '20210224 10:41:58 - NOTE -  Loaded the pymrio test sytem']