budget/national/changes/explanations/all
Failed

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-03-02 20:24:34.023030
Last Completion:
2021-03-03 00:38:08.300252
Last Success:
None
explanation_fetcher
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/explanations/explanation_fetcher.py", line 64, in <module>
    spew(dp, itertools.chain(res_iter, [get_explanations(parameters['url'])]))
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/explanations/explanation_fetcher.py", line 32, in get_explanations
    archive = gcl.download(url)
  File "/datapackage_pipelines_budgetkey/common/google_chrome.py", line 111, in download
    self.driver.get(url)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
  (Session info: chrome=88.0.4324.96)
DEBUG   :[chan 482] Max packet in: 32768 bytes
DEBUG   :[chan 482] Max packet out: 32768 bytes
DEBUG   :Secsh channel 482 opened.
DEBUG   :[chan 482] Sesch channel 482 request ok
DEBUG   :[chan 482] EOF received (482)
DEBUG   :[chan 482] EOF sent (482)
DEBUG   :Dropping user packet because connection is dead.

Pipeline ID: budget/national/changes/explanations/all

budget/national/changes/original/current-year-fixes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 04:19:59.047599
Last Completion:
2021-02-26 04:20:25.925852
Last Success:
2021-02-26 04:20:25.925852

Pipeline ID: budget/national/changes/original/current-year-fixes

budget/national/changes/original/national-budget-changes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 01:51:38.130631
Last Completion:
2021-02-26 01:56:16.911222
Last Success:
2021-02-26 01:56:16.911222

Pipeline ID: budget/national/changes/original/national-budget-changes

Changes, grouped by transaction_id, with explanations (national-budget-changes-aggregated)
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None
Dependency unsuccessful :Cannot run until dependency "./budget/national/changes/explanations/all" is successfully executed

Pipeline ID: budget/national/changes/processed/national-budget-changes-aggregated

budget/national/changes/processed/transactions
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 04:20:25.938037
Last Completion:
2021-02-26 11:09:55.794753
Last Success:
2021-02-26 11:09:55.794753

Pipeline ID: budget/national/changes/processed/transactions

National Yearly Budgets (national-budgets)
Succeeded

We get the data the Ministry of Finance publishes in data.gov.il every year. It comes in XLS format, with one row per TAKANA and phase (original, approved, executed). In this pipeline we create from each triplet a single row that has all the data.
Last Run triggered by:
manual
Last Run started at:
2021-02-11 11:30:59.021871
Last Completion:
2021-02-11 13:23:27.126360
Last Success:
2021-02-11 13:23:27.126360

Pipeline ID: budget/national/original/national-budgets

National Yearly Budgets - Processed (aggregated-yearly)
Succeeded

This pipeline joins the different phases of the budget (allocated, revised and executed). In the original file there's a separate row for each of the phases. We like it better as a single row with all phase info. Another thing this pipeline does is to rename the column titles - to more friendly English names. Also, it create rows for all hierarchies - where upper hierarchies (2, 4 & 6 digits) are plain aggregations of the 8-digit items they contain.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 15:46:11.498646
Last Completion:
2021-02-12 17:30:10.408804
Last Success:
2021-02-12 17:30:10.408804

Pipeline ID: budget/national/processed/aggregated-yearly

budget/national/processed/category-explanations
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-01-27 06:54:17.996534
Last Completion:
2021-01-27 06:54:19.159029
Last Success:
2021-01-27 06:54:19.159029

Pipeline ID: budget/national/processed/category-explanations

National Yearly Budgets - Joined with category-explanations (connected-items-explained)
Succeeded

This pipeline joins budget items that span across years.
Last Run triggered by:
dirty-task-update
Last Run started at:
2021-02-12 18:09:03.592044
Last Completion:
2021-02-12 18:11:51.305074
Last Success:
2021-02-12 18:11:51.305074

Pipeline ID: budget/national/processed/connected-items-explained

National Yearly Budgets - Connected Items (connected-national-budgets)
Succeeded

This pipeline joins budget items that span across years.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:58.246916
Last Completion:
2021-02-12 18:09:00.103959
Last Success:
2021-02-12 18:09:00.103959

Pipeline ID: budget/national/processed/connected-national-budgets

National Yearly Budgets - Just the total (item 00) (just-the-total)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:56.248465
Last Completion:
2021-02-12 17:56:34.897333
Last Success:
2021-02-12 17:56:34.897333

Pipeline ID: budget/national/processed/just-the-total

National Yearly Budgets - Main Item Names (roof-names)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:56.255357
Last Completion:
2021-02-12 17:56:29.915753
Last Success:
2021-02-12 17:56:29.915753

Pipeline ID: budget/national/processed/roof-names

National Yearly Budgets - With Extras (with-extras)
Succeeded

This pipeline joins the budget data to itself so that each item has a list of its immediate children.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:30:11.430173
Last Completion:
2021-02-12 17:47:55.459699
Last Success:
2021-02-12 17:47:55.459699

Pipeline ID: budget/national/processed/with-extras

Changes, grouped by transaction_id, with explanations (national-budget-changes-aggregated)
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None
Dependency unsuccessful :Cannot run until dependency "./budget/national/changes/explanations/all" is successfully executed

Pipeline ID: budget/national/changes/processed/national-budget-changes-aggregated

budget/national/changes/explanations/all
Failed

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-03-02 20:24:34.023030
Last Completion:
2021-03-03 00:38:08.300252
Last Success:
None
explanation_fetcher
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/explanations/explanation_fetcher.py", line 64, in <module>
    spew(dp, itertools.chain(res_iter, [get_explanations(parameters['url'])]))
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/explanations/explanation_fetcher.py", line 32, in get_explanations
    archive = gcl.download(url)
  File "/datapackage_pipelines_budgetkey/common/google_chrome.py", line 111, in download
    self.driver.get(url)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
  (Session info: chrome=88.0.4324.96)
DEBUG   :[chan 482] Max packet in: 32768 bytes
DEBUG   :[chan 482] Max packet out: 32768 bytes
DEBUG   :Secsh channel 482 opened.
DEBUG   :[chan 482] Sesch channel 482 request ok
DEBUG   :[chan 482] EOF received (482)
DEBUG   :[chan 482] EOF sent (482)
DEBUG   :Dropping user packet because connection is dead.

Pipeline ID: budget/national/changes/explanations/all

budget/national/changes/original/current-year-fixes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 04:19:59.047599
Last Completion:
2021-02-26 04:20:25.925852
Last Success:
2021-02-26 04:20:25.925852

Pipeline ID: budget/national/changes/original/current-year-fixes

budget/national/changes/original/national-budget-changes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 01:51:38.130631
Last Completion:
2021-02-26 01:56:16.911222
Last Success:
2021-02-26 01:56:16.911222

Pipeline ID: budget/national/changes/original/national-budget-changes

budget/national/changes/processed/transactions
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2021-02-26 04:20:25.938037
Last Completion:
2021-02-26 11:09:55.794753
Last Success:
2021-02-26 11:09:55.794753

Pipeline ID: budget/national/changes/processed/transactions

National Yearly Budgets (national-budgets)
Succeeded

We get the data the Ministry of Finance publishes in data.gov.il every year. It comes in XLS format, with one row per TAKANA and phase (original, approved, executed). In this pipeline we create from each triplet a single row that has all the data.
Last Run triggered by:
manual
Last Run started at:
2021-02-11 11:30:59.021871
Last Completion:
2021-02-11 13:23:27.126360
Last Success:
2021-02-11 13:23:27.126360

Pipeline ID: budget/national/original/national-budgets

National Yearly Budgets - Processed (aggregated-yearly)
Succeeded

This pipeline joins the different phases of the budget (allocated, revised and executed). In the original file there's a separate row for each of the phases. We like it better as a single row with all phase info. Another thing this pipeline does is to rename the column titles - to more friendly English names. Also, it create rows for all hierarchies - where upper hierarchies (2, 4 & 6 digits) are plain aggregations of the 8-digit items they contain.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 15:46:11.498646
Last Completion:
2021-02-12 17:30:10.408804
Last Success:
2021-02-12 17:30:10.408804

Pipeline ID: budget/national/processed/aggregated-yearly

budget/national/processed/category-explanations
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-01-27 06:54:17.996534
Last Completion:
2021-01-27 06:54:19.159029
Last Success:
2021-01-27 06:54:19.159029

Pipeline ID: budget/national/processed/category-explanations

National Yearly Budgets - Joined with category-explanations (connected-items-explained)
Succeeded

This pipeline joins budget items that span across years.
Last Run triggered by:
dirty-task-update
Last Run started at:
2021-02-12 18:09:03.592044
Last Completion:
2021-02-12 18:11:51.305074
Last Success:
2021-02-12 18:11:51.305074

Pipeline ID: budget/national/processed/connected-items-explained

National Yearly Budgets - Connected Items (connected-national-budgets)
Succeeded

This pipeline joins budget items that span across years.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:58.246916
Last Completion:
2021-02-12 18:09:00.103959
Last Success:
2021-02-12 18:09:00.103959

Pipeline ID: budget/national/processed/connected-national-budgets

National Yearly Budgets - Just the total (item 00) (just-the-total)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:56.248465
Last Completion:
2021-02-12 17:56:34.897333
Last Success:
2021-02-12 17:56:34.897333

Pipeline ID: budget/national/processed/just-the-total

National Yearly Budgets - Main Item Names (roof-names)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:47:56.255357
Last Completion:
2021-02-12 17:56:29.915753
Last Success:
2021-02-12 17:56:29.915753

Pipeline ID: budget/national/processed/roof-names

National Yearly Budgets - With Extras (with-extras)
Succeeded

This pipeline joins the budget data to itself so that each item has a list of its immediate children.
Last Run triggered by:
dirty-task-init
Last Run started at:
2021-02-12 17:30:11.430173
Last Completion:
2021-02-12 17:47:55.459699
Last Success:
2021-02-12 17:47:55.459699

Pipeline ID: budget/national/processed/with-extras