budget/municipal/fetcher
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:02:31.985104
Last Completion:
2024-03-29 00:10:10.374496
Last Success:
2024-03-29 00:10:10.374496

Pipeline ID: budget/municipal/fetcher

budget/national/changes/explanations/all
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2023-10-30 12:52:40.851010
Last Completion:
2023-10-30 12:54:56.895489
Last Success:
2023-10-30 12:54:56.895489

Pipeline ID: budget/national/changes/explanations/all

budget/national/changes/original/current-year-fixes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:14:59.451777
Last Completion:
2024-03-29 00:15:21.352178
Last Success:
2024-03-29 00:15:21.352178

Pipeline ID: budget/national/changes/original/current-year-fixes

budget/national/changes/original/national-budget-changes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:00:07.079796
Last Completion:
2024-03-29 00:02:20.580047
Last Success:
2024-03-29 00:02:20.580047

Pipeline ID: budget/national/changes/original/national-budget-changes

Changes, grouped by transaction_id, with explanations (national-budget-changes-aggregated)
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:12:33.477925
Last Completion:
2024-03-29 00:19:19.360882
Last Success:
2024-03-29 00:19:19.360882

Pipeline ID: budget/national/changes/processed/national-budget-changes-aggregated

budget/national/changes/processed/transactions
Failed

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:21.370973
Last Completion:
2024-03-29 00:15:45.173173
Last Success:
2024-03-22 02:35:12.860416

extract-change-groups
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 253, in <module>
    main()
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 250, in main
    spew(update_datapackage(datapackage), process_resources(res_iter))
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 232, in process_resource
    for transaction in get_transactions(get_changes(rows)):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 193, in assign_transactions
    changes = changes_by_trcode[trcode]
KeyError: '2011/83-029'

Pipeline ID: budget/national/changes/processed/transactions

National Yearly Budgets (national-budgets)
Succeeded

We get the data the Ministry of Finance publishes in data.gov.il every year. It comes in XLS format, with one row per TAKANA and phase (original, approved, executed). In this pipeline we create from each triplet a single row that has all the data.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 15:57:59.170600
Last Completion:
2024-03-14 18:03:17.779046
Last Success:
2024-03-14 18:03:17.779046

Pipeline ID: budget/national/original/national-budgets

National Yearly Budgets - Processed (aggregated-yearly)
Succeeded

This pipeline joins the different phases of the budget (allocated, revised and executed). In the original file there's a separate row for each of the phases. We like it better as a single row with all phase info. Another thing this pipeline does is to rename the column titles - to more friendly English names. Also, it create rows for all hierarchies - where upper hierarchies (2, 4 & 6 digits) are plain aggregations of the 8-digit items they contain.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 18:21:17.949862
Last Completion:
2024-03-14 20:13:53.924666
Last Success:
2024-03-14 20:13:53.924666

Pipeline ID: budget/national/processed/aggregated-yearly

budget/national/processed/category-explanations
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2023-10-30 12:54:56.900975
Last Completion:
2023-10-30 12:54:57.594178
Last Success:
2023-10-30 12:54:57.594178

Pipeline ID: budget/national/processed/category-explanations

National Yearly Budgets - Joined with category-explanations (connected-items-explained)
Succeeded

This pipeline joins budget items that span across years.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:46:11.555946
Last Completion:
2024-03-14 20:48:40.187135
Last Success:
2024-03-14 20:48:40.187135

Pipeline ID: budget/national/processed/connected-items-explained

National Yearly Budgets - Connected Items (connected-national-budgets)
Succeeded

This pipeline joins budget items that span across years.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.956897
Last Completion:
2024-03-14 20:46:11.189046
Last Success:
2024-03-14 20:46:11.189046

Pipeline ID: budget/national/processed/connected-national-budgets

National Yearly Budgets - Just the total (item 00) (just-the-total)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.770511
Last Completion:
2024-03-14 20:36:56.055236
Last Success:
2024-03-14 20:36:56.055236

Pipeline ID: budget/national/processed/just-the-total

National Yearly Budgets - Main Item Names (roof-names)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.778208
Last Completion:
2024-03-14 20:36:52.256096
Last Success:
2024-03-14 20:36:52.256096

Pipeline ID: budget/national/processed/roof-names

National Yearly Budgets - With Extras (with-extras)
Succeeded

This pipeline joins the budget data to itself so that each item has a list of its immediate children.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:13:54.003195
Last Completion:
2024-03-14 20:30:54.562845
Last Success:
2024-03-14 20:30:54.562845

Pipeline ID: budget/national/processed/with-extras

budgetkey/analysis/spending/publisher_entity_analysis
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:07:24.890781
Last Completion:
2024-03-29 10:44:44.791414
Last Success:
2024-03-29 10:44:44.791414

Pipeline ID: budgetkey/analysis/spending/publisher_entity_analysis

budgetkey/analysis/spending/publisher_foa_analysis
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:07:24.896407
Last Completion:
2024-03-29 10:11:11.646695
Last Success:
2024-03-29 10:11:11.646695

Pipeline ID: budgetkey/analysis/spending/publisher_foa_analysis

budgetkey/analysis/spending/united
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:00:01.637159
Last Completion:
2024-03-29 10:07:24.794283
Last Success:
2024-03-29 10:07:24.794283

Pipeline ID: budgetkey/analysis/spending/united

Main Page - Aggregated budget by functional classification (budget-functional-aggregates)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:13:54.078402
Last Completion:
2024-03-14 20:20:11.480477
Last Success:
2024-03-14 20:20:11.480477

Pipeline ID: budgetkey/budget-functional-aggregates

budgetkey/documents
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:20:11.873205
Last Completion:
2024-03-14 20:20:14.552842
Last Success:
2024-03-14 20:20:14.552842

Pipeline ID: budgetkey/documents

budgetkey/elasticsearch/index_activities
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:19:19.376004
Last Completion:
2024-03-29 00:19:54.355124
Last Success:
2024-03-29 00:19:54.355124

Pipeline ID: budgetkey/elasticsearch/index_activities

budgetkey/elasticsearch/index_budget
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:12:44.357769
Last Completion:
2024-03-29 00:40:35.017548
Last Success:
2024-03-29 00:40:35.017548

Pipeline ID: budgetkey/elasticsearch/index_budget

budgetkey/elasticsearch/index_calls_for_bids
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./procurement/calls_for_bids/calls-for-bids-all
From ./procurement/calls_for_bids/calls-for-bids-all :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-calls-for-bids']
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-all']]
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['From ./procurement/calls_for_bids/jobiz-all', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed']]]

Pipeline ID: budgetkey/elasticsearch/index_calls_for_bids

budgetkey/elasticsearch/index_contract_spending
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 01:51:33.556325
Last Completion:
2024-03-29 03:04:37.331504
Last Success:
2024-03-29 03:04:37.331504

Pipeline ID: budgetkey/elasticsearch/index_contract_spending

budgetkey/elasticsearch/index_entities
Invalid

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-01-01 02:29:59.341805
Last Completion:
2024-01-01 04:12:30.671371
Last Success:
2024-01-01 04:12:30.671371

Invalid dependency :Cannot run until dependency passes validation: ./entities/entities
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/spending-by-entity']
From ./entities/entities :['From ./procurement/spending/spending-by-entity', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]
From ./entities/entities :['From ./procurement/spending/spending-by-entity', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./supports/supports-by-entity']
From ./entities/entities :['From ./supports/supports-by-entity', ['Dependency unsuccessful', 'Cannot run until dependency "./supports/with-entities" is successfully executed']]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_suppliers']
From ./entities/entities :['From ./activities/social_services/social_service_suppliers', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/all']
From ./entities/entities :['From ./entities/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/companies/registrar/details']]
From ./entities/entities :['From ./entities/all', ['From ./entities/companies/registrar/details', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/companies/registrar/scraper" is successfully executed']]]
From ./entities/entities :['From ./entities/all', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/special/registry" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_entities

budgetkey/elasticsearch/index_gov_decisions
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 12:22:52.835048
Last Completion:
2024-03-29 12:24:23.864081
Last Success:
2024-03-29 12:24:23.864081

Pipeline ID: budgetkey/elasticsearch/index_gov_decisions

budgetkey/elasticsearch/index_muni_budgets
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:56.394972
Last Completion:
2024-03-29 00:32:49.075382
Last Success:
2024-03-29 00:32:49.075382

Pipeline ID: budgetkey/elasticsearch/index_muni_budgets

budgetkey/elasticsearch/index_muni_tenders
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:20:41.163373
Last Completion:
2024-03-29 00:21:20.470270
Last Success:
2024-03-29 00:21:20.470270

Pipeline ID: budgetkey/elasticsearch/index_muni_tenders

budgetkey/elasticsearch/index_national_budget_changes
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:34:23.759754
Last Completion:
2024-03-29 00:40:16.697335
Last Success:
2024-03-29 00:40:16.697335

Pipeline ID: budgetkey/elasticsearch/index_national_budget_changes

budgetkey/elasticsearch/index_people
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./people/aggregated
From ./people/aggregated :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/all']
From ./people/aggregated :['From ./people/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments']]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/state_owned/state-owned-nominations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/state_owned/state-owned-nominations', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/correlate" is successfully executed']]]]
From ./people/aggregated :['From ./people/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/procurement/procurement-individuals']]
From ./people/aggregated :['From ./people/all', ['From ./people/procurement/procurement-individuals', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]]
From ./people/aggregated :['From ./people/all', ['From ./people/procurement/procurement-individuals', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]]
From ./people/aggregated :['From ./people/all', ['Dependency unsuccessful', 'Cannot run until dependency "./people/political_donations/political_donations" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_people

budgetkey/elasticsearch/index_reports
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./reports/all
From ./reports/all :['Dependency unsuccessful', 'Cannot run until dependency "./reports/ngos/ngo-district-report" is successfully executed']
From ./reports/all :['Dependency unsuccessful', 'Cannot run until dependency "./reports/ngos/ngo-activity-report" is successfully executed']

Pipeline ID: budgetkey/elasticsearch/index_reports

budgetkey/elasticsearch/index_support_criteria
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./supports/criteria/support-criteria
From ./supports/criteria/support-criteria :['Dependency unsuccessful', 'Cannot run until dependency "./supports/criteria/support-criteria-doj" is successfully executed']

Pipeline ID: budgetkey/elasticsearch/index_support_criteria

budgetkey/elasticsearch/index_supports
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:37:37.414763
Last Completion:
2024-03-29 01:03:29.817999
Last Success:
2024-03-29 01:03:29.817999

Pipeline ID: budgetkey/elasticsearch/index_supports

budgetkey/elasticsearch/index_tenders
Invalid

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-08 00:25:46.861060
Last Completion:
2024-03-08 02:04:38.555490
Last Success:
2024-03-08 02:04:38.555490

Invalid dependency :Cannot run until dependency passes validation: ./procurement/tenders/processed
From ./procurement/tenders/processed :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/tenders/all" is successfully executed']
From ./procurement/tenders/processed :['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_tenders']
From ./procurement/tenders/processed :['From ./activities/social_services/social_service_tenders', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_tenders

budgetkey/elasticsearch/index_units
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:45.183121
Last Completion:
2024-03-29 00:16:15.774487
Last Success:
2024-03-29 00:16:15.774487

Pipeline ID: budgetkey/elasticsearch/index_units

budgetkey/elasticsearch/sitemaps
Invalid

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-01-01 15:36:52.442551
Last Completion:
2024-01-01 15:53:50.551880
Last Success:
2024-01-01 14:22:40.571748

Invalid dependency :Cannot run until dependency passes validation: ./budgetkey/elasticsearch/index_entities
From ./budgetkey/elasticsearch/index_entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/entities']
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/spending-by-entity']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./procurement/spending/spending-by-entity', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./procurement/spending/spending-by-entity', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./supports/supports-by-entity']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./supports/supports-by-entity', ['Dependency unsuccessful', 'Cannot run until dependency "./supports/with-entities" is successfully executed']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_suppliers']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./activities/social_services/social_service_suppliers', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/all']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/companies/registrar/details']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['From ./entities/companies/registrar/details', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/companies/registrar/scraper" is successfully executed']]]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/special/registry" is successfully executed']]]
Invalid dependency :Cannot run until dependency passes validation: ./budgetkey/elasticsearch/index_tenders
From ./budgetkey/elasticsearch/index_tenders :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/tenders/processed']
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/tenders/all" is successfully executed']]
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_tenders']]
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['From ./activities/social_services/social_service_tenders', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]]

Pipeline ID: budgetkey/elasticsearch/sitemaps

Send periodic emails (emails)
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 05:01:01.608784
Last Completion:
2024-03-29 12:31:57.047743
Last Success:
2024-03-29 12:31:57.047743

Pipeline ID: budgetkey/emails/emails

budgetkey/elasticsearch/index_calls_for_bids
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./procurement/calls_for_bids/calls-for-bids-all
From ./procurement/calls_for_bids/calls-for-bids-all :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-calls-for-bids']
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-all']]
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['From ./procurement/calls_for_bids/jobiz-all', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed']]]

Pipeline ID: budgetkey/elasticsearch/index_calls_for_bids

budgetkey/elasticsearch/index_entities
Invalid

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-01-01 02:29:59.341805
Last Completion:
2024-01-01 04:12:30.671371
Last Success:
2024-01-01 04:12:30.671371

Invalid dependency :Cannot run until dependency passes validation: ./entities/entities
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/spending-by-entity']
From ./entities/entities :['From ./procurement/spending/spending-by-entity', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]
From ./entities/entities :['From ./procurement/spending/spending-by-entity', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./supports/supports-by-entity']
From ./entities/entities :['From ./supports/supports-by-entity', ['Dependency unsuccessful', 'Cannot run until dependency "./supports/with-entities" is successfully executed']]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_suppliers']
From ./entities/entities :['From ./activities/social_services/social_service_suppliers', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]
From ./entities/entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/all']
From ./entities/entities :['From ./entities/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/companies/registrar/details']]
From ./entities/entities :['From ./entities/all', ['From ./entities/companies/registrar/details', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/companies/registrar/scraper" is successfully executed']]]
From ./entities/entities :['From ./entities/all', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/special/registry" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_entities

budgetkey/elasticsearch/index_people
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./people/aggregated
From ./people/aggregated :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/all']
From ./people/aggregated :['From ./people/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments']]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/state_owned/state-owned-nominations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/state_owned/state-owned-nominations', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/correlate" is successfully executed']]]]
From ./people/aggregated :['From ./people/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/procurement/procurement-individuals']]
From ./people/aggregated :['From ./people/all', ['From ./people/procurement/procurement-individuals', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]]
From ./people/aggregated :['From ./people/all', ['From ./people/procurement/procurement-individuals', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]]
From ./people/aggregated :['From ./people/all', ['Dependency unsuccessful', 'Cannot run until dependency "./people/political_donations/political_donations" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_people

budgetkey/elasticsearch/index_reports
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./reports/all
From ./reports/all :['Dependency unsuccessful', 'Cannot run until dependency "./reports/ngos/ngo-district-report" is successfully executed']
From ./reports/all :['Dependency unsuccessful', 'Cannot run until dependency "./reports/ngos/ngo-activity-report" is successfully executed']

Pipeline ID: budgetkey/elasticsearch/index_reports

budgetkey/elasticsearch/index_support_criteria
Invalid

Last Run triggered by:
None
Last Run started at:
None
Last Completion:
None
Last Success:
None

Invalid dependency :Cannot run until dependency passes validation: ./supports/criteria/support-criteria
From ./supports/criteria/support-criteria :['Dependency unsuccessful', 'Cannot run until dependency "./supports/criteria/support-criteria-doj" is successfully executed']

Pipeline ID: budgetkey/elasticsearch/index_support_criteria

budgetkey/elasticsearch/index_tenders
Invalid

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-08 00:25:46.861060
Last Completion:
2024-03-08 02:04:38.555490
Last Success:
2024-03-08 02:04:38.555490

Invalid dependency :Cannot run until dependency passes validation: ./procurement/tenders/processed
From ./procurement/tenders/processed :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/tenders/all" is successfully executed']
From ./procurement/tenders/processed :['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_tenders']
From ./procurement/tenders/processed :['From ./activities/social_services/social_service_tenders', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_tenders

budgetkey/elasticsearch/sitemaps
Invalid

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-01-01 15:36:52.442551
Last Completion:
2024-01-01 15:53:50.551880
Last Success:
2024-01-01 14:22:40.571748

Invalid dependency :Cannot run until dependency passes validation: ./budgetkey/elasticsearch/index_entities
From ./budgetkey/elasticsearch/index_entities :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/entities']
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/spending-by-entity']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./procurement/spending/spending-by-entity', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/spending/latest-contract-spending']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./procurement/spending/spending-by-entity', ['From ./procurement/spending/latest-contract-spending', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/quarterly-contract-spending-reports-data" is successfully executed']]]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./supports/supports-by-entity']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./supports/supports-by-entity', ['Dependency unsuccessful', 'Cannot run until dependency "./supports/with-entities" is successfully executed']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_suppliers']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./activities/social_services/social_service_suppliers', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/all']]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/companies/registrar/details']]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['From ./entities/companies/registrar/details', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/companies/registrar/scraper" is successfully executed']]]]
From ./budgetkey/elasticsearch/index_entities :['From ./entities/entities', ['From ./entities/all', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/special/registry" is successfully executed']]]
Invalid dependency :Cannot run until dependency passes validation: ./budgetkey/elasticsearch/index_tenders
From ./budgetkey/elasticsearch/index_tenders :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/tenders/processed']
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/tenders/all" is successfully executed']]
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['Invalid dependency', 'Cannot run until dependency passes validation: ./activities/social_services/social_service_tenders']]
From ./budgetkey/elasticsearch/index_tenders :['From ./procurement/tenders/processed', ['From ./activities/social_services/social_service_tenders', ['Dependency unsuccessful', 'Cannot run until dependency "./activities/social_services/social_services" is successfully executed']]]

Pipeline ID: budgetkey/elasticsearch/sitemaps

budget/national/changes/processed/transactions
Failed

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:21.370973
Last Completion:
2024-03-29 00:15:45.173173
Last Success:
2024-03-22 02:35:12.860416

extract-change-groups
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 253, in <module>
    main()
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 250, in main
    spew(update_datapackage(datapackage), process_resources(res_iter))
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 232, in process_resource
    for transaction in get_transactions(get_changes(rows)):
  File "/datapackage_pipelines_budgetkey/pipelines/budget/national/changes/processed/extract-change-groups.py", line 193, in assign_transactions
    changes = changes_by_trcode[trcode]
KeyError: '2011/83-029'

Pipeline ID: budget/national/changes/processed/transactions

budget/municipal/fetcher
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:02:31.985104
Last Completion:
2024-03-29 00:10:10.374496
Last Success:
2024-03-29 00:10:10.374496

Pipeline ID: budget/municipal/fetcher

budget/national/changes/explanations/all
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2023-10-30 12:52:40.851010
Last Completion:
2023-10-30 12:54:56.895489
Last Success:
2023-10-30 12:54:56.895489

Pipeline ID: budget/national/changes/explanations/all

budget/national/changes/original/current-year-fixes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:14:59.451777
Last Completion:
2024-03-29 00:15:21.352178
Last Success:
2024-03-29 00:15:21.352178

Pipeline ID: budget/national/changes/original/current-year-fixes

budget/national/changes/original/national-budget-changes
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:00:07.079796
Last Completion:
2024-03-29 00:02:20.580047
Last Success:
2024-03-29 00:02:20.580047

Pipeline ID: budget/national/changes/original/national-budget-changes

Changes, grouped by transaction_id, with explanations (national-budget-changes-aggregated)
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:12:33.477925
Last Completion:
2024-03-29 00:19:19.360882
Last Success:
2024-03-29 00:19:19.360882

Pipeline ID: budget/national/changes/processed/national-budget-changes-aggregated

National Yearly Budgets (national-budgets)
Succeeded

We get the data the Ministry of Finance publishes in data.gov.il every year. It comes in XLS format, with one row per TAKANA and phase (original, approved, executed). In this pipeline we create from each triplet a single row that has all the data.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 15:57:59.170600
Last Completion:
2024-03-14 18:03:17.779046
Last Success:
2024-03-14 18:03:17.779046

Pipeline ID: budget/national/original/national-budgets

National Yearly Budgets - Processed (aggregated-yearly)
Succeeded

This pipeline joins the different phases of the budget (allocated, revised and executed). In the original file there's a separate row for each of the phases. We like it better as a single row with all phase info. Another thing this pipeline does is to rename the column titles - to more friendly English names. Also, it create rows for all hierarchies - where upper hierarchies (2, 4 & 6 digits) are plain aggregations of the 8-digit items they contain.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 18:21:17.949862
Last Completion:
2024-03-14 20:13:53.924666
Last Success:
2024-03-14 20:13:53.924666

Pipeline ID: budget/national/processed/aggregated-yearly

budget/national/processed/category-explanations
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2023-10-30 12:54:56.900975
Last Completion:
2023-10-30 12:54:57.594178
Last Success:
2023-10-30 12:54:57.594178

Pipeline ID: budget/national/processed/category-explanations

National Yearly Budgets - Joined with category-explanations (connected-items-explained)
Succeeded

This pipeline joins budget items that span across years.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:46:11.555946
Last Completion:
2024-03-14 20:48:40.187135
Last Success:
2024-03-14 20:48:40.187135

Pipeline ID: budget/national/processed/connected-items-explained

National Yearly Budgets - Connected Items (connected-national-budgets)
Succeeded

This pipeline joins budget items that span across years.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.956897
Last Completion:
2024-03-14 20:46:11.189046
Last Success:
2024-03-14 20:46:11.189046

Pipeline ID: budget/national/processed/connected-national-budgets

National Yearly Budgets - Just the total (item 00) (just-the-total)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.770511
Last Completion:
2024-03-14 20:36:56.055236
Last Success:
2024-03-14 20:36:56.055236

Pipeline ID: budget/national/processed/just-the-total

National Yearly Budgets - Main Item Names (roof-names)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:30:54.778208
Last Completion:
2024-03-14 20:36:52.256096
Last Success:
2024-03-14 20:36:52.256096

Pipeline ID: budget/national/processed/roof-names

National Yearly Budgets - With Extras (with-extras)
Succeeded

This pipeline joins the budget data to itself so that each item has a list of its immediate children.


Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:13:54.003195
Last Completion:
2024-03-14 20:30:54.562845
Last Success:
2024-03-14 20:30:54.562845

Pipeline ID: budget/national/processed/with-extras

budgetkey/analysis/spending/publisher_entity_analysis
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:07:24.890781
Last Completion:
2024-03-29 10:44:44.791414
Last Success:
2024-03-29 10:44:44.791414

Pipeline ID: budgetkey/analysis/spending/publisher_entity_analysis

budgetkey/analysis/spending/publisher_foa_analysis
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:07:24.896407
Last Completion:
2024-03-29 10:11:11.646695
Last Success:
2024-03-29 10:11:11.646695

Pipeline ID: budgetkey/analysis/spending/publisher_foa_analysis

budgetkey/analysis/spending/united
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 10:00:01.637159
Last Completion:
2024-03-29 10:07:24.794283
Last Success:
2024-03-29 10:07:24.794283

Pipeline ID: budgetkey/analysis/spending/united

Main Page - Aggregated budget by functional classification (budget-functional-aggregates)
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:13:54.078402
Last Completion:
2024-03-14 20:20:11.480477
Last Success:
2024-03-14 20:20:11.480477

Pipeline ID: budgetkey/budget-functional-aggregates

budgetkey/documents
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2024-03-14 20:20:11.873205
Last Completion:
2024-03-14 20:20:14.552842
Last Success:
2024-03-14 20:20:14.552842

Pipeline ID: budgetkey/documents

budgetkey/elasticsearch/index_activities
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:19:19.376004
Last Completion:
2024-03-29 00:19:54.355124
Last Success:
2024-03-29 00:19:54.355124

Pipeline ID: budgetkey/elasticsearch/index_activities

budgetkey/elasticsearch/index_budget
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:12:44.357769
Last Completion:
2024-03-29 00:40:35.017548
Last Success:
2024-03-29 00:40:35.017548

Pipeline ID: budgetkey/elasticsearch/index_budget

budgetkey/elasticsearch/index_contract_spending
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 01:51:33.556325
Last Completion:
2024-03-29 03:04:37.331504
Last Success:
2024-03-29 03:04:37.331504

Pipeline ID: budgetkey/elasticsearch/index_contract_spending

budgetkey/elasticsearch/index_gov_decisions
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 12:22:52.835048
Last Completion:
2024-03-29 12:24:23.864081
Last Success:
2024-03-29 12:24:23.864081

Pipeline ID: budgetkey/elasticsearch/index_gov_decisions

budgetkey/elasticsearch/index_muni_budgets
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:56.394972
Last Completion:
2024-03-29 00:32:49.075382
Last Success:
2024-03-29 00:32:49.075382

Pipeline ID: budgetkey/elasticsearch/index_muni_budgets

budgetkey/elasticsearch/index_muni_tenders
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:20:41.163373
Last Completion:
2024-03-29 00:21:20.470270
Last Success:
2024-03-29 00:21:20.470270

Pipeline ID: budgetkey/elasticsearch/index_muni_tenders

budgetkey/elasticsearch/index_national_budget_changes
Succeeded

Last Run triggered by:
dirty-task-update
Last Run started at:
2024-03-29 00:34:23.759754
Last Completion:
2024-03-29 00:40:16.697335
Last Success:
2024-03-29 00:40:16.697335

Pipeline ID: budgetkey/elasticsearch/index_national_budget_changes

budgetkey/elasticsearch/index_supports
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:37:37.414763
Last Completion:
2024-03-29 01:03:29.817999
Last Success:
2024-03-29 01:03:29.817999

Pipeline ID: budgetkey/elasticsearch/index_supports

budgetkey/elasticsearch/index_units
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 00:15:45.183121
Last Completion:
2024-03-29 00:16:15.774487
Last Success:
2024-03-29 00:16:15.774487

Pipeline ID: budgetkey/elasticsearch/index_units

Send periodic emails (emails)
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2024-03-29 05:01:01.608784
Last Completion:
2024-03-29 12:31:57.047743
Last Success:
2024-03-29 12:31:57.047743

Pipeline ID: budgetkey/emails/emails