Last Run triggered by:

scheduled

Last Run started at:

2025-07-19 00:55:38.334084

Last Completion:

2025-07-19 00:56:03.722632

Last Success:

None

dump_to_es
ERROR   :DUMP TO ES ERROR 'any'
ERROR   :TB
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 146, in <module>
    spew_flow(flow(ctx.parameters), ctx)
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__
    spew(self.datapackage, self.resource_iterator, stats=self.stats)
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 68, in spew
    for res in resources_iterator:
  File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 68, in <genexpr>
    res_iter = (it if isinstance(it, ResourceWrapper) else ResourceWrapper(res, it)
  File "/usr/local/lib/python3.9/site-packages/dataflows/processors/dumpers/dumper_base.py", line 82, in process_resources
    ret = self.process_resource(
  File "/usr/local/lib/python3.9/site-packages/dataflows_elasticsearch/processors/dump_to_elasticsearch.py", line 69, in process_resource
    storage.create(index_name, res.descriptor['schema'],
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 123, in create
    self.put_mapping(bucket, descriptor, index_name, mapping_generator_cls)
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 79, in put_mapping
    mapping = mappers.descriptor_to_mapping(
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 139, in descriptor_to_mapping
    mapping_gen.generate_from_schema(descriptor)
  File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 55, in generate_from_schema
    super(BoostingMappingGenerator, self).generate_from_schema(schema)
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 126, in generate_from_schema
    self._update_properties(properties, schema)
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 116, in _update_properties
    dict(
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 117, in <genexpr>
    cls._convert_field(f, prefix)
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 109, in _convert_field
    prop = cls._convert_type(schema_type, field, prefix)
  File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 39, in _convert_type
    prop = super(BoostingMappingGenerator, cls)._convert_type(schema_type, field, prefix)
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 64, in _convert_type
    prop = {
KeyError: 'any'
ERROR   :Traceback (most recent call last):
ERROR   :File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 146, in <module>
ERROR   :spew_flow(flow(ctx.parameters), ctx)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__
ERROR   :spew(self.datapackage, self.resource_iterator, stats=self.stats)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 68, in spew
ERROR   :for res in resources_iterator:
ERROR   :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 68, in <genexpr>
ERROR   :res_iter = (it if isinstance(it, ResourceWrapper) else ResourceWrapper(res, it)
ERROR   :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/dumpers/dumper_base.py", line 82, in process_resources
ERROR   :ret = self.process_resource(
ERROR   :File "/usr/local/lib/python3.9/site-packages/dataflows_elasticsearch/processors/dump_to_elasticsearch.py", line 69, in process_resource
ERROR   :storage.create(index_name, res.descriptor['schema'],
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 123, in create
ERROR   :self.put_mapping(bucket, descriptor, index_name, mapping_generator_cls)
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 79, in put_mapping
ERROR   :mapping = mappers.descriptor_to_mapping(
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 139, in descriptor_to_mapping
ERROR   :mapping_gen.generate_from_schema(descriptor)
ERROR   :File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 55, in generate_from_schema
ERROR   :super(BoostingMappingGenerator, self).generate_from_schema(schema)
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 126, in generate_from_schema
ERROR   :self._update_properties(properties, schema)
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 116, in _update_properties
ERROR   :dict(
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 117, in <genexpr>
ERROR   :cls._convert_field(f, prefix)
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 109, in _convert_field
ERROR   :prop = cls._convert_type(schema_type, field, prefix)
ERROR   :File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 39, in _convert_type
ERROR   :prop = super(BoostingMappingGenerator, cls)._convert_type(schema_type, field, prefix)
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/mappers.py", line 64, in _convert_type
ERROR   :prop = {
ERROR   :KeyError
ERROR   ::
ERROR   :'any'

Pipeline ID: budgetkey/elasticsearch/index_activities

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Invalid dependency :Cannot run until dependency passes validation: ./procurement/calls_for_bids/calls-for-bids-all
From ./procurement/calls_for_bids/calls-for-bids-all :['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-calls-for-bids']
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['Invalid dependency', 'Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-all']]
From ./procurement/calls_for_bids/calls-for-bids-all :['From ./procurement/calls_for_bids/jobiz-calls-for-bids', ['From ./procurement/calls_for_bids/jobiz-all', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed']]]

Pipeline ID: budgetkey/elasticsearch/index_calls_for_bids

Last Run triggered by:

scheduled

Last Run started at:

2025-07-18 13:05:57.926426

Last Completion:

2025-07-18 14:03:13.139661

Last Success:

None

dump_to_es
ERROR   :DUMP TO ES ERROR ApiError(413, 'None')
ERROR   :TB
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 146, in <module>
    spew_flow(flow(ctx.parameters), ctx)
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__
    spew(self.datapackage, self.resource_iterator, stats=self.stats)
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/usr/local/lib/python3.9/site-packages/dataflows/processors/dumpers/dumper_base.py", line 69, in row_counter
    for row in iterator:
  File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 226, in write
    for result, row in iter:
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 453, in streaming_bulk
    for data, (ok, info) in zip(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 359, in _process_bulk_chunk
    yield from gen
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 293, in _process_bulk_chunk_error
    raise error
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 343, in _process_bulk_chunk
    resp = client.bulk(*args, operations=bulk_actions, **kwargs)  # type: ignore[arg-type]
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/utils.py", line 452, in wrapped
    return api(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/__init__.py", line 827, in bulk
    return self.perform_request(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 271, in perform_request
    response = self._perform_request(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 351, in _perform_request
    raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(
elasticsearch.ApiError: ApiError(413, 'None')
ERROR   :Traceback (most recent call last):
ERROR   :File "/datapackage_pipelines_budgetkey/processors/dump_to_es.py", line 146, in <module>
ERROR   :spew_flow(flow(ctx.parameters), ctx)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__
ERROR   :spew(self.datapackage, self.resource_iterator, stats=self.stats)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
ERROR   :for rec in res:
ERROR   :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/dumpers/dumper_base.py", line 69, in row_counter
ERROR   :for row in iterator:
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_elasticsearch/storage.py", line 226, in write
ERROR   :for result, row in iter:
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 453, in streaming_bulk
ERROR   :for data, (ok, info) in zip(
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 359, in _process_bulk_chunk
ERROR   :yield from gen
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 293, in _process_bulk_chunk_error
ERROR   :raise error
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/helpers/actions.py", line 343, in _process_bulk_chunk
ERROR   :resp = client.bulk(*args, operations=bulk_actions, **kwargs)  # type: ignore[arg-type]
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/utils.py", line 452, in wrapped
ERROR   :return api(*args, **kwargs)
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/__init__.py", line 827, in bulk
ERROR   :return self.perform_request(  # type: ignore[return-value]
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 271, in perform_request
ERROR   :response = self._perform_request(
ERROR   :File "/usr/local/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 351, in _perform_request
ERROR   :raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(
ERROR   :elasticsearch
ERROR   :.
ERROR   :ApiError
ERROR   ::
ERROR   :ApiError(413, 'None')

Pipeline ID: budgetkey/elasticsearch/index_gov_decisions

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Invalid dependency :Cannot run until dependency passes validation: ./people/aggregated
From ./people/aggregated :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/all']
From ./people/aggregated :['From ./people/all', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments']]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]]]
From ./people/aggregated :['From ./people/all', ['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]]]
From ./people/aggregated :['From ./people/all', ['Dependency unsuccessful', 'Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed']]

Pipeline ID: budgetkey/elasticsearch/index_people

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Dependency unsuccessful :Cannot run until dependency "./supports/criteria/support-criteria" is successfully executed

Pipeline ID: budgetkey/elasticsearch/index_support_criteria

Last Run triggered by:

scheduled

Last Run started at:

2025-06-26 21:35:33.161258

Last Completion:

2025-06-26 22:24:12.750689

Last Success:

2025-06-26 22:24:12.750689

Dependency unsuccessful :Cannot run until dependency "./budgetkey/elasticsearch/index_budget" is successfully executed
Dependency unsuccessful :Cannot run until dependency "./budgetkey/elasticsearch/index_gov_decisions" is successfully executed

Pipeline ID: budgetkey/elasticsearch/sitemaps