Invalid dependency :Cannot run until dependency passes validation: ./people/all From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments'] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/state-owned-nominations" is successfully executed']] From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/procurement/procurement-individuals'] From ./people/all :['From ./people/procurement/procurement-individuals', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed']] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed']
Pipeline ID: people/aggregated
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/all-appointments From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed'] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/state-owned-nominations" is successfully executed'] Invalid dependency :Cannot run until dependency passes validation: ./people/procurement/procurement-individuals From ./people/procurement/procurement-individuals :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed'] Dependency unsuccessful :Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed
Pipeline ID: people/all
Pipeline ID: people/association_founders/association-founders
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/all-appointments
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/collect-maya-forms-debug-stats
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/maya/correlate
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates From ./people/company_appointments/maya/maya-notification-updates :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']
Pipeline ID: people/company_appointments/maya/maya-list
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-nominations
scrape-notification-list
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 55, in _maya_api_call
return res.json()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 130, in <module>
spew(datapackage, [collect()])
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
for rec in res:
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 113, in collect
yield from _collect_date_range(year_start, year_end)
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 75, in _collect_date_range
res = _maya_api_call(date_from, date_to, current_page)
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 57, in _maya_api_call
raise Exception("Failed to Call Maya API for date_from:{} date_to:{} page_num:{}".format(date_from, date_to, page_num)) from e
Exception: Failed to Call Maya API for date_from:2006-01-01 date_to:2007-01-01 page_num:0
Pipeline ID: people/company_appointments/maya/maya-notification-list
id, date, company, notification_type, s3_object_name, url
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/maya-notification-updates
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-resignations
רשימת כל המינויים והחברות מאתר כלכליסט
Pipeline ID: people/company_appointments/media/calcalist/calcalist
Name, job and company description.
stream_remote_resources
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load
bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open
self.seek(0)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek
self.__response.raise_for_status()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 244, in <module>
rows = stream_reader(resource, url, ignore_missing or url == "", limit_rows,
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 199, in stream_reader
schema, headers, columns, stream, close = get_opener(url, _resource)()
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 175, in opener
_stream.open()
File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 425, in open
self.__parser.open(source, encoding=self.__encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/xlsx.py", line 81, in open
source_bytes = self.__loader.load(source, mode="b", encoding=encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load
raise exceptions.HTTPError(str(exception))
tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
Pipeline ID: people/company_appointments/media/themarker/themarker
Pipeline ID: people/company_appointments/state_owned/correlate
Pipeline ID: people/company_appointments/state_owned/parse_pdf_resources
columns: date, url
Pipeline ID: people/company_appointments/state_owned/reports
Pipeline ID: people/company_appointments/state_owned/state-owned-nominations
Pipeline ID: people/municipal_appointments/municipal_representitives
flow
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/set_type.py", line 71, in process_datapackage
assert added, 'Failed to find field {} in schema'.format(self.name)
AssertionError: Failed to find field re.compile('^party_letter$') in schema
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 4 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor set_type in position #3: Failed to find field re.compile('^party_letter$') in schema
Pipeline ID: people/political_candidates/election_candidates21
Pipeline ID: people/political_donations/political_donations
Dependency unsuccessful :Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed
Pipeline ID: people/procurement/procurement-individuals
Invalid dependency :Cannot run until dependency passes validation: ./people/all From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments'] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/state-owned-nominations" is successfully executed']] From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/procurement/procurement-individuals'] From ./people/all :['From ./people/procurement/procurement-individuals', ['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed']] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed']
Pipeline ID: people/aggregated
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/all-appointments From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed'] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/state_owned/state-owned-nominations" is successfully executed'] Invalid dependency :Cannot run until dependency passes validation: ./people/procurement/procurement-individuals From ./people/procurement/procurement-individuals :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed'] Dependency unsuccessful :Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed
Pipeline ID: people/all
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/all-appointments
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/collect-maya-forms-debug-stats
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/maya/correlate
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates From ./people/company_appointments/maya/maya-notification-updates :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']
Pipeline ID: people/company_appointments/maya/maya-list
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-nominations
id, date, company, notification_type, s3_object_name, url
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/maya-notification-updates
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-resignations
Dependency unsuccessful :Cannot run until dependency "./procurement/spending/latest-contract-spending" is successfully executed
Pipeline ID: people/procurement/procurement-individuals
scrape-notification-list
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 55, in _maya_api_call
return res.json()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 130, in <module>
spew(datapackage, [collect()])
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
for rec in res:
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 113, in collect
yield from _collect_date_range(year_start, year_end)
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 75, in _collect_date_range
res = _maya_api_call(date_from, date_to, current_page)
File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 57, in _maya_api_call
raise Exception("Failed to Call Maya API for date_from:{} date_to:{} page_num:{}".format(date_from, date_to, page_num)) from e
Exception: Failed to Call Maya API for date_from:2006-01-01 date_to:2007-01-01 page_num:0
Pipeline ID: people/company_appointments/maya/maya-notification-list
Name, job and company description.
stream_remote_resources
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load
bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open
self.seek(0)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek
self.__response.raise_for_status()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 244, in <module>
rows = stream_reader(resource, url, ignore_missing or url == "", limit_rows,
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 199, in stream_reader
schema, headers, columns, stream, close = get_opener(url, _resource)()
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 175, in opener
_stream.open()
File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 425, in open
self.__parser.open(source, encoding=self.__encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/xlsx.py", line 81, in open
source_bytes = self.__loader.load(source, mode="b", encoding=encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load
raise exceptions.HTTPError(str(exception))
tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
Pipeline ID: people/company_appointments/media/themarker/themarker
flow
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/set_type.py", line 71, in process_datapackage
assert added, 'Failed to find field {} in schema'.format(self.name)
AssertionError: Failed to find field re.compile('^party_letter$') in schema
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 4 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor set_type in position #3: Failed to find field re.compile('^party_letter$') in schema
Pipeline ID: people/political_candidates/election_candidates21
Pipeline ID: people/association_founders/association-founders
רשימת כל המינויים והחברות מאתר כלכליסט
Pipeline ID: people/company_appointments/media/calcalist/calcalist
Pipeline ID: people/company_appointments/state_owned/correlate
Pipeline ID: people/company_appointments/state_owned/parse_pdf_resources
columns: date, url
Pipeline ID: people/company_appointments/state_owned/reports
Pipeline ID: people/company_appointments/state_owned/state-owned-nominations
Pipeline ID: people/municipal_appointments/municipal_representitives
Pipeline ID: people/political_donations/political_donations