Invalid dependency :Cannot run until dependency passes validation: ./people/all From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments'] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_donations/political_donations" is successfully executed'] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed']
Pipeline ID: people/aggregated
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/all-appointments From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed'] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] Dependency unsuccessful :Cannot run until dependency "./people/political_donations/political_donations" is successfully executed Dependency unsuccessful :Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed
Pipeline ID: people/all
Pipeline ID: people/association_founders/association-founders
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/all-appointments
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/collect-maya-forms-debug-stats
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/maya/correlate
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates From ./people/company_appointments/maya/maya-notification-updates :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']
Pipeline ID: people/company_appointments/maya/maya-list
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-nominations
scrape-notification-list Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 55, in _maya_api_call return res.json() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 130, in <module> spew(datapackage, [collect()]) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew for rec in res: File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 113, in collect yield from _collect_date_range(year_start, year_end) File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 75, in _collect_date_range res = _maya_api_call(date_from, date_to, current_page) File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 57, in _maya_api_call raise Exception("Failed to Call Maya API for date_from:{} date_to:{} page_num:{}".format(date_from, date_to, page_num)) from e Exception: Failed to Call Maya API for date_from:2006-01-01 date_to:2007-01-01 page_num:0
Pipeline ID: people/company_appointments/maya/maya-notification-list
id, date, company, notification_type, s3_object_name, url
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/maya-notification-updates
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-resignations
רשימת כל המינויים והחברות מאתר כלכליסט
Pipeline ID: people/company_appointments/media/calcalist/calcalist
Name, job and company description.
stream_remote_resources Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open() File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open self.seek(0) File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek self.__response.raise_for_status() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 244, in <module> rows = stream_reader(resource, url, ignore_missing or url == "", limit_rows, File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 199, in stream_reader schema, headers, columns, stream, close = get_opener(url, _resource)() File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 175, in opener _stream.open() File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 423, in open self.__parser.open(source, encoding=self.__encoding) File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/xlsx.py", line 81, in open source_bytes = self.__loader.load(source, mode="b", encoding=encoding) File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load raise exceptions.HTTPError(str(exception)) tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
Pipeline ID: people/company_appointments/media/themarker/themarker
Pipeline ID: people/company_appointments/state_owned/correlate
Pipeline ID: people/company_appointments/state_owned/parse_pdf_resources
columns: date, url
Pipeline ID: people/company_appointments/state_owned/reports
Pipeline ID: people/company_appointments/state_owned/state-owned-nominations
Pipeline ID: people/municipal_appointments/municipal_representitives
flow Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process self.datapackage = self.process_datapackage(self.datapackage) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/set_type.py", line 71, in process_datapackage assert added, 'Failed to find field {} in schema'.format(self.name) AssertionError: Failed to find field re.compile('^party_letter$') in schema The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> spew_flow(flow, ctx) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow datastream = flow.datastream() File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 18, in datastream return self._chain(ds)._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() [Previous line repeated 4 more times] File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process self.raise_exception(exception) File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception raise error from cause dataflows.base.exceptions.ProcessorError: Errored in processor set_type in position #3: Failed to find field re.compile('^party_letter$') in schema
Pipeline ID: people/political_candidates/election_candidates21
flow Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage/helpers.py", line 59, in retrieve_descriptor the_descriptor = req.json() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 121, in process_datapackage return self.safe_process_datapackage(dp) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 159, in safe_process_datapackage self.load_dp = Package(self.load_source) File "/usr/local/lib/python3.9/site-packages/datapackage/package.py", line 102, in __init__ descriptor = helpers.retrieve_descriptor(descriptor) File "/usr/local/lib/python3.9/site-packages/datapackage/helpers.py", line 62, in retrieve_descriptor six.raise_from(exceptions.DataPackageException(message), error) File "<string>", line 3, in raise_from datapackage.exceptions.DataPackageException: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json" The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process self.datapackage = self.process_datapackage(self.datapackage) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 123, in process_datapackage raise SourceLoadError('Failed to load source {!r} and options {!r}: {}' dataflows.base.exceptions.SourceLoadError: Failed to load source 'https://next.obudget.org/datapackages/donations/transactions/datapackage.json' and options {}: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json" The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> spew_flow(flow, ctx) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow datastream = flow.datastream() File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 18, in datastream return self._chain(ds)._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process self.raise_exception(exception) File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception raise error from cause dataflows.base.exceptions.ProcessorError: Errored in processor load in position #1: Failed to load source 'https://next.obudget.org/datapackages/donations/transactions/datapackage.json' and options {}: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json"
Pipeline ID: people/political_donations/political_donations
Dirty dependency :Cannot run until dependency is executed: ./procurement/spending/latest-contract-spending
Pipeline ID: people/procurement/procurement-individuals
Invalid dependency :Cannot run until dependency passes validation: ./people/all From ./people/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/all-appointments'] From ./people/all :['From ./people/company_appointments/all-appointments', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed']] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations']] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]]] From ./people/all :['From ./people/company_appointments/all-appointments', ['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]]] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_donations/political_donations" is successfully executed'] From ./people/all :['Dependency unsuccessful', 'Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed']
Pipeline ID: people/aggregated
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/all-appointments From ./people/company_appointments/all-appointments :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed'] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-nominations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] From ./people/company_appointments/all-appointments :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations'] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list']] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']]] From ./people/company_appointments/all-appointments :['From ./people/company_appointments/maya/maya-resignations', ['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]] Dependency unsuccessful :Cannot run until dependency "./people/political_donations/political_donations" is successfully executed Dependency unsuccessful :Cannot run until dependency "./people/political_candidates/election_candidates21" is successfully executed
Pipeline ID: people/all
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/media/themarker/themarker" is successfully executed Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/all-appointments
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/collect-maya-forms-debug-stats
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-nominations From ./people/company_appointments/maya/maya-nominations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-nominations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]] Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-resignations From ./people/company_appointments/maya/maya-resignations :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list'] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates']] From ./people/company_appointments/maya/maya-resignations :['From ./people/company_appointments/maya/maya-list', ['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]]
Pipeline ID: people/company_appointments/maya/correlate
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates From ./people/company_appointments/maya/maya-notification-updates :['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']
Pipeline ID: people/company_appointments/maya/maya-list
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-nominations
id, date, company, notification_type, s3_object_name, url
Dependency unsuccessful :Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed
Pipeline ID: people/company_appointments/maya/maya-notification-updates
Name, job and company description.
Invalid dependency :Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-list From ./people/company_appointments/maya/maya-list :['Invalid dependency', 'Cannot run until dependency passes validation: ./people/company_appointments/maya/maya-notification-updates'] From ./people/company_appointments/maya/maya-list :['From ./people/company_appointments/maya/maya-notification-updates', ['Dependency unsuccessful', 'Cannot run until dependency "./people/company_appointments/maya/maya-notification-list" is successfully executed']]
Pipeline ID: people/company_appointments/maya/maya-resignations
Dirty dependency :Cannot run until dependency is executed: ./procurement/spending/latest-contract-spending
Pipeline ID: people/procurement/procurement-individuals
scrape-notification-list Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 55, in _maya_api_call return res.json() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 130, in <module> spew(datapackage, [collect()]) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew for rec in res: File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 113, in collect yield from _collect_date_range(year_start, year_end) File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 75, in _collect_date_range res = _maya_api_call(date_from, date_to, current_page) File "/datapackage_pipelines_budgetkey/pipelines/people/company_appointments/maya/scrape-notification-list.py", line 57, in _maya_api_call raise Exception("Failed to Call Maya API for date_from:{} date_to:{} page_num:{}".format(date_from, date_to, page_num)) from e Exception: Failed to Call Maya API for date_from:2006-01-01 date_to:2007-01-01 page_num:0
Pipeline ID: people/company_appointments/maya/maya-notification-list
Name, job and company description.
stream_remote_resources Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open() File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open self.seek(0) File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek self.__response.raise_for_status() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 244, in <module> rows = stream_reader(resource, url, ignore_missing or url == "", limit_rows, File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 199, in stream_reader schema, headers, columns, stream, close = get_opener(url, _resource)() File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 175, in opener _stream.open() File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 423, in open self.__parser.open(source, encoding=self.__encoding) File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/xlsx.py", line 81, in open source_bytes = self.__loader.load(source, mode="b", encoding=encoding) File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load raise exceptions.HTTPError(str(exception)) tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.themarker.com/st/inter/DB/tm/2015/minuyim30.xlsx
Pipeline ID: people/company_appointments/media/themarker/themarker
flow Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process self.datapackage = self.process_datapackage(self.datapackage) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/set_type.py", line 71, in process_datapackage assert added, 'Failed to find field {} in schema'.format(self.name) AssertionError: Failed to find field re.compile('^party_letter$') in schema The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> spew_flow(flow, ctx) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow datastream = flow.datastream() File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 18, in datastream return self._chain(ds)._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() [Previous line repeated 4 more times] File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process self.raise_exception(exception) File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception raise error from cause dataflows.base.exceptions.ProcessorError: Errored in processor set_type in position #3: Failed to find field re.compile('^party_letter$') in schema
Pipeline ID: people/political_candidates/election_candidates21
flow Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage/helpers.py", line 59, in retrieve_descriptor the_descriptor = req.json() File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 121, in process_datapackage return self.safe_process_datapackage(dp) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 159, in safe_process_datapackage self.load_dp = Package(self.load_source) File "/usr/local/lib/python3.9/site-packages/datapackage/package.py", line 102, in __init__ descriptor = helpers.retrieve_descriptor(descriptor) File "/usr/local/lib/python3.9/site-packages/datapackage/helpers.py", line 62, in retrieve_descriptor six.raise_from(exceptions.DataPackageException(message), error) File "<string>", line 3, in raise_from datapackage.exceptions.DataPackageException: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json" The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process self.datapackage = self.process_datapackage(self.datapackage) File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 123, in process_datapackage raise SourceLoadError('Failed to load source {!r} and options {!r}: {}' dataflows.base.exceptions.SourceLoadError: Failed to load source 'https://next.obudget.org/datapackages/donations/transactions/datapackage.json' and options {}: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json" The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> spew_flow(flow, ctx) File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow datastream = flow.datastream() File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 18, in datastream return self._chain(ds)._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process datastream = self.source._process() File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process self.raise_exception(exception) File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception raise error from cause dataflows.base.exceptions.ProcessorError: Errored in processor load in position #1: Failed to load source 'https://next.obudget.org/datapackages/donations/transactions/datapackage.json' and options {}: Unable to load JSON at "https://next.obudget.org/datapackages/donations/transactions/datapackage.json"
Pipeline ID: people/political_donations/political_donations
Pipeline ID: people/association_founders/association-founders
רשימת כל המינויים והחברות מאתר כלכליסט
Pipeline ID: people/company_appointments/media/calcalist/calcalist
Pipeline ID: people/company_appointments/state_owned/correlate
Pipeline ID: people/company_appointments/state_owned/parse_pdf_resources
columns: date, url
Pipeline ID: people/company_appointments/state_owned/reports
Pipeline ID: people/company_appointments/state_owned/state-owned-nominations
Pipeline ID: people/municipal_appointments/municipal_representitives