Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications" is successfully executed Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications2" is successfully executed Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications-ministry-of-immigration" is successfully executed
Pipeline ID: procurement/calls_for_bids/calls-for-bids-all
Pipeline ID: procurement/calls_for_bids/calls-for-bids-moh
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications-ministry-of-immigration
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications2
Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed
Pipeline ID: procurement/calls_for_bids/jobiz-all
Invalid dependency :Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-all From ./procurement/calls_for_bids/jobiz-all :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed']
Pipeline ID: procurement/calls_for_bids/jobiz-calls-for-bids
flow
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/lib/python3.9/socket.py", line 966, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -5] No address associated with hostname
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 716, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 404, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1061, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 363, in connect
self.sock = conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 644, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 802, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 594, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 112, in process_datapackage
raise self.exc
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 60, in describe
sample = list(itertools.islice(self.iterable, self.SAMPLE_SIZE))
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 89, in handle_iterable
for x in self.iterable:
File "/datapackage_pipelines_budgetkey/pipelines/procurement/calls_for_bids/jobiz.py", line 19, in fetch_results
content = requests.get(URL.format(index)).json()
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 677, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 5 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor iterable_loader in position #1: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
Pipeline ID: procurement/calls_for_bids/jobiz-scraper
Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications" is successfully executed Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications2" is successfully executed Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/gov-publications-ministry-of-immigration" is successfully executed
Pipeline ID: procurement/calls_for_bids/calls-for-bids-all
Dependency unsuccessful :Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed
Pipeline ID: procurement/calls_for_bids/jobiz-all
Invalid dependency :Cannot run until dependency passes validation: ./procurement/calls_for_bids/jobiz-all From ./procurement/calls_for_bids/jobiz-all :['Dependency unsuccessful', 'Cannot run until dependency "./procurement/calls_for_bids/jobiz-scraper" is successfully executed']
Pipeline ID: procurement/calls_for_bids/jobiz-calls-for-bids
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications-ministry-of-immigration
flow ERROR :Traceback (most recent call last): ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module> ERROR :spew_flow(flow, ctx) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__ ERROR :spew(self.datapackage, self.resource_iterator, stats=self.stats) ERROR :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew ERROR :for rec in res: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 55, in process_resource ERROR :yield from self.validator(res) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/validate.py", line 50, in func ERROR :yield from schema_validator(res.res, res, on_error=self.on_error) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/rows_processor.py", line 11, in process_resource ERROR :yield from self.func(resource) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 53, in func ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :[Previous line repeated 1 more time] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/filter_rows.py", line 19, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/add_computed_field.py", line 38, in process_resource ERROR :for row in rows: ERROR :[Previous line repeated 2 more times] ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 40, in process_resource ERROR :for row in resource: ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/schema_validator.py", line 68, in schema_validator ERROR :for i, row in enumerate(iterator): ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 41, in process_resource ERROR :yield self.process_row(row) ERROR :File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/row_processor.py", line 11, in process_row ERROR :ret = self.func(row) ERROR :File "/datapackage_pipelines_budgetkey/processors/gov_publications.py", line 70, in f ERROR :row[field] = row[field][0] ERROR :IndexError ERROR :: ERROR :list index out of range
Pipeline ID: procurement/calls_for_bids/gov-publications2
flow
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/local/lib/python3.9/socket.py", line 966, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -5] No address associated with hostname
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 716, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 404, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1061, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 363, in connect
self.sock = conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 644, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 802, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 594, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 112, in process_datapackage
raise self.exc
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 60, in describe
sample = list(itertools.islice(self.iterable, self.SAMPLE_SIZE))
File "/usr/local/lib/python3.9/site-packages/dataflows/helpers/iterable_loader.py", line 89, in handle_iterable
for x in self.iterable:
File "/datapackage_pipelines_budgetkey/pipelines/procurement/calls_for_bids/jobiz.py", line 19, in fetch_results
content = requests.get(URL.format(index)).json()
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 677, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 5 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor iterable_loader in position #1: HTTPSConnectionPool(host='jobiz.gov.il', port=443): Max retries exceeded with url: /ajax/results/%D7%94%D7%95%D7%93%D7%A2%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/0?ie=0&typeie=%D7%94%D7%95%D7%93%D7%A2%D7%94+%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA&search=Array (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f328fe3ffd0>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
Pipeline ID: procurement/calls_for_bids/jobiz-scraper
Pipeline ID: procurement/calls_for_bids/calls-for-bids-moh