flow
ERROR :/usr/local/lib/python3.9/site-packages/tabulator/stream.py:422: UserWarning: Not supported option(s) "http_headers" for scheme "https" and format "html"
warnings.warn(message, UserWarning)
DEBUG :Starting new HTTPS connection (1): parents.education.gov.il:443
DEBUG :https://parents.education.gov.il:443 "GET /prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448 HTTP/1.1" 404 None
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load
bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open
self.seek(0)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek
self.__response.raise_for_status()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 125, in process_datapackage
return self.safe_process_datapackage(dp)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 186, in safe_process_datapackage
stream: Stream = Stream(self.load_source, **self.options).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 425, in open
self.__parser.open(source, encoding=self.__encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/html.py", line 41, in open
self.__chars = self.__loader.load(source, encoding=encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load
raise exceptions.HTTPError(str(exception))
tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 127, in process_datapackage
raise SourceLoadError('Failed to load source {!r} and options {!r}: {}'
dataflows.base.exceptions.SourceLoadError: Failed to load source 'https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448' and options {'format': 'html', 'http_headers': {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0', 'Accept': 'application/json, text/plain, */*', 'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Content-Type': 'application/json', 'Alt-Used': 'parents.education.gov.il', 'Connection': 'keep-alive', 'Referer': 'https://parents.education.gov.il/prhnet/gov-education/kindergarten/search-daycare', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'Priority': 'u=0'}, 'custom_parsers': {'xml': <class 'dataflows.processors.parsers.xml_parser.XMLParser'>, 'excel-xml': <class 'dataflows.processors.parsers.excel_xml_parser.ExcelXMLParser'>, 'sql': <class 'dataflows.processors.parsers.sql_parser.ExtendedSQLParser'>, 'geojson': <class 'dataflows.processors.parsers.geojson_parser.GeoJsonParser'>}, 'ignore_blank_headers': True, 'skip_rows': [{'type': 'preset', 'value': 'auto'}], 'headers': 1, 'sample_size': 1000}: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 20 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor load in position #1: Failed to load source 'https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448' and options {'format': 'html', 'http_headers': {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0', 'Accept': 'application/json, text/plain, */*', 'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Content-Type': 'application/json', 'Alt-Used': 'parents.education.gov.il', 'Connection': 'keep-alive', 'Referer': 'https://parents.education.gov.il/prhnet/gov-education/kindergarten/search-daycare', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'Priority': 'u=0'}, 'custom_parsers': {'xml': <class 'dataflows.processors.parsers.xml_parser.XMLParser'>, 'excel-xml': <class 'dataflows.processors.parsers.excel_xml_parser.ExcelXMLParser'>, 'sql': <class 'dataflows.processors.parsers.sql_parser.ExtendedSQLParser'>, 'geojson': <class 'dataflows.processors.parsers.geojson_parser.GeoJsonParser'>}, 'ignore_blank_headers': True, 'skip_rows': [{'type': 'preset', 'value': 'auto'}], 'headers': 1, 'sample_size': 1000}: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
Pipeline ID: facilities/education/facilities_education
flow
ERROR :/usr/local/lib/python3.9/site-packages/tabulator/stream.py:422: UserWarning: Not supported option(s) "http_headers" for scheme "https" and format "html"
warnings.warn(message, UserWarning)
DEBUG :Starting new HTTPS connection (1): parents.education.gov.il:443
DEBUG :https://parents.education.gov.il:443 "GET /prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448 HTTP/1.1" 404 None
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 63, in load
bytes = _RemoteStream(source, self.__http_session, self.__http_timeout).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 121, in open
self.seek(0)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 143, in seek
self.__response.raise_for_status()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 125, in process_datapackage
return self.safe_process_datapackage(dp)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 186, in safe_process_datapackage
stream: Stream = Stream(self.load_source, **self.options).open()
File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 425, in open
self.__parser.open(source, encoding=self.__encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/html.py", line 41, in open
self.__chars = self.__loader.load(source, encoding=encoding)
File "/usr/local/lib/python3.9/site-packages/tabulator/loaders/remote.py", line 72, in load
raise exceptions.HTTPError(str(exception))
tabulator.exceptions.HTTPError: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
self.datapackage = self.process_datapackage(self.datapackage)
File "/usr/local/lib/python3.9/site-packages/dataflows/processors/load.py", line 127, in process_datapackage
raise SourceLoadError('Failed to load source {!r} and options {!r}: {}'
dataflows.base.exceptions.SourceLoadError: Failed to load source 'https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448' and options {'format': 'html', 'http_headers': {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0', 'Accept': 'application/json, text/plain, */*', 'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Content-Type': 'application/json', 'Alt-Used': 'parents.education.gov.il', 'Connection': 'keep-alive', 'Referer': 'https://parents.education.gov.il/prhnet/gov-education/kindergarten/search-daycare', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'Priority': 'u=0'}, 'custom_parsers': {'xml': <class 'dataflows.processors.parsers.xml_parser.XMLParser'>, 'excel-xml': <class 'dataflows.processors.parsers.excel_xml_parser.ExcelXMLParser'>, 'sql': <class 'dataflows.processors.parsers.sql_parser.ExtendedSQLParser'>, 'geojson': <class 'dataflows.processors.parsers.geojson_parser.GeoJsonParser'>}, 'ignore_blank_headers': True, 'skip_rows': [{'type': 'preset', 'value': 'auto'}], 'headers': 1, 'sample_size': 1000}: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
spew_flow(flow, ctx)
File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
datastream = flow.datastream()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/flow.py", line 19, in datastream
return self._chain(ds)._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
datastream = self.source._process()
[Previous line repeated 20 more times]
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
self.raise_exception(exception)
File "/usr/local/lib/python3.9/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor load in position #1: Failed to load source 'https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448' and options {'format': 'html', 'http_headers': {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0', 'Accept': 'application/json, text/plain, */*', 'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Content-Type': 'application/json', 'Alt-Used': 'parents.education.gov.il', 'Connection': 'keep-alive', 'Referer': 'https://parents.education.gov.il/prhnet/gov-education/kindergarten/search-daycare', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'Priority': 'u=0'}, 'custom_parsers': {'xml': <class 'dataflows.processors.parsers.xml_parser.XMLParser'>, 'excel-xml': <class 'dataflows.processors.parsers.excel_xml_parser.ExcelXMLParser'>, 'sql': <class 'dataflows.processors.parsers.sql_parser.ExtendedSQLParser'>, 'geojson': <class 'dataflows.processors.parsers.geojson_parser.GeoJsonParser'>}, 'ignore_blank_headers': True, 'skip_rows': [{'type': 'preset', 'value': 'auto'}], 'headers': 1, 'sample_size': 1000}: 404 Client Error: Not Found for url: https://parents.education.gov.il/prhnet/Api/MeonotController/GetExcel?0=2027&1=0&2=0&3=0&4=0&5=0&csrt=4274816473439949448
Pipeline ID: facilities/education/facilities_education