procurement/calls_for_bids/calls-for-bids-all
Failed

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 16:00:03.047099
Last Completion:
2022-01-20 16:00:11.930132
Last Success:
None

flow
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 426, in connect
    tls_in_tls=tls_in_tls,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 450, in ssl_wrap_socket
    sock, context, tls_in_tls, server_hostname=server_hostname
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/usr/local/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
    self.datapackage = self.process_datapackage(self.datapackage)
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 112, in process_datapackage
    raise self.exc
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 60, in describe
    sample = list(itertools.islice(self.iterable, self.SAMPLE_SIZE))
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 89, in handle_iterable
    for x in self.iterable:
  File "/datapackage_pipelines_budgetkey/pipelines/procurement/calls_for_bids/btl.py", line 27, in scrape
    page = requests.get(URL, headers=HEADERS).text
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
    spew_flow(flow, ctx)
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
    datastream = flow.datastream()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/flow.py", line 18, in datastream
    return self._chain(ds)._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
    self.raise_exception(exception)
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
    raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor iterable_loader in position #1: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))

Pipeline ID: procurement/calls_for_bids/calls-for-bids-all

procurement/calls_for_bids/calls-for-bids-moh
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 00:39:59.788125
Last Completion:
2022-01-21 00:40:02.216608
Last Success:
2022-01-21 00:40:02.216608

Pipeline ID: procurement/calls_for_bids/calls-for-bids-moh

procurement/calls_for_bids/gov-publications
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.236238
Last Completion:
2021-12-21 15:14:29.434836
Last Success:
2021-12-21 15:14:29.434836

Pipeline ID: procurement/calls_for_bids/gov-publications

procurement/calls_for_bids/gov-publications-ministry-of-immigration
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.248531
Last Completion:
2021-12-21 15:14:08.587316
Last Success:
2021-12-21 15:14:08.587316

Pipeline ID: procurement/calls_for_bids/gov-publications-ministry-of-immigration

procurement/calls_for_bids/gov-publications2
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.234812
Last Completion:
2021-12-21 15:14:16.135624
Last Success:
2021-12-21 15:14:16.135624

Pipeline ID: procurement/calls_for_bids/gov-publications2

procurement/calls_for_bids/jobiz-all
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:08.014135
Last Completion:
2022-01-20 15:00:10.579115
Last Success:
2022-01-20 15:00:10.579115

Pipeline ID: procurement/calls_for_bids/jobiz-all

procurement/calls_for_bids/jobiz-calls-for-bids
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:10.601324
Last Completion:
2022-01-20 15:00:12.492084
Last Success:
2022-01-20 15:00:12.492084

Pipeline ID: procurement/calls_for_bids/jobiz-calls-for-bids

procurement/calls_for_bids/jobiz-scraper
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:02.316830
Last Completion:
2022-01-20 15:00:07.988571
Last Success:
2022-01-20 15:00:07.988571

Pipeline ID: procurement/calls_for_bids/jobiz-scraper

התקשרויות רכש ממשלתיות (latest-contract-spending)
Succeeded

אנו סוכמים מידע מתוך הדוחות הרבעוניים לתוך טבלה המכילה את המידע העדכני ביותר.


Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 01:23:39.228098
Last Completion:
2022-01-21 04:57:36.348721
Last Success:
2022-01-21 04:57:36.348721

Pipeline ID: procurement/spending/latest-contract-spending

procurement/spending/quarterly-contract-spending-report-uris
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:00:02.321760
Last Completion:
2022-01-20 23:00:25.726146
Last Success:
2022-01-20 23:00:25.726146

Pipeline ID: procurement/spending/quarterly-contract-spending-report-uris

דוחות רבעוניים מתוך מרכבה, לפי נוהל היחידה לחופש המידע. (quarterly-contract-spending-reports)
Succeeded

כל משרדי הממשלה ויחידות הסמך (שמחוברות למערכת מרכבה) צריכים לפרסם אחת לרבעון (תוך 45 יום מסוף הרבעון) דו״ח התקשרויות לפי פורמט מוגדר היטב הממומש כדו״ח מובנה במערכת המרכבה. כל המשרדים מדווחים עצמאית והדו״חות עולים לאתר היחידה לחופש המידע, משם אנו אוספים אותם. התוצר של השרשר הזה הוא פירוט כל הדיווחים של כל המשרדים.


Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:00:25.881509
Last Completion:
2022-01-20 23:13:13.452292
Last Success:
2022-01-20 23:13:13.452292

Pipeline ID: procurement/spending/quarterly-contract-spending-reports

procurement/spending/quarterly-contract-spending-reports-data
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:13:13.545225
Last Completion:
2022-01-21 00:08:15.457867
Last Success:
2022-01-21 00:08:15.457867

Pipeline ID: procurement/spending/quarterly-contract-spending-reports-data

procurement/spending/spending-by-entity
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 05:02:27.848253
Last Completion:
2022-01-21 05:17:26.839732
Last Success:
2022-01-21 05:17:26.839732

Pipeline ID: procurement/spending/spending-by-entity

Load all tenders data from data.gov.il to flat file (all)
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 00:41:37.147677
Last Completion:
2022-01-21 00:44:29.485978
Last Success:
2022-01-21 00:44:29.485978

Pipeline ID: procurement/tenders/all

procurement/tenders/processed
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 04:12:13.686727
Last Completion:
2022-01-21 04:19:41.541753
Last Success:
2022-01-21 04:19:41.541753

Pipeline ID: procurement/tenders/processed

procurement/calls_for_bids/calls-for-bids-all
Failed

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 16:00:03.047099
Last Completion:
2022-01-20 16:00:11.930132
Last Success:
None

flow
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 426, in connect
    tls_in_tls=tls_in_tls,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 450, in ssl_wrap_socket
    sock, context, tls_in_tls, server_hostname=server_hostname
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/usr/local/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 79, in _process
    self.datapackage = self.process_datapackage(self.datapackage)
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 112, in process_datapackage
    raise self.exc
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 60, in describe
    sample = list(itertools.islice(self.iterable, self.SAMPLE_SIZE))
  File "/usr/local/lib/python3.7/site-packages/dataflows/helpers/iterable_loader.py", line 89, in handle_iterable
    for x in self.iterable:
  File "/datapackage_pipelines_budgetkey/pipelines/procurement/calls_for_bids/btl.py", line 27, in scrape
    page = requests.get(URL, headers=HEADERS).text
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/specs/../lib/flow.py", line 15, in <module>
    spew_flow(flow, ctx)
  File "/usr/local/lib/python3.7/site-packages/datapackage_pipelines/utilities/flow_utils.py", line 46, in spew_flow
    datastream = flow.datastream()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/flow.py", line 18, in datastream
    return self._chain(ds)._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 75, in _process
    datastream = self.source._process()
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 86, in _process
    self.raise_exception(exception)
  File "/usr/local/lib/python3.7/site-packages/dataflows/base/datastream_processor.py", line 96, in raise_exception
    raise error from cause
dataflows.base.exceptions.ProcessorError: Errored in processor iterable_loader in position #1: HTTPSConnectionPool(host='www.btl.gov.il', port=443): Max retries exceeded with url: /Funds/kolotkorim/Pages/default.aspx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))

Pipeline ID: procurement/calls_for_bids/calls-for-bids-all

procurement/calls_for_bids/calls-for-bids-moh
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 00:39:59.788125
Last Completion:
2022-01-21 00:40:02.216608
Last Success:
2022-01-21 00:40:02.216608

Pipeline ID: procurement/calls_for_bids/calls-for-bids-moh

procurement/calls_for_bids/gov-publications
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.236238
Last Completion:
2021-12-21 15:14:29.434836
Last Success:
2021-12-21 15:14:29.434836

Pipeline ID: procurement/calls_for_bids/gov-publications

procurement/calls_for_bids/gov-publications-ministry-of-immigration
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.248531
Last Completion:
2021-12-21 15:14:08.587316
Last Success:
2021-12-21 15:14:08.587316

Pipeline ID: procurement/calls_for_bids/gov-publications-ministry-of-immigration

procurement/calls_for_bids/gov-publications2
Succeeded

Last Run triggered by:
dirty-task-init
Last Run started at:
2021-12-21 15:14:01.234812
Last Completion:
2021-12-21 15:14:16.135624
Last Success:
2021-12-21 15:14:16.135624

Pipeline ID: procurement/calls_for_bids/gov-publications2

procurement/calls_for_bids/jobiz-all
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:08.014135
Last Completion:
2022-01-20 15:00:10.579115
Last Success:
2022-01-20 15:00:10.579115

Pipeline ID: procurement/calls_for_bids/jobiz-all

procurement/calls_for_bids/jobiz-calls-for-bids
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:10.601324
Last Completion:
2022-01-20 15:00:12.492084
Last Success:
2022-01-20 15:00:12.492084

Pipeline ID: procurement/calls_for_bids/jobiz-calls-for-bids

procurement/calls_for_bids/jobiz-scraper
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 15:00:02.316830
Last Completion:
2022-01-20 15:00:07.988571
Last Success:
2022-01-20 15:00:07.988571

Pipeline ID: procurement/calls_for_bids/jobiz-scraper

התקשרויות רכש ממשלתיות (latest-contract-spending)
Succeeded

אנו סוכמים מידע מתוך הדוחות הרבעוניים לתוך טבלה המכילה את המידע העדכני ביותר.


Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 01:23:39.228098
Last Completion:
2022-01-21 04:57:36.348721
Last Success:
2022-01-21 04:57:36.348721

Pipeline ID: procurement/spending/latest-contract-spending

procurement/spending/quarterly-contract-spending-report-uris
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:00:02.321760
Last Completion:
2022-01-20 23:00:25.726146
Last Success:
2022-01-20 23:00:25.726146

Pipeline ID: procurement/spending/quarterly-contract-spending-report-uris

דוחות רבעוניים מתוך מרכבה, לפי נוהל היחידה לחופש המידע. (quarterly-contract-spending-reports)
Succeeded

כל משרדי הממשלה ויחידות הסמך (שמחוברות למערכת מרכבה) צריכים לפרסם אחת לרבעון (תוך 45 יום מסוף הרבעון) דו״ח התקשרויות לפי פורמט מוגדר היטב הממומש כדו״ח מובנה במערכת המרכבה. כל המשרדים מדווחים עצמאית והדו״חות עולים לאתר היחידה לחופש המידע, משם אנו אוספים אותם. התוצר של השרשר הזה הוא פירוט כל הדיווחים של כל המשרדים.


Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:00:25.881509
Last Completion:
2022-01-20 23:13:13.452292
Last Success:
2022-01-20 23:13:13.452292

Pipeline ID: procurement/spending/quarterly-contract-spending-reports

procurement/spending/quarterly-contract-spending-reports-data
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-20 23:13:13.545225
Last Completion:
2022-01-21 00:08:15.457867
Last Success:
2022-01-21 00:08:15.457867

Pipeline ID: procurement/spending/quarterly-contract-spending-reports-data

procurement/spending/spending-by-entity
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 05:02:27.848253
Last Completion:
2022-01-21 05:17:26.839732
Last Success:
2022-01-21 05:17:26.839732

Pipeline ID: procurement/spending/spending-by-entity

Load all tenders data from data.gov.il to flat file (all)
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 00:41:37.147677
Last Completion:
2022-01-21 00:44:29.485978
Last Success:
2022-01-21 00:44:29.485978

Pipeline ID: procurement/tenders/all

procurement/tenders/processed
Succeeded

Last Run triggered by:
scheduled
Last Run started at:
2022-01-21 04:12:13.686727
Last Completion:
2022-01-21 04:19:41.541753
Last Success:
2022-01-21 04:19:41.541753

Pipeline ID: procurement/tenders/processed