Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Dependency unsuccessful :Cannot run until dependency "./entities/cooperatives/cooperatives" is successfully executed

Pipeline ID: entities/all

Last Run triggered by:

dirty-task-init

Last Run started at:

2023-05-27 00:28:40.414088

Last Completion:

2023-05-27 00:29:34.828143

Last Success:

None

dump.to_sql
ERROR   :Traceback (most recent call last):
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1880, in _execute_context
ERROR   :self.dialect.do_executemany(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 982, in do_executemany
ERROR   :context._psycopg2_fetched_rows = xtras.execute_values(
ERROR   :File "/usr/local/lib/python3.9/site-packages/psycopg2/extras.py", line 1299, in execute_values
ERROR   :cur.execute(b''.join(parts))
ERROR   :psycopg2.errors
ERROR   :.
ERROR   :UniqueViolation
ERROR   ::
ERROR   :duplicate key value violates unique constraint "cooperatives_pkey"
DETAIL:  Key (id)=(0) already exists.
ERROR   :The above exception was the direct cause of the following exception:
ERROR   :Traceback (most recent call last):
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/dump/to_sql.py", line 15, in <module>
ERROR   :spew_flow(flow(ctx.parameters), ctx)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 181, in __exit__
ERROR   :spew(self.datapackage, self.resource_iterator, stats=self.stats)
ERROR   :File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
ERROR   :for rec in res:
ERROR   :File "/usr/local/lib/python3.9/site-packages/dataflows/processors/dumpers/dumper_base.py", line 69, in row_counter
ERROR   :for row in iterator:
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_sql/writer.py", line 52, in write
ERROR   :for wr in self.__insert():
ERROR   :File "/usr/local/lib/python3.9/site-packages/tableschema_sql/writer.py", line 83, in __insert
ERROR   :statement.execute(self.__buffer)
ERROR   :File "<string>", line 2, in execute
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py", line 402, in warned
ERROR   :return fn(*args, **kwargs)
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/base.py", line 965, in execute
ERROR   :return e._execute_clauseelement(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ERROR   :ret = self._execute_context(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
ERROR   :self._handle_dbapi_exception(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
ERROR   :util.raise_(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
ERROR   :raise exception
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1880, in _execute_context
ERROR   :self.dialect.do_executemany(
ERROR   :File "/usr/local/lib/python3.9/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 982, in do_executemany
ERROR   :context._psycopg2_fetched_rows = xtras.execute_values(
ERROR   :File "/usr/local/lib/python3.9/site-packages/psycopg2/extras.py", line 1299, in execute_values
ERROR   :cur.execute(b''.join(parts))
ERROR   :sqlalchemy.exc
ERROR   :.
ERROR   :IntegrityError
ERROR   ::
ERROR   :(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "cooperatives_pkey"
DETAIL:  Key (id)=(0) already exists.
[SQL: INSERT INTO cooperatives (id, name, cooperative_registration_date, phone, primary_type_id, primary_type, legal_status, municipality_id, inspector, address) VALUES (%(id)s, %(name)s, %(cooperative_registration_date)s, %(phone)s, %(primary_type_id)s, %(primary_type)s, %(legal_status)s, %(municipality_id)s, %(inspector)s, %(address)s)]
[parameters: ({'id': '570014712', 'name': 'אלסביל - אגודה שיתופית לאספקת מים בריחניה בע"מ', 'cooperative_registration_date': datetime.datetime(1958, 2, 26, 0, 0), 'phone': None, 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'סגורה - בוטלה לאחר פירוק', 'municipality_id': 540, 'inspector': 'יעקב חליל', 'address': 'ריחאנייה 13818'}, {'id': '570025502', 'name': 'אלסד - אגודה שיתופית חקלאית למי השקאה בגוש חלב בע"מ', 'cooperative_registration_date': datetime.datetime(1978, 10, 1, 0, 0), 'phone': '04-6989217', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - מאושרת', 'municipality_id': 487, 'inspector': 'יעקב חליל', 'address': "ג'ש (גוש חלב) 13872 ת.ד: 5"}, {'id': '570019901', 'name': 'אלסהלה - אגודה שיתופית למי שתיה בעין-אלסהלה בע"מ', 'cooperative_registration_date': datetime.datetime(1968, 6, 9, 0, 0), 'phone': '050-6240762', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'סגורה - בוטלה לאחר פירוק', 'municipality_id': 657, 'inspector': 'יעקב חליל', 'address': 'עין א-סהלה 30026'}, {'id': '570034785', 'name': 'אלסואנה - אגודה שיתופית להשקיה חקלאית בע"מ', 'cooperative_registration_date': datetime.datetime(1998, 5, 4, 0, 0), 'phone': None, 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - מאושרת', 'municipality_id': 487, 'inspector': 'יעקב חליל', 'address': "ג'ש (גוש חלב) 13872 ת.ד: 232"}, {'id': '570027508', 'name': 'אלסוסן - אגודה שיתופית חקלאית למי השקאה באבו סנאן בע"מ', 'cooperative_registration_date': datetime.datetime(1981, 10, 6, 0, 0), 'phone': None, 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'סגורה - בוטלה לאחר פירוק', 'municipality_id': 473, 'inspector': 'יעקב חליל', 'address': 'אבו סנאן 24905'}, {'id': '570018911', 'name': 'אלסוקנה - אגודה שיתופית לשכון בנצרת בע"מ', 'cooperative_registration_date': datetime.datetime(1967, 3, 10, 0, 0), 'phone': None, 'primary_type_id': 12, 'primary_type': 'שיכון', 'legal_status': 'סגורה - בוטלה לאחר פירוק', 'municipality_id': 7300, 'inspector': None, 'address': '33 נצרת'}, {'id': '570035147', 'name': 'אלסחאב - אגודה שיתופית חקלאית למי השקאה בע"מ', 'cooperative_registration_date': datetime.datetime(1998, 12, 13, 0, 0), 'phone': '04-9944259', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - מאושרת', 'municipality_id': 8900, 'inspector': 'יעקב חליל', 'address': 'טמרה  טמרה 3081100 ת.ד: 4082 מיקוד ת.ד: 3081100'}, {'id': '570017004', 'name': 'אלסלאם - אגודה שיתופית חקלאית לאספקת מים בג\'ת בע"מ', 'cooperative_registration_date': datetime.datetime(1962, 4, 10, 0, 0), 'phone': '04-8578432', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - מאושרת', 'municipality_id': 628, 'inspector': 'יעקב חליל', 'address': "ג'ת 3009100 ת.ד: 621 מיקוד ת.ד: 3009100"}  ... displaying 10 of 1001 total bound parameter sets ...  {'id': '570052746', 'name': 'דגן גלבוע - אגודה שיתופית חקלאית בע"מ', 'cooperative_registration_date': datetime.datetime(2009, 11, 23, 0, 0), 'phone': '04-6798851', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - שינוי שם אגודה', 'municipality_id': 262, 'inspector': 'מזל שרעבי', 'address': 'גינוסר 1498000'}, {'id': '570052381', 'name': 'דגן גנוסר - אגודה שיתופית חקלאית בע"מ', 'cooperative_registration_date': datetime.datetime(2009, 7, 22, 0, 0), 'phone': '04-6798860', 'primary_type_id': 5, 'primary_type': 'חקלאית', 'legal_status': 'פעילה - מאושרת', 'municipality_id': 262, 'inspector': 'מזל שרעבי', 'address': 'גינוסר 1498000'})]
(Background on this error at: https://sqlalche.me/e/14/gkpj)

Pipeline ID: entities/cooperatives/cooperatives

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Invalid dependency :Cannot run until dependency passes validation: ./entities/all
From ./entities/all :['Dependency unsuccessful', 'Cannot run until dependency "./entities/cooperatives/cooperatives" is successfully executed']

Pipeline ID: entities/entities

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Invalid dependency :Cannot run until dependency passes validation: ./entities/all
From ./entities/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/companies/registrar/details']
From ./entities/all :['From ./entities/companies/registrar/details', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/companies/registrar/scraper" is successfully executed']]
From ./entities/all :['Invalid dependency', 'Cannot run until dependency passes validation: ./entities/associations/guidestar/guidestar']
From ./entities/all :['From ./entities/associations/guidestar/guidestar', ['Dependency unsuccessful', 'Cannot run until dependency "./entities/associations/guidestar/guidestar-scraper" is successfully executed']]
From ./entities/all :['Dependency unsuccessful', 'Cannot run until dependency "./entities/special/registry" is successfully executed']
From ./entities/all :['Dependency unsuccessful', 'Cannot run until dependency "./entities/cooperatives/cooperatives" is successfully executed']

Pipeline ID: entities/fingerprints

Last Run triggered by:

dirty-task-init

Last Run started at:

2023-05-27 00:28:38.018169

Last Completion:

2023-05-27 00:28:40.406921

Last Success:

None

scraper
Traceback (most recent call last):
  File "/datapackage_pipelines_budgetkey/pipelines/entities/moital/scraper.py", line 79, in <module>
    spew(datapackage, [scrape()])
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/wrapper/wrapper.py", line 75, in spew
    for rec in res:
  File "/datapackage_pipelines_budgetkey/pipelines/entities/moital/scraper.py", line 32, in scrape
    desired_capabilities=DesiredCapabilities.PHANTOMJS)
AttributeError: type object 'DesiredCapabilities' has no attribute 'PHANTOMJS'

Pipeline ID: entities/moital/moital_service_providers

Last Run triggered by:

scheduled

Last Run started at:

2023-06-06 00:08:25.724471

Last Completion:

2023-06-06 00:08:31.709071

Last Success:

None

stream_remote_resources
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 244, in <module>
    rows = stream_reader(resource, url, ignore_missing or url == "", limit_rows,
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 199, in stream_reader
    schema, headers, columns, stream, close = get_opener(url, _resource)()
  File "/usr/local/lib/python3.9/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 175, in opener
    _stream.open()
  File "/usr/local/lib/python3.9/site-packages/tabulator/stream.py", line 423, in open
    self.__parser.open(source, encoding=self.__encoding)
  File "/usr/local/lib/python3.9/site-packages/tabulator/parsers/xlsx.py", line 98, in open
    self.__book = openpyxl.load_workbook(
  File "/usr/local/lib/python3.9/site-packages/openpyxl/reader/excel.py", line 315, in load_workbook
    reader = ExcelReader(filename, read_only, keep_vba,
  File "/usr/local/lib/python3.9/site-packages/openpyxl/reader/excel.py", line 124, in __init__
    self.archive = _validate_archive(fn)
  File "/usr/local/lib/python3.9/site-packages/openpyxl/reader/excel.py", line 96, in _validate_archive
    archive = ZipFile(filename, 'r')
  File "/usr/local/lib/python3.9/zipfile.py", line 1266, in __init__
    self._RealGetContents()
  File "/usr/local/lib/python3.9/zipfile.py", line 1333, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Pipeline ID: entities/safety-orders/safety-orders

Last Run triggered by:

None

Last Run started at:

None

Last Completion:

None

Last Success:

None

Dependency unsuccessful :Cannot run until dependency "./entities/safety-orders/safety-orders" is successfully executed

Pipeline ID: entities/safety-orders/safety-orders-by-entity