Models

class import_export_extensions.models.ExportJob(*args, **kwargs)

Abstract model for managing celery export jobs.

Encapsulate all logic related to celery export.

Export steps:

  1. Create ExportJob with resource initialization parameters.

  2. Try to export all data to file.

  3. If everything correct - export data to file from database.

Export file saves in media files.

exception DoesNotExist
class ExportStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

ExportJob possible statuses.

  • CREATED:

    export job is just created, no exporting done

  • EXPORTING:

    export job started

  • EXPORT_ERROR:

    DB queryset not exported, errors

  • EXPORTED:

    DB queryset exported, no errors occurred

State diagrams:

CREATED
   |
EXPORTING  - (EXPORT_ERROR)
   |
EXPORTED
exception MultipleObjectsReturned
cancel_export() → None

Cancel current data export.

ExportJob can be CANCELLED only from following states:

  • CREATED

  • EXPORTING

export_data()

Export data to data_file from DB.

property export_filename: str

Get filename for export result file.

property file_format: Format

Get initialized format instance.

property progress: TaskStateInfo | None

Return dict with parsing state.

Example for sync mode:

{
    'state': 'EXPORTING',
    'info': None
}

Example for celery (celery) mode:

{
    'state': 'EXPORTING',
    'info': {'current': 15, 'total': 100}
}
Possible states:

  1. PENDING

  2. STARTED

  3. SUCCESS

  4. EXPORTING - custom status that also set export info

https://docs.celeryproject.org/en/latest/userguide/tasks.html#states

save(force_insert=False, force_update=False, using=None, update_fields=None)

Start task for data exporting when ExportJob is created.

Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.

class import_export_extensions.models.ImportJob(*args, **kwargs)

Abstract model for managing celery import jobs.

Encapsulate all logic related to celery import.

Import steps:

  1. Create ImportJob with resource initialization parameters and file

    with data to be imported from

  2. Dry run. Try to import all data from file and collect statistics (

    errors, new rows, updated rows).

  3. If data for import is correct - import data from file to database.

exception DoesNotExist
class ImportStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

ImportJob possible statuses.

  • CREATED:

    import job is just created, no parsing done

  • PARSING:

    parse job started

  • PARSED:

    data_file parsed, no errors in data occurred

  • INPUT_ERROR:

    data_file parsed, data contain errors

  • PARSE_ERROR:

    data_file can’t be parsed (invalid format, etc.)

  • IMPORT_CONFIRMED

    import confirmed but not started yet

  • IMPORTING:

    importing job started

  • IMPORTED:

    data from data_file imported to DB w/o errors

  • IMPORT_ERROR:

    unknown error during import

  • CANCELLED:

    import job has been cancelled (revoked)

State diagrams:

CREATED
   |
.parse_data()
   |
PARSING  - (INPUT_ERROR, PARSE_ERROR)
   |
PARSED
   |
.confirm_import()
   |
IMPORT_CONFIRMED
   |
.import_data()
   |
IMPORTING - IMPORT_ERROR
   |
IMPORTED
exception MultipleObjectsReturned
cancel_import() → None

Cancel current data import.

ImportJob can be CANCELLED only from following states:

  • CREATED

  • PARSING

  • CONFIRMED

  • IMPORTING

confirm_import()

Update task status to IMPORT_CONFIRMED and start parsing.

This is “intermediate” state between PARSED and IMPORTING and required because of possible latency of celery task start.

Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.

import_data()

Import data from data_file to DB.

parse_data()

Parse data_file and collect results.

Sets result and/or traceback and update status.

property progress: TaskStateInfo | None

Return dict with parsing state.

Example for sync mode:

{
    'state': 'PARSING',
    'info': None
}

Example for background (celery) mode:

{
    'state': 'PARSING',
    'info': {'current': 15, 'total': 100}
}
Possible states:

  1. PENDING

  2. STARTED

  3. SUCCESS

  4. PARSING - custom status that also set importing info

https://docs.celeryproject.org/en/latest/userguide/tasks.html#states

property resource

Get initialized resource instance.

save(force_insert=False, force_update=False, using=None, update_fields=None)

Start task for data parsing when ImportJob is created.

Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.

start_parse_data_task()

Start parsing task.