Models
- class import_export_extensions.models.ExportJob(*args, **kwargs)
Abstract model for managing celery export jobs.
Encapsulate all logic related to celery export.
Export steps:
Create ExportJob with resource initialization parameters.
Try to export all data to file.
If everything correct - export data to file from database.
Export file saves in media files.
- exception DoesNotExist
- class ExportStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
ExportJob possible statuses.
- CREATED:
export job is just created, no exporting done
- EXPORTING:
export job started
- EXPORT_ERROR:
DB queryset not exported, errors
- EXPORTED:
DB queryset exported, no errors occurred
State diagrams:
CREATED | EXPORTING - (EXPORT_ERROR) | EXPORTED
- exception MultipleObjectsReturned
- cancel_export() None
Cancel current data export.
- ExportJob can be CANCELLED only from following states:
CREATED
EXPORTING
- export_data()
Export data to data_file from DB.
- property export_filename: str
Get filename for export result file.
- property file_format: Format
Get initialized format instance.
- property progress: TaskStateInfo | None
Return dict with parsing state.
Example for sync mode:
{ 'state': 'EXPORTING', 'info': None }
Example for celery (celery) mode:
{ 'state': 'EXPORTING', 'info': {'current': 15, 'total': 100} }
- Possible states:
PENDING
STARTED
SUCCESS
EXPORTING - custom status that also set export info
https://docs.celeryproject.org/en/latest/userguide/tasks.html#states
- save(force_insert=False, force_update=False, using=None, update_fields=None)
Start task for data exporting when ExportJob is created.
Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.
- class import_export_extensions.models.ImportJob(*args, **kwargs)
Abstract model for managing celery import jobs.
Encapsulate all logic related to celery import.
Import steps:
- Create
ImportJob
with resource initialization parameters and file with data to be imported from
- Create
- Dry run. Try to import all data from file and collect statistics (
errors, new rows, updated rows).
If data for import is correct - import data from file to database.
- exception DoesNotExist
- class ImportStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
ImportJob possible statuses.
- CREATED:
import job is just created, no parsing done
- PARSING:
parse job started
- PARSED:
data_file parsed, no errors in data occurred
- INPUT_ERROR:
data_file parsed, data contain errors
- PARSE_ERROR:
data_file can’t be parsed (invalid format, etc.)
- IMPORT_CONFIRMED
import confirmed but not started yet
- IMPORTING:
importing job started
- IMPORTED:
data from data_file imported to DB w/o errors
- IMPORT_ERROR:
unknown error during import
- CANCELLED:
import job has been cancelled (revoked)
State diagrams:
CREATED | .parse_data() | PARSING - (INPUT_ERROR, PARSE_ERROR) | PARSED | .confirm_import() | IMPORT_CONFIRMED | .import_data() | IMPORTING - IMPORT_ERROR | IMPORTED
- exception MultipleObjectsReturned
- cancel_import() None
Cancel current data import.
- ImportJob can be CANCELLED only from following states:
CREATED
PARSING
CONFIRMED
IMPORTING
- confirm_import()
Update task status to IMPORT_CONFIRMED and start parsing.
This is “intermediate” state between PARSED and IMPORTING and required because of possible latency of celery task start.
Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.
- import_data()
Import data from data_file to DB.
- parse_data()
Parse data_file and collect results.
Sets result and/or traceback and update status.
- property progress: TaskStateInfo | None
Return dict with parsing state.
Example for sync mode:
{ 'state': 'PARSING', 'info': None }
Example for background (celery) mode:
{ 'state': 'PARSING', 'info': {'current': 15, 'total': 100} }
- Possible states:
PENDING
STARTED
SUCCESS
PARSING - custom status that also set importing info
https://docs.celeryproject.org/en/latest/userguide/tasks.html#states
- property resource
Get initialized resource instance.
- save(force_insert=False, force_update=False, using=None, update_fields=None)
Start task for data parsing when ImportJob is created.
Celery task is manually called with apply_async, to provide possibility of custom task_id with which task will be run.
- start_parse_data_task()
Start parsing task.