dvc.core package

Subpackages

• dvc.core.database package
  • Subpackages
    • dvc.core.database.bigquery package
      • Module contents
    • dvc.core.database.mysql package
      • Module contents
    • dvc.core.database.postgres package
      • Module contents
  • Module contents

Submodules

dvc.core.config module

class dvc.core.config.ConfigDefault

Bases: object

KEY__DATABASE_REVISION_SQL_FILES_FOLDER = 'DVC__DATABASE_REVISION_SQL_FILES_FOLDER'

KEY__DBFLAVOUR = 'DVC__DBFLAVOUR'

KEY__DBNAME = 'DVC__DBNAME'

KEY__HOST = 'DVC__HOST'

KEY__LOGGING_LEVEL = 'DVC__LOGGING_LEVEL'

KEY__PASSWORD = 'DVC__PASSWORD'

KEY__PORT = 'DVC__PORT'

KEY__TARGET_SCHEMA = 'DVC__TARGET_SCHEMA'

KEY__USER = 'DVC__USER'

VAL__DATABASE_REVISION_SQL_FILES_FOLDER = 'sample_revision_sql_files'

VAL__DBFLAVOUR = 'postgres'

VAL__DBNAME = ''

VAL__FILE_PATH: pathlib.Path = PosixPath('config.yaml')

VAL__FilE_NAME: str = 'config.yaml'

VAL__HOST = ''

VAL__LOGGING_LEVEL: str = 'INFO'

VAL__PASSWORD = ''

VAL__PORT = 5432

VAL__TARGET_SCHEMA = 'dvc'

VAL__USER = ''

classmethod get_config_dict(database_revision_sql_files_folder: str, target_schema: str, user: str, password: str, host: str, port: int, dbname: str, dbflavour: str, logging_level: int, as_file=False)

Parameters

• database_revision_sql_files_folder –

• user –

• password –

• host –

• port –

• dbname –

• dbflavour –

• logging_level – Assumed to be integer value

• as_file – whether to dump the dict as file.

Returns

class dvc.core.config.ConfigFileWriter(config_file_path: Union[pathlib.Path, str] = PosixPath('config.yaml'))

Bases: object

Read Config Files (in different formats) to Python Dictionary

write_to_yaml() → None

class dvc.core.config.ConfigReader(config_file_path: Union[pathlib.Path, str] = PosixPath('config.yaml'))

Bases: object

Read Config (in different formats) to Python Dictionary

Precedence in descending order 1. Config File 2. Environment Variable

class dvc.core.config.DatabaseConnectionFactory(config_reader: dvc.core.config.ConfigReader)

Bases: object

Return connections for various databases

MAPPING = {SupportedDatabaseFlavour.Postgres: 'self.pgconn'}

property conn: psycopg2.extensions.connection

Return the expected connection object for different database flavours :return:

property pgconn: psycopg2.extensions.connection

Return Postgres Database Connection

Returns

validate_requested_database_flavour() → dvc.core.database.SupportedDatabaseFlavour

Validate if requested database flavour is supported

Returns

class dvc.core.config.DatabaseRevisionFilesManager(config_file_reader: dvc.core.config.ConfigReader)

Bases: object

Manager all Database Revision Files

class Pointer

Bases: object

Head: ALl the way to the latest

BASE = 'base'

HEAD = 'head'

create_database_revision_files_folder() → None

Safely create the database revision files folder.

get_target_database_revision_files_by_pointer(current_database_version: dvc.core.struct.DatabaseVersion, candidate_database_revision_files: List[dvc.core.struct.DatabaseRevisionFile], pointer: dvc.core.config.DatabaseRevisionFilesManager.Pointer) → List[dvc.core.struct.DatabaseRevisionFile]

Given current database version and pointer, filter for target database revision files in the folder

Parameters

• current_database_version –

• candidate_database_revision_files –

Returns

get_target_database_revision_files_by_steps(current_database_version: dvc.core.struct.DatabaseVersion, steps: int, candidate_database_revision_files: List[dvc.core.struct.DatabaseRevisionFile]) → List[dvc.core.struct.DatabaseRevisionFile]

Given current database version and number of steps, filter for target database revision files in the folder.

Returns

dvc.core.exception module

exception dvc.core.exception.DatabaseConnectionFailureException

Bases: Exception

Exception raised when connection to the database fails

exception dvc.core.exception.EnvironmentVariableNotSetException(missing_env_var: str)

Bases: Exception

Exception raised when required environment variables are not found

exception dvc.core.exception.InvalidDatabaseRevisionFilesException(status: dvc.core.exception.InvalidDatabaseRevisionFilesException.Status, config_file_path: Optional[pathlib.Path], database_revision_file_paths: List[pathlib.Path])

Bases: Exception

Exception Raised when something is wrong with the DatabaseRevisionFiles

class Status(value)

Bases: enum.Enum

List of all reasons

FEWER_REVISION_SQL_FILES_FOUND_THAN_REQUIRED_STEPS_SPECIFIED = 103

MORE_REVISION_SQL_FILES_FOUND_THAN_REQUIRED_STEPS_SPECIFIED = 102

NONCONSECUTIVE_REVISION_SQL_FILES_FOR_HEAD_OR_BASE_POINTER = 104

NON_CONFORMANT_REVISION_FILE_NAME_EXISTS = 101

exception dvc.core.exception.InvalidDatabaseVersionException(database_version: str)

Bases: Exception

Exception raised when format of Database Version is wrong

exception dvc.core.exception.OperationNotAccountedForException(operation_type=<enum 'Operation'>)

Bases: Exception

Exception raised when operation is requested but is not yet developed

exception dvc.core.exception.RequestedDatabaseFlavourNotSupportedException(requested_database_flavour: str)

Bases: Exception

Exception raised when requested database flavour is not supported

dvc.core.file module

dvc.core.file.validate_file_exist(file_path: pathlib.Path) → None

Throw FileNotFileError if a given file does not exist

dvc.core.hash module

class dvc.core.hash.FileHasher

Bases: object

Hash content of any given file

md5(file_path: pathlib.Path) → str

Extract content from a file and hash its output

Parameters

file_path – Pathlib Path

Returns

string

dvc.core.logger module

class dvc.core.logger.SetRootLoggingLevel(func)

Bases: object

Used as a decorator to set the root logging level

set_logging_level(logging_level: int)

dvc.core.regex module

dvc.core.regex.get_matched_files_in_folder_by_regex(folder_path: pathlib.Path, file_name_regex: str) → List[pathlib.Path]

Loop recursively for all files in a given folder. Return those files whose name satisfy the regex.

Parameters

• folder_path – Path pointing tothe folder with files

• file_name_regex – regex used to filter for files with the desired file name

Returns

dvc.core.struct module

class dvc.core.struct.DatabaseRevisionFile(file_path: pathlib.Path)

Bases: object

Raise error when File Path does not conform to standard

STANDARD_RV_FILE_FORMAT_REGEX = '^RV[0-9]+__.*\\.(upgrade|downgrade)\\.sql$'

property description: str

Get the file description

Returns

property ending: str

Get the file ending

Returns

classmethod get_dummy_revision_file(revision: str, operation_type: dvc.core.struct.Operation) → dvc.core.struct.DatabaseRevisionFile

Return a dummy revision file

Parameters

• revision –

• operation_type –

Returns

property operation_type: dvc.core.struct.Operation

Get the operation type

Returns

property revision_number: int

Get the revision number

Returns

class dvc.core.struct.DatabaseVersion(version: str, created_at: Optional[datetime.datetime] = None)

Bases: object

STANDARD_DATABASE_VERSION_FORMAT_REGEX = '^V[0-9]+$'

property created_at

property next_downgrade_database_revision_file: dvc.core.struct.DatabaseRevisionFile

Get the database revision file for downgrade :return:

property next_upgrade_database_revision_file: dvc.core.struct.DatabaseRevisionFile

Get the database revision file for upgrade :return:

property version

property version_number: int

class dvc.core.struct.Operation(value)

Bases: enum.Enum

Database Operations Allowed

Downgrade = 'downgrade'

Upgrade = 'upgrade'

Module contents