Metadata-Version: 2.4
Name: core-cdc
Version: 2.0.0
Summary: This library provides the mechanism to implement Change Data Capture (CDC) services...
Author-email: Alejandro Cora González <alek.cora.glez@gmail.com>
Maintainer: Alejandro Cora González
License-Expression: MIT
Project-URL: Homepage, https://gitlab.com/bytecode-solutions/core/core-cdc
Project-URL: Repository, https://gitlab.com/bytecode-solutions/core/core-cdc
Project-URL: Documentation, https://core-cdc.readthedocs.io/en/latest/
Project-URL: Issues, https://gitlab.com/bytecode-solutions/core/core-cdc/-/issues
Project-URL: Changelog, https://gitlab.com/bytecode-solutions/core/core-cdc/-/blob/master/CHANGELOG.md
Classifier: Intended Audience :: Developers
Classifier: Development Status :: 5 - Production/Stable
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Utilities
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.9
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: core-mixins>=2.2.4
Requires-Dist: core-tests>=2.0.3
Provides-Extra: all
Requires-Dist: core-db[all]>=3.0.0; extra == "all"
Requires-Dist: mysql-replication>=1.0.9; extra == "all"
Provides-Extra: dev
Requires-Dist: core-dev-tools>=1.0.1; extra == "dev"
Provides-Extra: mongo
Requires-Dist: core-db[mongo]>=3.0.0; extra == "mongo"
Provides-Extra: mysql
Requires-Dist: core-db[mysql]>=3.0.0; extra == "mysql"
Requires-Dist: mysql-replication>=1.0.9; extra == "mysql"
Provides-Extra: snowflake
Requires-Dist: core-db[snowflake]>=3.0.0; extra == "snowflake"
Dynamic: license-file

core-cdc (CDC a.k.a Change Data Capture)
===============================================================================

It provides the core mechanism and required resources to 
implement "Change Data Capture" services...

===============================================================================


.. image:: https://img.shields.io/pypi/pyversions/core-cdc.svg
    :target: https://pypi.org/project/core-cdc/
    :alt: Python Versions

.. image:: https://img.shields.io/badge/license-MIT-blue.svg
    :target: https://gitlab.com/bytecode-solutions/core/core-cdc/-/blob/main/LICENSE
    :alt: License

.. image:: https://gitlab.com/bytecode-solutions/core/core-cdc/badges/release/pipeline.svg
    :target: https://gitlab.com/bytecode-solutions/core/core-cdc/-/pipelines
    :alt: Pipeline Status

.. image:: https://readthedocs.org/projects/core-cdc/badge/?version=latest
    :target: https://readthedocs.org/projects/core-cdc/
    :alt: Docs Status

.. image:: https://img.shields.io/badge/security-bandit-yellow.svg
    :target: https://github.com/PyCQA/bandit
    :alt: Security

|

Execution Environment
---------------------------------------

Install libraries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    pip install --upgrade pip
    pip install virtualenv
..

Create the Python Virtual Environment.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    virtualenv --python={{python-version}} .venv
    virtualenv --python=python3.11 .venv
..

Activate the Virtual Environment.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    source .venv/bin/activate
..

Install required libraries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    pip install .
..

Optional libraries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    pip install '.[all]'  # For all...
    pip install '.[mysql]'
    pip install '.[mongo]'
    pip install '.[snowflake]'
..

Check tests and coverage
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: shell

    python manager.py run-tests
    python manager.py run-tests --test-type integration
    python manager.py run-coverage

    # Having the docker containers up and running you can execute the functional
    # tests that ensure the CDC services are working as expected...
    python manager.py run-tests --test-type functional --pattern "*.py"
..

|

Engines
---------------------------------------

The following database engines have CDC implementations:

Fully Implemented
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

**MySQL** - Binary Log (BinLog) based CDC
    - Uses ``mysql-replication`` library
    - Captures INSERT, UPDATE, DELETE operations
    - Supports DDL events (CREATE, ALTER, DROP)
    - Fallback mechanism for column name resolution
    - See: `core_cdc/processors/mysql/ <core_cdc/processors/mysql/>`_

**MongoDB** - Change Streams based CDC
    - Uses native MongoDB Change Streams
    - Captures INSERT, UPDATE, DELETE operations
    - Requires replica set configuration
    - Real-time event streaming
    - See: `core_cdc/processors/mongo/ <core_cdc/processors/mongo/>`_

In Development
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

**MS SQL Server** - Abstract base class defined
    - See: `core_cdc/processors/mssql.py <core_cdc/processors/mssql.py>`_

**Oracle** - Abstract base class defined
    - See: `core_cdc/processors/oracle.py <core_cdc/processors/oracle.py>`_
