Folder Structure

Ml User Delete Service folder structure is designed to organize the different modules and files that constitute the ml user delete flink job. It follows a modular approach, facilitating easy management and development of the service.

The structure is as follows

.  
│
└── ml-jobs
    ├── pom.xml
    └── ml-user-delete
        ├── pom.xml
        └── src
            ├── main
            │   ├── resources
            │   │   └── user-delete.conf
            │   └── scala
            │       └── org
            │           └── sunbird
            │               └── job
            │                   └── userdelete
            │                       ├── domain
            │                       │   └── Event.scala
            │                       ├── functions
            │                       │   └── UserDeleteFunction.scala
            │                       └── task
            │                           ├── UserDeleteConfig.scala
            │                           └── UserDeleteStreamTask.scala                           
            └── test
                ├── resources
                │   └── test.conf
                └── scala
                    └── org
                        └── sunbird
                            └── job
                                ├── fixture
                                │   ├── EventFixture.scala
                                │   └── LoadMongoData.scala
                                └── spec
                                    └── UserDeleteFunctionTestSpec.scala
       

ml-jobs

This is the main directory that contains manage learn related Flink jobs.

ml-user-delete

This folder contains all the required directory for actual implementation of logics.

resources [main]

This folder contains application related configurations in a file called user-delete.conf

domain

This directory contains a file called Even.scala which is used to declare methods to get specific keys out of JSON.

functions

This folder contains the file UserDeleteFunction.scala which helps in processing and deleting user information from the MongoDB.

task

This folder contains a file called UserDeleteStreamTask.scala which is an entry point for this service. and another file called UserDeleteConfig.scala which holds the variable constants.

resources [test]

This folder contains test related configurations values in a file called test.conf.

fixture

This directory contains a file called EventFixture which is a hard coded kafka events for the testing purpose. Also contains LoadMongoData file which holds the Mongo Data for the testing purpose.

spec

In the spec folder we have the main test case file called UserDeleteFunctionTestSpec.scala that is used to test our stream processing task.

Source code

https://github.com/Sunbird-Lern/data-pipeline/tree/release-7.0.0/ml-jobs/ml-user-delete