📙
Sunbird Lern
AskDot
  • LEARN
    • Overview
    • Functional Capabilities
    • Technical Architecture
      • Technical Architecture Diagram
      • Tech Stack
    • Dependencies
    • Product Roadmap
  • USE
    • Overview
    • Release Notes
      • Release V 8.0.0 (Ongoing)
      • Release V 7.0.0
      • Release V 5.4.0
      • Release V 5.3.0
      • Release V 5.2.0
      • Release V 5.1.0
      • Release V 5.0.1
      • Release V 5.0.0
      • Release V 4.10.0
      • Release V 4.9.0
      • Release V 4.8.0
      • Release V 4.7.0
    • Developer Guide
      • USER & ORG SERVICE
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Keycloak Local setup
          • Additional Installation Dependencies:
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
              • User
              • Organisation
              • System Settings
              • Role
              • Bulk Upload Process
              • Tenant Preference
              • Cassandra Migration Version
              • User Consent
              • Email Template
              • OTP
              • Page Management (LMS Service)
              • Location
              • User Notes
              • Deprecated
          • Elastic Search
          • Redis
        • APIs
          • User Management
            • User Deletion API
            • Ownership Transfer API
          • Organisation Management
          • Location Management
          • Consent Management
          • OTP Services
          • Tenant Configurations
          • Bulk Upload
          • System Settings
          • API Management Service
          • Data Sync
          • Notification APIs
        • Flink Jobs
          • User Cache Updater
          • User Deletion Cleanup Flink Job
          • Ownership Transfer Flink Job
        • Reports
          • Standard Exhaust
            • State Admin Geo report
            • State Admin Report
          • Other Jobs
            • User Cache Indexer Job
            • Delete Users Assets Report
          • Data Products Developer Installation
            • System Requirements
            • Tech Stack
            • Installation Dependencies
            • Local installation of data-products
            • Server setup Guide
            • Installation Configuration
            • Data-product creation guide
            • Troubleshooting a data-product
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
          • System Settings
          • Email Configuration
          • SMS Configuration
          • OTP based SMS Configuration
        • Roles
          • System Roles
          • User Roles
          • RBAC
        • Dependencies
        • Keycloak on Sunbird
        • How to Guide
          • Create Organization
          • Create User
          • Add new user type & location
        • Single Sign-on with Sunbird
        • Caching and Denormalising User Metadata
          • ETLUserCacheUpdaterJob
          • UserCacheUpdaterFlinkJob
        • Unit Tests and Code Coverage
        • FAQs
      • LMS(BATCH) SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
        • Data Models
          • Cassandra
            • Data Dictionary
          • ElasticSearch
          • Redis
        • APIs
        • Flink Jobs
          • Merge User Courses
          • Relation Cache Updater
          • Activity Aggregate Updater
          • Assessment Aggregator
          • Enrolment Reconciliation
          • Collection Certificate Pre-Processor
          • Collection Certificate Generator
        • Reports
          • On-Demand Exhaust
            • Response Exhaust
            • User Info Exhaust
            • Progress Exhaust
          • Other Jobs
            • Collection Summary Job
            • Course Batch Status Updater Job
            • Cassandra Migrator Job
            • Score Metric Migration Job
            • Assessment Score Correction Job
            • Collection Reconciliation Job
            • Course Enrollment Job
            • Course Consumption Job
        • Logs, Telemetry Events
        • Configuration
          • System Configuration
          • Functional Configuration
        • Dependencies
        • Certificate Registry and Credentialing using Sunbird-RC
          • Configuring PublicKey in Sunbird-RC
          • Server Setup guide for Sunbird-RC
          • API Guide For Sunbird-RC
        • Certificate Flow
          • Certificates Creation and Configuration
        • FAQs
      • GROUPS
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
          • Redis
        • APIs
          • Create Group
        • Logs, Telemetry Events
          • UI Telemetry Events
          • Service Telemetry Events
        • Configuration
          • Functional Configurations
        • Design References
        • Dependencies
        • Unit Tests and Code Coverage
      • NOTIFICATION SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Data Dictionary
        • APIs
        • Flink Jobs
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
        • Dependencies
        • Unit Tests and Code Coverage
      • DISCUSSION FORUM
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
            • Nodebb setup
            • Discussion Middleware Setup
            • Discussion Middleware
            • Discussion forum integration with any application
            • Discussion-UI setup along with demo application.
            • Sunbird-lern portal for DF
          • Source Code
        • APIs
          • Category APIs
          • User APIs
          • Post APIs
          • Topic APIs
          • Forum APIs
        • Context Schema
        • Configurations
          • Nodebb Admin panel settings
          • Discussion MW & Nodebb System Config
        • Telemetry Events
        • Dependencies
      • ML SERVICE
        • DATA PIPELINE (Flink Jobs)
          • Program User Info
            • Component Diagram
            • Data Model
            • Folder Structure
          • Ml User Delete
            • Component Diagram
            • Data Model
            • Folder Structure
        • DATA PRODUCTS
          • Program Exhaust
            • Component Diagram
            • Folder Structure
    • Server Installation
    • Dependency setup
    • Deprecation
      • Release-5.4.0
    • Jenkins Jobs
    • Release Calendar
    • Learn More
      • Tech References
      • Telemetry Processing
      • Data Dictionary
        • Sample Data
      • Delete User Functionality
      • Asset Ownership Transfer
    • FAQs
  • Engage
    • Discuss
    • Contribute to Sunbird Lern
    • Raise an Issue
    • Contribution Guidelines
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. USE
  2. Developer Guide
  3. LMS(BATCH) SERVICE
  4. Flink Jobs

Merge User Courses

PreviousFlink JobsNextRelation Cache Updater

Last updated 1 year ago

Was this helpful?

'merge-user-courses' job will permit a user who owns multiple accounts on a sunbird platform (one on an Organisation tenant and one on the custodian tenant), to merge the course related usage history (course batch enrolments, course content consumption, course progress) into one primary account.

This job performs following tasks:

  • Merges the content consumption details from an account to primary account.

  • Merges the user batches from an account to primary account.

  • Merges the user activity aggregate data from an account to primary account.

  • Performs a batch-enrolment-sync to primary account

Configuration variables:

Variable
Default Value
Purpose

kafka.input.topic

{{env}}.lms.user.account.merge

Kafka topic from which messages/events are read to be processed.

kafka.output.failed.topic

{{env}}.learning.events.failed

Kafka topic to which message is written when an exception occurs while processing an event.

kafka.groupId

Kafka input topic group Id

kafka.output.course.batch.updater.topic

{{env}}.coursebatch.job.request

Kafka topic to which output message/event is inserted to perform user course progress activity reconciliation to merged Id

lms-cassandra.keyspace

sunbird_courses

Cassandra keyspace name

lms-cassandra.content_consumption.table

user_content_consumption

Cassandra table used to store content wise data for a collection of a batch by a user. Content progress, status etc

lms-cassandra.user_enrolments.table

user_enrolments

Cassandra table used to store user enrolment data in a collection of a particular batch. This also holds the consumption progress, enrolment status and issued certificate details

lms-cassandra.user_activity_agg.table

user_activity_agg

Cassandra table used to store user consumption aggregate details of a collection in a batch. Aggregates like the consumption completed content count

Sample event:

{
  "eid": "BE_JOB_REQUEST",
  "ets": 1648720639981,
  "mid": "LP.1648720639981.d6b1d8c8-7a4a-483a-b83a-b752bede648c",
  "actor": {
    "id": "Merge User Courses and Cert",
    "type": "System"
  },
  "context": {
    "pdata": {
      "ver": "1.0",
      "id": "org.sunbird.platform"
    },
    "channel": "0123221758376673287017",
    "env": "staging"
  },
  "object": {
    "type": "MergeUserCoursesAndCert",
    "id": "68b764efd14bb3b534984649"
  },
  "edata": {
    "action": "merge-user-courses-and-cert",
    "fromAccountId": "b964d83a-c40f-4b3c-85e7-68b764efd14b",
    "toAccountId": "191e6b5a-8581-46bd-b779-b3b534984649",
    "rootOrgId": "0123221758376673287017",
    "iteration": 1
  }
}

Source Code:

Logodata-pipeline/lms-jobs/merge-user-courses at release-5.3.0 · Sunbird-Lern/data-pipelineGitHub