📙
Sunbird Lern
AskDot
  • LEARN
    • Overview
    • Functional Capabilities
    • Technical Architecture
      • Technical Architecture Diagram
      • Tech Stack
    • Dependencies
    • Product Roadmap
  • USE
    • Overview
    • Release Notes
      • Release V 8.0.0 (Ongoing)
      • Release V 7.0.0
      • Release V 5.4.0
      • Release V 5.3.0
      • Release V 5.2.0
      • Release V 5.1.0
      • Release V 5.0.1
      • Release V 5.0.0
      • Release V 4.10.0
      • Release V 4.9.0
      • Release V 4.8.0
      • Release V 4.7.0
    • Developer Guide
      • USER & ORG SERVICE
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Keycloak Local setup
          • Additional Installation Dependencies:
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
              • User
              • Organisation
              • System Settings
              • Role
              • Bulk Upload Process
              • Tenant Preference
              • Cassandra Migration Version
              • User Consent
              • Email Template
              • OTP
              • Page Management (LMS Service)
              • Location
              • User Notes
              • Deprecated
          • Elastic Search
          • Redis
        • APIs
          • User Management
            • User Deletion API
            • Ownership Transfer API
          • Organisation Management
          • Location Management
          • Consent Management
          • OTP Services
          • Tenant Configurations
          • Bulk Upload
          • System Settings
          • API Management Service
          • Data Sync
          • Notification APIs
        • Flink Jobs
          • User Cache Updater
          • User Deletion Cleanup Flink Job
          • Ownership Transfer Flink Job
        • Reports
          • Standard Exhaust
            • State Admin Geo report
            • State Admin Report
          • Other Jobs
            • User Cache Indexer Job
            • Delete Users Assets Report
          • Data Products Developer Installation
            • System Requirements
            • Tech Stack
            • Installation Dependencies
            • Local installation of data-products
            • Server setup Guide
            • Installation Configuration
            • Data-product creation guide
            • Troubleshooting a data-product
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
          • System Settings
          • Email Configuration
          • SMS Configuration
          • OTP based SMS Configuration
        • Roles
          • System Roles
          • User Roles
          • RBAC
        • Dependencies
        • Keycloak on Sunbird
        • How to Guide
          • Create Organization
          • Create User
          • Add new user type & location
        • Single Sign-on with Sunbird
        • Caching and Denormalising User Metadata
          • ETLUserCacheUpdaterJob
          • UserCacheUpdaterFlinkJob
        • Unit Tests and Code Coverage
        • FAQs
      • LMS(BATCH) SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
        • Data Models
          • Cassandra
            • Data Dictionary
          • ElasticSearch
          • Redis
        • APIs
        • Flink Jobs
          • Merge User Courses
          • Relation Cache Updater
          • Activity Aggregate Updater
          • Assessment Aggregator
          • Enrolment Reconciliation
          • Collection Certificate Pre-Processor
          • Collection Certificate Generator
        • Reports
          • On-Demand Exhaust
            • Response Exhaust
            • User Info Exhaust
            • Progress Exhaust
          • Other Jobs
            • Collection Summary Job
            • Course Batch Status Updater Job
            • Cassandra Migrator Job
            • Score Metric Migration Job
            • Assessment Score Correction Job
            • Collection Reconciliation Job
            • Course Enrollment Job
            • Course Consumption Job
        • Logs, Telemetry Events
        • Configuration
          • System Configuration
          • Functional Configuration
        • Dependencies
        • Certificate Registry and Credentialing using Sunbird-RC
          • Configuring PublicKey in Sunbird-RC
          • Server Setup guide for Sunbird-RC
          • API Guide For Sunbird-RC
        • Certificate Flow
          • Certificates Creation and Configuration
        • FAQs
      • GROUPS
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
          • Redis
        • APIs
          • Create Group
        • Logs, Telemetry Events
          • UI Telemetry Events
          • Service Telemetry Events
        • Configuration
          • Functional Configurations
        • Design References
        • Dependencies
        • Unit Tests and Code Coverage
      • NOTIFICATION SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Data Dictionary
        • APIs
        • Flink Jobs
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
        • Dependencies
        • Unit Tests and Code Coverage
      • DISCUSSION FORUM
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
            • Nodebb setup
            • Discussion Middleware Setup
            • Discussion Middleware
            • Discussion forum integration with any application
            • Discussion-UI setup along with demo application.
            • Sunbird-lern portal for DF
          • Source Code
        • APIs
          • Category APIs
          • User APIs
          • Post APIs
          • Topic APIs
          • Forum APIs
        • Context Schema
        • Configurations
          • Nodebb Admin panel settings
          • Discussion MW & Nodebb System Config
        • Telemetry Events
        • Dependencies
      • ML SERVICE
        • DATA PIPELINE (Flink Jobs)
          • Program User Info
            • Component Diagram
            • Data Model
            • Folder Structure
          • Ml User Delete
            • Component Diagram
            • Data Model
            • Folder Structure
        • DATA PRODUCTS
          • Program Exhaust
            • Component Diagram
            • Folder Structure
    • Server Installation
    • Dependency setup
    • Deprecation
      • Release-5.4.0
    • Jenkins Jobs
    • Release Calendar
    • Learn More
      • Tech References
      • Telemetry Processing
      • Data Dictionary
        • Sample Data
      • Delete User Functionality
      • Asset Ownership Transfer
    • FAQs
  • Engage
    • Discuss
    • Contribute to Sunbird Lern
    • Raise an Issue
    • Contribution Guidelines
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. USE
  2. Developer Guide
  3. LMS(BATCH) SERVICE
  4. Flink Jobs

Collection Certificate Pre-Processor

PreviousEnrolment ReconciliationNextCollection Certificate Generator

Last updated 1 year ago

Was this helpful?

'collection-cert-pre-processor' job is used to perform the validation of certificate issuance criteria.

Configuration variables:

Variable
Default Value
Purpose

kafka.input.topic

{{env}}.issue.certificate.request

Kafka topic from which messages/events are read to be processed.

kafka.output.failed.topic

{{env}}.issue.certificate.failed

Kafka topic to which message is written when an exception occurs while processing an event.

kafka.output.topic

{{env}}.generate.certificate.request

Kafka topic used to trigger certificate generator job

kafka.groupId

{{env}}-collection-cert-pre-processor-group

Kafka input topic group Id

lms-cassandra.keyspace

sunbird_courses

Cassandra keyspace name

lms-cassandra.course_batch.table

course_batch

Cassandra table used to store batch details of a collection. Batch status, start date , end date , batch enrolment end date, enrolment type (open/invite-only), certificate templates etc are stored in this table.

lms-cassandra.user_enrolments.table

user_enrolments

Cassandra table used to store user enrolment data in a collection of a particular batch. This also holds the consumption progress, enrolment status and issued certificate details

lms-cassandra.assessment_aggregator.table

assessment_aggregator

Cassandra table used to store attempt wise data for each assessment in a collection of a batch by a user. Answer for each question in the assessment is also recorded.

lms-cassandra.user_activity_agg.table

user_activity_agg

Cassandra table used to store user consumption aggregate details of a collection in a batch. Aggregates like the consumption completed content count

redis.database.contentCache.id

5

Content cache is used to get assessment details from redis to compute the score details. This content cache is gettig updated by using content-cache-updater flink job

redis.database.collectionCache.id

0

Collection cache is used to get the course details. This collection cache is gettig updated by using content-service.

cert_domain_url

user_read_api

/private/user/v1/read

API route for fetching user profile details

content_read_api

/content/v4/read

API route for fetching content metadata

service.content.basePath

Content service URL

service.learner.basePath

User-Org service URL

assessment.metrics.supported.contenttype

["SelfAssess"]

variable used to identify assessment type of contents in a Course

enable.suppress.exception

true

Variable used to suppress exception if the content metadata is not available in Redis cache

cloud_storage_base_url

variable to identify the cloud storage base url. Used to replace the base url with variable mentioned in cloud_store_base_path_placeholder while storing to databases

cloud_store_base_path_placeholder

CLOUD_BASE_PATH

relative variable used to replace cloud storage base URLs and stored in database

content_cloud_storage_container

cloud storage container name

cloud_storage_cname_url

variable used to replace 'cloud_store_base_path_placeholder' value with cname or cloud storage url while reading data from database.

Sample event:

{
  "eid": "BE_JOB_REQUEST",
  "ets": 1683609537232,
  "mid": "LP.1683609537232.f04270f4-7d8b-4fc6-9315-dc5c495fb059",
  "actor": {
    "id": "Course Certificate Generator",
    "type": "System"
  },
  "context": {
    "pdata": {
      "ver": "1.0",
      "id": "org.sunbird.platform"
    }
  },
  "object": {
    "id": "013625297374085120185_do_2136252863013273601715",
    "type": "CourseCertificateGeneration"json
  },
  "edata": {
    "userIds": [
      "95c1f00b-c033-4ea2-acf6-e368e2a56bdb"
    ],
    "action": "issue-certificate",
    "iteration": 1,
    "trigger": "auto-issue",
    "batchId": "013625297374085120185",
    "reIssue": false,
    "courseId": "do_2136252863013273601715"
  }
}

Source code:

Logodata-pipeline/lms-jobs/credential-generator/collection-cert-pre-processor at release-5.3.0 · Sunbird-Lern/data-pipelineGitHub