📙
Sunbird Lern
AskDot
  • LEARN
    • Overview
    • Functional Capabilities
    • Technical Architecture
      • Technical Architecture Diagram
      • Tech Stack
    • Dependencies
    • Product Roadmap
  • USE
    • Overview
    • Release Notes
      • Release V 8.0.0 (Ongoing)
      • Release V 7.0.0
      • Release V 5.4.0
      • Release V 5.3.0
      • Release V 5.2.0
      • Release V 5.1.0
      • Release V 5.0.1
      • Release V 5.0.0
      • Release V 4.10.0
      • Release V 4.9.0
      • Release V 4.8.0
      • Release V 4.7.0
    • Developer Guide
      • USER & ORG SERVICE
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Keycloak Local setup
          • Additional Installation Dependencies:
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
              • User
              • Organisation
              • System Settings
              • Role
              • Bulk Upload Process
              • Tenant Preference
              • Cassandra Migration Version
              • User Consent
              • Email Template
              • OTP
              • Page Management (LMS Service)
              • Location
              • User Notes
              • Deprecated
          • Elastic Search
          • Redis
        • APIs
          • User Management
            • User Deletion API
            • Ownership Transfer API
          • Organisation Management
          • Location Management
          • Consent Management
          • OTP Services
          • Tenant Configurations
          • Bulk Upload
          • System Settings
          • API Management Service
          • Data Sync
          • Notification APIs
        • Flink Jobs
          • User Cache Updater
          • User Deletion Cleanup Flink Job
          • Ownership Transfer Flink Job
        • Reports
          • Standard Exhaust
            • State Admin Geo report
            • State Admin Report
          • Other Jobs
            • User Cache Indexer Job
            • Delete Users Assets Report
          • Data Products Developer Installation
            • System Requirements
            • Tech Stack
            • Installation Dependencies
            • Local installation of data-products
            • Server setup Guide
            • Installation Configuration
            • Data-product creation guide
            • Troubleshooting a data-product
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
          • System Settings
          • Email Configuration
          • SMS Configuration
          • OTP based SMS Configuration
        • Roles
          • System Roles
          • User Roles
          • RBAC
        • Dependencies
        • Keycloak on Sunbird
        • How to Guide
          • Create Organization
          • Create User
          • Add new user type & location
        • Single Sign-on with Sunbird
        • Caching and Denormalising User Metadata
          • ETLUserCacheUpdaterJob
          • UserCacheUpdaterFlinkJob
        • Unit Tests and Code Coverage
        • FAQs
      • LMS(BATCH) SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
        • Data Models
          • Cassandra
            • Data Dictionary
          • ElasticSearch
          • Redis
        • APIs
        • Flink Jobs
          • Merge User Courses
          • Relation Cache Updater
          • Activity Aggregate Updater
          • Assessment Aggregator
          • Enrolment Reconciliation
          • Collection Certificate Pre-Processor
          • Collection Certificate Generator
        • Reports
          • On-Demand Exhaust
            • Response Exhaust
            • User Info Exhaust
            • Progress Exhaust
          • Other Jobs
            • Collection Summary Job
            • Course Batch Status Updater Job
            • Cassandra Migrator Job
            • Score Metric Migration Job
            • Assessment Score Correction Job
            • Collection Reconciliation Job
            • Course Enrollment Job
            • Course Consumption Job
        • Logs, Telemetry Events
        • Configuration
          • System Configuration
          • Functional Configuration
        • Dependencies
        • Certificate Registry and Credentialing using Sunbird-RC
          • Configuring PublicKey in Sunbird-RC
          • Server Setup guide for Sunbird-RC
          • API Guide For Sunbird-RC
        • Certificate Flow
          • Certificates Creation and Configuration
        • FAQs
      • GROUPS
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Cassandra
            • Data Dictionary
          • Redis
        • APIs
          • Create Group
        • Logs, Telemetry Events
          • UI Telemetry Events
          • Service Telemetry Events
        • Configuration
          • Functional Configurations
        • Design References
        • Dependencies
        • Unit Tests and Code Coverage
      • NOTIFICATION SERVICE
        • Features/Core Capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
          • Source Code
          • Installation Configuration
        • Data Models
          • Data Dictionary
        • APIs
        • Flink Jobs
        • Logs, Telemetry Events
        • Configuration
          • Functional Configurations
        • Dependencies
        • Unit Tests and Code Coverage
      • DISCUSSION FORUM
        • Features/Core capabilities
        • Architecture
          • Code Flow
        • Developer Installation
          • System Requirements
          • Tech Stack
          • Installation Guide
            • Nodebb setup
            • Discussion Middleware Setup
            • Discussion Middleware
            • Discussion forum integration with any application
            • Discussion-UI setup along with demo application.
            • Sunbird-lern portal for DF
          • Source Code
        • APIs
          • Category APIs
          • User APIs
          • Post APIs
          • Topic APIs
          • Forum APIs
        • Context Schema
        • Configurations
          • Nodebb Admin panel settings
          • Discussion MW & Nodebb System Config
        • Telemetry Events
        • Dependencies
      • ML SERVICE
        • DATA PIPELINE (Flink Jobs)
          • Program User Info
            • Component Diagram
            • Data Model
            • Folder Structure
          • Ml User Delete
            • Component Diagram
            • Data Model
            • Folder Structure
        • DATA PRODUCTS
          • Program Exhaust
            • Component Diagram
            • Folder Structure
    • Server Installation
    • Dependency setup
    • Deprecation
      • Release-5.4.0
    • Jenkins Jobs
    • Release Calendar
    • Learn More
      • Tech References
      • Telemetry Processing
      • Data Dictionary
        • Sample Data
      • Delete User Functionality
      • Asset Ownership Transfer
    • FAQs
  • Engage
    • Discuss
    • Contribute to Sunbird Lern
    • Raise an Issue
    • Contribution Guidelines
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. USE
  2. Developer Guide
  3. USER & ORG SERVICE
  4. Flink Jobs

User Cache Updater

PreviousFlink JobsNextUser Deletion Cleanup Flink Job

Last updated 1 year ago

Was this helpful?

'user-cache-updater-2.0' job is used to generate the user-metadata information which is complied by fetching information from various Cassandra tables and are stored into the Redis cache. This user-metadata information is used by few exhaust reports.

Additional Reading:

,

Design guide:

Configuration variables:

Variable
Default Value
Purpose

kafka.input.topic

{{env}}.telemetry.audit

Kafka topic from which messages/events are read to be processed.

kafka.groupId

{{env}}-user-cache-updater-group

Kafka input topic group Id

redis-meta.database.userstore.id

12

Redis index to which user metadata is to be written to for caching

redis-meta.database.key.expiry.seconds

3600

Redis cache expiry in seconds

user-read.api.url

"/learner/private/user/v1/read"

API Endpoint for fetching User profile details

regd.user.producer.pid

learner-service

used to specify service providing user microservice

user.self.signin.types

["google","self"]

used to specify self sign-in modes available in application

user.validated.types

["sso"]

used to specify sign-in modes where user validation is signed from third party system

user.self.signin.key

"Self-Signed-In"

user.valid.key

"Validated"

user.read.url.fields

"locations,organisations"

used to specify the user metadata properties that are to be cached to Redis

user.read.api.error

["CLIENT_ERROR"]

Sample event:

{
  "eid": "AUDIT",
  "ets": 1573121861118,
  "ver": "3.0",
  "mid": "1573121861118.40f9136b-1cc3-458d-a04a-4459606df",
  "actor": {
    "id": "5609876543234567890987654345678",
    "type": "Request"
  },
  "context": {
    "channel": "01285019302823526477",
    "pdata": {
      "id": "dev.sunbird.portal",
      "pid": "learner-service",
      "ver": "2.5.0"
    },
    "env": "User",
    "did": "user-3",
    "cdata": [
      {
        "id": "google",
        "type": "SignupType"
      }
    ],
    "rollup": {
      "l1": "01285019302823526477"
    }
  },
  "object": {
    "id": "user-1",
    "type": "user"
  },
  "edata": {
    "state": "Update",
    "props": [
      "recoveryEmail",
      "recoveryPhone",
      "userId",
      "id",
      "externalIds",
      "updatedDate",
      "updatedBy"
    ]
  },
  "syncts": 1573121861125,
  "@timestamp": "2019-11-07T10:17:41.125Z",
  "flags": {
    "tv_processed": true,
    "dd_processed": true
  },
  "type": "events",
  "ts": "2019-11-07T10:17:41.118+0000"
}

Source code:

https://project-sunbird.atlassian.net/wiki/spaces/AN/pages/1520074753/Design+Denormalise+User+Metadata
https://lern.sunbird.org/learn/product-and-developer-guide/user-and-org-service/caching-and-denormalising-user-metadata/usercacheupdaterflinkjob
Logodata-pipeline/user-org-jobs/user-cache-updater-2.0 at release-5.3.0 · Sunbird-Lern/data-pipelineGitHub