Release V 5.0.0

Document Release Version

Project	Release Date	Version
Lern	18 Aug 22	V 5.0.0

Details of Released Tag

Components	Tags
Batch Service	sunbird-course-service : release-5.0.0_RC6 cert-service : release-5.0.0_RC4 certficate-registry : release-5.0.0_RC1 data-pipeline : release-5.0.0_RC6
User&Org Service	sunbird-lms-service : release-5.0.0_RC5
Group Service	groups-service : release-5.0.0_RC3
Discussion Forum Service	discussions-middleware : release-5.0.0_RC2 sunbird-nodebb : release-5.0.0_RC1
Notification Service	sunbird-notification-service : release-5.0.0_RC6
Data Products	data-products : release-5.0.0_RC4

Summary of the Changes

• Migrating component repositories in to Lern organisation in GitHub as a first step in making the installation and setup easy for adopters and contributors

• Increasing code coverage and unit test cases of all the components in Lern as part of stabilising the components

• Refactoring of the provisioning and deployment scripts of Lern BB

• Making SB Lern Cloud agnostic

Details of the Changes

User&Org Service

JIRA ID	Descriptions
SB-30076	UserOrg - increase code coverage and unit test cases
SB-30067	UserOrg - Deployment and Release processes
SB-29842	UserOrg Dataproducts Migration to Lern
SB-29826	Sunbird-apimanager-util migration to Lern
SB-29825	Sunbird-auth migration to Lern
SB-29824	Sunbird-utils - DB Migration to Lern
SB-29823	UserOrg Migration to Lern
SB-29813	OrgSearch to allow partial search and fuzzy Search
LR-103	Making SB Lern Cloud agnostic : Code changes to generalise CSP support in UserOrg
LR-124	Cassandra related changes for supporting multiple Data Centers
LR-232	CSP changes for cert, learner & course service
LR-128	Data Migration related to CSP changes : Report url in Job_request table needs to be updated
LR-125	Data Migration related to CSP changes : Template url in DB Certificate Objects needs to be updated - batch tables
LR-113	Data Migration related to CSP changes : Existing reports need to be migrated
LR-112	Data Migration related to CSP changes : Credential template, context files to be stored in to new CSP
LR-111	Data Migration related to CSP changes : Old Certificates in Azure needs to be migrated
LR-110	Data Migration related to CSP changes : Template url in DB Certificate Objects needs to be updated - RC tables and ES
LR-109	Data Migration related to CSP changes:Exisisting Certificate templates needs to be migrated.

Configurations:

Sunbird-lms-service:

isMultiDCEnabled={{cassandra_isMultiDCEnabled}}
sunbird_cassandra_consistency_level={{sunbird_cassandra_consistency_level}}
sunbird_user_cert_kafka_topic={{kafka_topic_lms_user_account}}
sunbird_cloud_service_provider={{cloud_service_provider}}
sunbird_account_name={{sunbird_public_storage_account_name}}
sunbird_account_key={{sunbird_public_storage_account_key}}

Batch Service

Discussion Forum

Group Service

Notification Service

Detailed Information is present in the JIRA list.

Configurations:

1. Nodebb upstream branch: v1.18.6

2. Jenkins build, deploy and upload related changes for Flink jobs are present in below link:

https://github.com/Sunbird-Lern/data-pipeline/tree/release-5.0.0/kubernetes/pipelinesgithub.com

3. There is a new variable added in devops repo to configure the bb name for kafka topics of flink jobs. This variable should be appended after the env name.

https://github.com/Sunbird-Lern/data-pipeline/blob/release-5.0.0/ansible/inventory/env/group_vars/all.yml#L10github.com

List of Lern Flink jobs:

collection-cert-pre-processor
collection-certificate-generator
activity-aggregate-updater
relation-cache-updater
merge-user-courses
assessment-aggregator
enrolment-reconciliation
notification-job

env variable changes are listed below: 
kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml

###  Merge User Courses Job related Vars
merge_user_courses_consumer_parallelism: 1
merge_user_courses_parallelism: 1
merge_user_courses_course_batch_parallelism: 1
merge_user_courses_course_date_format: "yyyy-MM-dd HH:mm:ss:SSSZ"

###  Notification Job related Vars
notification_job_consumer_parallelism: 1
notification_job_parallelism: 1

### assessment-aggregator related vars
assessaggregator_parallelism: 1
assessaggregator_consumer_parallelism: 1
assessaggregator_downstream_parallelism: 1
assessaggregator_scoreaggregator_parallelism: 1
middleware_cassandra_courses_keyspace: sunbird_courses
middleware_cassandra_assessment_aggregator_table: assessment_aggregator
middleware_cassandra_assessment_question_type : question
middleware_cassandra_user_enrolments_table: user_enrolments
middleware_cassandra_user_activity_agg_table: user_activity_agg
content_read_api_host: "http://dev.sunbirded.org"
content_read_api_endpoint: "/api/content/v1/read/"

merge-user-courses:
    job_class_name: 'org.sunbird.job.merge.user.courses.task.MergeUserCoursesStreamTask'
    replica: 1
    jobmanager_memory: 1024m
    taskmanager_memory: 1024m
    taskslots: 1
    cpu_requests: 0.3
  assessment-aggregator:
    job_class_name: 'org.sunbird.dp.assessment.task.AssessmentAggregatorStreamTask'
    replica: 1
    jobmanager_memory: 1024m
    taskmanager_memory: 1024m
    taskmanager_process_memory: 1700m
    jobmanager_process_memory: 1600m
    taskslots: 1
    cpu_requests: 0.3
    scale_enabled: false
  notification-job:
    job_class_name: 'org.sunbird.job.notification.task.NotificationStreamTask'
    replica: 1
    jobmanager_memory: 1024m
    taskmanager_memory: 1024m
    taskslots: 1
    cpu_requests: 0.3

Stop the existing Samza jobs - (merge-user-courses and notification-job)
Stop the assessment-aggregator job from sunbird-data-pipeline and remove it from the corresponding Jenkins job as well
All these 3 jobs will be running from LERN repo now

4. Jenkins build, deploy and upload related changes for data products are in below link:

https://github.com/Sunbird-Lern/data-products/tree/release-5.0.0/pipelinesgithub.com

LERN data-products list
  exhaust / progressexhaustjob
  exhaust / responseexhaust
  exhaust / userinfoexhaust
  job / course consumption
  job / course enrolment
  job / stateadminreport
  job / stateadmingeoreport
  job / collectionsummaryjobv2
  audit / collection reconcilation
  audit / coursebatch status updater
  updater / Cassandramigrator

5. Jenkins build, deploy and upload related changes for microservices are in below link:

https://github.com/project-sunbird/sunbird-devops/tree/learn-bbgithub.com

Devops config changes:

File Path: 
ansible/inventory/env/group_vars/all.yml
### Release-5.0.0 cloud service provider changes for supporting multiple providers ###
### cloud_service_provider value should be either (azure, aws, gcloud) as per cloud sdk dependency ###
cloud_service_provider: "azure"

## modified consistency levels from quorum to local_quorum for multiple data centers support
File Path: 
ansible/roles/stack-sunbird/templates/sunbird_cert-registry-service.env
Changes:
sunbird_cassandra_consistency_level=local_quorum

File Path: 
ansible/roles/stack-sunbird/templates/sunbird_groups-service.env
Changes:
sunbird_cassandra_consistency_level=local_quorum

File Path: 
ansible/roles/stack-sunbird/templates/sunbird_learner-service.env
Changes:
sunbird_cassandra_consistency_level=local_quorum
sunbird_user_cert_kafka_topic={{env_name}}{{bb}}.lms.user.account.merge

File Path: 
ansible/roles/stack-sunbird/templates/sunbird_lms-service.env
Changes:
sunbird_cassandra_consistency_level=local_quorum

File Path: 
ansible/roles/stack-sunbird/templates/sunbird_notification-service.env
Changes:
sunbird_cassandra_consistency_level=local_quorum
sunbird_notification_kafka_topic={{env_name}}{{bb}}.lms.notification

File Path:
kubernetes/helm_charts/sunbird-RC/registry/schemas/TrainingCertificate.json
Changes:
"credentialTemplate": "https://{{upstream_url}}/schema/credential_template.json"

File Path:
utils/sunbird-RC/schema/credential_template.json
Changes:
"https://{{upstream_url}}/schema/v1_context.json",
 "https://{{upstream_url}}/schema/sunbird_context.json"

Design Documentation:

https://project-sunbird.atlassian.net/wiki/spaces/UM/pages/3077767171/Sunbird-Lern+Code+Flink+Jobs+and+Data+Productsproject-sunbird.atlassian.net

Sunbird-Lern Code, Flink Jobs and Data Products

https://project-sunbird.atlassian.net/wiki/spaces/UM/pages/3149922315/Lern+BB+repositoriesproject-sunbird.atlassian.net

Lern BB repositories

https://project-sunbird.atlassian.net/wiki/spaces/UM/pages/3156869136/Sunbird-Lern+existing+jenkins+jobsproject-sunbird.atlassian.net

Sunbird-Lern existing jenkins jobs