Asset Ownership Transfer

Overview

The user deletion requirement in Lern has originated from the below requirement.

PRD: [PRD] Asset Ownership Transfer

What is changing?

The user can request for deletion of their account in Sunbird, this means two primary actions to happen.

  1. The user's Personal Identifiable Information (PII) needs to be removed

  2. The assets created by this user (such as questions, question sets, content, etc.) need to be transferred to an identified user.

Release Tags:

ComponentsBuild Jenkins JobBuild TagDeploy Jenkins JobDeploy TagComment

OnboardAPIs

NA

NA

Deploy/Kubernetes/OnboardAPIs

To onboard the delete user API

Cassandra Migration

Build/Core/Cassandra

Deploy/Kubernetes/Cassandra

Select the sunbird in cassandra_keyspace_to_migrate while deploying

script_repo_branch_or_tag: release-7.0.0_RC3

UserOrg Service

Build/Core/UserOrg

Deploy/Kubernetes/UserOrg

Kafka Setup

NA

NA

Deploy/Lern/KafkaSetup

DataPipeline

Build/Lern/LernFlinkJobs

Deploy/Lern/LernFlinkJobs

Define the core_vault_sunbird_keycloak_user_federation_provider_id in Lern inventory secret. Add user-deletion-cleanup and ml-user-delete into job list and deploy it.

Data Product

Build/Lern/LernDataProducts

Deploy/Lern/LernDataProducts

Configurations

Code changes

https://github.com/Sunbird-Lern/data-pipeline/tree/release-7.0.0/user-org-jobs/user-ownership-transfer

  • For making the ownership transfer functionality work through the flink job, the adapter needs to trigger the Kafka event, For more details about that refer to the below link,

  • For more details on the user deletion cleanup flink job, please visit

This flink job was introduced as part of the 7.0.0 release, if any adopter wants to use this feature before the release have to configure and use this flink job.

  1. Added below partition related settings and replication_factor related settings in ansible/roles/setup-lern-kafka/defaults/main.yml file of data pipeline repository.

  - name: user.ownership.transfer
    num_of_partitions: 2
    replication_factor: 1
    
    - name: user.ownership.transfer
    retention_time: 172800000
    replication_factor: 1
  1. Added below ownership transfer flinkjob related configuration in kubernetes/helm_charts/datapipeline_jobs/values.j2 file of data pipeline repository.

user-ownership-transfer:
  user-ownership-transfer: |+
    include file("/data/flink/conf/base-config.conf")
    kafka {
      input.topic = ${job.env}".user.ownership.transfer"
      groupId = ${job.env}"-user-ownership-transfer-group"
    }
    task {
      user.ownership.transfer.parallelism = {{ user_ownership_transfer_job_parallelism }}
    }
    lms-cassandra {
      course_batch.table = "{{ middleware_course_batch_table }}"
      keyspace = "{{ middleware_course_keyspace }}"
    }
    service {
         lms {
             basePath = "{{ lms_service_base_url }}"
         }
         userorg {
             basePath = "{{ userorg_service_base_url }}"
         }
    }
    user_read_api = "/private/user/v1/read/"
    batch_search_api = "/v1/course/batch/search"
    threshold.batch.write.size = {{ user_ownership_transfer_batch_write_size }}

  flink-conf: |+
    jobmanager.memory.flink.size: {{ flink_job_names['user-ownership-transfer'].jobmanager_memory }}
    taskmanager.memory.flink.size: {{ flink_job_names['user-ownership-transfer'].taskmanager_memory }}
    taskmanager.numberOfTaskSlots: {{ flink_job_names['user-ownership-transfer'].taskslots }}
    parallelism.default: 1
    jobmanager.execution.failover-strategy: region
    taskmanager.memory.network.fraction: 0.1
  1. Added below ownership transfer flinkjob related configuration in kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml of data pipeline repository

user_ownership_transfer_job_parallelism: 1
user_ownership_transfer_batch_write_size: 10


  user-ownership-transfer:
    job_class_name: 'org.sunbird.job.ownershiptransfer.task.UserOwnershipTransferStreamTask'
    replica: 1
    jobmanager_memory: 1024m
    taskmanager_memory: 1024m
    taskslots: 1
    cpu_requests: 0.3

Jenkins Job Details For the deployment of the above flink job:

user-ownership-transfer

Adoption of the feature Through API

This API was introduced as part of the 8.0.0 release and can be used directly from this release. Before this release adopters have to use either the flink job or extend this logic

Ownership Transfer API

  1. Added the below configuration in the user org service application.conf file of user org service

       "/user_ownership_transfer_actor"
             {
               router = smallest-mailbox-pool
               nr-of-instances = 5
               dispatcher = brr-usr-dispatcher
             }
           "/user_ownership_transfer_actor/*"
              {
                dispatcher = akka.actor.brr-usr-dispatcher
              }
  1. Added the below property in external resource.properties related to ownership transfer kafka topic in user org service

user-ownership-transfer-topic={{env_name}}.user.ownership.transfer
  1. Added the below configuration in ansible/roles/kong-api/defaults/main.yml for ownership transfer API in sunbird devops repository.

- name: ownershipTransfer
  uris: "{{ user_service_prefix }}/v1/ownership/transfer"
  upstream_url: "{{ userorg_service_url }}/v1/user/ownership/transfer"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
    - userUpdate
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"
  - name: opa-checks
    config.required: false
    config.enabled: false
  1. Added the user ownership transfer topic to the userorgservice.env file in Sunbird DevOps repository.

user-ownership-transfer-topic={{env_name}}.user.ownership.transfer

Last updated