Delete User Functionality
Overview
The user deletion requirement in Lern has originated from the below requirement.
PRD: [PRD] Delete Account functionality
BE Design Lern - [Design] Delete Account Functionality
FE Design Lern - [Design] [Front-end]Delete User Functionality
What is changing?
The user can request for deletion of their account in Sunbird, this means two primary actions to happen.
The user's Personal Identifiable Information (PII) needs to be removed
The assets created by this user (such as questions, question sets, content, etc.) need to be transferred to an identified user.
Release Tags:
Cassandra Migration
Build/Core/Cassandra
Deploy/Kubernetes/Cassandra
Select the sunbird in cassandra_keyspace_to_migrate while deploying
script_repo_branch_or_tag: release-7.0.0_RC3
DataPipeline
Build/Lern/LernFlinkJobs
Deploy/Lern/LernFlinkJobs
Define the core_vault_sunbird_keycloak_user_federation_provider_id in Lern inventory secret. Add user-deletion-cleanup and ml-user-delete into job list and deploy it.
Data Product
Build/Lern/LernDataProducts
Deploy/Lern/LernDataProducts
Adoption of the feature Through Flink Job
Code changes
https://github.com/Sunbird-Lern/data-pipeline/tree/release-7.0.0/user-org-jobs/user-deletion-cleanup
For making the delete user functionality work through the flink job, the adapter needs to trigger the kafka event, For more details about that refer the below link,
For more details on the user deletion cleanup flink job, please visit
User delete Flink job
This flink job was introduced as part of the 7.0.0 release, if any adopter wants to use this feature before the release have to configure and use this flink job.
Added below partition related settings and replication_factor related settings in ansible/roles/setup-lern-kafka/defaults/main.yml file of data pipeline repository.
- name: delete.user
retention_time: 172800000
replication_factor: 1
- name: delete.user
num_of_partitions: 2
replication_factor: 1Added below user delete flinkjob related configuration in kubernetes/helm_charts/datapipeline_jobs/values.j2 file of data pipeline repository.
user-deletion-cleanup:
user-deletion-cleanup: |+
include file("/data/flink/conf/base-config.conf")
kafka {
input.topic = ${job.env}".delete.user"
groupId = ${job.env}"-delete-user-group"
}
task {
user.deletion.cleanup.parallelism = {{ user_deletion_cleanup_job_parallelism }}
}
service {
lms {
basePath = "{{ lms_service_base_url }}"
}
userorg {
basePath = "{{ userorg_service_base_url }}"
}
}
sunbird_keycloak_user_federation_provider_id={{core_vault_sunbird_keycloak_user_federation_provider_id}}
user_read_api = "/user/v5/read/"
batch_search_api = "/course/v1/batch/list"
user {
keyspace = "sunbird"
lookup.table = "user_lookup"
table = "user"
externalIdentity.table = "usr_external_identity"
org.table = "user_organisation"
}
flink-conf: |+
jobmanager.memory.flink.size: {{ flink_job_names['user-deletion-cleanup'].jobmanager_memory }}
taskmanager.memory.flink.size: {{ flink_job_names['user-deletion-cleanup'].taskmanager_memory }}
taskmanager.numberOfTaskSlots: {{ flink_job_names['user-deletion-cleanup'].taskslots }}
parallelism.default: 1
jobmanager.execution.failover-strategy: region
taskmanager.memory.network.fraction: 0.1Added below user delete flinkjob related configuration in kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml of data pipeline repository
user_deletion_cleanup_job_consumer_parallelism: 1
user_deletion_cleanup_job_parallelism: 1
user-deletion-cleanup:
job_class_name: 'org.sunbird.job.deletioncleanup.task.UserDeletionCleanupStreamTask'
replica: 1
jobmanager_memory: 1024m
taskmanager_memory: 1024m
taskslots: 1
cpu_requests: 0.3Ownership transfer delete user assets report:
This report was introduced as part of the 8.0.0 release, if any adopter wants to use this feature before the release have to use the above flink job
Added below ownership transfer delete user assets report related configuration in ansible/roles/lern-data-products-deploy/templates/lern-model-config.j2 of data products repository.
if [ ! -z "$3" ]; then specificUserId=$3; configuredUserId=$3; fi #used as specfic user id in usercacheindxerjob,used as configured user id in deleteusersassetsreportjob
if [ ! -z "$5" ]; then refreshUserData=$5; configuredChannel=$5;fi #used as refreshUserData flag in usercacheindxerjob,used as configured channel flag in deleteusersassetsreportjob
"delete-users-assets-report-job")
echo '{"search":{"type":"none"},"model":"org.sunbird.userorg.job.report.DeletedUsersAssetsReportJob","modelParams":{"store":"{{dp_object_store_type}}","storageKeyConfig":"storage.key.config","storageSecretConfig":"storage.secret.config","storageContainer":"{{reports_container}}","storageEndpoint":"{{dp_storage_endpoint_config}}","configuredUserId":'$configuredUserId',"configuredOrganisationId":[],"configuredChannel":'$configuredChannel', "sparkCassandraConnectionHost":"{{ core_cassandra_host }}"}}'
;;Added below ownership transfer delete user assets report related configuration in ansible/roles/lern-data-products-deploy/templates/lern-run-job.j2 of data products repository
"delete-users-assets-report-job") echo 'org.sunbird.userorg.job.report.DeletedUsersAssetsReportJob'
;;Added below ownership transfer delete user assets report related configuration in lern-data-products/src/main/resources/application.conf of data products repository.
content.search.url="http://10.246.3.3/search/v3/search"
course.batch.search.url="http://10.246.3.3/lms/v1/course/batch/search"
sunbird_instance_name="Sunbird"
delete.user.cloud.objectKey="reports/"To learn more about Delete user Assets report visit here Jenkins Job Details For the deployment of the above flink job:
To trigger the job: Deploy/Lern/LernAnalyticsReplayJobs
job_id: Set to delete-users-assets-report-job.
specific_user_id / user_id_filter: Set to an empty string "". if you want to get the report for all deleted users you can provide multiple user IDs separated by commas within double quotes, like "a, b"
refresh_user_data / channel filter: Set to an empty string "". If you want to get the report for all channels or you can provide multiple channel IDs separated by commas within double quotes, like "a, b" then trigger
Adoption of the feature Through API
This API was introduced as part of the 7.0.0 release and can be used directly from this release. Before to this release adopters have to use either the flink job or extend this logic
Delete user API
Added the below configuration in the user org service application.conf file of user org service
"/user_deletion_background_job_actor"
{
router = smallest-mailbox-pool
nr-of-instances = 5
dispatcher = brr-usr-dispatcher
}
"/user_deletion_background_job_actor/*"
{
dispatcher = akka.actor.brr-usr-dispatcher
}Added the below property in external resource.properties related to user delete kafka topic in user org service
user-deletion-roles=public
user-deletion-broadcast-topic={{env_name}}.delete.userAdded the below configuration in ansible/roles/kong-api/defaults/main.yml for owuser delete API in sunbird devops repository.
- name: deleteUser
uris: "{{ user_service_prefix }}/v1/delete"
upstream_url: "{{ userorg_service_url }}/v1/user/delete"
strip_uri: true
plugins:
- name: jwt
- name: cors
- "{{ statsd_pulgin }}"
- name: acl
config.whitelist:
- userUpdate
- name: rate-limiting
config.policy: local
config.hour: "{{ medium_rate_limit_per_hour }}"
config.limit_by: credential
- name: request-size-limiting
config.allowed_payload_size: "{{ medium_request_size_limit }}"
- name: opa-checks
config.required: true
config.enabled: trueAdded the user delete topic to the userorgservice.env file in Sunbird DevOps repository.
user-deletion-roles=public
user-deletion-broadcast-topic={{env_name}}.delete.userLast updated
Was this helpful?