Component Diagram
Last updated
Last updated
Program Exhaust is a service that generates CSV file containing user details for those who has joined a program.
Database Layer:
PostgreSQL Database (job_request): This is where job requests are stored. These requests include information about which users have requested reports.
Cassandra Database (user_consent): This is where user consent data is stored. It contains information about whether users have agreed to share their PII data.
Cassandra Database (program_enrollment): This is where program enrollment data is stored when a user joins new program.
Redis Cache (user): Contains cached user data that can be quickly retrieved. Data provider
PostgreSQL
job_request
Redis Cache
user
Cassandra
program_enrolment, user_consent
Data Processing Layer: Apache Spark is used to perform transformations, sort columns, eliminate duplicates, and replace unknown values with null. This process enhances data quality, organizes data logically before storing to CSV.
This interaction diagram details the complete process of requesting and generating reports. The user can request a user info exhaust report through SunbirdEd from the program dashboard. Using exhaust APIs, this will map the request to SunbirdLern Which internally calls SunbirdObsrv service. userInfoExhaust data-product will be triggered by a scheduled cron task, which will query cassandra and redis to get data and process it using Spark to transform data and generate the report. The user receives the same report once it has been created.
Format
Nomenclature
Example
Security Levels
CSV zip (Password protected)
program-user-exhaust/<request_id>_<UpdatedDate>.zip
9CD107F48B5AF0D163F8AE8410829674_20230622.zip
L3 - Data encrypted with a user provided encryption key. Generally applicable to non PII data but can contain sensitive information which may not be considered open
Column Label
Column Type
Data Type
Description
User UUID
Static
String
The system generated unique user ID
User Name
Static
String
Name of the user(Firstname and Lastname)
Mobile Number
Static
String
User declared unmasked mobile number
Email ID
Static
String
User declared unmasked email ID
Consent Provided
Static
String
Yes/No. Flag to denote whether user has consented to the data sharing.
Consent Provided Date
Static
Date
Date when the user has consented to share the data
Program Name
Static
String
Program title
Program ID
Static
String
Unique Program Identifier.
State
Static
String
User declared state for self signed up users. If the user is a org validated user then the state as passed from org SSO or derived from sub-org ID.
District
Static
String
User declared district for self signed up users. If the user is a org validated user then the district as passed from org SSO or derived from sub-org ID.
Block
Static
String
Block name mapped to the user’s org/sub-org id
Cluster
Static
String
Cluster name mapped to the user’s org/sub-org id
School Id
Static
String
User declared school id for self signed up users. If the user is a org validated user then the school id as passed from org SSO or derived from sub-org ID
School Name
Static
String
School name mapped to the user’s org/sub-org id
User Type
Static
String
Type of the user
User Sub Type
Static
String
Sub Type of the user
Org Name
Static
String
Name of user org - Custodian for self signed up users and respective org name for org validated users
Following are the fields/columns that will be available in the CSV file only when the user consented for the data sharing.
On user consent
On user consent
On user consent
ef3733df-41e7-4bf1-b67c-091fd0762ee
opa5
7272272777
pao@test.com
false
01/01/23
'Prerak Head Teacher of the Block 19-20'
PGM-Prerak-Head-Teacher-of-the-Block-19-20-Feb2021
Uttar Pradesh
AGRA
ACHHNERA
ZPHS AGALI
9150206302
JHS NAGLA SADLE COMPOSITE
administrator
hm,dikshapreprodcustodian
