Respondent Training

Data Pipeline Respondent Overview

This page includes resources designed to support all data respondents, regardless of collection, with data pipeline. New data respondents are encouraged to review these resources before beginning a specific data collection.

General Information:

As a data respondent, it is important to note the expectations and requirements of this role.

First, determine which collection(s) fall within your responsibilities. Some data respondents may only have one collection to oversee, while others may be responsible for multiple collections. Either way, it is important to note the collection timeline(s) as listed on the specific collection website(s) and plan your schedule accordingly.

Next, ensure you have the appropriate skillset for this role and request additional training if needed. The skills deemed necessary for this role include:

Proficiency in Microsoft Excel
Proficiency in the LEA’s Student Information System and/or Staff Information System as required to create, produce or extract data files required for the assigned data collection(s).
Understanding of the privacy rules and legislation such as FERPA and Privacy Act of 1974.

Also attend all collection training and learning opportunities offered by CDE collection leads to best assist you with the process. Along with attending the trainings offered, review all website documentation provided and meet all expectations listed within the collection timeline.

Identity Management Role(s)

To complete any Data Pipeline collection, you must be provided with the necessary role(s) within the Identity Management system (IdM) by your Local Access Manager (LAM). All roles are expected to meet the collection timeline requirements and provide accurate and complete information.

If you are assigned the LEA Viewer role, you are responsible for reviewing the data reported to CDE and ensuring that the data is accurate. You cannot make updates to the data within the system, but you can view the data.
If you are assigned the LEA User role within IdM, you are responsible for creating the data file(s), uploading the file(s) into Data Pipeline, reviewing error reports to correct the data and repeating the process as needed until all errors are resolved, warnings are reviewed and data reported is determined to be complete and accurate.
If you are assigned the LEA Approver role within IdM, you not only have the same responsibilities as the LEA User role, but you are also responsible for marking the data file as finalized in the Data Pipeline system.

Data Collection Overview

There are four types of data collections:

Year-Round are files that are open throughout the school year and can be updated as needed. Our year-round collections consist of the Record Integration Tracking system (RITS), Educator Identification System (EDIS) and the Directory. RITS creates and updates the state assigned student identifiers (SASIDs); EDIS creates and updates the educator identifiers (EDIDs), and the Directory captures the Local Education Agency's (LEA's) general information and school information. Year-round files do not require the finalize submission process.
Interchanges are files uploaded by LEAs, such as districts, BOCES or administrative units. These files are placeholders of the data, which are fluid and updated by the LEAs often. Interchange files are the source information for the snapshots. There are generic business rules in place at the interchange level to ensure the data meets basic requirements such as having a valid code, not being blank, or requiring it to be reported as zero-filled. Interchange files do not require the finalize submission process.
Snapshots pull data from interchange files for reporting purposes. LEAs then create a snapshot that extracts the interchange data, which is a point in time and used for reports after the LEA approves the data. There are business rules in place in the snapshot that are more specific and can reference data from multiple interchange files. Snapshot collections require the finalize submission process.
Periodic collections are typically one-time per year collections that do not reference any interchange files. There are also periodic collections that have the possibility of referring to data reported in interchange file(s); such as the READ spring assessments. Periodic collections require the finalize submission process.

Data Pipeline Website Navigation

All collections have resources posted on that collection’s website. These resources include collection timelines, file layouts, business rules, file upload templates, and training material.

Upon navigating to the Data Pipeline Home Website, you will notice a menu on the right-hand side of the screen. Data collections are organized by the type of collection: Interchanges, Snapshots, Year-Round Collections and Periodic Collections. Once a collection type and specific collection is selected, the webpage has a similar layout with the following sections: Overview, Deadlines, File Layout and Definitions, Business Rules and Additional Resources.

Data Pipeline Reporting Process

The overall reporting process can vary by collection type and collection, so it is best to review the resources on the specific collection website for the full list of steps and overall process. However, there are a few processes that are the same for all collections:

Data File Upload:

When a file is required to be uploaded into Data Pipeline, the basic steps are the same regardless of the collection and file layout.

Year Round: File(s) uploaded as needed for updates to the data.
Interchanges: File(s) are uploaded into Data Pipeline on a regular basis.
Snapshots: This is created in Data Pipeline, using data uploaded as part of the Interchange process.
Periodic: File(s) are uploaded in Data Pipeline and finalized within the given timeline.

Reviewing Errors and Validating Data

As files are uploaded into the Data Pipeline, it is imperative to ensure the data reported is accurate and complete. To help with this process the system has built in business rules for each collection which assist in checking for inconsistencies within the data. The system also has data reports available for the LEA respondent(s) to review and check for accuracy as not all data inaccuracies can be discovered through business rules. Both the error reports and validation reports are available within the Data Pipeline system to use throughout the collection cycle.

Finalize Submission Process: Submit to CDE

Periodic files and Snapshots must be finalized and submitted to CDE. Each will have a deadline for this to be completed on or before. Some will also include a Sign Off Form that needs to be downloaded and signed and sent via email to the collection lead.

Additional Tools/Resources

Troubleshooting in Data Pipeline

Files must have specific data fields in a specific order and format to upload successfully into Data Pipeline. Snapshots require certain conditions are met to successfully process or include records in the snapshot. Sometimes there is an issue with one of these processes that requires additional troubleshooting. While CDE staff are available to assist, there are some general tips and steps that can be taken initially by the data respondent.

Syncplicity

Syncplicity is a tool utilized for sharing personally identifiable information securely between CDE staff and data respondents

Access Directions

Login to the Syncplicity website.
- Your username will be the email address listed in IdM.
- Your password is the password you created when you set up your Syncplicity account the first time a file was shared with you.
Navigate to the folder for the desired collection.
- Contact the specific Collection Lead if you do not have access to the necessary folder. Remember, each collection has an associated email address that can be found on the Contact Us webpage.
Select the upload button and select a file to upload from your computer.
Email the collection lead notifying them a document has been uploaded to Syncplicity. The collection lead will review your request and respond.

Contact

CDE staff are available to help with the data reporting process. Visit the Data Pipeline Contact page for a list of collection email addresses.