FAQs on SLDS

---

What is SLDS?

The State Longitudinal Data System Grant Program (SLDS) refers to the series of federal grants for states to develop or improve their data systems needed to effectively measure the success of educational programs. Longitudinal data is data that is collected over a period of time, typically over the course of multiple years. It allows CDE to evaluate the effectiveness of the many educational programs into which significant taxpayer funding is invested.

"Longitudinal analysis" is the measurement of a particular cohort of students from Kindergarten through high school and even college or career. For example, longitudinal analysis could tell communities and taxpayers if students from a particular program or elementary school eventually graduated from high school and college. It could also inform legislative decisions or policy making related to the effectiveness of specific programs during a student's K-12 career, by comparing and contrasting the aggregated results from various programs.

The SLDS grant provided funding for CDE to build the technology that would enable the department to better synthesize and evaluate data securely, while developing consistent and repeatable processes to reinforce CDE policies around data security and privacy. Please see more information on the SLDS Grant Program at http://nces.ed.gov/programs/slds/.

What is RISE (Relevant Information to Strengthen Education)?

RISE stands for Relevant Information to Strengthen Education, and it is CDE's name for the SLDO system. It includes the systems in use and the processes and procedures supporting those systems.

Where does RISE funding come from and what terms/conditions are associated with that funding?

SLDS grants are funded by the USDOE, and some SLDS grants were part of the 2009 American Recovery and Reinvestment Act (ARRA ) program. While a part of ARRA, there were additional reporting requirements imposed on states that received funding, but required no student data to be reported.

Initial SLDS grants were targeted more toward state infrastructure, enabling states to bring up their level of technology infrastructure to support current and future needs around educational program support and analyses necessary to demonstrate Return on Investment decisions and educational program outcome measurement.

Colorado used SLDS funds to build the Data Pipeline system, which is the replacement for the previous collection system for transmitting data between school districts and CDE. The previous system had been in place for almost 20 years. Other uses Colorado made of this funding were enhancing the data warehouse, adding and updating other infrastructure components, and building and implementing the security system CDE currently uses for district logins.

As SLDS initiatives have progressed, the requirements for use of USDOE funding have also evolved. For example, the most recent SLDS grant application was required to focus on the actual USE of data, and the states are required to use this round of funding only for enhancing the use of data, improving the user friendliness of the system, enhancing the process of requesting and receiving longitudinal analyses results, and any other effort that helps to make longitudinal analysis more meaningful for educational leaders and policy makers.

What data does RISE use?

The department's RISE system utilizes data already collected by CDE, via collections required by statute to measure the success of educational programs. Data is not collected by CDE solely for the purpose of RISE.

Who uses RISE?

Currently, only CDE's educational partners within the state, e.g., Department of Higher Education, Colorado Department of Human Services, etc. are able to use the RISE system. It is still in its infancy, so only a few analyses have been run through the system. As we formalize and define the process, and communicate it to all stakeholders for better understanding of the potential and the rules governing the process, its use will likely expand. Note that although members of other agencies may request data for educational research purposes, only CDE staff has access to the system to actually run the approved scenarios.

Who requests RISE data and how is it used?

RISE analysis is primarily requested by other state agencies. For example, the Colorado Department of Human Services requests data for the purpose of measuring the success of such educational programs as Colorado Child Care Assistance Program and Early Intervention, Part C IDEA. Other requests include state universities working together with non-profit organizations to research outcomes of educational programs. Some non-profit organizations perform longitudinal analysis on CDE's behalf, including the University of Northern Colorado.

For-profit commercial vendors do not request longitudinal data from CDE, nor would CDE provide it to them if they did. The data sharing agreements outlining all of the circumstances in which CDE has shared data with other entities for research purposes are all posted on CDE‘s website.

How are data requests processed within CDE?

All longitudinal analysis requests are proposed and reviewed individually, and evaluated based on the merits of each proposal. If PII is required to conduct the analysis, CDE's Research and Data Request process is followed to ensure the proposed research is beneficial to Colorado's education system and that the methodology to be followed conforms to required standards and controls.

In addition to the Research and Data Request process, all requestors must first establish the value of the query, specify the data needed and demonstrate the necessary adherence to security and privacy requirements. The requestor must explicitly document what data is needed, how it will be used, who will be authorized users, and when and how the data will be destroyed afterwards.

If the analysis is requested by another Colorado state agency, this documentation is attached to an agency-to-agency data sharing agreement which spells out very strict privacy and security requirements and regulations to be followed according to FERPA and other regulations. The documentation is all reviewed by key program personnel, privacy and security personnel, and other subject matter experts prior to approval.

All agreements are reviewed by CDE's senior leadership and Chief Privacy Officer on an annual basis and assessed to make sure the needs are still valid. The processes and procedures used for approving data requests for longitudinal analysis are the same as those used for any other data requests, and are described on CDE's privacy and security website.

How does RISE work?

While not yet operational, the RISE system is designed to pull data from a variety of sources, match the data to the right students and then eliminate any personally identifying information before any analysis is done. A cross agency governance committee has been monitoring progress of the RISE implementation efforts for the past three years.

Work is currently underway to formalize the process for requesting and reviewing analysis requests, so that going forward, it's clear to all what process is in place, how requests are prioritized and moved through the approval steps, and most importantly, what criteria is used to determine whether a particular analysis request adds enough value to the educational system to approve.

Efforts also continue to enhance the checks and balances necessary to ensure privacy and transparency in this type of cross-agency data analysis.More specifically, the system would work like this:

1. The interface will allow data from other sources, such as college completion data from Department of Higher Education, to be imported into temporary storage and matched against the CDE data.
2. Once the data has been matched to the right students, additional data and attributes can be brought in on a student. For example, comparing results of college level courses vs. program participation in high schools could indicate which programs most successfully prepare students for college. Adding additional attributes such as participation in STEM programs could also demonstrate a more complete picture of program effectiveness.
3. After the matching process is complete, any remaining work on that use case is typically done on aggregated or de-identified data.
4. Results are provided back to the requestor, as defined in the associated data sharing agreements, following whatever process was outlined and approved during the initial review and approval process. No further disclosure is allowed on any individualized data level once the matching process is complete.

Note that this is a very new process and only one use case is currently operational, which has to do with high school students who find their way to the Corrections system. The purpose of this use case is to identify those K-12 students who become incarcerated, receive education there, and then are released.

These connections can help to ensure that students who return to a K-12 environment can receive credit for the education they received while incarcerated. Without these connections, the student can go back to a K-12 environment without getting credit for the education they received while incarcerated and even worse, not make their way back into the K-12 records and are essentially "lost". The data shared helps give students the credit they deserve, and also helps districts identify students who graduated from their system.

The Department continues to work on formalizing the processes already in place, so they are understood and repeatable.

Is RISE run by a vendor?

The RISE system is run within CDE, by CDE employees who are authorized and who have received required training in data security and privacy protocols. Although some vendors were engaged to build components of the system when first developed, they are no longer involved and have no access to any CDE systems, including the RISE system.

When the 2009 SLDS grant was issued to Colorado, there were plans for the Governor's Office of Information Technology to build and house the RISE system. A variety of technical challenges were encountered with that approach, so prior to the end of that grant period in 2014, the RISE system was brought into CDE where it remains today.

Is RISE data in the cloud?

As already mentioned, CDE houses the RISE data system in its own data center, and does not use any cloud vendor to host student data. No vendors have access to RISE data, nor does any other agency or other entity have access to the RISE system.

How is data transmitted between agencies?

The method of transmission, format of data and individuals responsible for receiving analysis results are all defined during the approval process up front. Once approved and after the longitudinal data analyses are conducted by CDE with the RISE system, the approved process is followed and results are securely sent to the requestor.

Is CDE collecting specific data for use by RISE?

No additional data is collected for the purpose of longitudinal analyses. CDE uses only the data that is already housed in its Data Warehouse -- data collections mandated by state statute or federal law.

How would someone get access to data within the RISE system?

There are two reasons that someone may have access to data in the RISE system. The first is in the case of CDE employees who are working to either maintain the system or add new functionality to it. These individuals may be exposed to the data during testing or validation activities. Only authorized users within CDE have access to data within the RISE system. All personnel in this situation have received background checks, training in the use of confidential data, training in FERPA and also in CDE's privacy policies and protocols.

Occasionally there are vendor experts working on behalf of CDE who have access to the system itself, but care is taken not to expose them to actual data unless they also have received the training identified above. Any vendor that is allowed access to data is required to show the CDE how the respective data has been purged or deleted at the end of their respective engagements.

The second way a person could see RISE data is if it is formally "released" to them. In this situation, no vendors would ever be given RISE data. We only share data with other agencies, persons or entities and for limited reasons. When that occurs, we follow the review and approval processes mentioned above.

What are RISE reporting requirements?

Reporting requirements for SLDS grants are much like they would be for any large project initiative. Reports to USDOE require status reports, progress on each grant outcome as laid out in application, metrics to certify accomplishment of major milestones, and other similar information. Budget, schedule and quality are also addressed as part of the typical project management methodology. At no time is any actual student data, aggregated or otherwise, required to be sent to USDOE in return for SLDS funding.

Does RISE contain all the data included in the Common Education Data Standards (CEDS) model?

CEDS is a standard developed by the USDOE in collaboration with other partners that establishes common definitions and naming conventions for data elements used across the education community. A standard is helpful so when or if a state chooses to use a particular data element, common nomenclature and definitions may be used to enhance re-usability of systems developed. This increases efficiency and enhances collaboration across the different agencies or states that may be building similar products.

As an example, it is much like the standard road signs we use across the country. If each state had a different shape and color for the STOP sign, it would not be as easily recognizable. The number of STOP signs a state has doesn't matter, but every driver knows what it means to see the standard sign, and doesn't have to accommodate different ways to be told to stop. The CEDS standard is very similar but the common nomenclature is applied to technical development of data systems.

Colorado's RISE system is based on some of the CEDS standard, but our system was initially designed before CEDS was completed. Although RISE uses many of the data definitions within CEDS, RISE does not contain all the possible data identified within the CEDS standard. If a state follows this standard, it does not mean the state is sharing the data defined by CEDS with another state following the same standard. It just means that each state is using similar nomenclature and methods when discussing systems and data, and that makes more effective use of the taxpayers' investments.

Is RISE data shared with other states because it is funded with federal grant dollars?

No, just because a system is funded with federal dollars does not mean we are required to share our data with other states or the USDOE. Following similar processes and standards may enhance a state's ability to do so, but it is ultimately up to each individual state to decide whether to share data beyond its own borders.

Colorado has chosen not to, except in the case of using the National Student Clearinghouse. This organization collects and posts college enrollment and graduation data for the purposes of understanding the path that graduating seniors follow into the post-secondary arena. Reviewing cross state results can tell Colorado if we are losing our future teachers to out of state colleges and universities because their programs may be better.

Colorado just passed legislation (see HB 1170) that requires CDE and the Department of Higher Education to use this data for improving educator preparation courses and understanding how to better prepare teachers for careers within the state.

What checks and balances are in place for RISE?

The entire process for RISE contains many data governance and security-related checks and balances. Strict processes must be followed to request any longitudinal analysis using the RISE system, most of which are embedded in overall program governance and in the data governance in place that defines the review and approval of all data requests. This includes formalized processes for a requestor to ask for data for longitudinal analysis purposes, decision points throughout the process where there may be more information needed or a decision could be made to end the particular request workflow. It also includes the reviews necessary to decide if the purpose of the analysis meets the criteria set for "appropriate educational purpose," as well as scrutiny necessary for compliance with data security and privacy provisions.

CDE reviews all of these questions with every entity requesting data, ensuring that the "receiving" entity has adequate controls within their Information Security environment to safely manage and protect our data. Although CDE is already conducting these reviews and assessments on all entities requesting data, as mentioned above, formalization and communication of this process is still in progress. The goal is a repeatable, consistent and well understood process in place that CDE employees and partners fully understand and follow.

Are there different versions of the RISE system, and/or the associated Data Dictionary?

There is only one version of the RISE system, and one version of CDE's Data Dictionary. There are only a few CDE employees who manage and maintain the RISE system, as mentioned above, so there are no special "versions" of the system for different users, nor has there ever been.

CDE's Data Dictionary was built to provide information for the public to view the various data elements CDE collects, identify the authority to collect of each element, and to provide more "business" information for the interested layperson. The use of the term "business" here refers to internal CDE educational program information, and is meant to distinguish between information technology related system information and programmatic educational information.

Business users are those agency personnel who are subject matter experts on educational programs, not system level developers or coding experts. CDE's subject matter experts are those who evaluate the success of various programs, such as literacy or post-secondary programs.

CDE's Data Dictionary was built long before CDE built the RISE system to extend beyond K-12 longitudinal analysis. Because of this, it still reflects the pre-existing data collection system in some cases. It is being updated to reflect the current system, Data Pipeline, but that will take some time. CDE also understands that the Data Dictionary is challenging for members of the public to use. For these reasons, CDE is planning to upgrade the data dictionary to more clearly and easily provide information to the public.

What was the most recent SLDS Grant application focused on?

In March of 2015, the US Department of Education released another round of SLDS grant opportunities. This round is focused on effective use of data; this means that the states must demonstrate that the systems built over the past several years actually do allow effective longitudinal data analysis to be conducted, and the results are accurate, secure, provide value for investment, and most importantly, provide answers to the most commonly asked questions about effectiveness of public educational programs, such as whether a child enrolled in Early Intervention and CCCP programs perform better at the 3rd grade level than those only enrolled in one of these programs only.

The focus on this grant round is less on technology development, and more on the usefulness of the analyzed data. The maximum amount available to any one state is $7 million, over a period of four years.

What specifically did Colorado apply for?

Colorado applied for two categories, Evaluation & Research and Early Learning. The primary focus of the Evaluation and Research category is on enhancing Colorado's data request process, to include additional security measures, privacy controls, and enhanced tools to automate the overall request and review process. As mentioned above, there is an extensive review process required for anyone who requests longitudinal analysis through the RISE system.

Additional grant funding would allow CDE to apply more resources to that review and approval process, and expand security controls and privacy provisions in each review. Additional grant funding could also provide enhanced tools for CDE, schools and districts to access and examine data more efficiently.

The other category for which CDE applied for funding is Early Learning. Early Learning projects are currently underway, as required by statute. Additional SLDS funding would support and enhance current projects and allow CDE to demonstrate the benefits of those enhancements.

What was the outcome of the recent SLDS grant application?

CDE was notified on September 15th that Colorado was not one of the states chosen to receive additional SLDS funding at this time.

The 2015 grant narrative describes a multi-state collaborative (MSTC). What is this?

The Multi-State Collaborative is a group of ten participating states that will support each other whether the state receives funding or not. Each state committed to varying levels of participation, from providing review services and thought leadership, to jointly developing new products. The idea is to maximize use of federal funds so multiple states are not developing the same technologies. This makes much better use of taxpayer monies.

Colorado agreed to participate at a "reviewer" level, i.e., we will provide services only as needed to other states that solicit our input on processes and products they build. No data will be shared through this collaborative, only ideas. The Council of Chief State School Officers endorsed this collaborative, but did not support with funds.

What does "interoperability" mean in the context of longitudinal data analysis?

Interoperable systems refer to systems and processes that are built and function in similar ways so that components may be shared and/or interfaces can be built for communications. The term "interoperable systems" does not imply that data will be shared. States with interoperable systems may share design and technology expertise, but in no way does this require any sharing of data. Sharing data across states is a policy level decision and one that CDE has chosen not to engage in.