Synopsis
Data in your Concourse system is invaluable for a number of reasons, such as accreditation reporting and compliance reporting. While much of the information in your Concourse system can be found in other locations, such as the SIS, you may need to generate a list of syllabus-specific content that is only found in Concourse or you may just find it more convenient to extract this data from Concourse than from another system.
There are three main options available for extracting data from your Concourse system. Institutions may choose to download their syllabi as PDFs, generate reports manually in the Concourse Report Center, or implement the use of APIs. Downloading syllabi as PDFs results in a static record of entire syllabi as they existed at a moment in time. When only selected syllabus items or non-content data points are needed, institutions typically generate Concourse Reports or use APIs. In rare cases, such as when the data needed is not available in existing reports or via API, it may be necessary to request reports from Concourse Engineering. While it is possible to use a combination of bulk download, reports, and API to extract data from your Concourse system, it is strongly recommended that data extraction requests not be made to Concourse Engineering unless all other possibilities have been exhausted.
Key Terms
- Administer System Permission: the permission to process feeds, assign system permissions to other users, make changes to user accounts, purge users and courses, apply group permissions en masse, manage integrations, and manage domains. Some Concourse reports are only available to users with Administer System permission.
- Domain Permissions: the permission to view, manage, report upon, and/or modify syllabi for entire domains; four types of domain permissions are available: administer, edit, audit, and report.
- Bulk Download: a tool that allows domain users to download entire syllabi from their assigned domains; syllabi are saved as PDFs and accessed from Concourse as a zip file.
- Content Reports: reports that present data about syllabus content; content reports include the Item Report, Missing Item Report, Multi-item Tabular Report, Multi-item Snippets Report, and the Reading Level Report.
- Activity Reports: reports that present data about activity (access or modifications) on selected syllabi; activity reports include the Audit Report (if Audit is enabled in your system), Syllabus Access by Course Report, and Syllabus Changes by Course Report.
- User Reports: reports that present data about Concourse users; user reports include the User Report, Domain Permissions Report, and Curriculum Vitae Report (if Curriculum Vitae is enabled in your system).
- System Reports: reports that present system-wide data about syllabus access and modifications, user and course settings, and configuration settings; system reports include the Syllabus Access by Day Report, Syllabus Changes by Day Report, Tag Utilization Report, Group Permissions Report, Domain Structure Report, Hidden Courses Report, and Statistics Report.
- CSV: many Concourse reports can be downloaded as CSVs (i.e., spreadsheets) that can be shared with other stakeholders; these spreadsheets can be scrubbed or modified to focus on specific data or to emphasize the most relevant data.
- API: Application Programming Interfaces (APIs) allow your institution to automate the retrieval of data available in Concourse; Concourse supports four API endpoints: Syllabus Changes, Syllabus Access, Course Listing, and Syllabus Content.
- JSON: the data retrieved by APIs is formatted in JSON (JavaScript Object Notation); while this format is technically human readable, the data likely requires formatting adjustments before it can be meaningfully used by academic stakeholders.
Important to this Decision
Before making decisions about how your institution uses Concourse, you should consider how stakeholders, systems, timelines, and other factors are impacted. Review the important elements below to be better prepared to make changes to your Concourse configuration or implementation.
Prior to making this decision, you should consult academic stakeholders, personnel involved with accreditation and compliance, and IT staff involved with data management.
Institutions extract data from their Concourse systems for a variety of reasons, such as accreditation reporting, compliance reporting, transfer evaluation, quality assurance review, and more. The amount of data needed, the frequency/interval at which the data is needed, and the data’s intended purpose inform your institution’s decision about which method to use for data extraction.
Data extraction is an operation that likely occurs regularly at your institution, so it is important that you identify the method(s) that is most sustainable for long-term success. The decision about which extraction method to use can be made or updated at any time. We recommend consulting key players and reviewing relevant reporting timelines at your institution to inform your decision about what data extraction method to use and how often the operation should be performed.
Data extracted from Concourse via downloads, reports, and API is read-only; if data must be added to or modified in Concourse, feeds or manual entry should be used.
Selected data in Concourse may be extracted so that it can be added to another system, such as the SIS or transfer evaluation software. In the case of syllabus PDFs, these files may be uploaded to an accreditation portal or stored in an internal server.
- If your institution uses the Bulk Download tool:
- Domain users navigate to Admin > Tools.
- Select “Obtain an archive of syllabus PDFs.”
- Select Create Archive.
- Search for and select syllabi to include in the archive.
- Name the archive and select Create.
- The user generating the archive will receive an email notification when the archive is ready to be downloaded. Upon receiving the email, return to the bulk download area of Concourse and select the zip file from the list of archives.
- If your institution manually generates Concourse Reports:
- System and domain users select Reports in the main navigation menu.
- The reports displayed depend upon the user’s assigned permissions. The user can generate any report that is displayed; the data that presents in the report, however, may be limited based on the user’s group or domain permissions.
- If your institution uses APIs:
- Work with the API developer at your institution to determine what endpoints should be used, the frequency at which the data is needed (cadence of API queries), and how the data will be shared.
- Within Concourse, the System Administrator should navigate to Admin > System.
- Select “Obtain endpoint and key information to utilize the API.”
- On the subsequent page, the Base API endpoint, an example URL, and API key are available. Share this information and Concourse’s API documentation (in Related Topics below) with the API developer at your institution.
Choose the method(s) that best supports your other systems, processes, and staff; note that it is possible to use a combination of these methods for different purposes. If complete syllabi are required for accreditation, then the bulk download feature is a good choice. If selected syllabus content must be reviewed for compliance, then manual reports make sense. If your institution requires frequent reporting of extremely large amounts of data and your institution has the IT staff to support API usage, APIs may be the best choice. The long-term impact of data extraction is optimizing your reporting, view, and compliance processes, regardless of the method you choose.
Configuration Options
Concourse provides different options for extracting data to support different institutional contexts. We recommend that you review these options carefully before deciding which one best meets your needs.
- Option A: Generate Concourse Reports
- Option B: Bulk Download Syllabi
- Option C: Use APIs to Extract Data
- Option D: Request Data from Concourse Engineering
For the purposes of regularly extracting large amounts of selected system data, either generating Concourse Reports or using API is the best option. Requesting data from Concourse Engineering is reserved for situations when the amount of data or complexity of data needed is so unique that Concourse Reports and API do not produce the desired result; this is not an option for regular data extraction. If the choices below don’t fit your needs, reach out to Client Services for assistance.
- Approach: Users with Administer System permission can access user reports and system reports, and users with domain permissions can access a variety of content reports, activity reports, and user reports. Once your institution has provisioned system and domain permissions to appropriate users, communicate with those users what their role is in relation to data extraction and reporting. Users with high-level permissions should be trained on how to use Concourse reports, the type of reports they should pull and how often, and how this information should be shared with other stakeholders.
- Better For: This is the most commonly used option for extracting selective data from Concourse; this option works well if your institution does not need to retain static or offline copies of full syllabi. Using the Concourse Report Center does not require advanced technical skills, so this option is easy and convenient for all types of institutions to use.
- Advantages and Disadvantages of Generating Concourse Reports: A significant advantage of generating Concourse reports is that this operation can be delegated to multiple high-level users to reduce the workload on a single individual; subsequently, these users can generate Concourse reports on-demand, as frequently as desired. A minor disadvantage is that Concourse reports may require some adjustment (e.g., hiding irrelevant columns or using pivot tables) to make the data more useful before sharing CSVs with other stakeholders. It’s also possible–though unlikely–that reports containing very large amounts of data may time out. If this occurs, a member of Team Concourse can assist you.
- Effort: Generating Concourse reports manually is a quick and easy process for users who have been trained on the process. Most reports can be generated within a few minutes. The amount of effort and time increases with the frequency of report generation and the amount of data scrubbing that is required before sharing the data with other users; time estimates vary based on your institutional needs, amount of data, and intended usage.
- Approach: Users with domain permissions can bulk download syllabus PDFs. Once your institution has provisioned domain permissions to appropriate users, communicate with those users what their role is in relation to data extraction and archiving syllabi. Domain users should be trained on how to perform the bulk download operation, how to narrow search criteria to create organized archives, how often to bulk download syllabi, and where these files should be stored or shared with other stakeholders.
- Better For: This is the most commonly used option for extracting full syllabus data from Concourse. This option works well if your institution needs to keep or produce a record of full syllabus content.
- Advantages and Disadvantages of Bulk Downloading Syllabi: A significant advantage of bulk downloading syllabi is that this operation can be delegated to multiple domain users to reduce the workload on a single individual; these users can generate bulk downloads on-demand, as frequently as desired. A minor disadvantage of this operation is that bulk download is limited to 5,000 syllabus downloads at once; this limit may be lower in the case of large syllabi. If you need to download more than 5,000 syllabi simultaneously, it’s possible to narrow your criteria to download syllabi in smaller chunks. Bulk downloads also contain full syllabi as PDFs, so this operation is useful for creating archives or providing documentation to external bodies; if you only need small selections of syllabus content for compliance or quality review, bulk downloaded syllabi contain too much information to be effective for this process, and another method would be a better choice.
- Effort: Generating a bulk download of syllabi is a relatively quick and easy process. Users with domain permissions can select the download option from the Tools page, search for syllabi to include in the file, and name their archive. The system automatically generates a zip file of syllabus PDFs; the length of time it takes to generate the zip file varies based on how many syllabi are included in the archive. Typically, smaller archives are available within a few minutes while larger archives are available within a few hours. PDF archives are available in Concourse for 14 days; it is recommended that the domain user download the zip file before the archive expires so that the generation process does not have to be completed again.
- Approach: APIs are automated processes that must be developed and implemented by an API developer at your institution. It is important to note that APIs only generate information that is readily available in your Concourse system to users with the correct permissions. As a result, APIs generate the same or equivalent information as that available via Concourse Reports; APIs do not reveal any additional information. There are four API endpoints that can extract the following data from your Concourse system:
- Syllabus Changes: data available in the syllabus Change Log; a listing of change actions made to a syllabus
- Syllabus Access: data available in the syllabus Users list; a listing of the most recent date and time users viewed a syllabus
- Course Listing: data available via Search; a listing of courses in your system that meet certain criteria
- Syllabus Content: data available in Item Reports; a listing of content available on a group of syllabi
- Better For: This option is used most often by institutions that have very large or complex systems and have an API developer available to manage this project.
- Advantages and Disadvantages of Using APIs to Extract Data: API endpoints can be used to extract large amounts of data very quickly; API calls can be set to process as frequently as the institution wishes. APIs effectively generate large amounts of data with relatively little human intervention. However, there is a high degree of technical knowledge required to implement an API endpoint; we do not recommend that you attempt to utilize APIs unless there is an API developer at your institution who is available to lead the implementation process and provide maintenance support in the long-term.
- Effort: The time investment of setting up API endpoints varies. For members of the Concourse team, such as System Admins, the investment is minimal–typically just the amount of time it takes to decide what type of information is needed via API, how often it should be called from your system, and how it will be made available to stakeholders. For the API developer on your IT team, the time investment is tied up in the initial development of the endpoints; your API developer will have a better understanding of the time commitment and how it fits in with their workload. After the endpoints are implemented, the APIs are automated and require little intervention.
- Approach: To request data extraction from Concourse Engineering, email support@campusconcourse.com to explain the data that is needed, its intended purpose, preferred data format, and the timeline for delivery and usage. If at all possible, a member of Team Concourse will offer suggestions about ways that your institution can extract this data on demand so that you can take ownership of the process. If it is not possible for you to extract this data on your own, you may be asked to provide additional information to help our engineers extract your data in the most optimal way. Once Concourse Engineering develops a script to extract the data, you will be provided with an estimated timeline for delivery. When the data is available to you, it will be provided in the requested format; a copy of the data will be retained in the support ticket and in your institution’s Concourse shared drive for later reference.
- Better For: This option is used sporadically by institutions that have very specific data requests that cannot be fulfilled by Reports or API. This is not a method of data extraction that should be used regularly; it should be treated as a special exception or last resort.
- Advantages and Disadvantages of Requesting Data from Concourse Engineering: If a special data extraction request is submitted, it is possible that Concourse Engineers may be able to query the requested information from your system; for some very specific or granular requests, such as audit logs, Concourse does not track this information, and it therefore cannot be retrieved from your system. Because unique data extraction requires the development of a custom script, these requests may take some time to fulfill; additionally, data extraction requests are added to Concourse Engineering’s queue, which may be quite full depending on time of year, ongoing development projects, and other client needs.
- Effort: The amount of time to generate this report varies based on Concourse Engineering’s current workload and development queue; depending on the complexity of the request and current workload, estimates to fulfill these requests vary from two business days to a week or longer.
Post-Decision Changes
Your institution’s needs may change over time and you may need to make changes to your configuration settings. In other words, the configuration decisions you make during the implementation phase might not be as efficient five or ten years afterward. At some point, you may find that your chosen method for extracting data no longer works for you. Fortunately, this part of managing Concourse does not play a significant role in the end user experience, so changing your approach to data extraction later only requires communication and minimal training for the users whose roles change. Note that the motivations of bulk downloading syllabi and generating reports of selective information are not necessarily interchangeable, so it is unlikely that your institution would transition from bulk download to generating reports or bulk download to using API; your institution may opt to perform both processes simultaneously to meet multiple needs.
Review the guidance below to better understand the steps involved in shifting from one method to another.
If your institution initially extracts data by manually generating reports and later decides to automate this process through the use of APIs, you will need to:
- Discuss your data needs with academic stakeholders to determine which APIs should be implemented and how often the data is needed.
- Communicate with an API developer at your institution to initiate the use of the appropriate API(s) at the correct automated cadence.
- Determine how the information will be shared and used once API usage is fully implemented.
- Communicate with your domain users about the new data extraction process and how their role is changing regarding the generation of reports (i.e., the function is no longer needed).
If your institution initially extracts data via APIs and later decides to extract data through manual report generation, you will need to:
- Review the system and domain permissions assigned throughout your system to determine if they meet your needs for delegating report generation.
- Assign additional system and/or domain permissions as needed to support your reporting needs.
- Communicate with high-level users about institutional reporting expectations, especially clarifying their role and responsibilities (e.g., which reports, which domains, frequency).
- Train high-level users on generating, downloading, and sharing Concourse reports with other stakeholders.
- Communicate with your API developer about the shift in data extraction methodology so that the automated processes can be stopped.
- Inform other stakeholders that data will be generated manually in the future and how to get help if they need it.
Before making a decision about how you will extract data from Concourse, you should be familiar with some related issues and contexts. See the following articles for more information:
- Read about the types of reports available to you in the Report Center.
- Start out with API by learning about API - General Usage. If this seems like a good option for your institution, investigate the four different types of API endpoints that Concourse supports: Syllabus Changes, Syllabus Access, Course Listing, and Syllabus Content.
- To review the domain permissions assigned in your system (for report generation), generate a Domain Permissions Report.
Tell Us About Your Experience
Did this article answer your question? If you made a decision like this for your Concourse instance, let us know how it turned out. If we missed something in this article or if you have a question that isn’t addressed in our Knowledge Base, let us know how we can help by reaching out to
support@campusconcourse.com.