About the Customer:

Renaissance Learning is an educational technology company that provides cloud-based software and assessments for K-12 students and educators. Their products are designed to help students reach their academic potential and support teachers in delivering effective teaching strategies.
‍

Customer Challenge:

Renaissance Learning faced a significant challenge in aligning their Common Core-aligned test items to various state standards. This process was traditionally done manually, requiring substantial time and resources. The manual alignment of each item to the most appropriate state standard was not only time-consuming but also resource-intensive. If not addressed, this challenge could have limited Renaissance Learning's ability to efficiently adapt their item banks to multiple state standards, potentially impacting their competitiveness in the market and their ability to serve schools and districts with up-to-date, state-specific materials.

Partner Solution:

Add Value Machine (AVM) developed an innovative AI-assisted alignment pipeline to address Renaissance Learning's challenge. This solution leverages AWS AI/ML services, particularly AWS SageMaker JumpStart, to automate significant portions of the alignment process.

The pipeline utilizes multiple pre-trained Large Language Models (LLMs) from the SageMaker JumpStart model zoo to predict the most likely state standard alignment for each test item. Key components of the solution include:

1. Customized prompt templates that optimally frame the alignment task for the LLMs.

2. An ensemble of LLMs that predict alignments for each item/standard pair.

3. Alignment methods that synthesize LLM outputs into final alignment decisions.

4. Automatic alignment of high-confidence items to bypass manual review.

5. Identification of lower-confidence alignments for expert review.

This solution allows Renaissance Learning to automatically align items where the LLMs have high agreement, while flagging more ambiguous cases for human review. The pipeline's flexibility enables tuning for different subjects, standards, prompt templates, LLMs, and alignment methods.

AVM's implementation leveraged AWS SageMaker JumpStart for rapid prototyping and deployment, ensuring a scalable and reliable solution. The use of AWS services ensures that the solution can be quickly adapted to Renaissance Learning's specific needs.

Results and Benefits:

The implementation of AVM's AI-assisted alignment pipeline resulted in significant improvements for Renaissance Learning. The solution dramatically reduced the manual effort required for aligning test items to state standards, with high-confidence items being automatically aligned without the need for manual review.

While specific metrics were not provided, the case study indicates that the solution led to substantial time and cost savings for Renaissance Learning. The efficiency gains allowed the company to update their alignments more frequently and adapt their item banks to multiple state standards more easily.

For Renaissance Learning's customers (schools and districts), the solution resulted in more up-to-date and state-specific assessment content, potentially at a lower cost due to the reduced manual alignment effort. The combination of AI and expert review also ensured high alignment quality, providing educators with increased confidence in the assessment materials.

About the Partner:

Add Value Machine (AVM) is a leading provider of AI and machine learning solutions, specializing in developing innovative applications of generative AI for enterprises. As an AWS Advanced Tier Service Partner, AVM leverages AWS services to deliver scalable, secure, and efficient AI solutions across various industries, including education, finance, and healthcare. AVM's expertise in AI security and compliance ensures that their solutions meet the highest standards of data protection and regulatory requirements.

‍