GSoC 2025 - Testing for Code Generation Targets

[shubhambhar007]

Project Title: Testing for Code Generation Targets

Project Description: We have tools that allow users to generate code from their Concerto models, supporting several languages. To ensure the reliability and correctness of this generated code, we propose introducing a way of testing the code generation process. This involves compiling the generated code for each supported language and running tests to verify its correctness.

Motivation: The primary goal of this project is to enhance the reliability of our code generation tools by implementing a robust testing framework. By doing so, we aim to ensure that the generated code is correct and functional across all supported languages, thereby increasing user confidence and satisfaction.

Expected Outcomes:

Set of Docker Images: Create a set of Docker images for each code generation target.
Automated Code Generation Tests: Implement a system to run code generation tests within the correct Docker image using GitHub actions. For example, generate Java code and then compile and run it using javac to ensure the generated code is correct.
Technical Approach:

Set of Docker Images:

Develop Docker images for each supported language (e.g., Java, Python, JavaScript).
Ensure each image contains the necessary tools and dependencies to compile and run the generated code.
Automated Code Generation Tests:

Implement GitHub actions to automate the testing process.
Configure the actions to generate code from Concerto models, compile the code within the appropriate Docker image, and run tests to verify its correctness.
Ensure that the testing framework is extensible to support additional languages in the future.
Timeline:

Community Bonding Period:

Familiarize with the existing codebase and tools.
Engage with mentors and community members to gather insights and feedback.
Phase 1:

Develop Docker images for each supported language.
Test and validate the Docker images to ensure they contain the necessary tools and dependencies.
Phase 2:

Implement GitHub actions to automate the code generation and testing process.
Configure the actions to generate code, compile it within the Docker images, and run tests.
Phase 3:

Finalize the integration of the testing framework.
Perform extensive testing and debugging.
Prepare documentation and tutorials for users.
Deliverables:

Set of Docker images for each code generation target.
Automated code generation tests implemented using GitHub actions.
Comprehensive documentation and user guides.
Regular progress reports and final project report.
Skills Required/Preferred:

Systems engineering, CI/CD
Docker, Docker Compose
GitHub actions
Benefits to the Community:

Enhanced reliability and correctness of generated code.
Increased user confidence in the code generation tools.
Streamlined process for testing and validating generated code.
Future Work:

Explore additional testing frameworks and tools to further enhance the testing process.
Continuously gather user feedback and iterate on the testing framework to improve its effectiveness.
About Me: I am Shubham Bhardwaj, a Software Engineer with experience in systems engineering, CI/CD, and Docker. I am passionate about creating reliable and efficient solutions and am excited about the opportunity to contribute to this project through GSoc.

My Linkedin - www.linkedin.com/in/shubham-bhardwaj-ba02b616b

[PengJingzhao]

Is this related to the requirements for GSoC Question 6 'Testing for Code Generation Targets'? I am a student from Guangdong University of Technology. Since my freshman year, I joined a studio at my university that specializes in developing microservice projects. I led the automation integration and deployment processes, and I am very familiar with containerization technology, Docker, Jenkins CI/CD technology, and frequently use GitHub's workflow functionality for automated testing and deployment. I am very interested in this topic and hope to participate in it.

[PengJingzhao]

I would like to join your team. How can I contact you - through email or social groups? I'd like to get familiar with your team. Thank you very much.