Automated Program Synthesis and Optimization Using Foundation Models in Software Engineering
DOI:
https://doi.org/10.63282/3117-5481/AIJCST-V3I6P101Keywords:
Automated Program Synthesis, Foundation Models, Deep Learning, Software Optimization, Transformer Models, Code Generation, Semantic Understanding, Performance EnhancementAbstract
Synthesis and optimization of programs run on computers have become two key research topics in the current software engineering. The emergence of foundation models, which are trained on massive pretexts on large code and natural language datasets, presents new possibilities in interpreting, creating and optimizing code. The paper explores the use of foundation models in the synthesis of programs in a self-managed manner and in enhancing the performance of programs. We recommend a structural approach that incorporates deep learning models into existing software engineering pipelines to produce, execute, and improve code in an automated approach. The given framework uses transformer based models to represent codes, understand them and predict errors. Empirical experience has shown that foundation model-based synthesis can save a great deal of development time, result in better code quality, and higher accuracy as compared to the conventional heuristic-based methods. Besides, the framework offers information on optimization strategies such as minimization of computational resources and energy consumption. Based on an analysis we also draw attention to challenges that may be faced, such as model interpretability, data bias and scalability, which provides avenues upon which future research may be based in regard to developing intelligent software. This paper gives a broad outlook of how foundation models can be exploited to provide automated program synthesis and optimization, and offers viable implications to the academic and industry sectors
References
[1] Gulwani, S., Polozov, O., & Singh, R. (2017). Program Synthesis. Foundations and Trends® in Programming Languages, 4(1-2), 1–119.
[2] Jha, S., & Seshia, S. A. (2017). A Theory of Formal Synthesis via Inductive Learning.
[3] Xu, Y. (2016). Neural Program Synthesis by Self-Learning.
[4] The Role of Zero-Emission Telecom Infrastructure in Sustainable Network Modernization - Varinder Kumar Sharma - IJFMR Volume 2, Issue 5, September-October 2020. https://doi.org/10.36948/ijfmr.2020.v02i05.54991
[5] Thallam, N. S. T. (2020). The Evolution of Big Data Workflows: From On-Premise Hadoop to Cloud-Based Architectures.
[6] Enabling Mission-Critical Communication via VoLTE for Public Safety Networks - Varinder Kumar Sharma - IJAIDR Volume 10, Issue 1, January-June 2019. DOI 10.71097/IJAIDR.v10.i1.1539
[7] Manna, Z., & Waldinger, R. “A Deductive Approach to Program Synthesis.” Communications of the ACM, 14(3), 151-165, 1980.
[8] Smith, D. R. “KIDS: A Semi-Automatic Program Development System.” IEEE Transactions on Software Engineering, 16(9), 1024-1043, 1990.
[9] Fickas, S., & Feather, M. S. “Requirements Monitoring in Dynamic Environments.” Proceedings of the IEEE International Conference on Requirements Engineering, 1995, pp. 140-147.
[10] Biermann, A. W. “The Inference of Regular LISP Programs from Examples.” IEEE Transactions on Systems, Man, and Cybernetics, 8(8), 585-600, 1978. (Foundational pre-1980 but relevant for synthesis roots.)
[11] Goldberg, D. E. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, 1989.
[12] Mitchell, T. M. Machine Learning. McGraw-Hill, 1997.
[13] Ernst, M. D., Cockrell, J., Griswold, W. G., & Notkin, D. (2001). Dynamically discovering likely program invariants to support program evolution. IEEE Transactions on Software Engineering, 27(2), 99–123.
[14] Gulwani, S. “Dimensions in Program Synthesis.” Proceedings of the 12th International Symposium on Automated Analysis-Driven Program Transformation (AADPT), 2010.
[15] Solar-Lezama, A. “Program Synthesis by Sketching.” PhD Dissertation, University of California, Berkeley, 2008.
[16] Jha, S., Gulwani, S., Seshia, S. A., & Tiwari, A. “Oracle-Guided Component-Based Program Synthesis.” Proceedings of the 32nd International Conference on Software Engineering (ICSE), 2010, pp. 215-224.
[17] Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., & Tarlow, D. “DeepCoder: Learning to Write Programs.” Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
[18] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” Proceedings of NAACL-HLT, 2019, pp. 4171-4186.
[19] Brockschmidt, M., Allamanis, M., Gaunt, A. L., & Polu, S. “Generative Code Modeling with Graphs.” arXiv preprint arXiv:1805.08490, 2018.
[20] Maddison, C. J., Gaunt, A. L., Brockschmidt, M., Kusner, M. J., & Tarlow, D. “Structured Generative Models of Natural Source Code.” Proceedings of the 34th International Conference on Machine Learning (ICML), 2017, pp. 2407-2416.
[21] Singh, R., & Gulwani, S. “Synthesizing Number Transformations from Input-Output Examples.” Proceedings of the 24th International Conference on Computer Aided Verification (CAV), 2012, pp. 634-651.
[22] Alon, U., Brody, S., Levy, O., & Yahav, E. (2019). code2seq: Generating sequences from structured representations of code. International Conference on Learning Representations (ICLR).
[23] Wang, K., & Singh, R. (2018). Neural program synthesis from diverse demonstrations. Advances in Neural Information Processing Systems (NeurIPS), 31.
[24] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, 4171–4186.
[25] Ellis, K., Nye, M., Pu, Y., Sosa, F., Tenenbaum, J., & Solar-Lezama, A. (2018). Learning to infer graphics programs from hand-drawn images. Advances in Neural Information Processing Systems (NeurIPS), 31.
