In today's world, AI chatbots like ChatGPT and Claude can perform various functions, such as composing work emails and planning travel itineraries. These chatbots are systems built around large vision-language models (VLMs), trained on massive datasets that include books, websites, code, and images. AI algorithms are refined on extensive human feedback to follow instructions and avoid harmful or unwanted output, using that 'knowledge' to produce text or images based on user input.
Despite their clear limitations, chatbots can be very helpful for a wide range of tasks, including areas that traditionally require specialized skills, like computer programming. As part of a project for the U.S. Department of the Air Force–MIT AI Accelerator's Phantom Program, U.S. Air Force cadet Joshua Lynch, with the help of his mentor Laura Niss from MIT Lincoln Laboratory, aimed to determine if he could develop a fully functional program as a complete novice to coding. He employed a process called "vibe-coding," where a user relies entirely on prompts to guide a generative AI chatbot to write and refine code.
Lynch's motivation was to empower anyone familiar with military problem spaces, regardless of their technical background, to advance their ideas for useful software applications, effectively bypassing the time and cost constraints of the traditional military software development pipeline. He aimed to build his own application while Niss monitored his experience with the technology. "The Phantom student wanted to see if he could create a useful application through self-identified vibe-coding, without any previous experience," Niss states.
Lynch sought to determine if, starting with no coding skills and using chatbots, he could create an application specific to his tactical team to help reduce collateral damage while enhancing survivability in broader missions. This application would offer capabilities such as AI-assisted target recognition, modular intelligence, surveillance, and reconnaissance, autonomous striking, and battlefield communication management. Throughout the project, Lynch completed several professional development courses in AI and familiarized himself with both military and nonmilitary uses of the technology.
For code generation, he utilized paid models of three AI chatbots: Anthropic's Claude, OpenAI's ChatGPT, and Google's Gemini. Most of this work was done solely through the chatbots' main chat function in a web browser, rather than as an integrated system within a development environment, which is currently standard. The final application was produced using Google AI Studio App, which can create applications that interface with the Gemini application programming interface and has AI integrated into the development environment.
Over three months, Lynch worked with these models to build his application, called the Remote Operating Modular Augmentation Device (ROMAD-AI). During this time, he learned several methods to improve the code output. For example, he often encountered difficulties with the AI chatbots lacking hierarchical focus and modifying unrelated code sections. He discovered the importance of breaking problems into smaller parts, framing questions clearly, and steering conversations back on topic when they stray too far from the objective.
As Lynch gained more experience with the chatbots, limitations in AI capabilities and development time caused him to re-scope the project, moving it from an application that could assist on the battlefield to one that could perform basic document processing, such as analyzing tactical maps of battlefields and generating mission-planning documents through an interface with a VLM-powered chatbot. While the resulting prototype did not perform all capabilities Lynch originally intended to include (and in its current iteration was not secure for the desired use case), it demonstrated the capability and usefulness of such an application for service members. "I was quite impressed with this final product, and it showed me how powerful these systems can be at prototyping designs from nonexperts," Niss remarks.
Niss observed the change in Lynch's perspective of AI language models during his experience. Starting with an impressive goal, Lynch gained an understanding of the capabilities of current technology and significantly scaled down his expectations by the end of the project period. Measures of his perceptions of the different AI systems over time and across system updates were particularly interesting to Lynch and Niss, with Claude showing more stability than ChatGPT across traits such as likability, anthropomorphism, and perceived intelligence. Lynch found AI to be a helpful tutor but noted its inaccuracies on topics he knew well. The project demonstrated that AI chatbots can empower nontechnical service members to produce viable software applications for their unique problems, although they work better as prototyping assistants than as full production tools when handling sensitive information and critical applications. Improper vetting of code may lead to security risks, as demonstrated by an instance where Lynch was unaware that the final application was sending input documents to a Gemini AI model for analysis rather than parsing the documents locally on his computer. "For me, this project reinforced the expanse between experts in different fields," Niss concludes. "No matter how good AI gets, I think we'll always need to collaborate to get to the best solutions for the most important problems." This research was sponsored by the Department of the Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000.
Blogger's Review: This article showcases the immense potential of AI technology in military applications for novice programmers. Through the