SayIntentions.ai, a new AI-based air traffic control add-on currently in beta, takes an innovative approach to combining AI with traditional coding to build an ATC add-on for X-Plane and Microsoft Flight Simulator that promises to cater to the VFR sim pilot. The SayIntentions founder and lead developer, Brian, was kind enough to sit down with FSNews to share SayIntentions’ story and his thoughts about the potential of modern AI tools to augment our simming experience.
For pilots, Air Traffic Control (ATC) is a friend, a director, and sometimes a nuisance – but a constant companion. However, ATC is often the human element missing from immersive flight simulation, and multiplayer networks can be intimidating and restrictive. This is the problem that SayIntentions.ai aims to solve: bring humanity back to sim ATC.
Table of Contents
Action Initiation
“We started this project because we wanted it for ourselves.” Brian is an AI specialist and his co-founder is a C#/C++ programmer, and both of them have been flight simming since the mid-80s. Brian has been a customer of online ATC networks such as VATSIM and PilotEdge for a decade, and he feels that ATC is the missing link to a truly immersive sim experience, as well as an excellent educational tool.
“Flying without ATC is lonely and removes a giant element of immersion,” Brian states. “I feel like flight-simmers who don’t take the time to really learn the ins and outs of ATC comms are missing out on a big part of the aviation hobby,” says Brian. Although simmers can choose to fly alone, a little connection with someone on the other end of a radio can make a big impact. Brian states that interacting with ATC has taught him more about aviation than almost anything else.
Awkward Interactions
To Brian and his team, Air Traffic Control is indispensable to his simulator experience. However, he wasn’t fully satisfied with the current ATC options available – especially for the casual or VFR-oriented pilot. Although he has a tremendous amount of respect for VATSIM’s controllers, he acknowledges the problems inherent in their all-volunteer model. “The biggest issue is lack of coverage, and an ever-increasing pool of users who aren’t familiar with how things are supposed to operate. It’s a giant investment to learn to use it, and then to have your ATC services terminated […] because the controller is no longer available can be a real buzz-kill.“
Brian is also a PilotEdge veteran and has spent hundreds of hours flying with their controllers, whom he says provide an excellent experience. “I can’t say enough good things about them, and most of what I know about ATC came from […] their network. But they have their own drawbacks. Limited coverage area, and (frankly) you often feel like you’re the only person in the sky.” But above all else, Brian recognizes that many a pilot (real and sim) are hesitant to interact with these multiplayer ATC networks for one reason: they are intimidating.
“It’s funny,” he says, “the whole point of flight simming is to let us learn at our own pace with as much realism and immersion as we can handle. We learn as we go, but have no real fear of crashing or being judged for our terrible piloting skills. And yet when it comes to getting a realistic ATC experience, we’re back to the square-root of the problem. It’s real. There are REAL people listening. The WHOLE community is listening! And when you combine that with a serious learning curve, a huge number of flight simmers never quite get brave enough to push that transmit button.”
“With SayIntentions.ai,” he tells us, “We’ve solved that in a really fun way.”
App Inauguration
SayIntentions.ai’s GPT-based AI, says Brian, is the antidote to the fear and anxiety faced by countless simmers when they first key up the microphone. “The magic of what we’re offering is that you get a realistic experience combined with absolutely no intimidation factor.” With their high-quality voices backed by ChatGPT’s remarkable ability to produce natural language, SayIntentions is designed to let each pilot strike the balance that they prefer between realistic flight operations and consequences-free “roleplay.” “It doesn’t really matter if you get something wrong – which is the whole reason we love flight simming in the first place!”
Since its earliest versions, SayIntentions has become an add-on that Brian and his team cannot fly without. The sense of immersion that SayIntentions brings is second only to a multiplayer network on a good day, but without all the baggage associated with one. This enables Brian, and the beta testing team helping iron out the last few bugs, to take the ATC as lightly or as seriously as they wish on any given flight.
Almost Intelligent
Large Language Model Artificial Intelligences (LLM AIs) reached the mainstream with the release of OpenAI’s ChatGPT in November of 2022, immediately taking the tech industry by storm. An LLM AI is a generative AI (an AI capable of generating text, audio, video, or image content) specifically tailored for conversational text. They use specialized algorithms and ludicrously large datasets to understand, summarize, predict, and generate new content.
Since ChatGPT’s explosive entrance, flight simmers have theorized various ways to train the AI to become an air traffic controller. Early experiments were promising, but as Brian and his team discovered, ChatGPT was actually an awful air traffic controller. “They can role-play to some extent,” Brian tells us, “But it’s a complete disaster beyond that.”
SayIntentions’ implementation of GPT-4 strikes a powerful balance, leveraging AI not to create the first practical ATC add-on, but to create the first one that is truly intuitive and natural to interact with. “ATC itself is logical enough to fit on a flowchart and requires no real reasoning skills […] it’s perfect for a normal computer program,” says Brian. “But if that’s the case, how come nobody has built an ATC system that’s even remotely realistic?”
Without AI, even the best voice-driven sim ATC programs simply choose from a series of menus in a complicated state machine that relies on sim pilots using proper phrasing and keywords, as well as exceptional voice-to-text capabilities. Whether using voice or selecting options with the keyboard and mouse, the experience is not truly dynamic and if you get turned around, the program has no function to actually understand intent.
Understanding a pilot’s intent is the crux of the AI implementation in SayIntentions. A basic speech recognition algorithm can feed GPT-4 a pilot’s voice transmission, which is then interpreted to determine what the pilot wants to do. SayIntentions’ “secret sauce” handles the rest, and the pilot is provided with clearances and instructions that do not rely on specific, memorized commands.
In the real world, pilots (especially inexperienced, VFR-only pilots) often stray from standard phraseology. A human controller can understand these deviations and still provide ATC services, but a rigid “traditional” ATC add-on cannot. SayIntentions’ revolution is adding humanity back to sim ATC that naturally adapts to the pilot’s level of experience, just like real-world controllers would do.
Aerial Imperfection
“AI is definitely not 100% perfect, and probably never will be.” Tuning that balance between their own procedural framework and letting AI do the thinking is the primary goal of SayIntentions’ early testing phase. “Somewhere on that spectrum between ‘this is terrible‘ and ‘this is study-level perfect,‘ is ‘wow… this is pretty good’,” says Brian. “We’re getting closer and closer to ‘wow, this is pretty good!’ And of course, it’ll only continue to get better from here.”
Right now, SayIntentions.ai is in the midst of a soft launch to friends & family of the initial beta team. There are still improvements to be made, of course. “We’ve reached a stage where a real-world pilot who had a solid understanding of ATC comms will discover all ‘eyebrow-raising‘ moments.” But thanks to Brian’s tireless troubleshooting and the invaluable input from the beta team, improvements are made every single day. “I’m constantly surprised when people throw something at it that we haven’t explicitly trained it for, and the vast majority of the time, it gets it right.”
It may never be perfect, but then again – neither is any human controller. Brian has learned very well the limits of LLM AIs. “Anything that requires any level of predictability or consistency is where current language models turn into a total disaster. While we like to think that GPT-4 plays a critical role in how we’re able to create such a fun experience, it’s definitely been relegated to a very specific role in our process.”
GPT-4’s role in that process has been refined over many months and hundreds of man-hours. Initially, Brian was just going to train the AI on the FAA’s air traffic control publications directly. After all, those work for human controllers – so training the AI should have been a snap.
Perhaps predictably, it wasn’t that simple. “I’ve all but abandoned that idea,” Brian told us, “The AI does not do well with an overwhelming amount of context.” The issue, it turned out, was that the AI was trying to be a single “mega-controller.” With knowledge of all the intricacies of the ATC systems, it tripped over itself trying to satisfy all the complex conditions contained in the FAA’s legalese-encrusted texts.
Instead, SayIntentions.ai cleverly meters the amount of information that its “controllers” are able to see and understand at once. Brian has then “translated” the FAA tomes into a series of instructions that more closely match the way that real-world controllers think. In this improved form, Brian believes that it can improve others’ experiences just as much as it has improved his own, even in its infancy. “Frankly,” he says, “I’ll never fly without it, ever again.”
Advancing Iteration
It is proven that no software project survives contact with the user, and SayIntentions’ beta program has been no exception. The future of SayIntentions.ai has been shaped by the feedback of the beta testers, and the path that SayIntentions takes after its initial release is not exactly the one that Brian was anticipating.
Brian thought that IFR support would be the first thing on testers’ lists. As it turns out, though, the beta team mostly wants to increase the immersion of the VFR experience at which SayIntentions already excels. “People are much more excited about things like traffic-aware ATC and super accurate procedures that are customized to each airport,” discovered Brian. Customized procedures are an example of one area where SayIntentions has made great strides during the beta period. “It’s kind of magical when you’re sitting on the ground at John Wayne and they assign you the Orange Departure (VFR), or you’re inbound and they tell you to ‘report when over signal peak‘. There’s that reaction where you go, ‘holy smokes.. that’s exactly what a real-world controller would have said!'”
Currently, SayIntentions supports every major component of a VFR flight, with 24/7 coverage across the United States. Clearances, all types of U.S. VFR airspace, flight following and radar services, pattern work, and more have all been implemented thus far, along with “magical” features such as airport-specific departure, arrival, and noise abatement procedures. Traffic awareness, CTAF airports with busy patterns, highly accurate ground handling, and IFR are all on SayIntentions’ roadmap. The good news, Brian tells us, is that the technical details of how to handle these in SayIntentions’ AI framework are already solved – it’s only a question of allocating the small team’s time.
The team’s development process has evolved significantly throughout the process as well. New techniques to troubleshoot and respond quickly to tester feedback have been developed and implemented. “Sometimes even in the middle of someone’s flight, we’re able to watch and correct issues. This is unprecedented in software development,” says Brian, “And it’s a strong tribute to the platform we’ve built.“
Application Integration
SayIntentions is rapidly approaching a point at which the add-on is ready for its initial release. But like many modern software projects, the release is just the beginning – and according to Brian, there needs to come a point where an iterative project that will never truly be “done” simply becomes “done enough” to be set free of its initial development.
For him and his team, “It’s ready once two conditions are met. (1) It’s better than any other alternative out there for GA VFR, and (2) it’s a lot of fun. We believe we passed those milestones a long time ago.” Brian believes that a full public release by early 2024 is very much in the cards given the rapid pace that the team is squashing bugs. Currently, it is on an invite-only “soft launch” that will be slowly expanded over the coming weeks as the add-on is refined.
Pricing as well is yet to be determined. “People hate subscriptions,” says Brian, surprising nobody. “All of this stuff runs at a metered cost. The exceptional voices, robust AI experience – none of this is free [to integrate into SayIntentions].” However, as OpenAI continues to drop its prices – the recently announced updates to GPT-4 will be a third of the cost that is now, for example – Brian plans to be sure to price SayIntentions accordingly. “We’re not in this to get rich. We don’t need the money. But we do need to cover our costs, and maybe make enough to buy ourselves a new airplane in the Marketplace every now and again!”
Brian is open to partnering with an established developer to make SayIntentions’ “go-to-market” as smooth as it can be. “We’re not marketing or sales experts, and we’re not super well connected or known within the flight-sim community,” Brian says. “Most people don’t realize just how much work the ‘go-to-market‘ part of this really is. I’d rather get a partner who’s willing to help us with that, so we can keep our heads down on improving the product.”
Analyze Implications
It is unquestionable, at the end of 2023, that artificial intelligence will change the way that we approach technology – and this includes the hobby of flight simulation. It is easy to imagine that in a few years’ time, flight simulators ship with AI-based ATC as a standard feature. SayIntentions.ai, then, is poised to be the tip of the spear. This is a journey that may be fraught with difficulty, but according to Brian, it’s what he and the team are ready for.
The joy of flight simulation is the freedom to explore, learn, and indulge in aviation passion from the comfort of one’s own home. This has always been a hobby and industry on the cutting edge of technology. SayIntentions’ AI Air Traffic Control add-on is the latest example of this – a small, passionate team relentlessly pushing the boundaries of what is thought to be possible. And, just like the rest of us, they do it out of passion for aviation, and the joyous freedom of flight.
FSNews would like to thank the founder of SayIntentions.ai, Brian, for taking the time to talk with us today. To find out more about SayIntentions, visit their website. Join their Discord to stay up-to-date with development! If you are an industry member interested in partnering with SayIntentions.ai, please contact them at [email protected].
