Neurosymbolic AI: the 3rd wave Artificial Intelligence Review

Furthermore, issues related to adherence to principles of distinction, proportionality, and military necessity need to be addressed. Violations of international humanitarian law can result in legal consequences, and ensuring the adherence of Neuro-Symbolic AI systems to these principles poses a significant legal challenge in their military use. The integration of AI in military decision-making raises questions about who is ultimately accountable for the actions taken by autonomous systems. It is difficult to hold autonomous weapons systems accountable for their actions under international humanitarian and domestic law [120, 121].

• This approach has the potential to ultimately make medical AI systems more interpretable, reliable, and generalizable [72].
• If autonomous weapons systems cannot make this distinction accurately, they could lead to indiscriminate attacks and civilian casualties violating international humanitarian law [79, 87].
• Constraint logic programming can be used to solve scheduling problems, for example with constraint handling rules (CHR).

AI enhances cybersecurity by analyzing patterns, detecting anomalies, and responding rapidly to cyberattacks, thus protecting military networks and information systems [100]. Moreover, advanced AI techniques help in identifying vulnerabilities in these networks and systems, and to develop and implement security patches and mitigations. By leveraging the capabilities of AI, military experts in cybersecurity can contribute to the creation of expert systems that incorporate rules and insights for detecting and responding to cyber threats [100]. Experts in military intelligence can provide knowledge about patterns indicative of potential threats.

It uses deep learning neural network topologies and blends them with symbolic reasoning techniques, making it a fancier kind of AI Models than its traditional version. We have been utilizing neural networks, for instance, to determine an item’s type of shape or color. However, it can be advanced further by using symbolic reasoning to reveal more fascinating aspects of the item, such as its area, volume, etc. A key component of the system architecture for all expert systems is the knowledge base, which stores facts and rules for problem-solving.[53]

The simplest approach for an expert system knowledge base is simply a collection or network of production rules. The expert system processes the rules to make deductions and to determine what additional information it needs, i.e. what questions to ask, using human-readable symbols. For example, OPS5, CLIPS and their successors Jess and Drools operate in this fashion.

How does Neuro-Symbolic AI enhance traditional Symbolic AI ?

Autonomous weapons systems are weapons that can select and engage targets without human intervention [80]. While these systems are not yet widely deployed in real-world combat situations, these technologies have the potential to revolutionize warfare and defense. Autonomous weapons systems can be classified into the following two general categories. The key innovation underlying AlphaGeometry is its “neuro-symbolic” architecture integrating neural learning components and formal symbolic deduction engines.

It focuses on a narrow definition of intelligence as abstract reasoning, while artificial neural networks focus on the ability to recognize pattern. For example, NLP systems that use grammars to parse language are based on Symbolic AI systems. In conclusion, this paper highlights the transformative potential of Neuro-Symbolic AI for military applications. However, the careful development and deployment of Neuro-Symbolic AI require careful consideration of ethical issues, including data privacy, AI decision explainability, and potential unintended consequences of autonomous systems. Creating symbolic representations that accurately capture the complexities of real-world battlefield scenarios and their ethical implications is a challenging task [134, 106].

New deep learning approaches based on Transformer models have now eclipsed these earlier symbolic AI approaches and attained state-of-the-art performance in natural language processing. However, Transformer models are opaque and do not yet produce human-interpretable semantic representations for sentences and documents. Instead, they produce task-specific vectors where the meaning of the vector components is opaque. Parsing, tokenizing, spelling correction, part-of-speech tagging, noun and verb phrase chunking are all aspects of natural language processing long handled by symbolic AI, but since improved by deep learning approaches.

However, this also required much manual effort from experts tasked with deciphering the chain of thought processes that connect various symptoms to diseases or purchasing patterns to fraud. This downside is not a big issue with deciphering the meaning of children’s stories or linking common knowledge, but it becomes more expensive with specialized knowledge. Neural networks and other statistical techniques excel when there is a lot of pre-labeled data, such as whether a cat is in a video.

Applications of Symbolic AI

However, comprehensive testing and verification remain challenging due to the inherent complexity of military AI systems and their potential for unexpected emergent behaviors [154]. Recent advancements in Neuro-Symbolic AI have highlighted the importance of robust Verification and Validation (V&V) methods, Testing and Evaluations (T&E) processes. Renkhoff et al. [155] provide a comprehensive survey of the state-of-the-art symbolic ai vs neural networks techniques in Neuro-Symbolic T&E. Through the seamless integration of AI, particularly Neuro-Symbolic AI, military commanders gain immediate access to real-time data analysis and strategic understanding, enabling more informed and adaptable decision-making on complex battlefields [102]. Expert knowledge can be encoded into AI systems to assist military commanders in strategic planning [103].

For almost any type of programming outside of statistical learning algorithms, symbolic processing is used; consequently, it is in some way a necessary part of every AI system. Indeed, Seddiqi said he finds it’s often easier to program a few logical rules to implement some function than to deduce them with machine learning. It is also usually the case that the data needed to train a machine learning model either doesn’t exist or is insufficient. In those cases, rules derived from domain knowledge can help generate training data. An LNN consists of a neural network trained to perform symbolic reasoning tasks, such as logical inference, theorem proving, and planning, using a combination of differentiable logic gates and differentiable inference rules. These gates and rules are designed to mimic the operations performed by symbolic reasoning systems and are trained using gradient-based optimization techniques.

Today, many AI systems combine symbolic reasoning with machine learning techniques in a hybrid approach known as neurosymbolic AI. Both symbolic and neural network approaches date back to the earliest days of AI in the 1950s. On the symbolic side, the Logic Theorist program in 1956 helped solve simple theorems. The Perceptron algorithm in 1958 could recognize simple patterns on the neural network side. However, neural networks fell out of favor in 1969 after AI pioneers Marvin Minsky and Seymour Papert published a paper criticizing their ability to learn and solve complex problems. Hadayat Seddiqi, director of machine learning at InCloudCounsel, a legal technology company, said the time is right for developing a neuro-symbolic learning approach.

Meanwhile, many of the recent breakthroughs have been in the realm of “Weak AI” — devising AI systems that can solve a specific problem perfectly. But of late, there has been a groundswell of activity around combining the Symbolic AI approach with Deep Learning in University labs. And, the theory is being revisited by Murray Shanahan, Professor of Cognitive Robotics Imperial College London and a Senior Research Scientist at DeepMind. Shanahan reportedly proposes to apply the symbolic approach and combine it with deep learning. This would provide the AI systems a way to understand the concepts of the world, rather than just feeding it data and waiting for it to understand patterns. Shanahan hopes, revisiting the old research could lead to a potential breakthrough in AI, just like Deep Learning was resurrected by AI academicians.

Future innovations will require exploring and finding better ways to represent all of these to improve their use by symbolic and neural network algorithms. Some proponents have suggested that if we set up big enough neural networks and features, we might develop AI that meets or exceeds human intelligence. However, others, such as anesthesiologist Stuart Hameroff and physicist Roger Penrose, note that these models don’t necessarily capture the complexity of intelligence that might result from quantum effects in biological neurons. A research paper from University of Missouri-Columbia cites the computation in these models is based on explicit representations that contain symbols put together in a specific way and aggregate information.

In dynamic battlefield environments, accurately identifying combatants and non-combatants is a complex challenge [135]. Ensuring compliance with international humanitarian law and minimizing the risk of civilian casualties are important concerns [110]. Autonomous systems face challenges in low-light conditions, where cameras and advanced sensors may struggle, and radar may misinterpret objects, leading to potential misidentification and harm to civilians [135]. Furthermore, the use of ML algorithms trained on biased data introduces the risk of perpetuating discriminatory targeting patterns [136, 127]. For example, an algorithm trained on data identifying combatants with specific ethnicities or clothing styles may erroneously target individuals with similar appearances, regardless of their actual involvement in the conflict. Enhancing target discrimination in diverse conditions can be achieved through advanced sensors and multispectral imaging, coupled with training ML algorithms on unbiased and varied datasets [136, 135, 127].

This not only improves mission success and reduces collateral damage but also protects soldiers by enhancing potential threat and opportunity identification. By empowering commanders to track troop movements in real-time, analyze communication patterns, and anticipate enemy actions, AI contributes to a better understanding of the situation, ultimately leading to superior tactical choices. However, as imagined by Bengio, such a direct neural-symbolic correspondence was insurmountably limited to the aforementioned propositional logic setting. You can foun additiona information about ai customer service and artificial intelligence and NLP. Lacking the ability to model complex real-life problems involving abstract knowledge with relational logic representations (explained in our previous article), the research in propositional neural-symbolic integration remained a small niche. The concept of neural networks (as they were called before the deep learning “rebranding”) has actually been around, with various ups and downs, for a few decades already.

Alternatively, in complex perception problems, the set of rules needed may be too large for the AI system to handle. These soft reads and writes form a bottleneck when implemented in the conventional von Neumann architectures (e.g., CPUs and GPUs), especially for AI models demanding over millions of memory entries. Thanks to the high-dimensional geometry of our resulting vectors, their real-valued components can be approximated by binary, or bipolar components, taking up less storage. More importantly, this opens the door for efficient realization using analog in-memory computing. Ms. Dulari Bhatt is Assistant Professor (Big Data Analytics) at Adani Institute of Digital Technology Management (AIDTM). Her main research interests are in the field of Big Data Analytics, Computer Vision, Machine Learning, and Deep Learning.

Additionally, fostering diplomatic efforts to promote transparency and cooperation among nations regarding developing and deploying autonomous weapons can further mitigate this risk [79]. In the late 1980s and 1990s, symbolic AI began to lose ground to new AI paradigms, particularly connectionism (the basis of neural networks). The rise of machine learning, particularly deep learning, provided a more dynamic way of creating intelligent systems capable of processing vast amounts of unstructured data and learning from experience. These systems could recognize patterns in images, sounds, and other forms of data, something symbolic AI struggled with. In neural networks, the statistical processing is widely distributed across numerous neurons and interconnections, which increases the effectiveness of correlating and distilling subtle patterns in large data sets. On the other hand, neural networks tend to be slower and require more memory and computation to train and run than other types of machine learning and symbolic AI.

By proactively identifying potential issues in advance, organizations can reduce downtime, minimize unexpected maintenance costs, and optimize their maintenance schedules [99]. These old-school parallels between individual neurons and logical connectives might seem outlandish in the modern context of deep learning. The idea was based on the, now commonly exemplified, fact that logical connectives of conjunction and disjunction can be easily encoded by binary threshold units with weights — i.e., the perceptron, an elegant learning algorithm for which was introduced shortly. However, given the aforementioned recent evolution of the neural/deep learning concept, the NSI field is now gaining more momentum than ever. Symbolic AI has found applications in legal technology, where rule-based systems are used to interpret and process legal texts.

Neuro Symbolic AI is an interdisciplinary field that combines neural networks, which are a part of deep learning, with symbolic reasoning techniques. It aims to bridge the gap between symbolic reasoning and statistical learning by integrating the strengths of both approaches. This hybrid approach enables machines to reason symbolically while also leveraging the powerful pattern recognition capabilities of neural networks. LAWS are a class of autonomous weapons systems capable of independently identifying, targeting, and engaging adversaries without direct human control or intervention [80, 81]. These systems rely on a combination of sensor data, AI algorithms, and pre-programmed rules to make decisions [82].

Extensions to first-order logic include temporal logic, to handle time; epistemic logic, to reason about agent knowledge; modal logic, to handle possibility and necessity; and probabilistic logics to handle logic and probability together. Semantic networks, conceptual graphs, frames, and logic are all approaches to modeling knowledge such as domain knowledge, problem-solving knowledge, and the semantic meaning of language. DOLCE is an example of an upper ontology that can be used for any domain while WordNet is a lexical resource that can also be viewed as an ontology.

Neural-Symbolic Integration

Artificial Intelligence (AI) plays a significant role in enhancing the capabilities of defense systems, revolutionizing strategic decision-making, and shaping the future landscape of military operations. Neuro-Symbolic AI is an emerging approach that leverages and augments the strengths of neural networks and symbolic reasoning. These systems have the potential to be more impactful and flexible than traditional AI systems, making them well-suited for military applications. This paper comprehensively explores the diverse dimensions and capabilities of Neuro-Symbolic AI, aiming to shed light on its potential applications in military contexts. We investigate its capacity to improve decision-making, automate complex intelligence analysis, and strengthen autonomous systems.

Autonomy in military weapons systems refers to the ability of a weapon system, such as vehicles and drones, to operate and make decisions with some degree of independence from human intervention [79]. This involves the use of advanced technologies, often including AI, robotics, and ML, to enable military weapons to perceive, analyze, plan, and execute actions in a dynamic and complex environment. One of the most significant ways in which AI is changing the world in military settings is by enabling the development of autonomous weapons systems [10].

Researchers investigated a more data-driven strategy to address these problems, which gave rise to neural networks’ appeal. While symbolic AI requires constant information input, neural networks could train on their own given a large enough dataset. Although everything was functioning perfectly, as was already noted, a better system is required due to the difficulty in interpreting the model and the amount of data required to continue learning. Knowledge-based systems have an explicit knowledge base, typically of rules, to enhance reusability across domains by separating procedural code and domain knowledge. A separate inference engine processes rules and adds, deletes, or modifies a knowledge store.

Employing ensemble methods further enhances robustness and makes it challenging for attackers to craft effective adversarial inputs [142]. The training data used for Neuro-Symbolic AI models may contain biases, and these biases can be perpetuated in decision-making. This raises ethical concerns related to fairness, equity, and the potential for discriminatory actions, particularly in sensitive military operations [126]. Hence, ensuring that Neuro-Symbolic AI systems are free from bias potentially leading to discriminatory targeting is essential, especially in complex situations where decisions may impact diverse populations [127]. Implementing bias mitigation techniques during the training and deployment of AI models to ensure fairness and equity is crucial [127].

To better simulate how the human brain makes decisions, we’ve combined the strengths of symbolic AI and neural networks. Deep learning fails to extract compositional and causal structures from data, even though it excels in large-scale pattern recognition. While symbolic models aim for complicated Chat GPT connections, they are good at capturing compositional and causal structures. In contrast, a multi-agent system consists of multiple agents that communicate amongst themselves with some inter-agent communication language such as Knowledge Query and Manipulation Language (KQML).

Other work utilizes structured background knowledge for improving coherence and consistency in neural sequence models. In conclusion, neuro-symbolic AI is a promising field that aims to integrate the strengths of both neural networks and symbolic reasoning to form a hybrid architecture capable of performing a wider range of tasks than either component alone. With its combination of deep learning and logical inference, neuro-symbolic AI has the potential to revolutionize the way we interact with and understand AI systems. The Defense Advanced Research Projects Agency (DARPA) is funding the ANSR research program aimed at developing hybrid AI algorithms that integrate symbolic reasoning with data-driven learning to create robust, assured, and trustworthy systems [31]. Although the ANSR program is still in its early stages, we believe that it has the potential to revolutionize the application of AI use in military operations.

The second reason is tied to the field of AI and is based on the observation that neural and symbolic approaches to AI complement each other with respect to their strengths and weaknesses. For example, deep learning systems are trainable from raw data and are robust against outliers or errors in the base data, while symbolic systems are brittle with respect to outliers and data errors, and are far less trainable. It is therefore natural to ask how neural and symbolic approaches can be combined or even unified in order to overcome the weaknesses of either approach.

The greatest promise here is analogous to experimental particle physics, where large particle accelerators are built to crash atoms together and monitor their behaviors. In natural language processing, researchers have built large models with massive amounts of data using deep neural networks that cost millions of dollars to train. The next step lies in studying the networks to see how this can improve the construction of symbolic representations https://chat.openai.com/ required for higher order language tasks. The power of neural networks is that they help automate the process of generating models of the world. This has led to several significant milestones in artificial intelligence, giving rise to deep learning models that, for example, could beat humans in progressively complex games, including Go and StarCraft. But it can be challenging to reuse these deep learning models or extend them to new domains.

By the mid-1960s neither useful natural language translation systems nor autonomous tanks had been created, and a dramatic backlash set in. Autonomous weapons systems are considered a promising new technology with the potential to revolutionize warfare [108]. However, the development of autonomous weapons systems is raising several ethical and legal concerns [79, 87, 88]. For example, there is a concern that LAWS could be used to carry out indiscriminate attacks [79]. Furthermore, there is a growing fear that the development of LAWS could lead to a new arms race, as countries compete to develop the most advanced autonomous weapons systems [109].

The hybrid approach is gaining ground and there quite a few few research groups that are following this approach with some success. Noted academician Pedro Domingos is leveraging a combination of symbolic approach and deep learning in machine reading. Meanwhile, a paper authored by Sebastian Bader and Pascal Hitzler talks about an integrated neural-symbolic system, powered by a vision to arrive at a more powerful reasoning and learning systems for computer science applications. This line of research indicates that the theory of integrated neural-symbolic systems has reached a mature stage but has not been tested on real application data. Due to the shortcomings of these two methods, they have been combined to create neuro-symbolic AI, which is more effective than each alone. According to researchers, deep learning is expected to benefit from integrating domain knowledge and common sense reasoning provided by symbolic AI systems.

In contrast to the US, in Europe the key AI programming language during that same period was Prolog. Prolog provided a built-in store of facts and clauses that could be queried by a read-eval-print loop. The store could act as a knowledge base and the clauses could act as rules or a restricted form of logic.

• Metadata are a form of formally represented background knowledge, for example a knowledge base, a knowledge graph or other structured background knowledge, that adds further information or context to the data or system.
• Systems such as Lex Machina use rule-based logic to provide legal analytics, leveraging symbolic AI to analyze case law and predict outcomes based on historical data.
• Ensuring the reliability, safety, and ethical compliance of AI systems is important in military and defense applications.
• Deep learning is better suited for System 1 reasoning,  said Debu Chatterjee, head of AI, ML and analytics engineering at ServiceNow, referring to the paradigm developed by the psychologist Daniel Kahneman in his book Thinking Fast and Slow.

This section provides an overview of techniques and contributions in an overall context leading to many other, more detailed articles in Wikipedia. Sections on Machine Learning and Uncertain Reasoning are covered earlier in the history section. Our chemist was Carl Djerassi, inventor of the chemical behind the birth control pill, and also one of the world’s most respected mass spectrometrists. We began to add to their knowledge, inventing knowledge of engineering as we went along. Artificial intelligence software was used to enhance the grammar, flow, and readability of this article’s text.

Symbolic AI, also known as Good Old-Fashioned Artificial Intelligence (GOFAI), is a branch of artificial intelligence that uses symbols and symbolic reasoning to solve complex problems. Unlike modern machine learning techniques, which rely on data and statistical models, symbolic AI represents knowledge explicitly through symbols and rules. This approach has been foundational in the development of AI and remains relevant in various applications today. Current advances in Artificial Intelligence (AI) and Machine Learning have achieved unprecedented impact across research communities and industry. Nevertheless, concerns around trust, safety, interpretability and accountability of AI were raised by influential thinkers.

Non-symbolic AI is also known as “Connectionist AI” and the current applications are based on this approach – from Google’s automatic transition system (that looks for patterns), IBM’s Watson, Facebook’s face recognition algorithm to self-driving car technology. Language is a type of data that relies on statistical pattern matching at the lowest levels but quickly requires logical reasoning at higher levels. Pushing performance for NLP systems will likely be akin to augmenting deep neural networks with logical reasoning capabilities. According to Will Jack, CEO of Remedy, a healthcare startup, there is a momentum towards hybridizing connectionism and symbolic approaches to AI to unlock potential opportunities of achieving an intelligent system that can make decisions.

This understanding is vital to guarantee alignment with military objectives and adherence to ethical standards [93]. Neuro-Symbolic AI can be practically used in various military situations to make better decisions, analyze intelligence, and control autonomous systems [34]. It can provide more interpretable and explainable results for military decision-makers. However, it is important to consider the ethical and legal implications of using AI in the military including concerns related to transparency, accountability, and compliance with international laws and norms. This is easy to think of as a boolean circuit (neural network) sitting on top of a propositional interpretation (feature vector). However, the relational program input interpretations can no longer be thought of as independent values over a fixed (finite) number of propositions, but an unbound set of related facts that are true in the given world (a “least Herbrand model”).