in ,

Three Reasons Why Artificial Intelligence Wins With Specialization

Dr. Steven Gustafson is Noonum’s CTO and an AI scientist, passionate about solving hard problems while having fun and building great teams.

When I began learning about artificial intelligence (AI) and machine learning (ML), it was from the perspective of trying to help very smart people do challenging tasks. This is opposed to learning about an AI algorithm and trying to see how far I could push it for the sake of science (which has its own merits) and then searching for an application in the real world. Some early examples of AI systems were for fire prevention on naval ships, predicting component failure causes in jet engines and classifying reasons for claim denial by health insurance companies.

Some lessons I learned were that smart people solve these problems every day, so an AI system needs to augment their intelligence and not provide a poor replacement for it. When we hear about AI systems like chatbots giving illogical replies, large language models failing with edge cases and image-generating systems creating nonsensical outputs, I believe we are witnessing a lack of AI specialization or use-case customization. Here are three reasons why specialization is needed in real-world AI systems.

1. Experts rely upon a few high-quality, specialized sources to make inferences.

In my experience fine-tuning AI language models for finance tasks, I’ve found that many AI models are increasingly trained using massive datasets. These models represent an achievement of both engineering and mathematics, and although they show increased performance with representative datasets, they rarely achieve the same kind of performance with most industrial applications.

The sheer volume of data means it can never be thoroughly cleaned or sampled for a specific use case. For example, my main focus is on using AI to understand the material relationships between companies (what drives their performance now and in the future). We need to develop a specialized fine-tuning approach using both thousands of labeled data as well as data labeled using an “adversarial” model.

Additionally, specialization helps turn individual predictions into meaningful aggregations given different needs. For example, for my company’s financial tasks, on a given day, we rely upon over 3 billion predictions extracted from text that form a graph of relationships. The next day, we use a slightly different aggregation of approximately 3 billion predictions. We process almost 100,000 texts per day, all of which are likely talking about companies. When we aggregate these predictions, we select specific confidence thresholds and ways to infer from these relationships. If we included all possible texts about any topic, we would introduce a significant amount of noise that would require different aggregations and inferences to make them accurate and useful.

2. Experts refine their knowledge and judgment through reflection and feedback.

Recently, the New York Times (paywall) ran a story in which several of their writers looked back at a story that was wrong in hindsight and reflected on what they learned. This type of reflection is rare, but it’s precisely what leaders and experts with a “growth mindset” do on a regular basis.

When AI models are applied to a specific application, it often doesn’t take long before an edge case is detected, and the experts need to learn from it. In a simpler example, consuming news and social media can quickly lead one to realize some news sources or applications are more trustworthy or credible than others. In this case, filtering out those low-quality or low-integrity sites is required, just like an expert reflects on their experience and updates their knowledge for future cases.

While working with thousands of possible news sources, I realized that paying for a smaller set of a few thousand sources of higher quality should be the first step. We eventually determined that an even fewer number of sources provide unique knowledge that is often not repeated elsewhere. For these sources, we boost their importance. The processes of refining one’s knowledge and judgment should be ongoing to continue reflecting and improving.

3. Experts apply heuristics relevant to the context.

A classic issue in AI systems that involve natural language processing (NLP) is finding entities (also called “named entity recognition” or NER) in text and then linking them to a specific representation in the AI system. For example, in text, we might see someone mention a person named “Apple,” a local business named “Apple,” a city named “Apple” and perhaps an international company named “Apple, Inc.” Today, we have pretty good ML models that can recognize those names of entities and realize that “Apple” is typically recognized as an organization. However, in our system, like a knowledge graph, all the aforementioned name variants could be independent entities. So, how do we “link” the entities in the text to the entities in our graph?

Specialization requires knowledge of the domain, data sources and decision contexts. Because large datasets contain many contradictions, even large language models trained to do NER will still benefit from additional training using the target AI application data sources it will later predict on. Aside from that challenge, almost all AI model outputs still require a significant amount of rules to recognize contextual information to be successful.

We suggest employing custom entity linking after fine-tuning your NER models using custom-labeled datasets on top of large language models. This can significantly improve your AI pipeline in performing tasks like NER, relationship extraction and classification and entity mastering and linking to construct a knowledge graph. Would the same rules work for a different application of AI, say, for understanding diseases in healthcare? Probably not, and that is why specialization in AI applications can help them be successful.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?

Source link

What do you think?

New York City Proposes Rules to Clarify Upcoming Artificial Intelligence Law for Employers

2 Stocks to Play the Artificia