Knowledge engineering refers to all technical, scientific and social aspects involved in building, maintaining and using knowledge-based systems. Knowledge engineering is a field of study where we do the engineering of all such thought processes for specific domains.
It can be considered as the building blocks for (AI), which attempts to imitate the judgment of a human with experts in a specific domain.
The field concentrates on creating a knowledge base for a specific domain. It includes an in-depth investigation of a particular domain, learning all the important concepts about that domain, and then drafting out meaningful output. Knowledge Engineering is a way to process the information of how an expert in a specific domain would process and accordingly will act and make decisions.
In a general sense, it takes a problem to solve and then studies the factors which a human expert will consider while making a decision. A human expert will consider a number of parameters and some will be more important than others. After considering all the parameters human expert makes permutation and combinations using his prior experience with domain and give weightage to all the parameters and makes a decision. K.E. is the base of expert systems.
Knowledge engineering is not a new field, but it is one that has undergone significant changes since its original inception. Originally utilised for medical diagnosis and satellite control, today it has become an important part of a multitude of industries, which means new and exciting opportunities for AI engineers and systems and control engineers.
Knowledge engineers work to translate human expertise into what’s called a knowledge-based system that can replicate someone’s answer. These systems tackle complex, high-level problems where an industry expert would be called upon. Because it requires large amounts of information, knowledge engineering has also changed due to new technology like cloud storage systems. For example, an oncologist would search journals, textbooks and drug databases to find the right treatment for their patient, which would require a great deal of data for a program store. In addition to this knowledge itself, that system would require rules to decide how that medical information will be applied in different circumstances.
Knowledge engineering has been called a part of the fourth industrial revolution. It’s already being used in industries such as healthcare, customer service, financial services, manufacturing and law. These AI engineers are also responsible for technologies such as facial recognition and programs that understand human speech as well as product development like Amazon’s Alexa.
Knowledge engineering can increase the speed of decision-making for an organisation. More importantly, it has the potential to develop better solutions to more challenging problems.
Companies need to be able to handle larger and larger amounts of information at faster speeds. They can use machine learning and algorithms to help identify ways to improve productivity and quality, but these systems eventually need humans to take over the decision-making process. These are the problems that need an expert to solve. Having a system that can replicate that process can help reduce costs and make it so that knowledge is more readily available throughout an organisation, being used in different ways for different teams.
A major challenge is that systems need to be able to adapt to unpredictability. Data is constantly changing. Some data is difficult to understand or explain, while other information is relatively straightforward. Often multiple experts are needed to address an issue. Another challenge is that experts don’t always communicate the same way; they might express themselves verbally, through visualisations or by demonstration.
In addition to requiring an understanding of AI and machine learning, knowledge engineering also needs an understanding of human behavior and computer programming. Knowledge engineers are responsible for connecting AI with the experts, whether they’re in business, science or medicine. As the facilitator of an expert’s knowledge to the final product, a knowledge engineer is someone who builds and maintains personal connections, so having the communication skills and soft skills to work alongside the experts is critical.
One of the biggest barriers to knowledge engineering is known as “collateral knowledge.” This is information that may not be considered immediately relevant to solving the program but is still needed to make a final decision. Collateral knowledge is not particularly definitive or clear cut. Humans’ nonlinear thought process makes replacing the expert themselves extremely difficult, if not entirely impossible.
People don’t use linear lines of thought when making a decision, so replicating their thought process is extremely difficult. Knowledge engineers realised that they couldn’t duplicate intuition or gut instincts. Often, solutions to problems are gained through previous experiences or learning from past mistakes. For many organisations, it’s not worth the cost or storage needed to create a system smart enough to handle all of that data.
Our knowledge engineers pursue a different approach. They build systems that reach the same answer but don’t necessarily use the same information or logic a human does. Essentially, they leave our nonlinear thinking behind. They cannot replicate human thought, but systems can learn to reach similar conclusions as the experts. When the two don’t match, the knowledge engineers can go in and update the system. The model might get more complex, to the point where even the engineer isn’t sure how it’s reaching its correct answer. The eventual goal is that the system reaches the point where it is even smarter than the expert.
Knowledge acquisition is the process used to define the rules and ontologies required for a knowledge-based system. The phrase was first used in conjunction with expert systems to describe the initial tasks associated with developing an expert system, namely finding and interviewing domain experts and capturing their knowledge via rules, objects, and frame-based ontologies.
Expert systems were one of the first successful applications of artificial intelligence technology to real world business problems. Researchers at Stanford and other AI laboratories worked with doctors and other highly skilled experts to develop systems that could automate complex tasks such as medical diagnosis. Until this point computers had mostly been used to automate highly data intensive tasks but not for complex reasoning. Technologies such as inference engines allowed developers for the first time to tackle more complex problems.
As expert systems scaled up from demonstration prototypes to industrial strength applications it was soon realised that the acquisition of domain expert knowledge was one of if not the most critical task in the knowledge engineering process. This knowledge acquisition process became an intense area of research on its own. One of the earlier works on the topic used Batesonian theories of learning to guide the process.
One approach to knowledge acquisition investigated was to use natural language parsing and generation to facilitate knowledge acquisition. Natural language parsing could be performed on manuals and other expert documents and an initial first pass at the rules and objects could be developed automatically. Text generation was also extremely useful in generating explanations for system behavior. This greatly facilitated the development and maintenance of expert systems.
A more recent approach to knowledge acquisition is a re-use based approach. Knowledge can be developed in ontologies that conform to standards such as the Web Ontology Language (OWL). In this way knowledge can be standardised and shared across a broad community of knowledge workers. One example domain where this approach has been successful is bioinformatics.