What should an advanced, state-of-the-art information product look like to help businesses with information processing, including big data analytics? How autonomous from humans and how “smart” can such information systems be? Is it possible to use them without understanding anything in programming? These and other questions were answered by Olga Vedernikova, a graduate of MEPhI, the initiator of the Epsilon Workspace project (University Technological Accelerator of MEPhI and Rosatom).
What a modern business assistant should look like
My husband and I have worked all our lives in IT and Data Science divisions of different corporations, all the time facing tasks, where there was necessarily a part related to data analysis. At first, these were ordinary data and relational models, which we were taught at the institute. Then big data and tasks for their processing, analysis and predictive analytics began to appear. We have an understanding of what products and tools are needed in order to collect and process big data.
And then technologies related to artificial intelligence (AI) appeared, and thanks to them it became possible to collect, process and analyze big data without programmers.
I am sure that a modern product for a data processing business should look to the user like a set of visual elements, each of which is responsible for a particular operation on data, for example: combining, filtering, sorting, extracting, removing duplicates, and so on, starting with the simplest operations and ending with rather complex ones, such as machine learning models that can make a prediction based on this data, or some conclusions, or even make a recommendation. And all data analysis tasks are solved through building visual elements, that is, you simply “pull” different cubes on the screen with the cursor, connect them with each other and at each stage you see an intermediate result.
From these “cubes” you can design AI agents, assistants who help in business. It is very important that these are No-Code systems, that is, they do not require coding and programming knowledge.
There are different agents
The typical roles of AI agents have already been determined.
The first type is agents who are very good at working with unstructured information, such as documents scattered around, multi-page tables, charts, and so on. That is, they know how to analyze a document and pull out all the most valuable from it. New generations of information systems have advantages over solutions of previous years, which, as a rule, do not know how to work with tables and diagrams, they understand very poorly what is written there, what it refers to, especially if the diagrams are of different colors.
Solutions of new generations can understand the diagram, even if there is no data signature on it and somewhere below there is a legend, and the meaning of the columns of the diagram must be understood by their color. Modern large visual models are able to do all this.
This is demanded very often by companies that have a lot of engineering documentation, as well as auditors and consultants who are forced to work with a large number of different unstructured and complex documents.
Agents of the second type can recognize the document and understand into which tablets the extracted data must be decomposed so that they can then work with them, and then, on the basis of them, make some kind of recommendation or conduct an analysis. Such an agent does not just pull something out of PDF files, he also puts it all in the right places and gives some meaningful analytical result at the output.
As far as fantasy allows
The list of functions that a modern data processing system can perform is not closed, because AI can help the user automate a variety of operations, starting with some simple calculations, from preparing reports and writing letters to predictive models, as well as modeling or emulation of technological processes, up to the development of proposals for the chemical composition of drugs in the pharmaceutical industry. The list of possible industry tasks that can be entrusted to such a system is, in principle, limited only by the user’s imagination and the amount of data that he can “feed” to the system.
Of course, training a system on a large array of data is a good way, but very expensive one, inaccessible to most companies, except for the largest, with a large infrastructure, computing power and the funds to purchase data sets. 5 years ago, I would say that small companies are an absolutely inappropriate audience for AI systems. But now it is small companies that are even more interested in them than medium-sized businesses, because AI agents have the opportunity to perform functions that were previously inaccessible to them, for example, web scraping has become available to them, that is, an agent can “run” to different sites and from there pull out what you write to him in promt, he can conduct business analytics and market assessment of working with a certain partner.
In addition, for small and medium-sized companies, it is possible to train the system on internal corporate data – regulations, instructions, reference materials, data accumulated during production processes. This learning method is called augmented generation. Thanks to it, the system can provide assistance in the field of internal processes of the company.
Shall we talk?
The emergence of large language models and their connection to data processing systems gave rise to the possibility of AI agents that collect and process information and at the same time communicate with the user in natural language.
A very important type of AI agent is agent consultants working in question-answer mode. The feature of such an agent is that he understands what you are asking him. For example, you can ask him: what about my account under such and such an agreement, why haven’t you paid it yet? The agent can not just recognize this text, but also understand the context, understand what specific contract is in question, and, if necessary, ask a clarifying question. And this is all done by AI, without the participation of people or scripts. He himself decides which database he needs to get into, which sources he needs to look at, and then generates a response in colloquial format.
Do not forget about the human person
And we must also mention one mandatory requirement for modern business systems – Human-on-the-loop, the ability for people to provide feedback to the AI system. It is imperative to provide for the possibility of checking the results of the AI by humans. From this another obligatory property of the modern system follows: observability, that is, transparency of what the system does so that it does not turn into a “black box” for people.