Digital Commons Network^™

How To Best Retrain A Neural Network If We Added One More Input Variable, Saeid Tizpaz-Niari, Vladik Kreinovich

Departmental Technical Reports (CS)

Often, once we have trained a neural network to estimate the value of a quantity y based on the available values of inputs x1, ..., xn, we learn to measure the values of an additional quantity that have some influence on y. In such situations, it is desirable to re-train the neural network, so that it will be able to take this extra value into account. A straightforward idea is to add a new input to the first layer and to update all the weights based on the patterns that include the values of the new input. The problem with …

Why Bump Reward Function Works Well In Training Insulin Delivery Systems, Lehel Dénes-Fazakas, Lásló Szilágyi, Gyorgy Eigner, Olga Kosheleva, Vladik Kreinovich, Nguyen Hoang Phuong

Departmental Technical Reports (CS)

Diabetes is a disease when the body can no longer properly regulate blood glucose level, which can lead to life-threatening situations. To avoid such situations and regulate blood glucose level, patients with severe form of diabetes need insulin injections. Ideally, the system should automatically decide when best to inject insulin and how much to inject. To find the optimal control, researchers applied machine learning with different reward functions. It turns out that the most effective learning occurred when they used the so-called bump function. In this paper, we provide a possible explanation for this empirical result.

Fuzzy Techniques Explain The Effectiveness Of Relu Activation Function In Deep Learning, Julio C. Urenda, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In the last decades, deep learning has led to spectacular successes. One of the reasons for these successes was the fact that deep neural networks use a special Rectified Linear Unit (ReLU) activation function s(x) = max(0,x). Why this activation function is so successful is largely a mystery. In this paper, we show that common sense ideas -- as formalized by fuzzy logic -- can explain this mysterious effectiveness.

Natural Color Interpretation Of Interval-Valued Fuzzy Degrees, Victor L. Timchenko, Yury P. Kondratenko, Vladik Kreinovich, Olga Kosheleva

Departmental Technical Reports (CS)

Intuitively, interval-values fuzzy degrees are more adequate for representing expert uncertainty than the traditional [0,1]-based ones. Indeed, the very need for fuzzy degrees comes from the fact that experts often cannot describe their opinion not in terms of precise numbers, but by using imprecise ("fuzzy") words from natural language like "small". In such situations, it is strange to expect the same expert to be able to provide an exact number describing his/her degree of certainty; it is more natural to ask this expert to mark the whole interval (or even, more generally, a fuzzy set of possible degrees). In spite …

Logical Inference Inevitably Appears: Fuzzy-Based Explanation, Julio C. Urenda, Olga Kosheleva, Vladik Kreinovich, Orsolya Csiszar

Departmental Technical Reports (CS)

Many thousands years ago, our primitive ancestors did not have the ability to reason logically and to perform logical inference. This ability appeared later. A natural question is: was this appearance inevitable -- or was this a lucky incident that could have been missed? In this paper, we use fuzzy techniques to provide a possible answer to this question. Our answer is: yes, the appearance of logical inference in inevitable.

Why Softmax? Because It Is The Only Consistent Approach To Probability-Based Classification, Anatole Lokshin, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical problems, the most effective classification techniques are based on deep learning. In this approach, once the neural network generates values corresponding to different classes, these values are transformed into probabilities by using the softmax formula. Researchers tried other transformation, but they did not work as well as softmax. A natural question is: why is softmax so effective? In this paper, we provide a possible explanation for this effectiveness: namely, we prove that softmax is the only consistent approach to probability-based classification. In precise terms, it is the only approach for which two reasonable probability-based ideas -- Least …

Is Fully Explainable Ai Even Possible: Fuzzy-Based Analysis, Miroslav Svitek, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the main limitations of many current AI-based decision-making systems is that they do not provide any understandable explanations of how they came up with the produced decision. Taking into account that these systems are not perfect, that their decisions are sometimes far from good, the absence of an explanation makes it difficult to separate good decisions from suspicious ones. Because of this, many researchers are working on making AI explainable. In some applications areas -- e.g., in chess -- practitioners get an impression that there is a limit to understandability, that some decisions remain inhuman -- not explainable. …

Which Activation Function Works Best For Training Artificial Pancreas: Empirical Fact And Its Theoretical Explanation, Lehel Dénes-Fazakas, Lásló Szilágyi, György Eigner, Olga Kosheleva, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the most effective ways to help patients at the dangerous levels of diabetes is an artificial pancreas, a device that constantly monitors the patient's blood sugar level and injects insulin based on this level. Patient's reaction to insulin is highly individualized, so the artificial pancreas needs to be trained on each patient. It turns out that the best training results are attained when instead of the usual ReLU neurons, we use their minor modification known as Exponential Linear Units (ELU). In this paper, we provide a theoretical explanation for the empirically observed effectiveness of ELUs.

Why Fuzzy Control Is Often More Robust (And Smoother): A Theoretical Explanation, Orsolya Csiszar, Gábor Csiszar, Olga Kosheleva, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, practitioners use easier-to-compute fuzzy control to approximate the more-difficult-co-compute optimal control. As expected, for many characteristics, this approximate control is slightly worse than the optimal control it approximates, However, with respect to robustness or smoothness, the approximating fuzzy control is often better than the original one. In this paper, we provide a theoretical explanation for this somewhat mysterious empirical phenomenon.

Dialogs Re-Enacted Across Languages, Version 2, Nigel G. Ward, Jonathan E. Avila, Emilia Rivas, Divette Marco

Departmental Technical Reports (CS)

To support machine learning of cross-language prosodic mappings and other ways to improve speech-to-speech translation, we present a protocol for collecting closely matched pairs of utterances across languages, a description of the resulting data collection and its public release, and some observations and musings. This report is intended for:

• people using this corpus
• people extending this corpus
• people designing similar collections of bilingual dialog data.

Change Notes. This version supersedes UTEP-CS-22-108. There is some new information and numerous clarifications, mostly arising from our experiences diversifying our corpus and helping a vendor to use this protocol.

Selecting The Most Adequate Fuzzy Operation For Explainable Ai: Empirical Fact And Its Possible Theoretical Explanation, Orsolya Csiszar, Gábor Csiszar, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

A reasonable way to make AI results explainable is to approximate the corresponding deep-learning-generated function by a simple expression formed by fuzzy operations. Experiments on real data show that out of all easy-to-compute fuzzy operations, the best approximation is attained if we use an operation a + b − 0.5 ( limited to the interval [0,1]$. In this paper, we provide a possible theoretical explanation for this empirical result.

Why Inverse Layers In Pavement? Why Zipper Fracking? Why Interleaving In Education? A General Explanation, Edgar Daniel Rodriguez Velasquez, Aaron Velasco, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, if we split our efforts into two disconnected chunks, we get better results: a pavement is stronger if instead of a single strengthening layer, we place two parts of this layer separated by no-so-strong layers; teaching is more effective if instead of concentrating a topic in a single time interval, we split it into two parts separated in time, etc. In this paper, we provide a general explanation for all these phenomena.

Fast -- Asymptotically Optimal -- Methods For Determining The Optimal Number Of Features, Saied Tizpaz-Niari, Luc Longpré, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In machine learning -- and in data processing in general -- it is very important to select the proper number of features. If we select too few, we miss important information and do not get good results, but if we select too many, this will include many irrelevant ones that only bring noise and thus again worsen the results. The usual method of selecting the proper number of features is to add features one by one until the quality stops improving and starts deteriorating again. This method works, but it often takes too much time. In this paper, we propose …

Causality: Hypergraphs, Matter Of Degree, Foundations Of Cosmology, Cliff Joslyn, Andres Ortiz-Muñoz, Edgar Daniel Rodriguez Velasquez, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

The notion of causality is very important in many applications areas. Because of this importance, there are several formalizations of this notion in physics and in AI. Most of these definitions describe causality as a crisp ("yes"-"no") relation between two events or two processes -- cause and effect. However, such descriptions do not fully capture the intuitive idea of causality: first, often, several conditions are needed to be present for an effect to occur, and, second, the effect is often a matter of degree. In this paper, we show how to modify the current description of causality so as to …

Everything Is A Matter Of Degree: The Main Idea Behind Fuzzy Logic Is Useful In Geosciences And In Authorship, Christian Servin, Aaron Velasco, Edgar Daniel Rodriguez Velasquez, Vladik Kreinovich

Departmental Technical Reports (CS)

This paper presents two applications of the general principle -- the everything is a matter of degree -- the principle that underlies fuzzy techniques. The first -- qualitative -- application helps explain the fact that while most earthquakes occur close to faults (borders between tectonic plates or terranes), earthquakes have also been observed in areas which are far away from the known faults. The second -- more quantitative -- application is to the problem of which of the collaborators should be listed as authors and which should be simply thanked in the paper. We argue that the best answer to …

Foundations Of Neural Networks Explain The Empirical Success Of The "Surrogate" Approach To Ordinal Regression -- And Recommend What Next, Salvador Robles, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

Recently, a new efficient semi-heuristic statistical method -- called Surrogate Approach -- has been proposed for dealing with regression problems. How can we explain this empirical success? And since this method is only an approximation to reality, what can we recommend if there is a need for a more accurate approximation? In this paper, we show that this empirical success can be explained by the same arguments that explain the empirical success of neural networks -- and these arguments can also provide us with possible more general techniques (that will hopefully lead to more accurate approximation to real-life phenomena).

Towards Decision Making Under Interval Uncertainty, Juan A. Lopez, Vladik Kreinovich

Departmental Technical Reports (CS)

In many real-life situations, we need to make a decision. In many cases, we know the optimal decision in situations when we know the exact value of the corresponding quantity x. However, often, we do not know the exact value of this quantity, we only know the bounds on the value x -- i.e., we know the interval containing $x$. In this case, we need to select a decision corresponding to some value from this interval. The selected value will, in general, be different from the actual (unknown) value of this quantity. As a result, the quality of our decision …

Conflict Situations Are Inevitable When There Are Many Participants: A Proof Based On The Analysis Of Aumann-Shapley Value, Sofia Holguin, Vladik Kreinovich

Departmental Technical Reports (CS)

When collaboration of several people results in a business success, an important issue is how to fairly divide the gain between the participants. In principle, the solution to this problem is known since the 1950s: natural fairness requirements lead to the so-called Shapley value. However, the computation of Shapley value requires that we can estimate, for each subset of the set of all participants, how much gain they would have gained if they worked together without others. It is possible to perform such estimates when we have a small group of participants, but for a big company with thousands of …

Integrity First, Service Before Self, And Excellence: Core Values Of Us Air Force Naturally Follow From Decision Theory, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

By analyzing data both from peace time and from war time, the US Air Force came with three principles that determine success: integrity, service before self, and excellent. We show that these three principles naturally follow from decision theory, a theory that describes how a rational person should make decisions.

People Prefer More Information About Uncertainty, But Perform Worse When Given This Information: An Explanation Of The Paradoxical Phenomenon, Jieqiong Zhao, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In a recent experiment, decision makers were asked whether they would prefer having more information about the corresponding situation. They confirmed this preference, and such information was provided to them. However, strangely, the decisions of those who received this information were worse than the decisions of the control group -- that did not get this information. In this paper, we provide an explanation for this paradoxical situation.

Low-Probability High-Impact Events Are Even More Important Than It Is Usually Assumed, Aaron Velasco, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

A large proportion of undesirable events like earthquakes, floods, tornados occur in zones where these events are frequent. However, a significant number of such events occur in other zones, where such events are rare. For example, while most major earthquakes occur in a vicinity of major faults, i.e., on the border between two tectonic plates, some strong earthquakes also occur inside plates. We want to mitigate all undesirable events, but our resources are limited. So, to allocate these resources, we need to decide which ones are more important. For this decision, a natural idea is to use the product of …

Wormholes, Superfast Computations, And Selivanov's Theorem, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

While modern computers are fast, there are still many practical problems that require even faster computers. It turns out that on the fundamental level, one of the main factors limiting computation speed is the fact that, according to modern physics, the speed of all processes is limited by the speed of light. Good news is that while the corresponding limitation is very severe in Euclidean geometry, it can be more relaxed in (at least some) non-Euclidean spaces, and, according to modern physics, the physical space is not Euclidean. The differences from Euclidean character are especially large on micro-level, where quantum …

How People Make Decisions Based On Prior Experience: Formulas Of Instance-Based Learning Theory (Ilbt) Follow From Scale Invariance, Palvi Aggarwal, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

To better understand human behavior, we need to understand how people make decisions, how people select one of possible actions. This selection is usually based on predicting consequences of different actions, and these predictions are, in their turn, based on the past experience. For example, consequences that occur more frequently in the past are viewed as more probable. However, this is not just about frequency: recent observations are usually given more weight that past ones. Researchers have discovered semi-empirical formulas that describe our predictions reasonably well; these formulas form the basis of the Instance-Based Learning Theory (ILBT). In this paper, …

What Do Goedel's Theorem And Arrow's Theorem Have In Common: A Possible Answer To Arrow's Question, Miroslav Svitek, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Kenneth Arrow, the renowned author of the Impossibility Theorem that explains the difficulty of group decision making, noticed that there is some commonsense similarity between his result and Goedel's theorem about incompleteness of axiomatic systems. Arrow asked if it is possible to describe this similarity in more precise terms. In this paper, we make the first step towards this description. We show that in both cases, the impossibility result disappears if we take into account probabilities. Namely, we take into account that we can consider probabilistic situations, that we can make probabilistic conclusions, and that we can make probabilistic decisions …

Why Gliding Symmetry Used To Be Prevalent In Biology But Practically Disappeared, Julio C. Urenda, Vladik Kreinovich

Departmental Technical Reports (CS)

At present, many living creatures have symmetries; in particular, the left-right symmetry is ubiquitous. Interestingly, 600 million years ago, very fee living creatures had the left-right symmetry: most of them had a gliding symmetry, symmetry with respect to shift along a line followed by reflection in this line. This symmetry is really seen in living creatures today. In this paper, we provide a physical-based geometric explanation for this symmetry change: we explain both why gliding symmetry was ubiquitous, and why at present, it is rarely observed, while the left-right symmetry is prevalent.

The World Is Cognizable: An Argument Based On Hoermander's Theorem, Miroslav Svitek, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Is the world cognizable? Is it, in principle, possible to predict the future state of the world based on the measurements and observations performed in a local area -- e.g., in the Solar system? In this paper, we use general physicists' principles and a mathematical theorem about partial differential equations to show that such prediction is indeed, theoretically possible.

Success (Studying Underlying Characteristics Of Computing And Engineering Student Success) Survey: Non-Cognitive And Affective Profiles In Engineering And Computing Students At Utep (2018-2022), Sanga Kim, Christian Teran Lopez, Andres Segura, Gabriel Miki

Departmental Technical Reports (CS)

No abstract provided.

Designing An Optimal Medicine Cocktail Is Np-Hard, Luc Longpre, Vladik Kreinovich

Departmental Technical Reports (CS)

In many cases, a combination of different drugs -- known as a medicine cocktail -- is more effective against a disease than each individual drug. It is desirable to find the most effective cocktail. This problem can be naturally formulated as a problem of maximizing a quadratic expression under the condition that all the unknowns (concentrations of different medicines) are non-negative. At first glance, it may seem that this problem is feasible -- since a similar economic problem of finding the optimal investment portfolio is known to be feasible. However, it turns out that the cocktail problem is different: it …

Interval-Valued And Set-Valued Extensions Of Discrete Fuzzy Logics, Belnap Logic, And Color Optical Computing, Victor L. Timchenko, Yury P. Kondratenko, Vladik Kreinovich

Departmental Technical Reports (CS)

It has been recently shown that in some applications, e.g., in ship navigation near a harbor, it is convenient to use combinations of basic colors -- red, green, and blue -- to represent different fuzzy degrees. In this paper, we provide a natural explanation for the efficiency of this empirical fact: namely, we show that it is reasonable to consider discrete fuzzy logics, it is reasonable to consider their interval-valued and set-valued extensions, and that a set-valued extension of the 3-values logic is naturally equivalent to the use of color combinations.

Why Fractional Fuzzy, Mehran Mazandarani, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situation, control experts can only formulate their experience by using imprecise ("fuzzy") words from natural language. To incorporate this knowledge in automatic controllers, Lotfi Zadeh came up with a methodology that translate the informal expert statements into a precise control strategy. This methodology -- and its following modifications -- is known as fuzzy control. Fuzzy control often leads to a reasonable control -- and we can get an even better control results by tuning the resulting control strategy on the actual system. There are many parameters that can be changes during tuning, so tuning usually is rather …