Physical Sciences and Mathematics Commons^™

Every Relu-Based Neural Network Can Be Described By A System Of Takagi-Sugeno Fuzzy Rules: A Theorem, Barnabas Bede, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

While modern deep-learning neural networks are very successful, sometimes they make mistakes, and since their results are "black boxes" -- no explanation is provided -- it is difficult to determine which recommendations are erroneous. It is therefore desirable to make the resulting computations explainable, i.e., to describe their results by using commonsense rules. In this paper, we use "fuzzy" techniques -- techniques developed by Lotfi Zadeh to deal with commonsense rules formulated by using imprecise ("fuzzy") words from natural language -- to show that such a rule-based representation is always possible. Our result does not yet provide the desired explainability, …

Smooth Non-Additive Integrals And Measures And Their Potential Applications, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we explain why non-additive integrals and measures are needed, how non-additive integrals and measures are related, how to use them in decision making, and how they can help in fundamental physics. These four topics are covered, correspondingly, in Sections 2-5 of this paper.

When Is A Single "And"-Condition Enough?, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, there are several possible decisions. Any general recommendation means specifying, for each possible decision, conditions under which this decision is recommended. In some cases, a single "and"-condition is sufficient: e.g., a condition under which a patient is recommended to take aspirin is that "the patient has a fever and the patient does not have stomach trouble". In other cases, conditions are more complicated. A natural question is: when is a single "and"-condition enough? In this paper, we provide an answer to this question.

If We Add Axiom Of Choice To Constructive Analysis, We Get Classical Arithmetic: An Exercise In Reverse Constructive Mathematics, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

A recent paper in Bulletin of Symbolic Logic reminded that the Axiom of Choice is, in general, false in constructive analysis. This result is an immediate consequence of a theorem -- first proved by Tseytin -- that every computable function is continuous. In this paper, we strengthen the result about the Axiom of Choice by proving that this axiom is as non-constructive as possible: namely, that if we add this axiom to constructive analysis, then we get full classical arithmetic.

Why Sigmoid Transformation Helps Incorporate Logic Into Deep Learning: A Theoretical Explanation, Chitta Baral, Vladik Kreinovich

Departmental Technical Reports (CS)

Traditional neural networks start from the data, they cannot easily handle prior knowledge -- this is one of the reasons why they often take very long to train. It is desirable to incorporate prior knowledge into deep learning. For the case when this knowledge consists of propositional statements, a successful way to incorporate this knowledge was proposed in a recent paper by van Krieken et al. That paper uses the fact that a neural network does not directly return a truth value, it returns a real value -- in effect, the degree of confidence in the corresponding statement -- from …

Integrating Machine Learning Methods For Medical Diagnosis, Jazmin Quezada

Open Access Theses & Dissertations

Abstract:The rapid advancement of machine learning techniques has revolutionized the field of medical diagnosis by offering powerful tools to analyze complex data sets and make accurate predictions. In this proposed method, we present a novel approach that integrates machine learning and optimization models to enhance the accuracy of medical diagnoses. Our method focuses on fine-tuning and optimizing the parameters of machine learning algorithms commonly used in medical diagnosis, such as logistic regression, support vector machines, and neural networks. By employing optimization techniques, we systematically explore the parameter space of these algorithms to discover the most optimal configurations. Moreover, by representing …

Usually, Either Left And Right Brains Are Equally Active Or Only One Of Them Is Active: First-Principles Explanation, Julio C. Urenda, Vladik Kreinovich

Departmental Technical Reports (CS)

It is known that in most practical situations, either both left and right brains are equally active, or only one of them is active. A recent paper showed that this empirical phenomenon can be explained by a realistic model of the brain effectiveness. In this paper, we show that this conclusion can be made without any specific assumptions about the brain, based on first principles.

From Type-2 Fuzzy To Type-2 Intervals And Type-2 Probabilities, Vladik Kreinovich, Olga Kosheleva, Luc Longpré

Departmental Technical Reports (CS)

Our knowledge comes from observations, measurements, and expert opinions. Measurements and observations are never 100% accurate, there is always a difference between the measurement result and the actual value of the corresponding quantity. We gauge the resulting uncertainty either by an interval of possible values, or by a probability distribution on the set of possible values, or by a membership function that describes to what extent different values are possible. The information about uncertainty also comes either from measurements or from expert estimates and is, therefore, also uncertain. It is important to take such "type-2" uncertainty into account. This is …

Which Random-Set Representation Of A Fuzzy Set Is The Simplest?, Vladik Kreinovich, Olga Kosheleva, Hung T. Nguyen

Departmental Technical Reports (CS)

One of the ways to elicit membership degrees is by polling. For example, we ask a group of people how many believe that 30 C is hot. If 8 out of ten say that it is hot, we assign the degree 8/10 to the statement "30 C is hot". In precise mathematical terms, polling can be described via so-called random sets. It is known that every fuzzy set can be obtained this way, i.e., that every fuzzy set can be represented by an appropriate random set. Moreover, it is known that for many fuzzy sets, there are several different random-set …

How To Efficiently Propagate P-Box Uncertainty, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, to get the desired estimate or prediction, we need to process existing data. This data usually comes from measurements, and measurements are never 100% accurate. Because we only know the input values with uncertainty, the results of processing this data also comes with uncertainty. To make an appropriate decision, we need to know how accurate is the resulting estimate, i.e., how the input uncertainty "propagates" through the data processing algorithm. In the ideal case, when we know the probability distribution of each measurement error, we can, in principle, use Monte-Carlo simulations to describe the uncertainty of …

Uncertainty Quantification For Results Of Ai-Based Data Processing: Towards More Feasible Algorithms, Christoph Q. Lauter, Martine Ceberio, Vladik Kreinovich, Olga Kosheleva

Departmental Technical Reports (CS)

AI techniques have been actively and successfully used in data processing. This tendency started with fuzzy techniques, now neural network techniques are actively used. With each new technique comes the need for the corresponding uncertainty quantification (UQ). In principle, for both fuzzy and neural techniques, we can use the usual UQ methods -- however, these techniques often require an unrealistic amount of computation time. In this paper, we show that in both cases, we can use specific features of the corresponding techniques to drastically speed up the corresponding computations.

Giant Footprints Of Buddha And Generalized Limits, Julio C. Urenda, Vladik Kreinovich

Departmental Technical Reports (CS)

In many places in Asia, there are footprints claimed to be left by Buddha. Many of them are much larger than the usual size of human feet, up to 150 cm and more in length. In this paper, we provide a possible mathematical explanation for such unusual sizes.

How To Deal With Inconsistent Intervals: Utility-Based Approach Can Overcome The Limitations Of The Purely Probability-Based Approach, Kittawit Autchariyapanitkul, Tomoe Entani, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many application areas, we rely on experts to estimate the numerical values of some quantities. Experts can provide not only the estimates themselves, they can also estimate the accuracies of their estimates -- i.e., in effect, they provide an interval of possible values of the quantity of interest. To get a more accurate estimate, it is reasonable to ask several experts -- and to take the intersection of the resulting intervals. In some cases, however, experts overestimate the accuracy of their estimates, their intervals are too narrow -- so narrow that they are inconsistent: their intersection is empty. In …

Why Micro-Funding? Why Small Businesses Are Important? Analysis Based On First Principles, Hein D. Tran, Edwin Tomy George, Vladik Kreinovich

Departmental Technical Reports (CS)

On the one hand, in economics, there is a well-known and well-studied economy of scale: when two smaller companies merge, it lowers their costs and thus, makes them more effective and therefore more competitive. At first glance, this advantage of big size would make economy dominated by big companies -- but in reality, small business remain a significant and important economic sector. Similarly, it is well known and well studied that research collaboration enhances researchers' productivity -- but still a significant portion of important results come from individual efforts. In several applications areas, there are area-specific explanations for this seemingly …

Local-Global Support For Earth Sciences: Economic Analysis, Uyen Hoang Pham, Aaron Velasco, Vladik Kreinovich

Departmental Technical Reports (CS)

Most funding for science comes from taxpayers. So, it is very important to be able to convince taxpayers that this funding is potentially beneficial for them. This task is easier in Earth sciences, e.g., in meteorology, where there are clear local benefits. The problem is that while many people support local studies focused on their region, they do not always have a good understanding of the fact that effective local benefits require also studying surrounding areas -- and what should be the optimal balance between local and (more) global studies. In this paper, on a (somewhat) simplified model of the …

How To Make Machine Learning Financial Recommendations More Fair: Theoretical Explanation, Tho M. Nguyen, Saeid Tizpaz-Niari, Vladik Kreinovich

Departmental Technical Reports (CS)

Machine learning has been actively and successfully used to make financial decisions. In general, these systems work reasonably well. However, in some cases, these systems show unexpected bias towards minority groups -- the bias that is sometime much larger than the bias in the data on which they were trained. A recent paper analyzed whether a proper selection of hyperparameters can decrease this bias. It turned out that while the selection of hyperparameters indeed affect the system's fairness, only a few of the hyperparameters lead to consistent improvement of fairness: the number of features used for training and the number …

Approximate Stochastic Dominance Revisited, Chon Van Le, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

According to decision theory, in general, to recommend the best of possible actions, we need to know, for each possible action, the probabilities of different outcomes, and we also need to know the decision maker's utility function -- that describes his/her preferences. For some pairs of probability distributions, however, we can make such a recommendation without knowing the exact form of the utility function -- e.g., in financial applications, we only need to know that a larger amount is preferable to a smaller one. Such situations, when we can make decisions based only on the information about probabilities, are known …

Just-In-Accuracy: Mobile Approach To Uncertainty, Martine Ceberio, Christoph Q. Lauter, Vladik Kreinovich

Departmental Technical Reports (CS)

To make a mobile device last longer, we need to limit computations to a bare minimum. One way to do that, in complex control and decision making problems, is to limit precision with which we do computations, i.e., limit the number of bits in the numbers' representation. A problem is that often, we do not know with what precision should we do computations to get the desired accuracy of the result. What we propose is to first do computations with very low precision, then, based on these computations, estimate what precision is needed to achieve the given accuracy, and then …

When Is It Beneficial To Merge Two Companies? When Is It Beneficial To Start A Research Collaboration?, Miroslav Svitek, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Merging two companies or splitting a company into two, teaming of two researchers or two research groups -- or splitting a research group into two -- these are frequent occurrences. Sometimes these actions lead to increased effectiveness, but sometimes, contrary to the optimistic expectations, the overall effectiveness decreases. To minimize the possibility of such failures, it is desirable to replace the current semi-intuitive way of making the corresponding decisions with a more objective approach. In this paper, we propose such an approach.

Linear Regression Under Partial Information, Tho M. Nguyen, Saeid Tizpaz-Niari, Vladik Kreinovich

Departmental Technical Reports (CS)

Often, we need to know how to estimate the value of a difficult-to-directly estimate quantity y -- e.g., tomorrow's temperature -- based on the known values of several quantities x1, ..., xn. In many practical situations, we know that the relation between y and xi can be accurately described by a linear function. So, to find this dependence, we need to estimate the coefficients of this linear dependence based on the known cases in which we know both y and xi; this is known as linear regression. In the ideal situation, when in each case, we know all the inputs …

Why Unit Two-Variable-Per-Inequality (Utvpi) Constraints Are So Efficient To Handle: Intuitive Explanation, Saeid Tizpaz-Niari, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In general, integer linear programming is NP-hard. However, there exists a class of integer linear programming problems for which an efficient algorithm is possible: the class of so-called unit two-variable-per-inequality (UTVPI) constraints. In this paper, we provide an intuitive explanation for why an efficient algorithm turned out to be possible for this class. Namely, the smaller the class, the more probable it is that a feasible algorithm is possible for this class, and the UTVPI class is indeed the smallest -- in some reasonable sense described in this paper.

Industry-Academia Collaboration: Main Challenges And What Can We Do, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

How can we bridge the gap between industry and academia? How can we make them collaborate more effectively? In this essay, we try to come up with answers to these important questions.

Why Attitudes Are Usually Mutual: A Possible Mathematical Explanation, Julio C. Urenda, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we provide a possible mathematical explanation of why people's attitude to each other is usually mutual: we usually have good attitude who those who have good feelings towards us, and we usually have negative attitudes towards those who have negative feelings towards, Several mathematical explanations of this mutuality have been proposed, but they are based on specific approximate mathematical models of human (and animal) interaction. It is desirable to have a solid mathematical explanation that would not depend on such approximate models. In this paper, we show that a recent mathematical result about relation algebras can lead …

Towards A Psychologically Natural Relation Between Colors And Fuzzy Degrees, Victor L. Timchenko, Yuriy P. Kondratenko, Olga Kosheleva, Vladik Kreinovich, Nguyen Hoang Phuong

Departmental Technical Reports (CS)

A natural way to speed up computations -- in particular, computations that involve processing fuzzy data -- is to use the fastest possible communication medium: light. Light consists of components of different color. So, if we use optical color computations to process fuzzy data, we need to associate fuzzy degrees with colors. One of the main features -- and of the main advantages -- of fuzzy technique is that the corresponding data has intuitive natural meaning: this data comes from words from natural language. It is desirable to preserve this naturalness as much as possible. In particular, it is desirable …

Algebraic Product Is The Only "And-Like"-Operation For Which Normalized Intersection Is Associative: A Proof, Thierry Denœx, Vladik Kreinovich

Departmental Technical Reports (CS)

For normalized fuzzy sets, intersection is, in general, not normalized. So, if we want to limit ourselves to normalized fuzzy sets, we need to normalize the intersection. It is known that for algebraic product, the normalized intersection is associative, and that for many other "and"-operations (t-norms), normalized intersection is not associative. In this paper, we prove that algebraic product is the only "and"-operation (even the only "and-like" operation) for which normalized intersection is associative.

How To Select A Model If We Know Probabilities With Interval Uncertainty, Vladik Kreinovich

Departmental Technical Reports (CS)

Purpose: When we know the probability of each model, a natural idea is to select the most probable model. However, in many practical situations, we do not know the exact values of these probabilities, we only know intervals that contain these values. In such situations, a natural idea is to select some probabilities from these intervals and to select a model with the largest selected probabilities. The purpose of this study is to decide how to most adequately select these probabilities.

Design/methodology/approach: We want the probability-selection method to preserve independence: If, according to the probability intervals, the two …

If Everything Is A Matter Of Degree, Why Do Crisp Techniques Often Work Better?, Miroslav Svitek, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Numerous examples from different application domain confirm the statement of Lotfi Zadeh -- that everything is a matter of degree. Because of this, one would expect that in most -- if not all -- practical situations taking these degrees into account would lead to more effective control, more effective prediction, etc. In practice, while in many cases, this indeed happens, in many other cases, "crisp" methods -- methods that do not take these degrees into account -- work better. In this paper, we provide two possible explanations for this discrepancy: an objective one -- explaining that the optimal (best-fit) model …

How To Propagate Interval (And Fuzzy) Uncertainty: Optimism-Pessimism Approach, Vinícius F. Wasques, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, inputs to a data processing algorithm are known with interval uncertainty, and we need to propagate this uncertainty through the algorithm, i.e., estimate the uncertainty of the result of data processing. Traditional interval computation techniques provide guaranteed estimates, but from the practical viewpoint, these bounds are too pessimistic: they take into account highly improbable worst-case situations when all the measurement and estimation errors happen to be strongly correlated. In this paper, we show that a natural idea of having more realistic estimates leads to the use of so-called interactive addition of intervals, techniques that has already …

How To Combine Probabilistic And Fuzzy Uncertainty: Theoretical Explanation Of Clustering-Related Empirical Result, Lázló Szilágyi, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In contrast to crisp clustering techniques that assign each object to a class, fuzzy clustering algorithms assign, to each object and to each class, a degree to which this object belongs to this class. In the most widely used fuzzy clustering algorithm -- fuzzy c-means -- for each object, degrees corresponding to different classes add up to 1. From this viewpoint, these degrees act as probabilities. There exist alternative fuzzy-based clustering techniques in which, in line with the general idea of the fuzzy set, the largest of the degrees is equal to 1. In some practical situations, the probability-type fuzzy …

Which Fuzzy Implications Operations Are Polynomial? A Theorem Proves That This Can Be Determined By A Finite Set Of Inequalities, Sebastia Massanet, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

To adequately represent human reasoning in a computer-based systems, it is desirable to select fuzzy operations that are as close to human reasoning as possible. In general, every real-valued function can be approximated, with any desired accuracy, by polynomials; it is therefore reasonable to use polynomial fuzzy operations as the appropriate approximations. We thus need to select, among all polynomial operations that satisfy corresponding properties -- like associativity -- the ones that best fit the empirical data. The challenge here is that properties like associativity mean satisfying infinitely many constraints (corresponding to infinitely many possible triples of values), while most …