Computer Sciences Commons^™

Why Skew Normal: A Simple Pedagogical Explanation, José Guadalupe Flores Muñiz, Vyacheslav Kalashnikov, Nataliya Kalashnykova, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, we only know a few first moments of a random variable, and out of all probability distributions which are consistent with this information, we need to select one. When we know the first two moments, we can use the Maximum Entropy approach and get normal distribution. However, when we know the first three moments, the Maximum Entropy approach doe snot work. In such situations, a very efficient selection is a so-called skew normal distribution. However, it is not clear why this particular distribution should be selected. In this paper, we provide an explanation for this selection.

Why 70/30 Or 80/20 Relation Between Training And Testing Sets: A Pedagogical Explanation, Afshin Gholamy, Vladik Kreinovich, Olga Kosheleva

Departmental Technical Reports (CS)

When learning a dependence from data, to avoid overfitting, it is important to divide the data into the training set and the testing set. We first train our model on the training set, and then we use the data from the testing set to gauge the accuracy of the resulting model. Empirical studies show that the best results are obtained if we use 20-30% of the data for testing, and the remaining 70-80% of the data for training. In this paper, we provide a possible explanation for this empirical result.

Why Burgers Equation: Symmetry-Based Approach, Leobardo Valera, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

In many application areas ranging from shock waves to acoustics, we encounter the same partial differential equation known as the Burgers' equation. The fact that the same equation appears in different application domains, with different physics, makes us conjecture that it can be derived from the fundamental principles. Indeed, in this paper, we show that this equation can be uniquely determined by the corresponding symmetries.

Lotfi Zadeh: A Pioneer In Ai, A Pioneer In Statistical Analysis, A Pioneer In Foundations Of Mathematics, And A True Citizen Of The World, Vladik Kreinovich

Departmental Technical Reports (CS)

Everyone knows Lotfi Zadeh as the Father of Fuzzy Logic. There have been -- and will be -- many papers on this important topic. What I want to emphasize in this paper is that his ideas go way beyond fuzzy logic:

• he was a pioneer in AI;
• he was a pioneer in statistical analysis; and
• he was a pioneer in foundations of mathematics.

My goal is to explain these ideas to non-fuzzy folks. I also want to emphasize that he was a true Citizen of the World.

From Traditional Neural Networks To Deep Learning: Towards Mathematical Foundations Of Empirical Successes, Vladik Kreinovich

Departmental Technical Reports (CS)

How do we make computers think? To make machines that fly, it is reasonable to look at the creatures that know how to fly: the birds. To make computers think, it is reasonable to analyze how we think -- this is the main origin of neural networks. At first, one of the main motivations was speed -- since even with slow biological neurons, we often process information fast. The need for speed motivated traditional 3-layer neural networks. At present, computer speed is rarely a problem, but accuracy is -- this motivated deep learning. In this paper, we concentrate on the …

Italian Folk Multiplication Algorithm Is Indeed Better: It Is More Parallelizable, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Traditionally, many ethnic groups had their own versions of arithmetic algorithms. Nowadays, most of these algorithms are studied mostly as pedagogical curiosities, as an interesting way to make arithmetic more exciting to the kids: by applying to their patriotic feelings -- if they are studying the algorithms traditionally used by their ethic group -- or simply to their sense of curiosity. Somewhat surprisingly, we show that one of these algorithms -- a traditional Italian multiplication algorithm -- is actually in some reasonable sense better than the algorithm that we all normally use -- namely, it is easier to parallelize.

A New Kalman Filter Model For Nonlinear Systems Based On Ellipsoidal Bounding, Ligang Sun, Hamza Alkhatib, Boris Kargoll, Vladik Kreinovich, Ingo Neumann

Departmental Technical Reports (CS)

In this paper, a new filter model called set-membership Kalman filter for nonlinear state estimation problems was designed, where both random and unknown but bounded uncertainties were considered simultaneously in the discrete-time system. The main loop of this algorithm includes one prediction step and one correction step with measurement information, and the key part in each loop is to solve an optimization problem. The solution of the optimization problem produces the optimal estimation for the state, which is bounded by ellipsoids. The new filter was applied on a highly nonlinear benchmark example and a two-dimensional simulated trajectory estimation problem, in …

Why Learning Has Aha-Moments And Why We Should Also Reward Effort, Not Just Results, Gerargo Uranga, Vladik Kreinovich, Olga Kosheleva

Departmental Technical Reports (CS)

Traditionally, in machine learning, the quality of the result improves steadily with time (usually slowly but still steadily). However, as we start applying reinforcement learning techniques to solve complex tasks -- such as teaching a computer to play a complex game like Go -- we often encounter a situation in which for a long time, then is no improvement, and then suddenly, the system's efficiency jumps almost to its maximum. A similar phenomenon occurs in human learning, where it is known as the aha-moment. In this paper, we provide a possible explanation for this phenomenon, and show that this explanation …

Can Mass Be Negative?, Vladik Kreinovich, Sergei Soloviev

Departmental Technical Reports (CS)

Overcoming the force of gravity is an important part of space travel and a significant obstacle preventing many seemingly reasonable space travel schemes to become practical. Science fiction writers like to imagine materials that may help to make space travel easier. Negative mass -- supposedly causing anti-gravity -- is one of the popular ideas in this regard. But can mass be negative? In this paper, we show that negative masses are not possible -- their existence would enable us to create energy out of nothing, which contradicts to the energy conservation law.

Type-2 Fuzzy Analysis Explains Ubiquity Of Triangular And Trapezoid Membership Functions, Olga Kosheleva, Vladik Kreinovich, Shahnaz Shahbazova

Departmental Technical Reports (CS)

In principle, we can have many different membership functions. Interestingly, however, in many practical applications, triangular and trapezoidal membership functions are the most efficient ones. In this paper, we use fuzzy approach to explain this empirical phenomenon.

How To Efficiently Compute Ranges Over A Difference Between Boxes, With Applications To Underwater Localization, Luc Jaulin, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

When using underwater autonomous vehicles, it is important to localize them. Underwater localization is very approximate. As a result, instead of a single location x, we get a set X of possible locations of a vehicle. Based on this set of possible locations, we need to find the range of possible values of the corresponding objective function f(x). For missions on the ocean floor, it is beneficial to take into account that the vehicle is in the water, i.e., that the location of this vehicle is not in a set X' describing the under-floor matter. Thus, the actual set of …

How To Detect Crisp Sets Based On Subsethood Ordering Of Normalized Fuzzy Sets? How To Detect Type-1 Sets Based On Subsethood Ordering Of Normalized Interval-Valued Fuzzy Sets?, Christian Servin, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

If all we know about normalized fuzzy sets is which set is a subset of which, will we be able to detect crisp sets? It is known that we can do it if we allow all possible fuzzy sets, including non-normalized ones. In this paper, we show that a similar detection is possible if we only allow normalized fuzzy sets. We also show that we can detect type-1 fuzzy sets based on the subsethood ordering of normalized interval-valued fuzzy sets.

Towards Foundations Of Fuzzy Utility: Taking Fuzziness Into Account Naturally Leads To Intuitionistic Fuzzy Degrees, Christian Servin, Vladik Kreinovich

Departmental Technical Reports (CS)

The traditional utility-based decision making theory assumes that for every two alternatives, the user is either absolutely sure that the first alternative is better, or that the second alternative is better, or that the two alternatives are absolutely equivalent. In practice, when faced with alternatives of similar value, people are often not fully sure which of these alternatives is better. To describe different possible degrees of confidence, it is reasonable to use fuzzy logic techniques. In this paper, we show that, somewhat surprisingly, a reasonable fuzzy modification of the traditional utility elicitation procedure naturally leads to intuitionistic fuzzy degrees.

How Many Monte-Carlo Simulations Are Needed To Adequately Process Interval Uncertainty: An Explanation Of The Smart Electric Grid-Related Simulation Results, Afshin Gholamy, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the possible ways of dealing with interval uncertainty is to use Monte-Carlo simulations. A recent study of using this technique for the analysis of different smart electric grid-related algorithms shows that we need approximately 500 simulations to compute the corresponding interval range with 5% accuracy. In this paper, we provide a theoretical explanation for these empirical results.

Measures Of Specificity Used In The Principle Of Justifiable Granularity: A Theoretical Explanation Of Empirically Optimal Selections, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

To process huge amounts of data, one possibility is to combine some data points into granules, and then process the resulting granules. For each group of data points, if we try to include all data points into a granule, the resulting granule often becomes too wide and thus rather useless; on the other case, if the granule is too narrow, it includes only a few of the corresponding point -- and is, thus, also rather useless. The need for the trade-off between coverage and specificity is formalized as the principle of justified granularity. The specific form of this principle …

Why Rectified Linear Neurons Are Efficient: Symmetry-Based, Complexity-Based, And Fuzzy-Based Explanations, Olac Fuentes, Justin Parra, Elizabeth Y. Anthony, Vladik Kreinovich

Departmental Technical Reports (CS)

Traditionally, neural networks used a sigmoid activation function. Recently, it turned out that piecewise linear activation functions are much more efficient -- especially in deep learning applications. However, so far, there have been no convincing theoretical explanation for this empirical efficiency. In this paper, we show that, by using different uncertainty techniques, we can come up with several explanations for the efficiency of piecewise linear neural networks. The existence of several different explanations makes us even more confident in our results -- and thus, in the efficiency of piecewise linear activation functions.

How To Make A Proof Of Halting Problem More Convincing: A Pedagogical Remark, Benjamin W. Robertson, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

As an example of an algorithmically undecidable problem, most textbooks list the impossibility to check whether a given program halts on given data. A usual proof of this result is based on the assumption that the hypothetical halt-checker works for all programs. To show that a halt-checker is impossible, we design an auxiliary program for which the existence of such a halt-checker leads to a contradiction. However, this auxiliary program is usually very artificial. So, a natural question arises: what if we only require that the halt-checker work for reasonable programs? In this paper, we show that even with such …

Why Triangular Membership Functions Are Often Efficient In F-Transform Applications: Relation To Interval Uncertainty\\ And Haar Wavelets, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Fuzzy techniques describe expert opinions. At first glance, we would therefore expect that the more accurately the corresponding membership functions describe the expert's opinions, the better the corresponding results. In practice, however, contrary to these expectations, the simplest -- and not very accurate -- triangular membership functions often work the best. In this paper, on the example of the use of membership functions in F-transform techniques, we provide a possible theoretical explanation for this surprising empirical phenomenon.

How To Store Tensors In Computer Memory: An Observation, Martine Ceberio, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, after explaining the need to use tensors in computing, we analyze the question of how to best store tensors in computer memory. Somewhat surprisingly, with respect to a natural optimality criterion, the standard way of storing tensors turns out to be one of the optimal ones.

Beyond Integration: A Symmetry-Based Approach To Reaching Stationarity In Economic Time Series, Songsak Sriboonchitta, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Many efficient data processing techniques assume that the corresponding process is stationary. However, in areas like economics, most processes are not stationery: with the exception of stagnation periods, economies usually grow. A known way to apply stationarity-based methods to such processes -- integration -- is based on the fact that often, while the process itself is not stationary, its first or second differences are stationary. This idea works when the trend polynomially depends on time. In practice, the trend is usually non-polynomial: it is often exponentially growing, with cycles added. In this paper, we shod how integration techniques can be …

Why Sparse?, Thongchai Dumrongpokaphan, Olga Kosheleva, Vladik Kreinovich, Aleksandra Belina

Departmental Technical Reports (CS)

In many situations, a solution to a practical problem is sparse, i.e., corresponds to the case when most of the parameters describing the solution are zeros, and only a few attain non-zero values. This surprising empirical phenomenon helps solve the corresponding problems -- but it remains unclear why this phenomenon happens. In this paper, we provide a possible theoretical explanation for this mysterious phenomenon.

Z-Numbers: How They Describe Student Confidence And How They Can Explain (And Improve) Laplacian And Schroedinger Eigenmap Dimension Reduction In Data Analysis, Vladik Kreinovich, Olga Kosheleva, Michael Zakharevich

Departmental Technical Reports (CS)

Experts have different degrees of confidence in their statements. To describe these different degrees of confidence, Lotfi A. Zadeh proposed the notion of a Z-number: a fuzzy set (or other type of uncertainty) supplemented by a degree of confidence in the statement corresponding to fuzzy sets. In this chapter, we show that Z-numbers provide a natural formalization of the competence-vs-confidence dichotomy, which is especially important for educating low-income students. We also show that Z-numbers provide a natural theoretical explanation for several empirically heuristic techniques of dimension reduction in data analysis, such as Laplacian and Schroedinger eigenmaps, and, moreover, show how …

Why Taylor Models And Modified Taylor Models Are Empirically Successful: A Symmetry-Based Explanation, Mioara Joldes, Christoph Lauter, Martine Ceberio, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we show that symmetry-based ideas can explain the empirical success of Taylor models and modified Taylor models in representing uncertainty.

How To Best Apply Neural Networks In Geosciences: Towards Optimal "Averaging" In Dropout Training, Afshin Gholamy, Justin Parra, Vladik Kreinovich, Olac Fuentes, Elizabeth Y. Anthony

Departmental Technical Reports (CS)

The main objectives of geosciences is to find the current state of the Earth -- i.e., solve the corresponding inverse problems -- and to use this knowledge for predicting the future events, such as earthquakes and volcanic eruptions. In both inverse and prediction problems, often, machine learning techniques are very efficient, and at present, the most efficient machine learning technique is deep neural training. To speed up this training, the current learning algorithms use dropout techniques: they train several sub-networks on different portions of data, and then "average" the results. A natural idea is to use arithmetic mean for this …

Why Deep Learning Methods Use Kl Divergence Instead Of Least Squares: A Possible Pedagogical Explanation, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In most applications of data processing, we select the parameters that minimize the mean square approximation error. The same Least Squares approach has been used in the traditional neural networks. However, for deep learning, it turns out that an alternative idea works better -- namely, minimizing the Kullback-Leibler (KL) divergence. The use of KL divergence is justified if we predict probabilities, but the use of this divergence has been successful in other situations as well. In this paper, we provide a possible explanation for this empirical success. Namely, the Least Square approach is optimal when the approximation error is normally …

Sudoku App: Model-Driven Development Of Android Apps Using Ocl?, Yoonsik Cheon, Aditi Barua

Departmental Technical Reports (CS)

Model driven development (MDD) shifts the focus of software development from writing code to building models by developing an application as a series of transformations on models including eventual code generation. Can the key ideas of MDD be applied to the development of Android apps, one of the most popular mobile platforms of today? To answer this question, we perform a small case study of developing an Android app for playing Sudoku puzzles. We use the Object Constraint Language (OCL) as the notation for creating precise models and translate OCL constraints to Android Java code. Our findings are mixed in …

Propagation Of Probabilistic Uncertainty: The Simplest Case (A Brief Pedagogical Introduction), Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

The main objective of this text is to provide a brief introduction to formulas describing the simplest case of propagation of probabilistic uncertainty -- for students who have not yet taken a probability course.

Impacts Of Java Language Features On The Memory Performances Of Android Apps, Yoonsik Cheon, Adriana Escobar De La Torre

Departmental Technical Reports (CS)

Android apps are written in Java, but unlike Java applications they are resource-constrained in storage capacity and battery lifetime. In this document, we perform an experiment to measure quantitatively the impact of Java language and standard API features on the memory efficiency of Android apps. We focus on garbage collection because it is a critical process for performance affecting user experience. We learned that even Java language constructs and standard application programming interfaces (APIs) may be a source of a performance problem causing a significant memory overhead for Android apps. Any critical section of code needs to be scrutinized on …

Need For A Large-N Array (And Wavelets And Differences) To Determine The Assumption-Free 3-D Earth Model, Solymar Ayala Cortez, Aaron A. Velasco, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the main objectives of geophysical seismic analysis is to determine the Earth's structure. Usually, to determine this structure, geophysicists supplement the measurement results with additional geophysical assumptions. An important question is: when is it possible to reconstruct the Earth's structure uniquely based on the measurement results only, without the need to use any additional assumptions? In this paper, we show that for this, one needs to use large-N arrays -- 2-D arrays of seismic sensors. To actually perform this reconstruction, we need to use differences between measurements by neighboring sensor and we need to apply wavelet analysis to …

Efficient Parameter-Estimating Algorithms For Symmetry-Motivated Models: Econometrics And Beyond, Vladik Kreinovich, Anh H. Ly, Olga Kosheleva, Songsak Sriboonchitta

Departmental Technical Reports (CS)

It is known that symmetry ideas can explain the empirical success of many non-linear models. This explanation makes these models theoretically justified and thus, more reliable. However, the models remain non-linear and thus, identification or the model's parameters based on the observations remains a computationally expensive nonlinear optimization problem. In this paper, we show that symmetry ideas can not only help to select and justify a nonlinear model, they can also help us design computationally efficient almost-linear algorithms for identifying the model's parameters.