Posts

Neural Networks as Morphisms

-
Image
Abstract This set of notes attempts to study possibility of constructing categories with neural nets as morphism and functors. The reason for such construction comes stems from possible enriching connections between category theory and deep learning. Usually Category Theory is used to formalize abstractions of the given concrete objects. On the other hand, Deep Learning is mostly used in constructing explicit neural network based mappings between various spaces given by sampled datasets. The notes explores if it is possible to leverage on strengths of category theory and deep learning in order to represent more abstract concepts and processes. Introduction Motivation: The main motivation for development of framework if two folds. Firstly, it might shed a new light for representing more abstract objects, such as movements and processes. Secondly, we leverage on rich experience of building deep neural nets accumulated in Machine Learning community. Abs...
Post a Comment
Read more

On learning C++

-
"You are number of languages that you know" said one of the quotes attributed to Al-Farabi, which my school teacher used to tell us. Indeed almost all languages describe the same objects: humans, animals, nature and universe in general. However, the way each language does so might differ from another one. This difference makes learning new languages fun. A somewhat similar statement can be said about programming languages. Almost all of them perform some kind of logical and computational transformations on given data, however the ways they do so might significantly differ. Having learnt some Python to play with Machine Learning (ML) algorithms, I wanted to explore other programming languages. The first language which came to mind was C++. It is one of the languages used to code massive and complex software systems. So did I explored using a website  https://www.programiz.com/cpp-programming . How it went The installation part went easy. The application XCode does a solid ...
Post a Comment
Read more

On ideas behind Bayesian Learning

-
When one reads about Machine Learning (ML) or Artificial Intelligence (AI) it is common to come across a list of approaches to AI such as: symbolic reasoning, Bayesian learning, artificial neural networks and so on. Seeing this list I was puzzled with a notion of Bayesian Learning. The only thing I knew with a name Bayes in its name was a famous Bayes's theorem which states that \[ P(H\mid E) = \frac{P(E\mid H)P(H)}{P(E)} \] where $P(H)$ and $P(H\mid E)$ denote prior and posterior probabilities of hypothesis $H$, respectively. What puzzled me is the application of such a simple looking formula to draw patterns from data. In order to learn basic principles of Bayesian learning, I decided to consult Wikipedia and few online tutorials. This post is about results of this exploration. Wikipedia says Bayesian learning (inference?) is a method of statistical inference in which Bayes's theorem is used to update the probability for a hypothesis as more evidence or informatio...
Post a Comment
Read more

On objects of study of Classical Algebraic Geometry

-
A subject of Algebraic Geometry (AG) is one of central branches of Mathematics, yet I have almost no knowledge of objects and spaces that algebraic geometers study. The mere look at the name suggests some kind of relationship between algebra and geometry. So does algebra lends its hand to solve geometric problems, or is it other way around. I did not know. Realising my ignorance of the matter, I decided to rectify situation I decided to read some Wikipedia articles along with some introductory books on the subject matter. In this post I intend to share fruits of my explorations. How AG was born? AG was born out of desire to solve systems of polynomial equations. Suppose we have a system \[ f_i(X_1,\ldots,X_n) = 0, \quad i=1,\ldots, m \] where each $f_i$ is a polynomial of $n$ variables with coefficients drawn from some field $k$, which can succinctly be written as $f_i\in k[X_1,\ldots,X_n]$. What to do next? Mostly it is pretty difficult to solve such systems. However, i...
Post a Comment
Read more

From Gales to Covers

-
This post continues to study relationship between definitions of Hausdorff dimension obtained through covers and gales. In the last post, we have shown how to switch from covers to gales, which allowed us to conclude that \[ \inf G(X) \le \dim_H(X) \] So now our aim should be verification of reverse inequality, i.e. $\dim_H(X)\le \inf G(X)$. In order to do this, we go back to definitions. Let us start with basics $\mathcal{C} = \{0,1\}^{\mathbb{N}}$ is Cantor space with some subset $X$. Let $d$ be an $s$-gale which succeeds on $X$, i.e. $X\subseteq S^{\infty}[d]$. This means that $\inf G(X)\le s$. Furthermore, for any given $\epsilon>0$, one could choose $s$ such that $\inf G(X)\ge s-\epsilon$. Given such $s$, we claim that $\mathcal{H}^s(X)=0$, i.e. \[ \inf\{\sum_{i=1}^{\infty}diam(C_{w_i})^s \} = 0 \] where $X\subseteq \bigcup C_{w_i}$. This would give us $\dim_H(X)\le s \le \inf G(X) + \epsilon$. By letting $\epsilon\to 0$, the desired inequality follows. So our main...
Post a Comment
Read more

From Covers to Gales

-
In this post we attempt to understand a relationship between Hausdorff dimension defined in a classical way and Hausdorff dimension defined using gales. Recall that classical Hausdorff dimension is defined in terms of covers. So, in order to understant that relationship, we need to be able to 'switch' back and forth from covers and gales. In this post we will try to switch from covers to gales. As before, we work in Cantor space $\mathcal{C} = \{0,1\}^{\mathbb{N}}$. We start with some subset $X\subseteq \mathcal{C}$. For any given $\varepsilon>0$, one can choose $s\ge 0$ such that $s-\varepsilon < \dim_H(X) < s$. Then we go on to show that there $s$-gale succeeding on $X$, i.e. $\inf G(X)\le s$. Combining this with above inequality, we have \[ \inf G(X)\le s < \dim_H(X) + \varepsilon \] By letting $\varepsilon \to 0$, one shows that $\inf G(X)\le \dim_H(X)$. So, we are left to show that it is indeed possible to construct $s$-gale given the fact that $\dim...
Post a Comment
Read more

On Gales

-
In this post we aim to provide characterization of Hausdorff Dimension in Cantor space in terms of special function called gales. Our presentation mainly follows that of Mayordomo [1], while the notion itself is from Lutz [2]. Gales Let us first go through definitions of supergales and gales. Take $s\in [0, \infty)$. An $s$-supergale is a function $d:\{0,1\}^*\to [0,\infty)$ such that: \[ d(w) \ge 2^{-s}\left[ d(w0) + d(w1) \right] \] for all $w\in \{0,1\}^*$. A definition $s$-gale is similar to the above, except for the equality sign in place of the inequality sign for each $w\in \{0,1\}^*$. A supermartingale is $1$-supergale, while a martingale is an $1$-gale. As for names, they seem to derive from a notion of martingale from probability theory. A parameter $s$ describes fairness of the betting game. Observe that the condition for $s$-gale can be rewritten as: \[ E[d(wb)] = \frac{1}{2}d(w0) + \frac{1}{2}d(w1) = 2^{s-1}d(w) \] Depending on the value of $s$, the game ca...
Post a Comment
Read more