Back to TOC

# Bei's Study Notes

## Types of Machine Learning Problems (wiki)

ML tasks can be classified into three broad categories based on the nature of learning signal or feedback avialable to a learning system:

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

Between supervised and unsupervised learning is semi-supervised learning.

Depending on the output:

1. Classification
2. Regression
3. Clustering
4. Ranking
5. Recommendation

Also:

1. Density estimation
2. Dimensionality reduction

## Statistics

1. Paramatric methods - Methods that make assumptions that the data comes from a distribution and it intends to estimate these parameters.
2. Nonparamatric methods - Methods that does not assume certain form of distributions. (Or rather, a free-form of distributions. "nonparametric really means many parametric.").

## Machine Learning Approaches

1. Linear Model
2. Kernel machines
3. Decision tree
4. Bayesian Estimation
5. Hidden Markov Models
6. Ensemble learning
1. Random Forest
7. Graphical models (Bayesian networks)
8. Clustering
9. Association rules learning
10. Artificial neural network
11. Reinforcement learning
13. Inductive logic programming
14. Feature learning
1. Sparse dictionary learning
15. Genetic algorithms
16. Rule-based machine learning
17. Learning classifier systems (LCS)

## Model Selection

Hypotheses set . Bias - underfitting. Variance - overfitting. Generalization. Triple trade-of.

Cross validation. Test set. Analogy with exams.

## General Relativity

### General Relativity 02 - Manifold and tensor fieldsLast updated: 2017-04-19 21:55:03 PDT.

(This is from Charpter 2.2 of Wald)

### Vectors

In GR, we discuss about spaces that are not exactly flat. If we can split the space into parts and each part is continously corresponsive to a region of , then it is called a manifold. is called toe dimension of the manifold.

Normally, a finite dimensional manifold can be embeded in a higher dimensional Euclidean space. Whereas such embedding might not be natural. In GR, the spacetime does not naturally live in a higher dimensional space, so an abstract definition of a manifold is necessary.

Definition (Manifold): An n-dimensional, , real manifold is a set together with a collection of subsets satisfying the following properties:

1. is a open cover of .
2. For each , there is a one-to-one, onto, map where is an open subset of .
3. For each and that , the map maps to . Then both sets must be open and the map must be .

Each map is called a "chart" or a "coordinate system". The definition of manifold and complex manifold simply is the same with some natural changes.

Through out this book, the manifold involed are all assumed to be Hausdorff and paracomact.

NOTE By this definition, the charts, by no means, are required to provide a "straight coordinate" to , by which I mean the tangent vector can change through out each .

Example Euclidean space with one chart and map to be identity function.

Example 2-sphere (a 2-dimensional spherical surface embedded in ).

In SR, each coordinate system is applicable to the whole universe. but in GR, a coordinate system is only usable inside its corresponding open set.

With the mapping to , we can easily define differentiability and smoothness of maps between manifolds.

Defintion (Diffeomorphism): Let and to be manifolds and and to be chart maps. A map is said to be if for each is in Euclidean spaces. If is , one-to-one and onto, then it is called a diffeomorphism, and and are said to be diffeomorphic.

NOTE diffeomorphism requires the manifolds to have the same dimension.

### Tangent vector in a manifold (without embedding in \mathbb R^n ):

We can define tangent vectors as directional derivatives. In , the mapping between vectors and directional derivatives is one-to-one. defines derivative operator . Directional derivatives are characterized by Leibnitz rules.

Let denote the collections of all functions from manifold into . We define a tangent vector at point to be a map which is (1) linear and (2) obeys Leibnitz rules:

NOTE: Be very careful that the second rule also applies the function and at .

Prop:

Proof.

If , since linearlity, otherwise divide the equation by , and .

The maps of a tangent vectors of a point forms a vector space by adding this addition law: .

Prop:

Theorem 2.2.1: Let be an -dimensional manifold. Let and let denote the tangent space at , then .

Proof. Given a chart of open set where , If , then is . For define functional by

This means, the th component of is a functional that takes the th partial derivative of function in , and apply the point to it. It is clear that is a derivative from the chain rule. Now we need to prove that .

For any function , if is ,

especially, we have

Let , and we have

where denotes , that is, picking the th element of the result.

For any , apply the functional,

This means is a linear combination of .

The basis is called a coordinate basis, frequently denoted as simply . For each different chart chosen, there is a different coordinate basis , and

We can also get the vector transformation law from it:

A smooth curve, on a manifold is a map of into , . At each point , there is a tangent vector associated with as follows.

Therefore the components of is given by

If and are on the manifold, there is no way to correlate them in a general manifold. Another construct ("connection", or "parallel transportation") must be introduced to do so. However, if the curvation is nonzero, the identification of with obtained in this manner will depend on the choice of curve.

A tangent field, , on a manifold is an assignment of a tangent vector, for each point . Despite the fact that the tangent spaces and at different points are different vector spaces, there is a natural notion of what it means for to vary smoothy from point to point.

A one-parameter group of diffeomorphisms is a map from . In particular:

1. is a diffeomorphism, and
2. .

At each point , is a curve, called the orbit of which passes through at . Define to be the tangent vector at . Thus we can consider the vector field to be the generator of such a group.

Conversely, we can ask a question, that if given a vector field , can we find a family of curves s.t. for each point , there is one and only one curve that passes through the point with the tangent vector equals to . The answer is yes.

Therefore we have a one-to-one mapping between a tangent field and a diffeomorphism. We can thus consider a tangent field to be a mapping of type :

Given two smooth vector fields and , we can define the commutator field as follows:

next >

next >

### Topological space (definition dump)

Definition (Topological space, open set): A topological space is a set with a collection of subsets of that satisfies:

1. ;
2. If , then ;
3. If , then

Sets in are called "open set"s.

An example of a topology on contains all the open intervals in . Thus the name "open set"s.

Definition (Induced topology): If is a topological space and is a subset of , we may make into a topological space by defining the topology , then forms a topology space. is called induced (or relative) topology.

Definition (Product topology): If and are both topological spaces, the direct prodct of both naturally forms a topological space . is called the product topology.

NOTE This lifts the dimension of the topological space.

Open balls on naturally form a topology.

Definition (Continuous mapping): If and are topological spaces, a map is continuous if the inverse image maps every open set in to an open set in .

Definition (Homeomorphism): If is continuous, one-to-one, onto, and its inverse is continues, then is called a homeomorphism, and the spaces are said to be "homeomorphic".

NOTE Not to be confused with homomorphism and homomorphic.

Definition (Closed set): The complement of an open set is called a "closed set". Sets in a topology can be open, close, both, or neither.

Definition (Connected): The topology is said to be conneted if the only subsets that are both open and closed are and . is connected.

Definition (Closure): If is a topological space, , the closure is the intersection of all open sets that contains .

Properties:

1. is closed;
2. ;
3. is closed.

NOTE Meaning "to make a set closed". Closure of a set is unique and is necessarily in the topology.

Definition (Interior, Boundary): Interior of is defined as the union of all the open sets contained in . The boundy of , denoted (or ), is defined as elements in but not the interior of , .

NOTE alternatively, .
NOTE alternatively, .
NOTE alternatively, .

Definition (Hausdorff): A topological space is Hausdorff if any two distinct points can be included in two disjoint open sets.

is Hausdorff.

### Compactness

One of the most powerful notions in topology is that of compactness, which is defined as follows.

Definition (Open cover): If is a topological space and a collection of open sets has , then is said to be an open cover of , and "covers" . Also if is a subset of , and , then is said to be an open cover of and "covers" . A subcollection of forms a subcover if it also covers (or ).

Definition (Compact space): If every open cover can be written as finite subcover, then the topological space is compact.

Alternative definitions of compact space. The following are equivalent:

1. A topological space is compact.
2. Every open cover a has a finite subcover.
3. has a sub-base such that every cover of the space by members of the sub-base has a finite subcover (Alexander's sub-base theorem).
4. Any collection of closed subsets of with the finite intersection property has nonempty intersection.
5. Every net on X has a convergent subnet (see the article on nets for a proof).
6. Every filter on X has a convergent refinement.
7. Every ultrafilter on X converges to at least one point.
8. Every infinite subset of X has a complete accumulation point.

Definition (Open cover of a set, subcover of a set): If is a topological space and is a subset of . A open cover is a open cover of if . A subcover that also covers is called a subcover of .

Definition (Compact subset): is said to be compact if every open cover of A has a finite subcover.

The relation ship between compact space and compact subset is given by these two theorems:

1. Compact subset of a Hausdorff space is closed.
2. Closed subset of a compact space is compect.

Heine-Borel Theorem. A closed interval of is compact.

Open interval is not compact (since the open cover has no finite subcover).

A subset of is compact iff it is closed and bounded.

NOTE A unbounded set can totally be closed. For example, is obviously unbounded and closed.

Let and be topological spaces. Suppose is compact and is continuous. Then is compact.

NOTE This transfers compactness through homeomorphisms.

A continuous function from a compact topological space into is bounded and attains its maximum and minimum values.

Tychonoff theorem: Product of compact topological spaces is compact. Given the axiom of choice, the number of such spaces can be infinite.

An application of these is that is compact, because 1) the sphere in is closed and bounded, therefore compact; 2) there is a continuous function from to .

### Convergence of sequences

To extend the normal definition of sequence convergence, a sequence of points in a topological space is said to converge to point if . is called the limit of the sequence.

A point is said to be a accumulation point of if every open neiborhood of contains infinitely many points of the sequence.

NOTE The difference between a limit and an accumulation point is that the former requires a particular set of infinite points in . For example, the alternating sequence has two accumulation points and , but it does not have a limit.

Definition (First countable): For every point in , if there is a countable collection of open sets that for every neiborhood of , contains at least one element in .

Definition (Second countable): There is a countable collection of open sets that every open set can be written as the union of some of the sets in the collections. The sets in that collection are called basis.

NOTE The basis of a linear space is a collection of vectors, s.t. every vector in the space is a linear combination of the basis. The basis of a topological space is a collection of open sets, s.t. every opens set in the space is a union of the basis.

NOTE is second countable. Open balls with rational radii centered on rational coordinates can form a countable collection of open sets.
NOTE Every second countable space is first countable.

The relationship between compactness and convergence of sequences is expressed by Bolzano-Weierstrass theorem:

Bolzano-Weierstrass theorem Let be a topological space and let

1. If is compact, then every infinite sequence of points in has a accumulation point lying in ;
2. Conversely, if is second countable and every sequence in has an accumulation point in , then is compact.

Thus, in particular, if is second countable, is compact iff every sequence in has a convergent subsequence whose limit lies in .

### Paracompactness

Definition (Neighborhood) Given in topological space , a neighborhood of is a subset of that includes an open set containing :

NOTE: may not be open, but it contains a open set that contains .

Definition (Refinement of an open cover): Open cover of is said to be a refinement of open cover of if .

NOTE Refinements forms a partially ordered set.
NOTE Subcover is always a refinement of a open cover. A refinement of a open cover is not always a subcover.

Definition (Locally finite): is locally finite if each has an open neighborhood such that only finitely many satisfy .

NOTE Compactness requires a finite subcover, locally finiteness only requires a finte refinement. It is a weaker requirement.

Definition (Paracompactness): A space is paracompact if every open cover has a locally finite refinement.

NOTE: Locally finiteness is weaker than finitenes of subcovers. Therefore every compact space is paracompact.

NOTE: (wiki) Every metric space is paracompact. A topological space is metrizable if and only if it is a paracompact and locally metrizable Hausdorff space. Paracompactness has little to do with the notion of compactness, but rather more to do with breaking up topological space entities into manageable pieces.

A paracomact manifold implies:

1. admits a Riemannian metric and
2. is second countable.

The most important implication is that a paracompact manifold will have a partition of unity.

Definition (Partition of unity): If is a topological space, and is a set of continuous functions from to unit interval , such that for every point :

1. there is a neighborhood of where all but finite number of functions in are , and
2. the sum of all the functions at is :

This is for the ease of defining integrals on the manifold.

next >

### Starting to Learn General RelativityLast updated: 2017-04-19 21:55:03 PDT.

I was told that usually, a physics student can only master either GR or QFT, but not both at the same time. It will be an interesting experiment if I can start both since I'm not in a rush right now. So this is it. This is the textbook I will use: General Relativity by Robert M. Wald.

## General Relativity - UCI OCW

### Perfect Fluid in Special RelativityLast updated: 2016-06-08 11:28:33 PDT.

I was watching a OCW from UC Irvine on GR. It starts with some discussions about special relativistic. This one about perfect fluid caught me because is not obvious how the Euler equation is derived. So I did it here.

It is an interesting topic in astrophysics since most of the objects we study can be considered (general) relativistic perfect fluid. As a first step, perfect fluid in special relativity is studied.

Starting by write down the stress-energy tensor at a point . First create the reference frame at that point, and the tensor looks like

Then generalize it using Lorentz transformation:

where consider only the boost

where is the Lorentz factor, then the tensor is

Combine the equations:

To get the equation of motion, we can use the energy-momemtum conservation law:

For ,