eth-janishutz/eth-summaries
1
0
Fork 0
mirror of https://github.com/janishutz/eth-summaries.git synced 2025-11-25 02:24:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add A&W summary. Still WIP for fixing errors and more proofs

Browse Source
This commit is contained in:
Admin
2025-09-12 17:07:40 +02:00
parent 395c71b962
commit a4309f0ede
22 changed files with 3370 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
semester2/algorithms-and-probability/assets/binomialExpansion.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 190 KiB

BIN
semester2/algorithms-and-probability/assets/overview.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 442 KiB

BIN
semester2/algorithms-and-probability/aw-summary.pdf Normal file
View File

Binary file not shown.

59
semester2/algorithms-and-probability/aw-summary.tex Normal file
View File

@@ -0,0 +1,59 @@
\documentclass{article}
\newcommand{\dir}{~/projects/latex} % IMPORTANT: No trailing slashes!
\input{\dir/include.tex}
\load{recommended}
\setup{Algorithms and Probability}
\begin{document}
\startDocument
\usetcolorboxes
\setcounter{numberingConfig}{3}
\vspace{2cm}
\begin{Huge}
\begin{align*}
& \Pr[X|Y] \geq \Pr[X|\overline{Y}] \\
X & := \text{I pass the exam} \\
Y & := \text{I read this summary}
\end{align*}
\end{Huge}
\vspace{4cm}
\begin{center}
\begin{Large}
``\textit{Dieses Algorithmus mapped $E$ nach $E$ Strich}''
\end{Large}
\hspace{3cm} - Rasmus Kyng, 2025
\end{center}
\vspace{3cm}
\begin{center}
FS2025, ETHZ\\[0.2cm]
\begin{Large}
Summary of the Script and Lectures
\end{Large}\\[0.2cm]
\end{center}
\newpage
\printtoc{ForestGreen}
\input{parts/graphs.tex}
\input{parts/combinatorics.tex}
\input{parts/probability.tex}
\input{parts/algorithms.tex}
\input{parts/coding.tex}
\end{document}

588
semester2/algorithms-and-probability/parts/algorithms.tex Normal file
View File

@@ -0,0 +1,588 @@
\newsection
\section{Algorithms}
\subsection{Graph algorithms}
\subsubsection{Long path problem}
Given a tuple $(G, B)$ where $G$ is a graph and $B \in \N_0$, we need to determine whether there exists a path of length $B$ in $G$.
This problem is another one of the infamous $\mathcal{N}\mathcal{P}$-Complete problems, for which there \textit{supposedly} doesn't exist an algorithm that can solve the problem in polynomial time.
We can show that this problem belongs to said group if we can show that we can \textit{efficiently} construct a graph $G'$ with $n' \leq 2n - 2$ vertices such that $G$ has a Hamiltonian Cycle if and only if $G'$ has a path of length $n$.
We construct a graph $G'$ from graph $G$ by selecting a vertex $v$ and replacing each edge incident to that vertex with edges that lead to newly added vertices $\hat{w_1}, \hat{w_2}, \ldots$.
$G'$ has $(n - 1) + \deg(v) \leq 2n - 2$ vertices. All the vertices $\hat{w_1}, \ldots, \hat{w_{\deg(v)}}$ all have degree $1$.
The graph $G'$ fulfills the above implication because
\begin{enumerate}[label=(\roman*)]
\item Let $\langle v_1, v_2, \ldots, v_n, v_1 \rangle$ be a Hamiltonian cycle.
Assume for contradiction that the resulting graph, constructed according to the description above does not contain a path of length $n$.
Let's assume $v_1 = v$ (the vertex removed during construction). However, $\langle \hat{v_2}, v_2, \ldots, \hat{v_n}, v_n \rangle$ is a path of length $n$
\item Let $\langle u_0, u_1, \ldots, u_n \rangle$ be a path of length $n$ in $G'$ and let $\deg(u_i) \geq 2 \smallhspace \forall i \in \{1, \ldots, n - 1\}$ These vertices hence have to be the $n - 1$ remaining vertices of $G$, thus we have $u_0 = \hat{w_i}$ and $u_n = \hat{w_j}$ two different ones of new vertices of degree $1$ in $G'$. Thus, we have $u_1 = w_i$ and $u_{n - 1} = w_j$ and we have $\langle v, u_1, \ldots, u_{n - 1}, v \rangle$, which is a Hamiltonian cycle in $G$
\end{enumerate}
Due to the construction of the graph $G'$ we can generate it from $G$ in \tco{n^2} steps. We thus have:
\begin{theorem}[]{Long Path Problem}
If we can find a \textit{long-path} in a graph with $n$ vertices in time $t(n)$, we can decide if a graph with $n$ vertices has a Hamiltonian cycle in $t(2n - 2) + \text{\tco{n^2}}$
\end{theorem}
\fhlc{Cyan}{Short long paths}
In biological applications, the long paths searched are usually small compared to $n$. It is possible to solve this problem in polynomial time if for the tuple $(G, B)$ $B = \log(n)$.
Notation and useful properties:
\begin{itemize}
\item $[n] := \{1, 2, \ldots, n\}$. $[n]^k$ is the set of sequences over $[n]$ of length $k$ and we have $\left| [n]^k \right| = n^k$. ${[n] \choose k}$ is the set of subsets of $[n]$ of cardinality $k$ and we have $\left| {[n] \choose k} \right| = {n \choose k}$
\item For every graph $G = (V, E)$ we have $\sum_{v \in V} \deg(v) = 2|E|$
\item $k$ vertices (no matter if part of a path or not) can be coloured using $[k]$ in exactly $k^k$ ways whereas $k!$ of said colourings use each colour exactly once.
\item For $c, n \in \R^+$ we have $c^{\log(n)} = n^{\log(c)}$
\item For $n \in \N_0$ we have $\sum_{i = 0}^{n} {n \choose i} = 2^n$. (Application of binomial expansion, see \ref{sec:binomial-expansion})
\item For $n \in \N_0$ we have $\frac{n!}{n^n} \geq e^{-n}$
\item If we repeat an experiment with probability of success $p$ until success, $\E[\mathcal{X}] = \frac{1}{p}$ where $\mathcal{X} :=$ number of trials
\end{itemize}
\newpage
\shade{ForestGreen}{Colourful paths}
A path is called \textit{colourful} if all vertices on it are coloured differently.
For $v \in V$ and $i \in \N_0$ let's define
\begin{align*}
P_i(v) := \left\{ S \in {[k] \choose i + 1} \smallhspace \Bigg| \smallhspace \exists \text{ path ending in $v$ coloured with $S$ colours } \right\}
\end{align*}
Thus, $P_i(v)$ contains a set $S$ of $i + 1$ colours if and only if there exists a path with vertex $v$ whose colours are the colours in $S$.
It is important to note that such a path has to be always \textit{exactly} of length $i$.
If we solve this problem for every $v \in V$, we solved our problem and we have
\begin{align*}
\exists \text{ colourful path of length $k - 1$ } \Longleftrightarrow \bigcup_{v \in V} P_{k - 1}(v) \neq \emptyset
\end{align*}
For the algorithm, we need to also define $N(v)$ which returns the neighbours of $v$ and $\gamma: V \rightarrow [k]$ which assigns a colour to each vertex.
\begin{algorithm}
\caption{Colourful path algorithm}
\begin{algorithmic}[1]
\Procedure{Colourful}{$G, i$}
\For{\textbf{all} $v \in V$}
\State $P_i(v) \gets \emptyset$
\For{\textbf{all} $x \in N(v)$}
\For{\textbf{all} $R \in P_{i - 1}(x)$ with $\gamma(v) \notin R$}
\State $P_i(v) \gets P_i(v) \cup \{R \cup \{\gamma(v)\}\}$
\EndFor
\EndFor
\EndFor
\EndProcedure
\end{algorithmic}
\end{algorithm}
The time complexity of this algorithm is \tco{2^k km}. If we now have $k = \text{\tco{\log(n)}}$, the algorithm is polynomial.
\shade{ForestGreen}{Random colouring}
The idea to solve the short long path problem in polynomial time is to randomly colour the vertices using colours $[k]$, whereby $k := B + 1$ and check if there is a colourful path of length $k - 1$.
Since we are guaranteed to have a colourful path if we find one, we can develop a Las Vegas Algorithm that solves this problem. But first
\begin{theorem}[]{Random Colouring}
Let $G$ be a graph with a path of length $k - 1$
\begin{enumerate}[label=(\arabic*)]
\item We have $p_{\text{success}} = \Pr[\exists \text{ colourful path of length } k - 1] \geq \Pr[P \text{ is colourful}] = \frac{k!}{k^k} \geq e^{-k}$
\item The expected number of trials required to get a colourful path of length $k - 1$ is $\frac{1}{p_{\text{success}}} \leq e^k$
\end{enumerate}
\end{theorem}
For our algorithm we choose a $\lambda > 1 \in \R$ and we repeat the test at most $\ceil{\lambda e^k}$. If we succeed once, we abort and output ``\textsc{Yes}''. If we haven't succeeded in any of the trials, we output ``\textsc{No}''
\begin{theorem}[]{Random Colouring Algorithm}
\begin{itemize}
\item Time complexity: \tco{\lambda(2e)^k km}
\item If we return ``\textsc{Yes}'', the graph is \textit{guaranteed} to contain a path of length $k - 1$
\item If we return ``\textsc{No}'', the probability of false negative is $e^{-\lambda}$
\end{itemize}
\end{theorem}
\newpage
\setcounter{all}{4}
\subsubsection{Flows}
\begin{definition}[]{Network}
A \textit{Network} is a tuple $N = (\mathcal{V}, \mathcal{A}, c, s, t)$ whereby
\begin{itemize}
\item $(\mathcal{V}, \mathcal{A})$ is a directed graph
\item $c: \mathcal{A} \rightarrow \R_0^+$ the \textit{capacity function}
\item $s \in \mathcal{V}$ is the \textit{source}
\item $t \in \mathcal{V}\backslash \{s\}$ is the \textit{target}
\end{itemize}
\end{definition}
The capacity function hereby describes the maximum flow through each edge. For each vertex that is not the source or target, the flow is constant, i.e. the total amount entering vertex $v$ has to be equal to the amount exiting it again.
\begin{definition}[]{Flow}
Given a network $N = (\mathcal{V}, \mathcal{A}, c, s, t)$, a flow in said network is a function $f: \mathcal{A} \rightarrow \R$ where
\begin{align*}
0 \leq f(e) \leq c(e) \smallhspace \forall e \in \mathcal{A} \text{ The acceptability }
\end{align*}
\begin{align*}
\forall v \in \mathcal{V} \backslash \{s, t\} \sum_{u \in \mathcal{V}: (u, v) \in \mathcal{A}} f(u, v) = \sum_{u \in \mathcal{V}: (v, u) \in \mathcal{A}} f(v, u) \text{ the conservation of flow }
\end{align*}
The value of a flow $f$ is given by
\begin{align*}
\text{val}(f) := \sum_{u \in \mathcal{V}: (s, u) \in \mathcal{A}} f(s, u) - \sum_{u \in \mathcal{V}: (u, s) \in \mathcal{A}} f(u, s)
\end{align*}
We call a flow \textit{integeral} if $f(e) \in \Z \smallhspace \forall e \in \mathcal{A}$
\end{definition}
\begin{lemma}[]{Flow}
The total flow to the target equals the value of the flow, i.e.
\begin{align*}
\text{netinflow}(f) = \text{val}(f) = \sum_{u \in \mathcal{V} : (u, t) \in \mathcal{A}} f(u, t) - \sum_{u \in \mathcal{V}: (t, u) \in \mathcal{A}} f(t, u)
\end{align*}
\end{lemma}
\begin{definition}[]{$s$-$t$-cut}
An $s$-$t$-cut of a network $N = (\mathcal{V}, \mathcal{A}, c, s, t)$ is a partition $P = (\mathcal{S}, \mathcal{T})$ of $\mathcal{V}$ (i.e. $\mathcal{S} \cup \mathcal{T} = \mathcal{V}$ and $\mathcal{S} \cap \mathcal{T} = \emptyset$) where $s \in \mathcal{S}$ and $t \in \mathcal{T}$.
The capacity of a $s$-$t$-cut is then given by
\begin{align*}
\text{cap}(\mathcal{S}, \mathcal{T}) = \sum_{(u, w) \in (\mathcal{S} \times \mathcal{T}) \cap \mathcal{A}} c(u, w)
\end{align*}
\end{definition}
\begin{lemma}[]{$s$-$t$-cut}
Given $f$ is a flow and $(\mathcal{S}, \mathcal{T})$ a $s$-$t$-cut in $N = (\mathcal{V}, \mathcal{A}, c, s, t)$, we have
\begin{align*}
\text{val}(f) \leq \text{cap}(\mathcal{S}, \mathcal{T})
\end{align*}
\end{lemma}
\begin{theorem}[]{Max-flow - min-cut}
Every network $N = (\mathcal{V}, \mathcal{A}, c, s, t)$ fulfills ($f$ a flow, $(\mathcal{S}, \mathcal{T})$ an $s$-$t$-cut)
\begin{align*}
\max_{f \in N} \text{val}(f) = \min_{(\mathcal{S}, \mathcal{T}) \in N} \text{cap}(S, T)
\end{align*}
\end{theorem}
It is easier to calculate the min-cut, since there are only a finite number of $s$-$t$-cuts (albeit exponentially many, i.e. $2^{|\mathcal{V}| - 2}$), thus the importance of the above theorem.
It forms the basis of the algorithms that calculate a solution to the max-flow problem.
An approach to solve the max-flow problem is to use augmenting paths, but the issue with that is that it isn't guaranteed that we will find a max flow in a finite number of steps.
\fhlc{Cyan}{Residual capacity network}
Using a residual capacity graph also known as a residual network, we can solve the max-flow problem in polynomial time.
The concept is simple, yet ingenious: It is simply a network of the remaining capacity (or the exceeded capacity) of an edge $e \in \mathcal{A}$ for a flow $f$
\begin{definition}[]{Residual Network}
Let $N = (\mathcal{V}, \mathcal{A}, c, s, t)$ be a network without bidirectional edges and let $f$ be a flow in said network $N$. The residual network $N_f = (\mathcal{V}, \mathcal{A}_f, r_f, s, t)$ is given by
\begin{enumerate}[label=(\arabic*)]
\item If $e \in \mathcal{A}$ with $f(e) < c(e)$, then edge $e$ is also $\in \mathcal{A}_f$ whereas $r_f(e) := c(e) - f(e)$
\item If $e \in \mathcal{A}$ with $f(e) > 0$ then edge $e^{\text{opp}}$ in $\mathcal{A}_f$ whereas $r_f(e^{\text{opp}}) = f(e)$
\item Only edges as described in (1) and (2) can be found in $\mathcal{A}_f$
\end{enumerate}
We call $r_f(e), e \in \mathcal{A}$ the \textit{residual capacity} of edge $e$
\end{definition}
When reconstructing a network from the residual network, the original network is given by:
\begin{itemize}
\item The capacity of an edge $(u, v)$ in the original network is the value of $(u, v)$ and $(v, u)$ in the residual network added (if applicable), where $(u, v)$ is directed \textit{towards} the target.
\item The flow of an edge is a bit more complicated: An edge that might appear to intuitively not be part of the flow may be and vice-versa.
Flow preservation is the key: The same amount of fluid has to enter each vertex (that is not $s$ or $t$) has to also exit it again.
To check if an edge is part of the flow, simply see if the start vertex of the edge has an excess of fluid.
\item If just one edge is present, the edge is flipped, if two are present, figure out which edge was part of the flow and which one is the remaining capacity.
\end{itemize}
Note: A vertex in the residual network directed towards the \textit{target} is usually the remaining capacity!
\begin{theorem}[]{Maximum Flow}
A flow $f$ in a network $N = (\mathcal{V}, \mathcal{A}, c, s, t)$ is a maximum flow if and only if there does not exist a directed path between the source and target of the residual network.
For every such maximum flow there exists a $s$-$t$-cut with $\text{val}(f) = \text{cap}(S, T)$
\end{theorem}
\newpage
\fhlc{Cyan}{Algorithms}
Most algorithms for the max-flow problem use a residual network, where $\mathcal{A}_f$ is the edge-set in the residual network.
\begin{algorithm}
\caption{\textsc{Ford-Fulkerson}}
\begin{algorithmic}[1]
\Procedure{Ford-Fulkerson}{$\mathcal{V}, \mathcal{A}, c, s, t$}
\State $f \gets 0$ \Comment{Flow is constantly $0$}
\While{$\exists s$-$t$-path $P$ in $(\mathcal{V}, \mathcal{A}_f)$} \Comment{Augmenting path}
\State Increase flow along $P$ by minimum residual capacity in $P$
\EndWhile
\State \Return $f$ \Comment{Maximum flow}
\EndProcedure
\end{algorithmic}
\end{algorithm}
The problem with this algorithm is that it may not terminate for irrational capacities. If we however only consider integral networks without bidirectional edges, it can be easily seen that if we denote $U \in \N$ the upper bound for capacities, the time complexity of this algorithm is \tco{nUm} where \tco{m} is the time complexity for constructing residual network.
\begin{theorem}[]{Max-Flow Algorithm}
If in a network without bidirectional edges and all capacities integral and no larger than $U$, there is an integral max-flow we can compute in \tco{mnU}, whereas $m$ is the number of edges and $n$ the number of vertices in the network.
\end{theorem}
There are more advanced algorithms than this one that can calculate solutions to this problem faster or also for irrational numbers.
For the following two proposition, $m = |E|$ and $n = |V|$, i.e. $m$ is the number of edges and $n$ the number of vertices
\begin{proposition}[]{Capacity-Scaling}
If in a network all capacities are integral and at most $U$, there exists an integral max-flow that can be computed in \tco{mn(1 + \log(U))}
\end{proposition}
\begin{proposition}[]{Dynamic-Trees}
The max-flow of a flow in a network can be calculated in \tco{mn\log(n)}
\end{proposition}
\newpage
\fhlc{Cyan}{Bipartite Matching as Flow-Problem}
We can use the concepts of flows to determine matchings in bipartite graphs.
Let $G = (V, E)$ be a bipartite graph, i.e. $\exists$ Partition $(\mathcal{U}, \mathcal{W})$ of $V$ such that $E = \{\{u, w\} \divides u \in \mathcal{U}, w \in \mathcal{W}\}$.
We construct a network $N = (V \dot{\cup} \{s, t\}, \mathcal{A}, c, s, t)$, i.e. to the vertices of $G$, we added a source and a target.
The capacity function is $c(e) = 1$.
We copy the edges from $G$, having all edges be directed ones from vertices in $\mathcal{U}$ to ones in $\mathcal{W}$.
We add edges from the source $s$ to every vertex in $\mathcal{U}$ and from every vertex in $\mathcal{W}$ to the target $t$.
\begin{lemma}[]{Bipartite Matching - Max-Flow}
The maximum matching in a bipartite graph $G$ is equal to the maximum flow in the network $N$ as described above
\end{lemma}
\fhlc{Cyan}{Edge- and Vertex-disjoint paths}
We can determine the degree of the connectivity of a graph (i.e. the number of vertices / edges that have to be removed that the graph becomes disconnected) by determining how many edge- or vertex-disjoint paths exist between two vertices.
Again, using max-flow, we can solve this problem as follows:
Given an undirected graph $G = (V, E)$ and two vertices $u, v \in V : u \neq v$, we need to define our network $N = (\mathcal{V}, \mathcal{A}, c, s, t)$:
\begin{itemize}
\item Copy the vertex set $V$ and union it with two new vertices $s$ and $t$ and we thus have $\mathcal{V} = V \cup \{s, t\}$
\item Add two edges for each undirected edge in $G$, i.e. $\mathcal{A} = E \cup E'$ where $E'$ is has all edge directions reversed
\item We define the capacity function as $c(e) = 1$
\item We add two edges $(s, u)$ and $(v, t)$ and set the capacity of these edges to $|V|$. These are the two vertices between which we evaluate the edge-disjoint paths
\end{itemize}
If instead of edge-disjoint paths, we want to find \textit{vertex}-disjoint paths, we simply replace each vertex $x \in V\backslash\{u, v\}$ by $x_{\text{in}}$ and $x_{\text{out}}$ and connect all input-edges to $x_{\text{in}}$ and all output-edges of $x$ to $x_{\text{out}}$
\fhlc{Cyan}{Image segmentation}
We can also use cuts to solve image segmentation, i.e. to split background from foreground. We can translate an image to an undirected graph, since every pixel has four neighbours.
Whatever the pixel values mean in the end, we assume we can deduce two non-negative numbers $\alpha_p$ and $\beta_p$ denoting the probability that $p$ is in the foreground or background respectively.
Since this topic looks to not be too relevant for the exam, a full explanation of this topic can be found in the script on page 186-189
\fhlc{Cyan}{Flows and convex sets}
From the definition of flows we have seen, there is always \textit{at least} one flow, the flow \textbf{0}.
\begin{lemma}[]{Flows}
Let $f_0$ and $f_1$ be flows in a network $N$ and let $\lambda \in \R : 0 < \lambda < 1$, then the flow $f_{\lambda}$ given by
\begin{align*}
\forall e \in \mathcal{A} : f_{\lambda}(e) := (1 - \lambda)f_0(e) + \lambda f_1(e)
\end{align*}
is also a flow in $N$. We have
\begin{align*}
\text{val}(f_{\lambda}) = (1 - \lambda) \cdot \text{val}(f_0) + \lambda \cdot \text{val}(f_1)
\end{align*}
\end{lemma}
\begin{corollary}[]{Number of flows in networks}
\begin{enumerate}[label=(\roman*)]
\item A network $N$ has either exactly \textit{one} flow (the flow \textbf{0}) or infinitely many flows
\item A network $N$ has either exactly \textit{one} maximum flow or infinitely many maximum flows
\end{enumerate}
\end{corollary}
\shade{ForestGreen}{Convex sets}
We define a function $f: \mathcal{A} \rightarrow \R$ that induces a vector $v_f := (f(e_1), f(e_2), \ldots, f(e_m)) \in \R^m$ whereas $e_1, \ldots, e_m$ is an ordering of the vertices of $\mathcal{A}$ where $m = |\mathcal{A}|$.
We can interpret the set of (maximum) flows as a subset of $\R^m$
\begin{definition}[]{Convex set}
Let $m \in \N$
\begin{enumerate}[label=(\roman*)]
\item For $v_0, v_1 \in \R^m$ let $\overline{v_0v_1}:=\{(1 - \lambda v_0) + \lambda v_1 \divides \lambda \in \R, 0 \leq \lambda \leq 1\}$ be the \textit{line segment} connecting $v_0$ and $v_1$
\item A set $\mathcal{C} \subseteq \R^m$ is called \textit{convex} if for all $v_0, v_1 \in \mathcal{C}$ the whole line segment $\overline{v_0 v_1}$ is in $\mathcal{C}$
\end{enumerate}
\end{definition}
\textbf{Examples:} Spheres or convex Polytopes (e.g. dice or tetrahedra in $\R^3$)
\begin{theorem}[]{Convex sets}
The set of flows of a network with $m$ edges, interpreted as vectors is a convex subset of $\R^m$. The set of all maximum flows equally forms a convex subset of $\R^m$
\end{theorem}
\newpage
\subsubsection{Min-Cuts in graphs}
In the following section we use \textit{multigraphs}.
\begin{recall}[]{Multigraph}
A multigraph is an undirected, unweighted and acyclic graph $G = (V, E)$, where multiple edges are allowed to exist between the same pair of vertices.
\textit{(Instead of multiple edges, we could also allow weighted edges, but the algorithms and concepts presented here are more easily understandable using multiple edges)}
\end{recall}
\fhlc{Cyan}{Min-Cut Problem}
We define $\mu(G)$ to be the cardinality of the \textit{min-cut} (this is the problem).
This problem is similar to the min-cut problem for flows, only that we have a multigraph now. We can however replace multiple edges with a single, weighted edge, allowing us to use the algorithms discussed above.
Since we need to compute $(n - 1)$ $s$-$t$-cuts, our total time complexity is \tco{n^4 \log(n)}, since we can compute $s$-$t$-cuts in \tco{n^3\log(n)} = \tco{n\cdot m\log(n)}
\fhlc{Cyan}{Edge contraction}
Let $e = \{u, v\}$ be an edge of our usual multigraph $G$.
The \textit{contraction of} $e$ replaces the two vertices $u$ and $v$ with a single vertex denoted $x_{u, v}$, which is incident to all edges any of the two vertices it replaced were incident to, apart from the ones between the two vertices $u$ and $v$.
We call the new graph $G/e$ and $\deg_{G/e}(x_{u, v}) = \deg_G(u) + \deg_G(v) - 2k$ where $k$ denotes the number of edges between $u$ and $v$.
Of note is that there is a bijection: $\text{Edges in G without the ones between $u$ and } v \leftrightarrow \text{Edges in $G/e$}$
\begin{lemma}[]{Edge contraction}
Let $G$ be a graph and $e$ be an edge of $G$. Then we have that $\mu(G/e) \geq \mu(G)$ and we have equality if $G$ contains a min-cut $\mathcal{C}$ with $e \notin \mathcal{C}$.
\end{lemma}
\fhlc{Cyan}{Random edge contraction}
\begin{algorithm}
\caption{Random Cut where $G$ is a connected Multigraph}
\begin{algorithmic}[1]
\Procedure{Cut}{$G$}
\While{$|V(G)| > 2$} \Comment{Vertices of $G$}
\State $e \gets$ uniformly random edge in $G$
\State $G \gets G/e$
\EndWhile
\State \Return Size of a unique cut of $G$
\EndProcedure
\end{algorithmic}
\end{algorithm}
If we assume that we can perform edge contraction in \tco{n} and we can choose a uniformly random edge in $G$ in \tco{n} as well, it is evident that we can compute \textsc{Cut}($G$) in \tco{n^2}
\begin{lemma}[]{Random edge contraction}
If $e$ is uniformly randomly chosen from the edges of multigraph $G$, then we have
\begin{align*}
\Pr[\mu(G) = \mu(G/e)] \geq 1 - \frac{2}{n}
\end{align*}
\end{lemma}
\newpage
\begin{lemma}[]{Correctness of \textsc{Cut}$(G)$}
To evaluate the correctness of \textsc{Cut}$(G)$, we define
\begin{align*}
\hat{p}(G) := \text{Probability that \textsc{Cut}($G$) returns the value } \mu(G)
\end{align*}
and let
\begin{align*}
\hat{p}(n) := \inf_{G=(V, E), |V| = n}\hat{p}(G)
\end{align*}
Then, for all $n \geq 3$ we have
\begin{align*}
\hat{p} \geq \left( 1 - \frac{2}{n} \right) \cdot \hat{p}(n - 1)
\end{align*}
\end{lemma}
\begin{lemma}[]{Probability of Correctness of \textsc{Cut}$(G)$}
For all $n \geq 2$ we have $\displaystyle \hat{p}(n) \geq \frac{2}{n(n - 1)} = \frac{1}{{n \choose 2}}$
\end{lemma}
Thus, we repeat the algorithm \textsc{Cut}$(G)$ $\lambda {n \choose 2}$ times for $\lambda > 0$ and we return the smallest value we got.
\begin{theorem}[]{\textsc{Cut}$(G)$}
For the algorithm that runs \textsc{Cut}$(G)$ $\lambda{n \choose 2}$ times we have the following properties:
\begin{enumerate}[label=(\arabic*)]
\item Time complexity: \tco{\lambda n^4}
\item The smallest found value is with probability at least $1 - e^{-\lambda}$ equal to $\mu(G)$
\end{enumerate}
\end{theorem}
If we choose $\lambda = \ln(n)$, we have time complexity \tco{n^4 \ln(n)} with error probability \textit{at most} $\frac{1}{n}$
Of note is that for low $n$, it will be worth it to simply deterministically determine the min-cut
\newpage
\subsection{Geometric Algorithms}
\subsubsection{Smallest enclosing circle}
\newcommand{\cplane}{C^\text{\textbullet}}
Given a set $P$ of $n$ points in a plane, we want to find a circle $C(P)$ such that $C(P)$ encloses all points of $P$ and whose \textit{radius} is minimal (SEC).
For a circle $C$ we use $\cplane$ for the circular plane enclosed by $C$, including $C$ and we also allow the points in $P$ to lay on $C(P)$.
\begin{lemma}[]{Existence of distinct SEC}
For each (finite) set of points $P$ in $\R^2$ there exists a distinct smallest enclosing circle $C(P)$
\end{lemma}
The below lemma forms the basis of our algorithm
\begin{lemma}[]{Subset of set of points}
For each (finite) set of points $P$ in $\R^2$ with $|P| \geq 3$ there exists a subset $Q \subseteq P$ such that $|Q| = 3$ and $C(Q) = C(P)$
\end{lemma}
\begin{algorithm}
\caption{Smallest Enclosing Circle $C(P)$}
\begin{algorithmic}[1]
\Procedure{CompleteEnumeration}{$P$}
\For{\textbf{all} $Q \subseteq P$ with $|Q| = 3$}
\State determine $C(Q)$
\If{$P \subseteq \cplane(Q)$}
\State \Return $C(Q)$
\EndIf
\EndFor
\EndProcedure
\end{algorithmic}
\end{algorithm}
The above algorithm has time complexity \tco{n^4}, which is due to having to check at most ${n \choose 3}$ sets $Q$, for which we can determine $C(Q)$ in constant time, since $|Q| = 3$. We then have to check for each point in $P$ if it is contained in $\cplane(Q)$, which we can do in \tco{n}, since we can check for each point in constant time.
This is quite inefficient and using a randomized algorithm, we can achieve \tco{n \ln(n)}
\begin{algorithm}
\caption{Smallest Enclosing Circle $C(P)$ randomized}
\begin{algorithmic}[1]
\Procedure{CompleteEnumerationRandomized}{$P$}
\While{\textit{always}}
\State select $Q \subseteq P$ with $|Q| = 11$ uniformly randomly
\State determine $C(Q)$
\If{$P \subseteq \cplane(Q)$}
\State \Return $C(Q)$
\EndIf
\State double all points in $P$ outside $C(Q)$
\EndWhile
\EndProcedure
\end{algorithmic}
\end{algorithm}
To implement this algorithm, we also need the doubling function: We initialize an array \verb|num| to length $n$ to $1$. \verb|num[i]| now contains the number of copies of the $i$-th point. If the point lays outside $C(Q)$, we can simply set \verb|num[i] = 2 * num[i]|
\newpage
To select $11$ points uniformly randomly in \tco{n}, consider
\begin{lemma}[]{Uniformly random $Q$}
Let $n_1, \ldots, n_t \in \N$ and $N := \sum_{i = 1}^{t} n_i$. We generate $X \in \{1, \ldots, t\}$ randomly as seen in Algorithm \ref{alg:t-uniformly-random-numbers}
Then we have $\Pr[X = i] = \frac{n_i}{N}$ for all $i = 1, \ldots, t$
\end{lemma}
\begin{algorithm}
\caption{Choose uniformly random number}\label{alg:t-uniformly-random-numbers}
\begin{algorithmic}[1]
\Procedure{ChooseUniformlyRandom}{$t$}
\State \Call{UniformInt}{$1, N$}
\State $x \gets 1$
\While{$\sum_{i = 1}^{x} n_i < k$}
\State $x \gets x + 1$
\EndWhile
\State \Return $x$
\EndProcedure
\end{algorithmic}
\end{algorithm}
\begin{lemma}[]{Uniformly random $Q$}
Let $P$ be a set of $n$ not necessarily different points and for $r \in \N$ let $R$ be uniformly randomly chosen from ${P \choose r}$. Then, the expected number of points of $P$ that are outside $C(R)$ is at most $3 \frac{n - r}{r + 1} \leq 3\frac{n}{r + 1}$
\end{lemma}
\begin{theorem}[]{Correctness of \textsc{CompleteEnumerationRandomized}}
The algorithm \textsc{CompleteEnumerationRandomized} computes the smallest enclosing circle of $P$ in time \tco{n \log(n)}
\end{theorem}
\fhlc{Cyan}{Sampling Lemma}
\begin{definition}[]{Sampling Lemma helpers}
Given a finite set $S$, $n := |S|$ and $\phi$ be an arbitrary function on $2^S$ in an arbitrary range. We define
\begin{align*}
V(R) = V_{\phi}(R) & := \{ s \in S \divides \phi(R) \cup \{s\} \neq \phi(R) \} \\
X(R) = X_{\phi}(R) & := \{ s \in S \divides \phi(R) \backslash \{s\} \neq \phi(R) \}
\end{align*}
We call elements in $V(R)$ \textit{violators of} $R$ and elements in $X(R)$ we \textit{call external in} $R$
\end{definition}
The sampling lemma connects the expected number of violating elements to the number of external elements.
\begin{lemma}[]{Sampling Lemma}
Let $k \in \N, 0 \leq k \leq n$. We set $v_k := \E[|V(R)|]$ and $x_k := \E[|X(R)|]$ where $R$ is a subset of $S$ of cardinality $k$, uniformly randomly selected from ${s \choose k}$. Then we have for $r \in \N, 0 \leq r <n$
\begin{align*}
\frac{v_r}{n - r} = \frac{x_{r + 1}}{r + 1}
\end{align*}
\end{lemma}
\begin{corollary}[]{Expected numbers / ranks}
If we choose $r$ elements $R$ from a set $A$ of $n$ numbers randomly, the expected rank of the minimum of $R$ in $A$ is exactly $\frac{n - r}{r + 1} + 1 = \frac{n + 1}{r + 1}$.
If we choose $r$ points $R$ of a set $P$ of $n$ points in a plane randomly, the expected number of points of $P$ that are outside $C(R)$ is at most $3 \frac{n - r}{r + 1}$
\end{corollary}
\newpage
\subsubsection{Convex hull}
\begin{definition}[]{Convex hull}
Let $S \subseteq \R^d, d \in \N$. The \textit{convex hull}, $\text{conv}(S)$ of $S$ is the cut of all convex sets that contains $S$, i.e.
\begin{align*}
\text{conv}(S) := \bigcap_{S\subseteq C \subseteq \R^d} C
\end{align*}
where $C$ is convex
\end{definition}
A convex hull is always convex again.
\setcounter{all}{37}
\begin{theorem}[]{JarvisWrap}
The \verb|JarvisWrap| algorithm can find the convex hull in \tco{nh}, where $h$ is the number of corners of $P$ and $P$ is a set of $n$ points in any position in $\R^2$.
\end{theorem}
\begin{algorithm}
\caption{JarvisWrap}
\begin{algorithmic}[1]
\Procedure{FindNext}{$q$}
\State Choose $p_0 \in P\backslash\{q\}$ arbitrarily
\State $q_{next} \gets p_0$
\For{\textbf{all} $p \in P\backslash \{q, p_0\}$}
\If{$p$ is to right of $q q_{next}$}
$q_{next} \gets p$ \Comment{$qq_{next}$ is vector from $q$ to $q_{next}$}
\EndIf
\EndFor
\State \Return $q_{next}$
\EndProcedure
\Procedure{JarvisWrap}{$P$}
\State $h \gets 0$
\State $p_{now} \gets$ point in $P$ with lowest $x$-coordinate
\While{$p_{now} \neq q_0$}
\State $q_h \gets p_{now}$
\State $p_now \gets$ \Call{FindNext}{$q_h$}
\State $h \gets h + 1$
\EndWhile
\State \Return $(q_0, q_1, \ldots, q_{h - 1})$
\EndProcedure
\end{algorithmic}
\end{algorithm}
\newpage
\fhlc{Cyan}{LocalRepair}
\begin{algorithm}
\caption{LocalRepair}
\begin{algorithmic}[1]
\Procedure{LocalRepair}{$p_1, \ldots, p_n$}
\State $q_0 \gets q_1$ \Comment{input sorted according to $x$-coordinate}
\State $h \gets 0$
\For{$i \in \{2, \ldots, n\}$} \Comment{Lower convex hull, left to right}
\While{$h > 0$ and $q_h$ is to the left of $q_{h - 1}p_i$} \Comment{$q_{h - 1}p_i$ is a vector}
\State $h \gets h - 1$
\EndWhile
\State $h \gets h + 1$
\State $q_h \gets p_i$ \Comment{$(q_0, \ldots, q_h)$ is lower convex hull of $\{p_1, \ldots, p_i\}$}
\EndFor
\State $h' \gets h$
\For{$i \in \{n - 1, \ldots, 1\}$} \Comment{Upper convex hull, right to left}
\While{$h > h'$ and $q_h$ is to the left of $q_{h - 1}p_i$} \Comment{$q_{h - 1}p_i$ is a vector}
\State $h \gets h - 1$
\EndWhile
\State $h \gets h + 1$
\State $q_h \gets p_i$
\EndFor
\State \Return $(q_0, \ldots, q_{h - 1})$ \Comment{The corners of the convex hull, counterclockwise}
\EndProcedure
\end{algorithmic}
\end{algorithm}
\setcounter{all}{42}
\begin{theorem}[]{LocalRepair}
The \verb|LocalRepair| algorithm can find the convex hull of a (in respect to the $x$-coordinate of each point) sorted set $P$ of $n$ points in $\R^2$ in \tco{n}.
\end{theorem}
The idea of the \verb|LocalRepair| algorithm is to repeatedly correct mistakes in the originally randomly chosen polygon.
The improvement steps add edges to the polygon such that eventually all vertices lay withing the smallest enclosing circle of the polygon.

10
semester2/algorithms-and-probability/parts/coding.tex Normal file
View File

@@ -0,0 +1,10 @@
\newsection
\section{Coding}
\label{sec:implementation}
\subsection{Tricks}
\begin{itemize}
\item Encoding subsets is very easy using bitshift operators. We can use \verb|(i & (1 << j)) != 0| in combination with a loop to go through all subsets of the set. The for-loop will have to go from $i = 1$ (if we want to exclude the empty set) to $i = 2^k - 1$
\item Inclusion-Excluison is very powerful
\item Always check the what the input values are and expect them to provide bad code (e.g. provided code reads ints, even though we need to read doubles (or long)). An easy way to check if everything is correct, is to print the data import's results and compare with the input files if something looks incorrect
\item DP (as always) can come in very handy for solving probabilities related problems
\end{itemize}

120
semester2/algorithms-and-probability/parts/combinatorics.tex Normal file
View File

@@ -0,0 +1,120 @@
\newsection
\section{Combinatorics}
\subsection{Introduction}
Combinatorics was developed from the willingness of humans to gamble and the fact that everybody wanted to win as much money as possible.
\subsection{Simple counting operations}
The easiest way to find the best chance of winning is to write down all possible outcomes. This can be very tedious though when the list gets longer.
We can note this all down as a list or as a tree diagram. So-called Venn Diagrams might also help represent the relationship between two sets or events. Essentially a Venn Diagram is a graphical representation of set operations such as $A \cup B$.
\subsection{Basic rules of counting}
\subsubsection{Multiplication rule}
If one has $n$ possibilities for a first choice and $m$ possibilities for a second choice, then there are a total of $n \cdot m$ possible combinations.
When we think about a task, and we have an \textbf{and} in between e.g. properties, we need to multiply all the options.
\subsubsection{Addition rule}
If two events are mutually exclusive, the first has $n$ possibilities and the second one has $m$ possibilities, then both events together have $n+m$ possibilities.
When we think about a task, and we have an \textbf{or} in between e.g. properties, then we need to add all the options.
\newpage
\subsection{Factorial}
\begin{definition}[]{Factorial}
The factorial stands for the product of the first $n$ natural numbers where $n \ge 1$. Notation: $!$
\[
n! = n \cdot (n - 1) \cdot (n - 2) \cdot \ldots \cdot 3 \cdot 2 \cdot 1
\]
Additionally, $0! = 1$. We read $n!$ as ``\textit{n factorial}''
\end{definition}
\subsubsection{Operations}
We can rewrite $n!$ as $n \cdot (n - 1)!$ or $n \cdot (n - 1) \cdot (n - 2)!$ and so on.
It is also possible to write $7 \cdot 6 \cdot 5$ with factorial notation: $\displaystyle \frac{7!}{4!}$, or in other words, for any excerpt of a factorial sequence: \[n \cdot (n - 1) \cdot \ldots \cdot m = \frac{n!}{(m - 1)!}\]
\subsection{Permutations}
\begin{definition}[]{Permutations}
A permutation of a group is any possible arrangement of the group's elements in a particular order\\
\textbf{Permutation rule without repetition:} The number of $n$ \textbf{\textit{distinguishable}} elements is defined as: $n!$
\end{definition}
\subsubsection{Permutation with repetition}
For $n$ elements $n_1,n_2,\ldots,n_k$ of which some are identical, the number of permutations can be calculated as follows:
\[
p = \frac{n!}{n_1! \cdot n_2! \cdot \ldots \cdot n_k!}
\]
where $n_k$ is the number of times a certain element occurs.
As a matter of fact, this rule also applies to permutations without repetition, as each element occurs only once, which means the denominator is $1$, hence $\displaystyle \frac{n!}{(1!)^n} = n!$
\inlineex \smallhspace CANADA has $6$ letters, of which $3$ letters are the same. So the word consists of $3$ A's, which can be arranged in $3!$ different ways, a C, N and D, which can be arranged in $1!$ ways each. Therefore, we have:
\[
\frac{6!}{3!\cdot 1! \cdot 1! \cdot 1!} = \frac{6!}{3!} = 6 \cdot 5 \cdot 4 = 120
\]
Since $1!$ equals $1$, we can always ignore all elements that occur only once, as they won't influence the final result.
\newpage
\subsection{Variations}
\begin{definition}[]{Variations}
A \textbf{\textit{variation}} is a selection of $k$ elements from a universal set that consists of $n$ \textit{distinguishable} elements.\\
\textbf{Variation rule without repetition:} The $_n\mbox{P}_k$ function is used to \textit{\textbf{place}} $n$ elements on $k$ places. In a more mathematical definition:
The number of different variations consisting of $k$ different elements selected from $n$ distinguishable elements can be calculated as follows:
\[
\frac{n!}{(n - k)!} = _n\mbox{P}_k
\]
\end{definition}
\subsubsection{Variations with repetition}
If an element can be selected more than once and the order matters, the number of different variations consisting of $k$ elements selected from $n$ distinguishable elements can be calculated using $n^k$
\subsection{Combinations}
\begin{definition}[]{Combination}
A combination is a selection of $k$ elements from $n$ elements in total without any regard to order or arrangement.
\textbf{Combination rule without repetition:} \[
_n\mbox{C}_k = {n\choose k} = \frac{_n\mbox{P}_k}{k!} = \frac{n!}{(n - k)! \cdot k!}
\]
\end{definition}
\subsubsection{Combination with repetition}
In general the question to ask for combinations is, in how many ways can I distribute $k$ objects among $n$ elements?
\[
_{n + k - 1}\mbox{C}_k = {n + k - 1\choose k} = \frac{(n + k - 1)!}{k!(n - 1)!}
\]
\subsection{Binomial Expansion}
\label{sec:binomial-expansion}
Binomial expansion is usually quite hard, but it can be much easier than it first seems. The first term of the expression of $(a + b)^n$ is always $1 a^n b^0$. Using the formula for combination without repetition, we can find the coefficients of each element:
\begin{center}
\includegraphics[width=0.6\linewidth]{./assets/binomialExpansion.png}
\end{center}
This theory is based on the Pascal's Triangle and the numbers of row $n$ correspond to the coefficients of each element of the expanded term.
We can calculate the coefficient of each part of the expanded term $k$ with combinatorics as follows: $\displaystyle {n\choose k}$
\begin{formula}[]{Binomial Expansion}
\textbf{\textit{\underbar{In general:}}}
\[
(a + b)^n = 1a^nb^0 + {n\choose 1} a^{n-1}b^{1} + {n\choose 2} a^{n-2}b^{2} + \ldots + {n\choose n - 1} a^{1}b^{n - 1} + {n\choose n} a^{0}b^{n}
\]
\end{formula}
\subsection{Overview}
\includegraphics[width=1\linewidth]{./assets/overview.png}

566
semester2/algorithms-and-probability/parts/graphs.tex Normal file
View File

@@ -0,0 +1,566 @@
\newsection
\section{Graphs}
\subsection{Repetition}
The graph algorithms of Kruskal, Prim, Boruvka, Dijkstra, Bellman-Ford, Floyd-Warshall and Johnson are still of importance. See A \& D summary for them
\begin{definition}[]{$\alpha$-approximation algorithm}
This kind of algorithm approximates the result to level $\alpha$, where $\alpha$ is the factor by which the result is off. Ideally, $\alpha = 1$, for a perfect approximation, which in reality is hardly ever, if ever possible.
\end{definition}
\setcounter{subsection}{3}
\subsection{Connectivity}
\setcounter{all}{23}
% Page 37
\begin{definition}[]{$k$-Connected graph}
A graph $G = (V, E)$ is called $k$\textit{-connected} if $|V| \geq k + 1$ and for all subsets $X \subseteq V$ with $|X| < k$ we have that the graph $G[V\backslash X]$ is connected
\end{definition}
If $G[V\backslash X]$ is \bi{not} connected, we call it a \textit{(vertex)-separator}.
A $k$-connected graph is also $k - 1$ connected, etc
\begin{definition}[]{$k$-edge-connected}
A graph $G = (V, E)$ is called $k$\textit{-edge-connected} if for all subsets $X \subseteq E$ with $|X| < k$ we have: The graph $(V, E\backslash X)$ is connected
\end{definition}
We also have \textit{edge-separators}, defined analogously to above, i.e. the set $X$ above is called a $u$-$v$-edge-separator if by removing it from the graph, $u$ and $v$ no longer lay in the same connected component.
If a graph is $k$-edge-connected, it is also $k - 1$-edge-connected, etc
\begin{theorem}[]{Menger's theorem}
Let $G = (V, E)$ be a graph and $u \neq v \in V$. Then we have
\begin{enumerate}
\item Every $u$-$v$-vertex-separator has size at least $k \Leftrightarrow$ Exist at least $k$ internally vertex-disjoint $u$-$v$-paths
\item Every $u$-$v$-edge-separator has size at least $k \Leftrightarrow$ Exist at least $k$ edge-disjoint $u$-$v$-paths
\end{enumerate}
\end{theorem}
\newpage
\subsubsection{Articulation points}
If a graph is connected, but not $2$-connected, there exists at least one vertex $v$ for which, if removed, the graph is not connected, called \textit{articulation points}. Using a modified DFS, we can find these vertices. Instead of just setting a flag for each vertex we visit, we set a number indicating the order in which the vertices were visited. The first vertex we visit, for instance, gets number $1$.
We also add another value, which we call \verb|low[v]| $:=$ the smallest DFS-number that can be reached from $v$ using a path of arbitrarily many edges of the DFS-tree and at most one residual edge.
We also have (where $s$ is the start / source vertex and $E(T)$ is the edge set of the DFS-tree):
$v$ is an articulation point $\Leftrightarrow$ ($v = s$ and $s$ has degree at least $2$ in $T$) or ($v \neq s$ and there exists $w \in V$ with $\{(v, w)\} \in E(T)$ and $\verb|low[w]| \geq \verb|dfs[v]|$)
\begin{algorithm}
\caption{\textsc{FindArticulationPoints}($G, s$)}
\begin{algorithmic}[1]
\State $\forall v \in V$: \texttt{dfs[v]}$\gets 0$ \Comment{Stores dfs number for vertex $v$ and also flag for visit}
\State $\forall v \in V$: \texttt{low[v]}$\gets 0$ \Comment{Stores low point for vertex $v$}
\State $\forall v \in V$: \texttt{isArticulationPoint[v]}$\gets \texttt{false}$ \Comment{Indicates if vertex $v$ is articulation point}
\State \texttt{num} $\gets 0$
\State $T \gets \emptyset$ \Comment{Depth-First-Search Tree}
\State \Call{DFS-Visit}{$G, s$}
\If{$s$ has degree at least two in $T$} \Comment{Start vertex classification could be incorrect from \textsc{DFS-Visit}}
\State $\texttt{isArticulationPoint[s]} \gets \texttt{true}$
\Else
\State $\texttt{isArticulationPoint[s]} \gets \texttt{false}$
\EndIf
\Procedure{DFS-Visit}{$G, v$}
\State \texttt{num} $\gets$ \texttt{num} $+ 1$
\State \texttt{dfs[v]} $\gets$ \texttt{num}
\State \texttt{low[v]} $\gets$ \texttt{dfs[v]}
\For{\textbf{all} $\{v, w\} \in E$}
\If{\texttt{dfs[w]} = 0}
\State $T \gets T \cup \{\{v, w\}\}$
\State $\texttt{val} \gets$ \Call{DFS-Visit}{$G, w$}
\If{$\texttt{val} \geq \texttt{dfs[v]}$} \Comment{Check articulation point condition}
\State $\texttt{isArticulationPoint[v]} \gets \texttt{true}$
\EndIf
\State $\texttt{low[v]} \gets \min \{\texttt{low[v]}, \texttt{val}\}$
\ElsIf{$\texttt{dfs[w]} \neq 0$ \textbf{and} $\{v, w\} \notin T$} \Comment{Update low if already visited}
\State $\texttt{low[v]} \gets \min \{\texttt{low[v]}, \texttt{dfs[w]}\}$
\EndIf
\EndFor
\State \Return \texttt{low[v]}
\EndProcedure
\end{algorithmic}
\end{algorithm}
\stepcounter{all}
\begin{theorem}[]{Articulation points Computation}
For a connected graph $G = (V, E)$ that is stored using an adjacency list, we can compute all articulation points in \tco{|E|}
\end{theorem}
\newpage
\subsubsection{Bridges}
While articulation points show that a graph is not $2$-connected, bridges shows that a graph isn't $2$-edge-connected.
In other words, they are certificates for the graph \textit{not} being $2$-edge-connected or more formally:
\begin{center}
\fbox{
\textit{
An edge $e \in E$ in a connected graph $G = (V, E)$ is called a bridge if the graph $(V, E\backslash \{e\})$ is not connected.
}
}
\end{center}
From the definition of bridges we immediately have that a spanning tree has to contain all bridges of a graph, and we can also state that only edges of a Depth-First-Search-Tree are possible candidates for a bridge.
The idea now is that every vertex contained in a bridge is either an articulation point or has degree $1$ in $G$. Or more formally:
\begin{center}
\fbox{
\textit{
A directed edge $(v, w)$ of Depth-First-Search-Tree $T$ is a bridge if and only if $\texttt{low[w]} > \texttt{dfs[v]}$
}
}
\end{center}
\begin{theorem}[]{Bridges Computation}
For a connected graph $G = (V, E)$ that is stored using an adjacency list, we can compute all bridges and articulation points in \tco{|E|}
\end{theorem}
\subsubsection{Block-Decomposition}
\begin{definition}[]{Block-Decomposition}
Let $G = (V, E)$ be a connected graph. For $e, f \in E$ we define a relation by
\begin{align*}
e \sim f \Longleftrightarrow e = f \text{ or exists a common cycle through $e$ and } f
\end{align*}
Then this relation is an equivalence relation and we call the equivalence classes \textit{blocks}, sometimes also known as $2$-connectivity-components
\end{definition}
It is now evident that two blocks, if even, can only intersect in one articulation point.
The Block-Decomposition is given by:
\begin{center}
\fbox{
\parbox{15cm}{
Let $T$ be a bipartite graph (in this case a tree), with $V = A \uplus B$ where $A$ is the set of articulation points of $G$ and $B$ the set of blocks of $G$ (This means that every block in $G$ is a vertex in $V$).
We connect a vertex $a \in A$ with a block $b \in B$ if and only if $a$ is incident to an edge in $b$.
$T$ is connected if $G$ is and it is free of cycles if $G$ is free of cycles, since every cycle is translatable to a cycle in $G$
}
}
\end{center}
The algorithm to determine bridges and articulation points can again be reused and allows us to determine a Block-Decomposition in linear time.
\newpage
\subsection{Cycles}
\subsubsection{Eulerian cycles / circuits}
\begin{definition}[]{Eulerian cycle}
A \textit{eulerian cycle} in a graph $G = (V, E)$ is a circuit (closed cycle) that contains each edge exactly once.
If a graph contains a eulerian cycle, we call it \textit{eulerian}
\end{definition}
If $G$ contains a eulerian cycle, $\deg(v)$ of all vertices $v \in V$ is even. For connected graph, we even have a double-sided implication.
If we combine the entirety of the explanations of pages 43-45 in the script, we reach the following algorithm, where $N_G(v)$ is the function returning the neighbours of vertex $v$ in graph $G$:
\begin{algorithm}
\caption{\textsc{EulerianCycle}$(G, v_{start})$}
\begin{algorithmic}[1]
\Procedure{RandomCycle}{$G, v_{start}$}
\State $v \gets v_{start}$
\State $W \gets \langle v \rangle$ \Comment{Prepare the cycle (add the start vertex to it)}
\While{$N_G(v) \neq \emptyset$}
\State Choose $v_{next}$ arbitrarily from $N_G(v)$ \Comment{Choose arbitrary neighbour}
\State Attach $v_{next}$ to the cycle $W$
\State $e \gets \{v, v_{next}\}$
\State Delete $e$ from $G$
\State $v \gets v_{next}$
\EndWhile
\State \Return $W$
\EndProcedure
\State $W \gets$ \Call{RandomCycle}{$v_{start}$} \Comment{Fast runner}
\State $v_{slow} \gets$ start vertex of $W$
\While{$v_{slow}$ is not the last vertex in $W$}
\State $v \gets$ successor of $v_{slow}$ in $W$
\If{$N_G(v) \neq \emptyset$}
\State $W' \gets$ \Call{RandomCycle}{v}
\State $W \gets W_1 + W' + W_2$ \Comment{We union the different branches of the Euler cycle}
\EndIf
\State $v_{slow} \gets$ successor of $v_{slow}$ in $W$
\EndWhile
\State \Return $W$
\end{algorithmic}
\end{algorithm}
\begin{theorem}[]{Eulerian Graph}
\begin{enumerate}[label=\alph*)]
\item A connected graph $G$ is eulerian if and only if the degree of all vertices is even
\item In a connected eulerian graph, we can find a eulerian cycle in \tco{|E|}
\end{enumerate}
\end{theorem}
\newpage
\subsubsection{Hamiltonian Cycles}
\begin{definition}[]{Hamiltonian Cycle}
A \textit{Hamiltonian Cycle} in a graph $G = (V, E)$ is a cycle passing through each vertex \textit{exactly once}.
If a graph contains a Hamiltonian cycle, we call it \textit{Hamiltonian}
\end{definition}
A classic example here is the Travelling Salesman Problem (TSP), covered later on.
The issue with Hamiltonian cycles is that the problem is $\mathcal{N}\mathcal{P}$-complete, thus it is assumed that there does not exist an algorithm that can determine if a graph is Hamiltonian in polynomial time.
\stepcounter{all}
\begin{theorem}[]{Hamiltonian Cycle Algorithm}
The algorithm \textsc{HamiltonianCycle} is correct and has space complexity \tco{n \cdot 2^n} and time complexity \tco{n^2 \cdot 2^n}, where $n = |V|$
\end{theorem}
In the below algorithm, $G = (V, E)$ is a graph for which $V = [n]$ and $N(v)$ as usual the neighbours of $v$ and we define $S$ as a subset of the vertices of $G$ with $1 \in S$.
We define
\begin{align*}
P_{S, x} := \begin{cases}
1 & \text{exists a $1$-$x$-path in $G$ that contains exactly the vertices of $S$} \\
0 & \text{else}
\end{cases}
\end{align*}
We then have:
\begin{align*}
G \text{ contains a Hamiltonian cycle } \Longleftrightarrow \exists x \in N(1) \text{ with } P_{[n], x} = 1
\end{align*}
Or in words, a graph contains a Hamiltonian Cycle if and only if for any of the neighbours of vertex $1$, our predicate $P_{S, x} = 1$ for $S = V = [n]$ and $x$ being that vertex in the neighbours set $N(1)$.
This means, we have found a recurrence relation, albeit an exponential one.
\begin{algorithm}
\caption{\textsc{HamiltonianCycle}$(G=([n], E))$}
\begin{algorithmic}[1]
\For{\textbf{all} $x \in [n], x \neq 1$} \Comment{Initialization}
\State $\displaystyle P_{\{1, x\}, x} :=
\begin{cases}
1 & \text{if } \{1, x\} \in E \\
0 & \text{else}
\end{cases}$
\EndFor
\For{$s = 3, \ldots, n$} \Comment{Recursion}
\For{\textbf{all} $S \subseteq [n]$ with $1 \in S$ and $|S| = s$} \Comment{See implementation notes in Section \ref{sec:implementation}}
\For{\textbf{all} $x \in S, x\neq 1$} \Comment{Fill table for all $x$ in the subset}
\State $P_{S, x} = \max\{P_{S\backslash \{x\}, x'} \divides x' \in S \cap N(x), x' \neq 1\}$
\EndFor
\EndFor
\EndFor
\If{$\exists x \in N(1)$ with $P_{[n], x} = 1$} \Comment{Check condition}
\State \Return \verb|true|
\Else
\State \Return \verb|false|
\EndIf
\end{algorithmic}
\end{algorithm}
\newpage
\fhlc{Cyan}{Improved algorithm}
There are algorithms that can find Hamiltonian cycles without using exponential memory usage.
The concept for that is the inclusion-exclusion principle (more on that in Section \ref{sec:prob-basics})
\begin{theorem}[]{Inclusion-Exclusion-Principle}
For finite sets $A_1, \ldots, A_n$ ($n \geq 2$) we have
\begin{align*}
\left| \bigcup_{i = 1}^n A_i \right| & = \sum_{l = 1}^{n}\left((-1)^{l + 1} \sum_{1 \leq i_1 < \dots < i_l \leq n} |A_{i_1} \cap \ldots \cap A_{i_l}|\right) \\
& = \sum_{i = 1}^{n}|A_i| - \sum_{1 \leq i_1 < i_2 \leq n}|A_{i_1} \cap A_{i_2}| + \sum_{1 \leq i_1 < i_2 < i_3 \leq n} |A_{i_1} \cap A_{i_2} \cap A_{i_3}| - \ldots + (-1)^{n + 1} \cdot |A_1 \cap \ldots \cap A_n|
\end{align*}
\end{theorem}
Since it is easier to find walks compared to paths, we define for all subsets $S \subseteq [n]$ with $v \notin S$ for a start vertex $s \in V$
\begin{align*}
W_S := \{ \text{\textit{walks} of length $n$ in $G$ with start and end vertex $s$ that doesn't visit any vertices of }S \}
\end{align*}
We thus reach the following algorithm:
\begin{algorithm}
\caption{\textsc{CountHamiltionianCycles}$(G = ([n], E))$}
\begin{algorithmic}[1]
\State $s \gets 1$ \Comment{Start vertex, can be chosen arbitrarily}
\State $Z \gets |W_{\emptyset}|$ \Comment{All possible paths with length $n$ in $G$}
\For{\textbf{all} $S \subseteq [n]$ with $s \notin S$ and $S \neq \emptyset$}
\State Compute $|W_S|$ \Comment{With adjacency matrix of $G[V\backslash S]$}
\State $Z \gets Z + (-1)^{|S|}|W_S|$ \Comment{Inclusion-Exclusion}
\EndFor
\State $Z \gets \frac{Z}{2}$ \Comment{There are two cycles for each \textit{true} cycle (in both directions, we only care about one)}
\State \Return $Z$ \Comment{The number of Hamiltonian cycles in $G$}
\end{algorithmic}
\end{algorithm}
\begin{theorem}[]{Count Hamiltionian Cycles Algorithm}
The algorithm computes the number of Hamiltonian cycles in $G$ with space complexity \tco{n^2} and time complexity \tco{n^{2.81}\log(n) \cdot 2^n}, where $n = |V|$
\end{theorem}
The time complexity bound comes from the fact that we need \tco{\log(n)} matrix multiplications to compute $|W_S|$, which can be found in entry $(s, s)$ in $(A_S)^n$, where $A_S$ is the adjacency matrix of the induced subgraph $G[V\backslash S]$.
Each matrix multiplication can be done in \tco{n^{2.81}} using Strassen's Algorithm.
The $2^n$ is given by the fact that we have that many subsets to consider.
\newpage
\subsubsection{Special cases}
\stepcounter{all}
\begin{lemma}[]{Bipartite graph}
If $G = (A \uplus B, E)$ is a bipartite graph with $|A| \neq |B|$, $G$ cannot contain a Hamiltonian cycle
\end{lemma}
A hypercube $H_d$ with dimension $d$ has the vertex set $\{0, 1\}^d$.
Two vertices are connected if and only if their $0$-$1$-sequences differ in exactly one bit.
\begin{center}
\fbox{
\textit{
Every hypercube of dimension $d \geq 2$ has a Hamiltonian cycle
}
}
\end{center}
Grid graphs (also known as mesh graphs) are graphs laid out in a (typically) square grid of size $m \times n$
\begin{center}
\fbox{
\parbox{15cm}{
A grid graph contains a Hamiltonian cycle if and only if $n$ or $m$ (or both) are even. If both are odd, there is no Hamiltonian cycle
}
}
\end{center}
\stepcounter{all}
\begin{theorem}[]{Dirac}
If $G$ is a graph with $|V| \geq 3$ vertices, for which every vertex has at least $\frac{|V|}{2}$ neighbours, $G$ is Hamiltonian.
\end{theorem}
In other words, every graph with minimum degree $\frac{|V|}{2}$ is Hamiltonian.
\subsubsection{Travelling Salesman Problem}
Given a graph $K_n$ and a function $\displaystyle l : {[n] \choose 2} \rightarrow \N_0$ that assigns a length to each edge of the graph, we are looking for a Hamiltonian cycle $C$ in $K_n$ with
\begin{align*}
\sum_{e \in C} l(e) = \min \left\{ \sum_{e \in C'} l(e) \divides C' \text{ is a Hamiltonian cycle in } K_n \right\}
\end{align*}
In words, we are looking for the hamiltonian cycle with the shortest length among all hamiltonian cycles.
\stepcounter{all}
\begin{theorem}[]{Travelling Salesman Problem}
If there exists for $\alpha > 1$ a $\alpha$-approximation algorithm for the travelling salesman problem with time complexity \tco{f(n)}, there also exists an algorithm that can decide if a graph with $n$ vertices is Hamiltonian in \tco{f(n)}.
\end{theorem}
This obviously means that this problem is also $\mathcal{N}\mathcal{P}$-complete.
If we however use the triangle-inequality $l(\{x, z\}) \leq l(\{x, y\}) + l(\{y, z\}))$, which in essence says that a direct connection between two vertices has to always be shorter or equally long compared to a direct connection (which intuitively makes sense),
we reach the metric travelling salesman problem, where, given a graph $K_n$ and a function $l$ (as above, but this time respecting the triangle-inequality), we are again looking for the same answer as for the non-metric problem.
\begin{theorem}[]{Metric Travelling Salesman Problem}
There exists a $2$-approximation algorithm with time complexity \tco{n^2} for the metric travelling salesman problem.
\end{theorem}
\shortproof This algorithm works as follows: Assume we have an MST and we walk around the outside of it.
Thus, the length of our path is $2$ \verb|mst|($K_n, l$).
If we now use the triangle inequality, we can skip a few already visited vertices and at least not lengthen our journey around the outside of the MST.
Any Hamiltonian cycle can be transformed into an MST by removing an arbitrary edge from it.
Thus, for the optimal length (minimal length) of a Hamiltonian cycle, we have $\text{opt}(K_n, l) \geq \verb|mst|(K_n, l)$.
If we now double the edge set (by duplicating each edge), then, since for $l(C) = \sum_{e \in C} l(e)$ for our Hamiltonian cycle $C$, we have $l(C) \leq 2 \text{opt}(K_n, l)$, we can simply find a eulerian cycle in the graph in \tco{n}, and since it takes \tco{n^2} to compute an MST, our time complexity is \tco{n^2}
\newpage
\subsection{Matchings}
Matchings are assignment problems, which could take the form of assigning a job to a specific CPU core or system.
That system will have to fulfill the performance requirements of the task, but performance should not be wasted, as there could be a different job with higher performance requirements that would not be processable simultaneously otherwise.
\begin{definition}[]{Matching}
An edge set $M \subseteq E$ of a graph $G$ is called a \textit{matching} if no vertex of the graph is assigned to more than one vertex, or more formally:
\begin{align*}
e \cap f = \emptyset \text{ for all } e, f \in M \text{ with } e \neq f
\end{align*}
We call a vertex $v$ \textit{covered} by $M$ if there exists an edge $e \in M$ that contains $v$.
A matching is called a \textit{perfect matching} if every vertex is covered by an edge of $M$, or equivalently $|M| = \frac{|V|}{2}$
\end{definition}
\stepcounter{all}
\begin{definition}[]{Maxima}
Given a graph $G$ and matching $M$ in $G$
\begin{itemize}
\item $M$ is called a \textit{maxim\underbar{al} matching} (or in German ``inklusionsmaximal'') if we have $M \cup \{e\}$ is no matching for all $e \in E \backslash M$
\item $M$ is called a \textit{maxim\underbar{um} matching} (or in German ``kardinalitätsmaximal'') if we have $|M| \geq |M'|$ for all matchings $M'$ in $G$
\end{itemize}
\end{definition}
\subsubsection{Algorithms}
\begin{algorithm}
\caption{\textsc{Greedy-Matching}$(G)$}
\begin{algorithmic}[1]
\State $M \gets \emptyset$
\While{$E \neq \emptyset$}
\State choose an arbitrary edge $e \in E$ \Comment{Randomly choose from $E$}
\State $M \gets M \cup \{e\}$
\State delete $e$ and all incident edges (to both vertices) in $G$
\EndWhile
\end{algorithmic}
\end{algorithm}
The above algorithm doesn't return the maximum matching, just a matching
\begin{theorem}[]{Greedy-Matching}
The \textsc{Greedy-Matching} determines a maximal matching $M_{Greedy}$ in \tco{|E|} for which we have
\begin{align*}
|M_{Greedy}| \geq \frac{1}{2} |M_{\max}|
\end{align*}
where $M_{\max}$ is a maximum matching
\end{theorem}
\begin{theorem}[]{Berge's Theorem}
If $M$ is a not a maximum matching in $G$, there exists an augmenting path to $M$
\end{theorem}
\inlineproof If $M$ is not a maximum matching, there exists a matching $M'$ with higher cardinality, where $M \oplus M'$ ($M$ xor $M'$) has a connected component that contains more edges of $M'$ than $M$. Said connected component is the augmenting path for $M$
This idea leads to an algorithm to determine a maximum matching: As long as a matching isn't a maximum matching, there exists an augmenting path that allows us to expand the matching. After \textit{at most} $\frac{|V|}{2} - 1$ steps, we have a maximum matching. For bipartite graphs, we can use modified BFS with time complexity \tco{(|V| + |E|) \cdot |E|} to determine the augmenting paths.
\begin{algorithm}
\caption{\textsc{AugmentingPath}$(G = (A \uplus B, E), M)$}
\begin{algorithmic}[1]
\State $L_0 := \{ \text{set of all non-covered vertices in $A$} \}$
\If{$L_0 = \emptyset$}
\State \Return $M$ is a maximum matching
\EndIf
\State Mark all vertices in $L_0$ as visited
\For{$i = 1, \ldots, n$}
\If{$i$ is odd} \Comment{We start with an unmatched vertex (by definition)}
\State $L_i := \{\text{all unvisited neighbours of $L_{i - 1}$ using edges in } E \backslash M\}$
\Else
\State $L_i := \{\text{all unvisited neighbours of $L_{i - 1}$ using edges in } M\}$
\EndIf
\State Mark all vertices in $L_i$ as visited \Comment{We used them in our augmenting path, note that}
\If{$L_i$ contains non-covered vertex $v$}
\State Find path $P$ starting at $L_0$ and ending at $v$ using backtracking
\State \Return $P$
\EndIf
\EndFor
\State \Return $M$ is already a maximum matching
\end{algorithmic}
\end{algorithm}
\begin{center}
\fbox{
\parbox{15cm}{
\textbf{Augmenting Path}: An \textit{alternating path} that (here) starts from unmatched vertices, where an alternating path is a path that starts with an unmatched vertex and whose edges alternately belong to the matching and not.
}
}
\end{center}
The algorithm discussed above uses layers $L_i$ to find the augmenting paths. Each of the layers is alternatingly part of the matching and not part of it, where the first one is not part of the matching. Augmenting paths also always have length $m$ odd, so the last layer is \textit{not} part of the matching.
\fhlc{Cyan}{Hopcroft and Karp Algorithm}
\begin{algorithm}
\caption{\textsc{MaximumMatching}$(G = (A \oplus B, E))$}
\begin{algorithmic}[1]
\State $M \gets \{e\}$ for any edge $e \in E$ \Comment{Initialize Matching with simple one of just one edge (trivially a matching)}
\While{there are still augmenting paths in $G$}
\State $k \gets$ length of the shortest augmenting path
\State find a maximal set $S$ of pairwise disjunct augmenting paths of length $k$
\For{\textbf{all} $P$ of $S$}
\State $M \gets M \oplus P$ \Comment{Augmenting along all paths of $S$}
\EndFor
\EndWhile
\State \Return $M$
\end{algorithmic}
\end{algorithm}
To find the shortest augmenting path, we observe that if the last layer has more than one non-covered vertex, we can potentially (actually, likely) find more than one augmenting path.
We find one first, remove it from the data structure and find more augmenting paths by inverting the tree structure (i.e. cast \textit{flippendo} on the edges) and using DFS to find the all augmenting paths.
We always delete each visited vertex and we thus have time complexity \tco{|V| + |E|}, since we only visit each vertex and edge once.
\begin{theorem}[]{Hopcroft and Karp Algorithm}
The algorithm of Hopcroft and Karp's while loop is only executed \tco{\sqrt{|V|}} times.
Hence, the maximum matching is computed in \tco{\sqrt{|V|} \cdot (|V| + |E|)}
\end{theorem}
\newpage
\fhlc{Cyan}{Other matching algorithms}
In Section 4, using flows to compute matchings is discussed.
\begin{theorem}[]{Weighted Matching problem}
Let $n$ be even and $l: {[n] \choose 2} \rightarrow \N_0$ be a weight function of a complete graph $K_n$. Then, we can compute, in time \tco{n^3}, a minimum perfect matching with
\begin{align*}
\sum_{e \in M} l(e) = \min\left\{ \sum_{e \in M'} l(e) \smallhspace \Big| \smallhspace M' \text{ is a perfect matching in } K_n \right\}
\end{align*}
\end{theorem}
\begin{theorem}[]{MTSP with approximation}
There is a $\frac{3}{2}$-approximation algorithm with time complexity \tco{n^3} for the metric travelling salesman problem
\end{theorem}
\subsubsection{Hall's Theorem}
\begin{theorem}[]{Hall's Theorem}
For a bipartite graph $G = (A \uplus B, E)$ there exists a matching $M$ with cardinality $|M| = |A|$ if and only if
\begin{align*}
|N(X)| \geq |X| \smallhspace \forall X \subseteq{A}
\end{align*}
\end{theorem}
The following theorem follows from Hall's Theorem immediately. We remember that a graph is $k$-regular if and only if every vertex of the graph has degree exactly $k$
\begin{theorem}[]{Matching in $k$-regular bipartite graphs}
Let $G$ be a $k$-regular bipartite graph. Then there exists $M_1, \ldots, M_k$ such that $E = M_1 \uplus \ldots \uplus M_k$ where each $M_i$ is a perfect matching
\end{theorem}
\begin{theorem}[]{Algorithm for the problem}
If $G$ is a $2^k$-regular bipartite graph, we can find a perfect matching in time \tco{|E|}
\end{theorem}
It is important to note that the algorithms to determine a perfect matching in bipartite graph do not work for non-bipartite graphs, due to the fact that when we remove every other edge from the eulerian cycle, it is conceivable that the graph becomes disconnected. While this is no issue for bipartite graphs (as we can simply execute the graph on all connected components), for $k = 1$, such a connected component can could contain an odd number of vertices, thus there would be a eulerian cycle of odd length from which not every other edge can be deleted. Since that component has odd length, no perfect matching can exist.
\stepcounter{all}
\newpage
\subsection{Colourings}
Good examples for problems that can be solved using colourings are the channel picking of wireless devices, for compilers to pick registers and for creating timetables and exam schedules.
\begin{definition}[]{Colouring}
A \textit{(vertex-)colouring} of a graph $G$ with $k$ colours is an image $c: V \rightarrow [k]$ such that
\begin{align*}
c(u) \neq c(v) \smallhspace \forall \{u, v\} \in E
\end{align*}
The chromatic number $\mathscr{X}(G)$ is the minimum number of colours that are required to colour the graph $G$
\end{definition}
\stepcounter{all}
Graphs with chromatic number $k$ are also called $k$-\textit{partite}, where from the naming of a \textit{bipartite} graph comes, since we can \textit{partition} the graph into $k$ separate sets
\begin{theorem}[]{Bipartite graph}
A graph $G = (V, E)$ is bipartite if and only if it does not contain a cycle of uneven length as sub-graph
\end{theorem}
On political maps, two neighbouring countries are coloured in different colours.
We assume that the territories of every country is connected and that all countries that only touch at one point can be coloured with the same colour.
\begin{theorem}[]{Four colour theorem}
Every land map can be coloured in four colours
\end{theorem}
Again, the problem of determining if the chromatic number of a graph is smaller than a value $t$ is another $\mathcal{N}\mathcal{P}$-complete problem. We thus have to again proceed with an approximation.
The following algorithm correctly computes \textit{\textbf{a}} valid colouring.
\begin{theorem}[]{Greedy-Colouring Algorithm}
For the number of colours $C(G)$ the \textsc{Greedy-Colouring} needs to colour the connected graph $G$ we have
\begin{align*}
\mathscr{X}(G) \leq C(G) \leq \Delta(G) + 1
\end{align*}
where $\Delta(G) := \max_{v\in V}\deg(v)$ is the maximum degree of a vertex in $G$. If the graph is stored as an adjacency list, the algorithm finds a colouring \tco{|E|}
\end{theorem}
\begin{algorithm}
\caption{\textsc{Greedy-Colouring}$(G)$}
\begin{algorithmic}[1]
\State Choose an arbitrary order of vertices $V = \{v_1, \ldots, v_n\}$
\State $c[v_1] \gets 1$
\For{$i = 2, \ldots, n$}
\State $c[v_i] \gets \min \{k \in \N \divides k \neq c(u) \smallhspace \forall u \in N(v_i) \cap \{v_1, \ldots, v_{i - 1}\}\}$ \Comment{Find minimum available colour}
\EndFor
\end{algorithmic}
\end{algorithm}
\begin{theorem}[]{Brook's theorem}
Let $G$ be a connected graph that is neither complete nor a cycle of uneven length (uneven cycle), we have
\begin{align*}
\mathscr{X}(G) \leq \Delta(G)
\end{align*}
and there is an algorithm that colours the graph using $\Delta(G)$ colours in \tco{|E|}. Otherwise $\mathscr{X}(G) \leq \Delta(G) + 1$
\end{theorem}
Of note is that a graph with an even number of vertices and edges does not contain an uneven cycle, so for an incomplete graph with an even number of edges and vertices, we always have that $\mathscr{X}(G) \leq \Delta(G)$
\begin{theorem}[]{Maximum degree}
Let $G$ be a graph and $k \in \N$ the number representing the maximum degree of any vertex of any induced subgraph of $G$. Then we have $\mathscr{X}(G) \leq k + 1$ and a $(k + 1)$-coloring can be found in \tco{|E|}
\end{theorem}
\begin{theorem}[]{Mycielski-Construction}
For all $k \geq 2$ there exists a triangle-free graph $G_k$ with $\mathscr{X}(G_k) \geq k$
\end{theorem}
It is even possible to show that for all $k, l \geq 2$ there exists a graph $G_{k, l}$ such that said graph doesn't contain a cycle of length \textit{at most} $l$, but we still have $\mathscr{X}(G_{k, l}) \geq k$.
To conclude this section, one last problem:
We are given a graph $G$ which we are told has $\mathscr{X}(G) = 3$. This means, we know that there is exists an order of processing for the \textsc{Greedy-Colouring} algorithm that only uses three colours. We don't know the colours, but we can find an upper bound for the number of colours needed
\begin{theorem}[]{$3$-colourable graphs}
Every $3$-colourable graph $G$ can be coloured in time \tco{|E|} using at most \tco{\sqrt{|V|}} colours
\end{theorem}
Since the graph has to be bipartite (because for each vertex $v$, its neighbours can only be coloured in $2$ other colours, because the graph can be $3$-coloured), we can use BFS and thus have linear time. The algorithm works as follows: We choose the vertices with the largest degree and apply three colours to them. For the ones of smaller degree, we apply Brook's theorem.

809
semester2/algorithms-and-probability/parts/probability.tex Normal file
View File

@@ -0,0 +1,809 @@
\newsection
\section{Probability}
\subsection{Basics}
\begin{definition}[]{Discrete Sample Space}
A sample space $S$ consists of a set $\Omega$ consisting of \textit{elementary events} $\omega_i$. Each of these elementary events has a probability assigned to it, such that $0 \leq \Pr[\omega_i] \leq 1$ and
\[
\sum_{\omega \in \Omega} \Pr[\omega] = 1
\]
We call $E \subseteq \Omega$ an \textit{event}. The probability $\Pr[E]$ of said event is given by
\[
\Pr[E] := \sum_{\omega \in E} \Pr[\omega]
\]
If $E$ is an event, we call $\overline{E} := \Omega \backslash E$ the \textit{complementary event}
\end{definition}
\begin{lemma}[]{Events}
For two events $A, B$, we have:
\begin{multicols}{2}
\begin{enumerate}
\item $\Pr[\emptyset] = 0, \Pr[\Omega] = 1$
\item $0 \leq \Pr[A] \leq 1$
\item $\Pr[\overline{A}] = 1 - \Pr[A]$
\item If $A \subseteq B$, we have $\Pr[A] \leq \Pr[B]$
\end{enumerate}
\end{multicols}
\end{lemma}
\begin{theorem}[]{Addition law}
If events $A_1, \ldots, A_n$ are relatively disjoint (i.e. $\forall (i \neq j) : A_i \cap A_j = \emptyset$), we have (for infinite sets, $n = \infty$)
\[
\Pr\left[ \bigcup_{i = 1}^{n} A_i \right] = \sum_{i = 1}^{n} \Pr[A_i]
\]
\end{theorem}
\newpage
\label{sec:prob-basics}
\setcounter{all}{5}
The below theorem is known as the Inclusion-Exclusion-Principle, or in German the ``Siebformel'' and is the general case of the addition law, where the events don't have to be disjoint.
\begin{theorem}[]{Inclusion/Exclusion}
Let $A_1, \ldots, A_n$ be events, for $n \geq 2$. Then we we have
\begin{align*}
\Pr\left[ \bigcup_{i = 1}^{n} A_i \right] & = \sum_{l = 1}^{n} (-1)^{l + 1} \sum_{1 \leq i_1 < \ldots < i_l \leq n} \Pr[A_{i_1} \cap \ldots \cap A_{i_l}] \\
& = \sum_{i = 1}^{n} \Pr[A_i] - \sum_{1 \leq i_1 < i_2 \leq n} \Pr[A_{i_1} \cap A_{i_2}]
+ \sum_{1\leq i_1 < i_2 < i_3 \leq n} \Pr[A_{i_1} \cap A_{i_2} \cap A_{i_3}] -\ldots \\
& + (-1)^{n + 1} \cdot \Pr[A_1 \cap \ldots \cap A_n]
\end{align*}
\end{theorem}
What is going on here? We add all intersections where an even number of $\cap$-symbols are used and subtract all those who have and odd number of intersections.
\fhlc{Cyan}{Use:} This is useful for all kinds of counting operations where some elements occur repeatedly, like counting the number of integers divisible by a list of integers (see Code-Expert Task 04)
Of note here is that we sum up with e.g. $\displaystyle\sum_{1 \leq i_1 < j_1 \leq n} \Pr[A_{i_1} \cap A_{i_2}]$ is all subsets of the whole set $\Omega$, where two events are intersected / added.
If $\Omega = A_1 \cup \ldots \cup A_n$ and $\Pr[\omega] = \frac{1}{|\Omega|}$, we get
\[
\left|\bigcup_{i = 1}^{n}A_i\right| = \sum_{l = 1}^{n} (-1)^{l + 1} \sum_{1 \leq i_1 < \ldots < i_l \leq n} |A_{i_1} \cap \ldots \cap A_{i_l}
\]
Since for $n \geq 4$ the Inclusion-Exclusion-Principle formulas become increasingly long and complex, we can use a simple approximation, called the \textbf{Union Bound}, also known as the \textit{Boolean inequality}
\begin{corollary}[]{Union Bound}
For events $A_1, \ldots, A_n$ we have (for infinite sequences of events, $n = \infty$)
\[
\Pr\left[ \bigcup_{i = 1}^{n} A_i \right] \leq \sum_{i = 1}^{n} \Pr[A_i]
\]
\end{corollary}
\vspace{1cm}
\begin{center}
\fbox{\textbf{Laplace principle}: We can assume that all outcomes are equally likely if nothing speaks against it}
\end{center}
\vspace{1cm}
Therefore, we have $\Pr[\omega] = \displaystyle \frac{1}{|\Omega|}$ and for any event $E$, we get $\displaystyle \Pr[E] = \frac{|E|}{|\Omega|}$
\newpage
\subsection{Conditional Probability}
\setcounter{all}{8}
\begin{definition}[]{Conditional Probability}
Let $A, B$ be events, with $\Pr[B] > 0$. The \textit{conditional probability} $\Pr[A|B]$ of $A$ given $B$ is defined as
\[
\Pr[A|B] := \frac{\Pr[A \cap B]}{\Pr[B]}
\]
We may also rewrite the above as
\[
\Pr[A \cap B] = \Pr[B|A] \cdot \Pr[A] = \Pr[A|B] \cdot \Pr[B]
\]
\end{definition}
\setcounter{all}{10}
\begin{theorem}[]{Multiplication law}
Let $A_1, \ldots, A_n$ be events. If $\Pr[A_1 \cap \ldots \cap A_n] > 0$, we have
\[
\Pr[A_1 \cap \ldots \cap A_n] = \Pr[A_1] \cdot \Pr[A_2|A_1] \cdot \Pr[A_3|A_1 \cap A_2] \cdot \ldots \cdot \Pr[A_n|A_1 \cap \ldots \cap A_n]
\]
\end{theorem}
The proof of the above theorem is based on the definition of conditional probability. If we rewrite $\Pr[A_1] = \frac{\Pr[A_1]}{1}$, apply the definition of $\Pr[A_2 | A_1]$, and do the same to all subsequent terms, the equation simplifies to $\Pr[A_1 \cap \ldots \cap A_n]$
\fhlc{Cyan}{Use:} The law of total probability is used, as the name implies, to calculate the total probability of all possible ways in which an even $B$ can occur.
\setcounter{all}{13}
\begin{theorem}[]{Law of total probability}
Let $A_1, \ldots, A_n$ be relatively disjoint events and let $B \subseteq A_1 \cup \ldots \cup A_n$. We then have
\[
\Pr[B] = \sum_{i = 1}^{n} \Pr[B|A_i] \cdot \Pr[A_i]
\]
The same applies for $n = \infty$. Then, $B = \bigcup_{i = 1}^{\infty} A_i$
\end{theorem}
Using the previously defined theorem, we get Bayes' Theorem
\setcounter{all}{15}
\begin{theorem}[]{Bayes' Theorem}
Let $A_1, \ldots, A_n$ be relatively disjoint events and let $B \subseteq A_1 \cup \ldots \cup A_n$ be an event with $\Pr[B] > 0$. Then for each $i = 1, \ldots, n$, we have
\[
\Pr[A_i|B] = \frac{\Pr[A_i \cap B]}{\Pr[B]} = \frac{\Pr[B|A_i] \cdot \Pr[A_i]}{\sum_{j = 1}^{n} \Pr[B|A_j] \cdot \Pr[A_j]}
\]
The same applies for $n = \infty$. Then $B = \bigcup_{i = 1}^{\infty} A_i$
\end{theorem}
\fhlc{Cyan}{Use:} Bayes' Theorem is commonly used to calculate probabilities on different branches or in other words, to rearrange conditional probabilities. The sum in the denominator represents all posible paths to the event summed up
\inlineex \hspace{0mm} Assume we want to find the probability that event $X$ happened given that event $Y$ happened. \textbf{Important:} Event $X$ happened \textit{before} event $Y$ happened and we do \textit{not} know the probability of $X$. Therefore we have $\Pr[X|Y]$ as the probability. But we don't actually know that probability, so we can use Bayes' Theorem to restate the problem in probabilities we can (more) easily determine.
\newpage
\subsection{Independence}
\setcounter{all}{18}
\fancydef{Independence of two events} Two events $A$ and $B$ are called \textbf{independent} if
\[
\Pr[A \cap B] = \Pr[A] \cdot \Pr[B]
\]
\setcounter{all}{22}
\begin{definition}[]{Independence}
Events $A_1, \ldots, A_n$ are called \textit{independent}, if for all subsets $I \subseteq \{1, \ldots, n\}$ with $I = \{i_1, \ldots, i_k\}$ and $|I| = k$, we have that
\[
\Pr[A_{i_1} \cap \ldots \cap A_{i_k}] = \Pr[A_{i_1}] \cdot \ldots \cdot \Pr[A_{i_k}]
\]
\end{definition}
The same in simpler terms: If all events $A_1, \ldots, A_n$ are relatively disjoint, they are independent. We can determine if they are, if the probability of the intersection of all events is simply their individual probabilities multiplied with each other.
\begin{lemma}[]{Independence}
Events $A_1, \ldots, A_n$ are independent if and only if for all $(s_1, \ldots, s_n) \in \{0, 1\}^n$ we have
\[
\Pr[A_1^{s_1} \cap \ldots \cap A_n^{s_n}] = \Pr[A_1^{s_1}] \cdot \ldots \cdot \Pr[A_n^{s_n}]
\]
where $A_i^{0} = \overline{A_i}$ (i.e. $s_i = 0$) and $A_i^{1} = A_i$ (i.e. $s_i = 1$)
\end{lemma}
$\{0, 1\}^n$ is the space of $n$-bit binary numbers, representing subsets of the sample space, each of them being any of the subsets intersected with up to $n$ other subsets
The $s_i$ in this expression are very straight forward to understand as simply indicating if we consider the event or its complement.
\fancylemma{Let $A$, $B$ and $C$ be independent events. Then, $A\cap B$ and $C$ as well $A \cup B$ and $C$ are independent}
In this lecture, we are always going to assume that we can use actual random numbers, not just pseudo random numbers that are generated by PRNGs (Pseudo Random Number Generators).
\newsection
\subsection{Random Variables}
\setcounter{all}{25}
\begin{definition}[]{Random Variable}
A \textit{random variable} is an image $\mathcal{X}: \Omega \rightarrow \R$ that maps the sample space to a real number.
The range $W_{\mathcal{X}} := \mathcal{X}(\Omega) = \{ x \in \R : \forall \omega \in \Omega \text{ with } \mathcal{X}(\omega) = x \}$'s countability depends on the countability of $\Omega$, and is either \textit{countable} or \textit{countably infinite}
\end{definition}
\begin{scriptsize}
\textit{For those who don't have an intuition for what a random variable actually is: See Section \ref{sec:random-var-details}.}
\end{scriptsize}
Often times when looking at random variables, we are interested in the probabilities at which $\mathcal{X}$ takes certain values.
We write either $\mathcal{X}^{-1}(x_i)$ or more intuitively $\mathcal{X} = x_i$. Analogously, we have (short: $\Pr[``\mathcal{X} \leq x_i'']$ as $\Pr[\mathcal{X} \leq x_i]$)
\[
\Pr[``\mathcal{X} \leq x_i''] = \sum_{x \in W_{\mathcal{X}} : x \leq x_i} \Pr[``\mathcal{X} = x''] = \Pr[\{ \omega \in \Omega : \mathcal{X}(\omega) \leq x_i \}]
\]
From this notation, we easily get two real functions.
We call $f_{\mathcal{X}}: \R \rightarrow [0, 1]$ for which $x \mapsto \Pr[\mathcal{X} = x]$ the \textbf{\textit{probability mass function}} (PMF, Dichtefunktion) of $\mathcal{X}$, which maps a real number to the probability that the random variable takes this value.
The \textbf{\textit{cumulative distribution function}} (CDF, Verteilungsfunktion) of $\mathcal{X}$ is a function, which maps a real number to the probability that the value taken by the random variable is lower than, or equal to, the real number.
Often times it suffices to state the PMF of the random variable (since we can easily derive the CDF from it)
\[
F_{\mathcal{X}} : \R \rightarrow [0, 1], \mediumhspace x\rightarrow \Pr[\mathcal{X} \leq x] = \sum_{x' \in W_{\mathcal{X}} : x' \leq x} \Pr[\mathcal{X} = x'] = \sum_{x' \in W_{\mathcal{X}} : x' \leq x} f_{\mathcal{X}}(x')
\]
\subsubsection{Expected value}
\setcounter{all}{27}
\begin{definition}[]{Expected Value}
The \textit{expected value} $\E[\mathcal{X}]$ describes the average value the random variable $\mathcal{X}$ takes.
We define the \textit{expected value} $\E[\mathcal{X}]$ as
\[
\E[\mathcal{X}] := \sum_{x \in W_{\mathcal{X}}} x \cdot \Pr[\mathcal{X} = x]
\]
only if the sum converges absolutely. Otherwise, the \textit{expected value} is undefined. This is trivially true for finite sample spaces.
\end{definition}
\begin{scriptsize}
In this lecture, only random variables with an expected value are covered, so that condition does not need to be checked here
\end{scriptsize}
\setcounter{all}{29}
Alternative to the above definition over the elements of the range of the random variable, we can also define it as
\begin{lemma}[]{Expected Value}
If $\mathcal{X}$ is a random variable, we have
\[
\E[\mathcal{X}] = \sum_{\omega \in \Omega} \mathcal{X}(\omega) \cdot \Pr[\omega]
\]
\end{lemma}
If the range of the random variable consists only of non-zero integers, we can calculate the expected value with the following formula
\begin{theorem}[]{Expected Value}
Let $\mathcal{X}$ be a random variable with $W_{\mathcal{X}} \subseteq \N_0$. We then have
\begin{align*}
\E[\mathcal{X}] = \sum_{i = 1}^{\infty} \Pr[\mathcal{X} \geq i]
\end{align*}
\end{theorem}
\newpage
\fhlc{Cyan}{Conditional Random Variables}
\begin{definition}[]{Conditional Random Variable}
Let $\mathcal{X}$ be a random variable and let $A$ be an event with $\Pr[A] > 0$
\begin{align*}
\Pr[(\mathcal{X} | A) \leq x] = \Pr[X \leq x | A] = \frac{\Pr[\{\omega \in A : \mathcal{X}(\omega) \leq x\}]}{\Pr[A]}
\end{align*}
\end{definition}
\begin{theorem}[]{Expected Value (Conditional)}
Let $\mathcal{X}$ be a random variable. For relatively disjoint events $A_1, \ldots, A_n$ with $A_1 \cup \ldots \cup A_n = \Omega$ and $\Pr[A_1], \ldots, \Pr[A_n] > 0$, we have (analogously for $n = \infty$)
\begin{align*}
\E[\mathcal{X}] = \sum_{i = 1}^{n} \E[\mathcal{X} | A_i] \cdot \Pr[A_i]
\end{align*}
\end{theorem}
\fhlc{Cyan}{Linearity of the expected value}
We can calculate the expected value of a sum of any number of random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n : \Omega \rightarrow \R$ simply by summing the expected values of each of the random variables $\mathcal{X}_i$
\begin{theorem}[]{Linearity of expected value}
Given random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n$ and let $\mathcal{X} := a_1 \mathcal{X}_1 + \ldots + a_n \mathcal{X}_n + b$ for any $a_1, \ldots, a_n, b \in \R$, we have
\begin{align*}
\E[\mathcal{X}] = a_1 \cdot \E[\mathcal{X}_1] + \ldots + a_n \cdot \E[\mathcal{X}_n] + b
\end{align*}
\end{theorem}
Very simply with two random variables $X$ and $Y$, we have $\E[X + Y] = \E[X] + \E[Y]$
\setcounter{all}{35}
\begin{definition}[]{Indicator Variable}
We use \textit{indicator variables} to formalize the probability that an event $A$ occurs using the expected value
For an event $A \subseteq \Omega$ the accompanying indicator variable $\mathcal{X}_A$ is given by
\begin{align*}
\mathcal{X}_A(\omega) := \begin{cases}
1 & \text{if } \omega \in A \\
0 & \text{else }
\end{cases}
\end{align*}
For the expected value of $\mathcal{X}_A$ we have: $\E[\mathcal{X}_A] = \Pr[A]$
\end{definition}
We can now prove the Inclusion-Exclusion-Principle using a fairly simple proof. See Example 2.36 in the script for it.
\fhlc{Cyan}{Use:} We use the indicator variable for experiments where we perform a certain action numerous times where each iteration does not (or does for that matter) depend on the previous outcome.
\newpage
\subsubsection{Variance}
Even though two random variables may have the same expected value, they can still be significantly different. The Variance describes the dispersion of the results, or how far off the expected value the different values are maximally (up to a certain limit, that is)
\setcounter{all}{39}
\begin{definition}[]{Variance}
For a random variable $\mathcal{X}$ with $\mu = \E[\mathcal{X}]$, the \textit{variance} $\text{Var}[\mathcal{X}]$ is given by
\begin{align*}
\text{Var}[\mathcal{X}] := \E[(\mathcal{X} - \mu)^2] = \sum_{x \in W_{\mathcal{X}}} (x - \mu)^2 \cdot \Pr[\mathcal{X} = x]
\end{align*}
$\sigma := \sqrt{\text{Var}[\mathcal{X}]}$ is called the \textit{standard deviation} of $\mathcal{X}$
\end{definition}
\begin{theorem}[]{Variance (easier)}
For any random variable $\mathcal{X}$ we have
\[
\text{Var}[\mathcal{X}] = \E[\mathcal{X}^2] - \E[\mathcal{X}]^2
\]
\end{theorem}
We also have
\begin{theorem}[]{Variance}
For any random variable $\mathcal{X}$ and $a, b \in \R$ we have
\[
\text{Var}[a \cdot \mathcal{X} + b] = a^2 \cdot \text{Var}[\mathcal{X}]
\]
\end{theorem}
The moments of a random variable are given by the expected value and the variance.
\begin{definition}[]{Moment}
The \textbf{\textit{$k$th moment}} of a random variable $\mathcal{X}$ is $\E[\mathcal{X}^k]$ whereas $\E[(\mathcal{X} - \E[\mathcal{X}])^k]$ is called the \textbf{\textit{$k$th central moment}}.
\end{definition}
\shade{gray}{Note} The expected value is thus the first moment and the variance the second central moment.
\subsubsection{Intuition}
\label{sec:random-var-details}
If you struggle to imagine what a random variable $\mathcal{X}$ is, or what for example $\mathcal{X}^2$ is, read on.
As definition 3.25 states, a random variable is a function, which is why people tend to get confused.
It is not a variable in the normal way of understanding.
With that in mind, things like $\mathcal{X}^2$ makes much more sense, as it's simply the result of the function squared, which then makes theorem 3.40 make much more sense, given the definition of the expected value.
Of note is that remembering the summation formulas for the variance (or knowing how to get to it) is handy for the exam, as that formula is not listed on the cheat-sheet provided by the teaching team as of FS25.
Deriving it is very easy though, as it's simply applying the expected value definition to the initial definition, which is listed on the cheat-sheet.
% Page 126 (actual)
\newpage
\subsection{Discrete distribution}
\subsubsection{Bernoulli-Distribution}
A random variable $\mathcal{X}$ with $W_{\mathcal{X}}$ is called \textit{\textbf{Bernoulli distributed}} if and only if its probability mass function is of form
\[
f_{\mathcal{X}}(x) = \begin{cases}
p & x = 1 \\
1 - p & x = 0 \\
0 & \text{else}
\end{cases}
\]
The parameter $p$ is called the probability of success (Erfolgswahrscheinlichkeit). Bernoulli distribution is used to describe boolean events (that can either occur or not). It is the trivial case of binomial distribution with $n = 1$. If a random variable $\mathcal{X}$ is Bernoulli distributed, we write
\[
\mathcal{X} \sim \text{Bernoulli}(p)
\]
and we have
\[
\E[\mathcal{X}] = p \hspace{1cm} \text{and} \hspace{1cm} \text{Var}[\mathcal{X}] = p(1 - p)
\]
\subsubsection{Binomial Distribution}
If we perform a Bernoulli trial repeatedly (e.g. we flip a coin $n$ times), the number of times we get one of the outcomes is our random variable $\mathcal{X}$ and it is called \textbf{\textit{binomially distributed}} and we write
\[
\mathcal{X} \sim \text{Bin}(n, p)
\]
and we have
\[
\E[\mathcal{X}] = np \hspace{1cm} \text{and} \hspace{1cm} \text{Var}[\mathcal{X}] = np(1 - p)
\]
\subsubsection{Geometric Distribution}
If we have an experiment that is repeated until we have achieved success, where the probability of success is $p$, the number of trials (which is described by the random variable $\mathcal{X}$) is \textbf{\textit{geometrically distributed}}. We write
\[
\mathcal{X} \sim \text{Geo}(p)
\]
The density function is given by
\[
f_{\mathcal{X}}(i) = \begin{cases}
p(1 - p)^{i - 1} & \text{for } i \in \N \\
0 & \text{else}
\end{cases}
\]
whilst the expected value and variance are defined as
\[
\E[\mathcal{X}] = \frac{1}{p} \hspace{1cm} \text{and} \hspace{1cm} \text{Var}[\mathcal{X}] = \frac{1 - p}{p^2}
\]
The cumulative distribution function is given by
\[
F_{\mathcal{X}}(n) = \Pr[\mathcal{X} \leq n] = \sum_{i = 1}^{n} \Pr[\mathcal{X} = i] = \sum_{i = 1}^{n} p(1 - p)^{i - 1} = 1 - (1 - p)^n
\]
\shade{gray}{Note} Every trial in the geometric distribution is unaffected by the previous trials
\setcounter{all}{45}
\begin{theorem}[]{Geometric Distribution}
If $\mathcal{X} \sim \text{Geo}(p)$, for all $s, t \in \N$ we have
\[
\Pr[\mathcal{X} \geq s + t | X > s] = \Pr[X \geq t]
\]
\end{theorem}
\newpage
\fhlc{cyan}{Coupon Collector problem}
First some theory regarding waiting for the $n$th success. The probability mass function is given by $f_{\mathcal{X}}(x) = \begin{pmatrix}z - 1\\ n - 1\end{pmatrix} \cdot p^n \cdot (1- p)^{z - n}$ whereas the expected value is given by $\displaystyle\E[\mathcal{X}] = \sum_{i = 1}^{n} \E[\mathcal{X}_i] = \frac{n}{p}$
The coupon collector problem is a well known problem where we want to collect all coupons on offer. How many coupons do we need to obtain on average to get one of each? We will assume that the probability of getting coupon $i$ is equal to all other coupons and getting a coupon doesn't depend on what coupons we already have (independence)
Let $\mathcal{X}$ be a random variable representing the number of purchases to the completion of the collection. We split up the time into separate phases, where $\mathcal{X}$ is the number of coupons needed to end phase $i$, which ends when we have found one of the $n - i + 1$ coupons not previously collected (i.e. we got a coupon we haven't gotten yet)
Logically, $\mathcal{X} = \sum_{i = 1}^{n} \mathcal{X}_i$. We can already tell from the experiment we are conducting that it is going to be geometrically distributed and thus the probability of success is going to be $p = \frac{n - i + 1}{n}$ and we have $\E[\mathcal{X}_i] = \frac{n}{n - i + 1}$
With that, let's determine
\[
\E[\mathcal{X}] = \sum_{i = 1}^{n} \E[\mathcal{X}_i] = \sum_{i = 1}^{n} \frac{n}{n - i + 1} = n \cdot \sum_{i = 1}^{n} \frac{1}{i} = n \cdot H_n
\]
where $H_n := \sum_{i = 1}^{n} \frac{1}{i}$ is the $n$th harmonic number, which we know (from Analysis) is $H_n = \ln(n) +$\tco{1}, thus we have $\E[\mathcal{X}] = n \cdot \ln(n) +$\tco{n}.
The idea of the transformation is to reverse the $(n - i + 1)$, so counting up instead of down, massively simplifying the sum and then extracting the $n$ and using the result of $H_n$ to fully simplify
\subsubsection{Poisson distribution}
The \textbf{\textit{Poisson distribution}} is applied when there is only a small likelihood that an event occurs, but since the cardinality of the sample space in question is large, we can expect at least a few events to occur.
We write
\[
\mathcal{X} \sim \text{Po}(\lambda)
\]
An example for this would be for a person to be involved in an accident over the next hour. The probability mass function is given by
\[
f_{\mathcal{X}}(i) = \begin{cases}
\frac{e^{-\lambda}\lambda^i}{i!} & \text{for } i \in \N_o \\
0 & \text{else}
\end{cases}
\hspace{1cm} \text{and} \hspace{1cm} \E[\mathcal{X}] = \text{Var}[\mathcal{X}] = \lambda
\]
\shade{cyan}{Using the Poisson distribution as limit for the binomial distribution}
We can approximate the binomial distribution using the Poisson distribution if we have large $n$ and small constant $np$. $\lambda = \E[\mathcal{X}] = np$ in that case.
\newpage
\subsection{Multiple random variables}
There are times when we are interested in the outcomes of multiple random variables simultaneously. For two random variables $\mathcal{X}$ and $\mathcal{Y}$, we evaluate probabilities of type
\[
\Pr[\mathcal{X} = x, \mathcal{Y} = y] = \Pr[\{ \omega \in \Omega : \mathcal{X}(\omega) = x, \mathcal{Y}(\omega) = y \}]
\]
Here $\Pr[\mathcal{X} = x, \mathcal{Y} = y]$ is a shorthand notation for $\Pr[``\mathcal{X} = x'' \cap ``\mathcal{Y} = y'']$
We define the \textit{common probability mass function} $f_{\mathcal{X}, \mathcal{Y}}$ by
\[
f_{\mathcal{X}, \mathcal{Y}}(x, y) := \Pr[\mathcal{X} = x, \mathcal{Y} = y]
\]
We can also get back to the individual probability mass of each random variable
\begin{align*}
f_{\mathcal{X}} = \sum_{y \in W_{\mathcal{Y}}} f_{\mathcal{X}, \mathcal{Y}}(x, y) \hspace{1cm} \text{or} \hspace{1cm} f_{\mathcal{Y}}(y) = \sum_{x \in W_{\mathcal{X}}} f_{\mathcal{X}, \mathcal{Y}}(x, y)
\end{align*}
We hereby call $f_{\mathcal{X}}$ and $f_{\mathcal{Y}}$ \textit{marginal density} (Randdichte)
We define the \textbf{\textit{common cumulative distribution function}} by
\begin{align*}
F_{\mathcal{X}, \mathcal{Y}}(x, y) := \Pr[\mathcal{X} \leq x, \mathcal{Y} \leq y] = \Pr[\{ \omega \in \Omega : \mathcal{X}(\omega) \leq x, \mathcal{Y} \leq y \}] = \sum_{x' \leq x} \sum_{y' \leq y} f_{\mathcal{X}, \mathcal{Y}}(x', y')
\end{align*}
Again, we can use marginal density
\begin{align*}
F_{\mathcal{X}}(x) = \sum_{x'\leq x} f_{\mathcal{X}}(x') = \sum_{x' \leq x} \sum_{y \in W_{\mathcal{Y}}} f_{\mathcal{X}, \mathcal{Y}}(x', y)
\hspace{5mm} \text{and} \hspace{5mm}
F_{\mathcal{Y}}(y) = \sum_{y'\leq y} f_{\mathcal{Y}}(y') = \sum_{y' \leq y} \sum_{x \in W_{\mathcal{X}}} f_{\mathcal{X}, \mathcal{Y}}(x, y')
\end{align*}
\subsubsection{Independence of random variables}
\setcounter{all}{52}
\begin{definition}[]{Independence}
Random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n$ are called \textbf{\textit{independent}}
if and only if for all $(x_1, \ldots, x_n) \in W_{\mathcal{X}_1} \times \ldots \times W_{\mathcal{X}_n}$ we have
\begin{align*}
\Pr[\mathcal{X}_1 = x_1, \ldots, \mathcal{X}_n = x_n] = \Pr[\mathcal{X}_1 = x_1] \cdot \ldots \cdot \Pr[\mathcal{X}_n = x_n]
\end{align*}
Or alternatively, using probability mass functions
\begin{align*}
f_{\mathcal{X}_1, \ldots \mathcal{X}_n}(x_1, \ldots, x_n) = f_{\mathcal{X}_1}(x_1) \cdot \ldots \cdot f_{\mathcal{X}_n}(x_n)
\end{align*}
In words, this means that for independent random variables, their common density is equal to the product of the individual marginal densities
\end{definition}
The following lemma shows that the above doesn't only hold for specific values, but also for sets
\begin{lemma}[]{Independence}
Let $\mathcal{X}_1, \ldots, \mathcal{X}_n$ be independent random variables and let $S_1, \ldots, S_n \subseteq \R$ be any set, then we have
\begin{align*}
\Pr[\mathcal{X}_1 \in S_1, \ldots, \mathcal{X}_n \in S_n] = \Pr[\mathcal{X}_1 \in S_1] \cdot \ldots \cdot \Pr[\mathcal{X}_n \in S_n]
\end{align*}
\end{lemma}
\begin{corollary}[]{Independence}
Let $\mathcal{X}_1, \ldots, \mathcal{X}_n$ be independent random variables and let $I = \{i_1, \ldots, i_k\} \subseteq [n]$, then $\mathcal{X}_{i_1}, \ldots, \mathcal{X}_{i_k}$ are also independent
\end{corollary}
\begin{theorem}[]{Independence}
Let $f_1, \ldots, f_n$ be real-valued functions ($f_i : \R \rightarrow \R$ for $i = 1, \ldots, n$). If the random variables $X_1, \ldots, X_n$ are independent, then this also applies to $f_1(\mathcal{X}_1), \ldots, f_n(\mathcal{X}_n)$
\end{theorem}
\subsubsection{Composite random variables}
Using functions we can combine multiple random variables in a sample space.
\setcounter{all}{58}
\begin{theorem}[]{Two random variables}
For two independent random variables $\mathcal{X}$ and $\mathcal{Y}$, let $\mathcal{Z} := \mathcal{X} + \mathcal{Y}$. Then we have
\begin{align*}
f_{\mathcal{Z}}(z) = \sum_{x \in W_{\mathcal{X}}} f_{\mathcal{X}} \cdot f_{\mathcal{Y}}(z - x)
\end{align*}
\end{theorem}
We call, analogously to the terms used for power series, $f_{\mathcal{Z}}(z)$ ``convolution''
\subsubsection{Moments of composite random variables}
\setcounter{all}{60}
\begin{theorem}[]{Linearity of the expected value}
For random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n$ and $\mathcal{X} := a_1 \mathcal{X}_1 + \ldots + a_n \mathcal{X}_n$ with $a_1, \ldots, a_n \in \R$ we have
\begin{align*}
\E[\mathcal{X}] = a_1 \E[\mathcal{X}_1] + \ldots + a_n \E[\mathcal{X}_n]
\end{align*}
\end{theorem}
There are no requirements in terms of independence of the random variables, unlike for the multiplicativity
\begin{theorem}[]{Multiplicativity of the expected value}
For independent random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n$ we have
\begin{align*}
\E[\mathcal{X}_1 \cdot \ldots \cdot \mathcal{X}_n] = \E[\mathcal{X}_1] \cdot \ldots \cdot \E[\mathcal{X}_n]
\end{align*}
\end{theorem}
\begin{theorem}[]{Variance of multiple random variables}
For independent random variables $\mathcal{X}_1, \ldots, \mathcal{X}_n$ and $\mathcal{X} = \mathcal{X}_1 + \ldots + \mathcal{X}_n$ we have
\begin{align*}
\text{Var}[\mathcal{X}] = \text{Var}[\mathcal{X}_1] + \ldots + \text{Var}[\mathcal{X}_n]
\end{align*}
\end{theorem}
\subsubsection{Wald's Identity}
Wald's identity is used for cases where the number of summands is not a constant, commonly for algorithms that repeatedly call subroutines until a certain result is attained.
The time complexity of such an algorithm can be approximated by splitting up the algorithm into phases, where each phase is a call of the subroutine.
The number of calls to the subroutine, thus the number of phases, is usually not deterministic in that case but rather bound to a random variable.
\setcounter{all}{65}
\begin{theorem}[]{Wald's Identity}
Let $\mathcal{N}$ and $\mathcal{X}$ be two independent random variables with $W_{\mathcal{N}} \subseteq \N$. Let
\begin{align*}
\mathcal{Z} := \sum_{i = 1}^{\mathcal{N}}\mathcal{X}_i
\end{align*}
where $\mathcal{X}_1, \mathcal{X}_2, \ldots$ are independent copies of $\mathcal{X}$. Then we have
\begin{align*}
\E[\mathcal{Z}] = \E[\mathcal{N}] \cdot \E[\mathcal{X}]
\end{align*}
\end{theorem}
\newpage
\subsection{Approximating probabilities}
Since it can be very expensive to calculate the true probabilities in some cases, we will now cover some tools that allow us to approximate the probabilities using upper or lower bounds.
\subsubsection{Markov's \& Chebyshev's inequalities}
\setcounter{all}{67}
\begin{theorem}[]{Markov's inequality}
Let $\mathcal{X}$ be a random variable that may only take non-negative values. Then for all $t > 0 \in \R$, we have
\begin{align*}
\Pr[\mathcal{X} \geq t] \leq \frac{\E[\mathcal{X}]}{t} \Longleftrightarrow \Pr[\mathcal{X} \geq t \cdot \E[\mathcal{X}]] \leq \frac{1}{t}
\end{align*}
\end{theorem}
Markov's inequality is fairly straight forward to prove, and it already allows us to make some useful statements, like that for the coupon collector problem, we only need to make more than $100 n \log(n)$ purchases with probability $\frac{1}{100}$. The following inequality usually gives a much more precise bound than Markov's inequality
\begin{theorem}[]{Chebyshev's inequality}
Let $\mathcal{X}$ be a random variable and $t > 0 \in\R$. Then we have
\begin{align*}
\Pr[|\mathcal{X} - \E[\mathcal{X}| \geq t]] \leq \frac{\text{Var}[\mathcal{X}]}{t^2} \Longleftrightarrow \Pr[|\mathcal{X} - \E[\mathcal{X}]| \geq t \cdot \sqrt{\text{Var}[\mathcal{X}]}] \leq \frac{1}{t^2}
\end{align*}
\end{theorem}
A common tactic when using these is to restate the original probability $\Pr[X \geq t]$ as $\Pr[|X - \E[X]| \geq t - \E[X]]$ and then set $t = t'$ for $t' = t - \E[X]$
\subsubsection{Chernoff bounds}
The Chernoff bounds are specifically designed for Bernoulli-variables
\setcounter{all}{70}
\begin{theorem}[]{Chernoff bounds}
Let $\mathcal{X}_1, \ldots, \mathcal{X}_n$ be independent Bernoulli-distributed random variables with $\Pr[\mathcal{X}_i = 1] = p_i$ and $\Pr[\mathcal{X}_i = 0] = 1 - p_i$. Then we have for $\mathcal{X} := \sum_{i = 1}^{n} \mathcal{X}_i$
\begin{enumerate}[label=(\roman*)]
\item $\Pr[\mathcal{X} \geq (1 + \delta)\E[\mathcal{X}]] \leq e^{-\frac{1}{3}\delta^2\E[\mathcal{X}]}$ \largehspace for all $0 < \delta \leq 1$
\item $\Pr[\mathcal{X} \leq (1 - \delta)\E[\mathcal{X}]] \leq e^{-\frac{1}{2}\delta^2\E[\mathcal{X}]}$ \largehspace for all $0 < \delta \leq 1$
\item $\Pr[\mathcal{X} \geq t] \leq 2^{-t}$ \largehspace for $t \geq 2e\E[\mathcal{X}]$
\end{enumerate}
\end{theorem}
We determine the $\delta$ in the inequality by finding it such that $t = (1 + \delta)\E[X]$ or, for the second one, $t = (1 - \delta)\E[X]$. For the third one, no $\delta$ is required
\newpage
\subsection{Randomized Algorithms}
In comparison to \textit{deterministic} algorithms, here, the output is \textbf{\textit{not}} guaranteed to be equal for the same input data after reruns.
While this can be an issue in some cases, it allows us to usually \textit{significantly} reduce time complexity and thus allows us to solve $\mathcal{N}\mathcal{P}$-complete problems in some cases in polynomial time even.
The problem with \textit{true} randomness is that it is hardly attainable inside computers, some kind of predictability will always be there in some form or another, especially if the random number generator is algorithm-based, not on random events from the outside.
In this course, we will though assume that random numbers generated by random number generators provided by programming languages actually provide independent random numbers
In the realm of randomized algorithms, one differentiates between two approaches
\begin{tables}{lcc}{Types of randomized algorithms}
& \shade{ForestGreen}{Monte-Carlo-Algorithm} & \shade{ForestGreen}{Las-Vegas-Algorithm} \\
Always correct & \ding{55} & \ding{51} \\
Constant runtime & \ding{51} & \ding{55} \\
\end{tables}
While this is the normal case for Las-Vegas Algorithms, we can also consider the following:
Let the algorithm terminate if a certain runtime bound has been exceeded and return something like ``???'' if it has not found a correct answer just yet. We will most commonly use this definition of a Las-Vegas-Algorithm in this course
\subsubsection{Reduction of error}
\setcounter{all}{72}
\begin{theorem}[]{Error reduction Las-Vegas-Algorithm}
Let $\mathcal{A}$ be a Las-Vegas-Algorithm, where $\Pr[\mathcal{A}(I) \text{correct}] \geq \varepsilon$
Then, for all $\delta > 0$ we call $\mathcal{A}_{\delta}$ an algorithm that calls $\mathcal{A}$ until we either get a result that is not ``???'' or we have executed $N = \varepsilon^{-1} \ln(\delta^{-1})$ times. For $\mathcal{A}_{\delta}$ we then have
\begin{align*}
\Pr[\mathcal{A}_{\delta}(I) \text{correct}] \geq 1 - \delta
\end{align*}
\end{theorem}
On the other hand, for Monte-Carlo-Algorithms, the probability of error decreases rapidly. It is not easy though to determine whether or not an answer is correct, unless the algorithm only outputs two different values \textit{and} we know that one of these values is \textit{always} correct
\setcounter{all}{74}
\begin{theorem}[]{Error reduction}
Let $\mathcal{A}$ be a randomized algorithm, outputting either \textsc{Yes} or \textsc{No}, whereas
\begin{align*}
\Pr[\mathcal{A}(I) = \textsc{Yes}] & = 1 \mediumhspace \text{if $I$ is an instance of \textsc{Yes}} \\
\Pr[\mathcal{A}(I) = \textsc{No}] & \geq \varepsilon \mediumhspace \text{if $I$ is an instance of \textsc{No}}
\end{align*}
Then, for all $\delta > 0$ we call $\mathcal{A}_{\delta}$ the algorithm that calls $\mathcal{A}$ until either \textsc{No} is returned or until we get \textsc{Yes} $N = \varepsilon^{-1} \ln(\delta^{-1})$ times. Then for all instances $I$ we have
\begin{align*}
\Pr[\mathcal{A}_{\delta}(I) \text{correct}] \geq 1 - \delta
\end{align*}
\end{theorem}
This can also be inverted and its usage is very straight forward.
\newpage
If we however have Monte-Carlo-Algorithms that have two-sided errors, i.e. there is error in both directions, we have
\begin{theorem}[]{Two-Sided Error reduction}
Let $\varepsilon > 0$ and $\mathcal{A}$ be a randomized algorithm, that always outputs either \textsc{Yes} or \textsc{No} whereas
\begin{align*}
\Pr[\mathcal{A}(I) \text{correct}] \geq \frac{1}{2} + \varepsilon
\end{align*}
Then, for all $\delta > 0$ we call $\mathcal{A}_{\delta}$ the algorithm that calls $\mathcal{A}$ $N = 4 \varepsilon^{-2} \ln(\delta^{-1})$ independent times and then returns the largest number of equal responses. Then we have
\begin{align*}
\Pr[\mathcal{A}_{\delta}(I) \text{correct}] \geq 1 - \delta
\end{align*}
\end{theorem}
For randomized algorithms for optimization problems like the calculation of a largest possible stable set, as seen in Example 2.37 in the script, we usually only consider if they achieve the desired outcome.
\begin{theorem}[]{Optimization problem algorithms}
Let $\varepsilon > 0$ and $\mathcal{A}$ be a randomized algorithm for a maximization problem, for which
\begin{align*}
\Pr[\mathcal{A}(I) \geq f(I)] \geq \varepsilon
\end{align*}
Then, for all $\delta > 0$ we call $\mathcal{A}_{\delta}$ the algorithm that calls $\mathcal{A}$ $N = 4 \varepsilon^{-2} \ln(\delta^{-1})$ independent times and then returns the \textit{best} result. Then we have
\begin{align*}
\Pr[\mathcal{A}_{\delta}(I) \geq f(I)] \geq 1 - \delta
\end{align*}
\end{theorem}
For minimization problems, analogously, we can replace $\geq f(I)$ with $\leq f(I)$
% P154
\subsubsection{Sorting and selecting}
The QuickSort algorithm is a well-known example of a Las-Vegas algorithm. It is one of the algorithms that \textit{always} sorts correctly, but its runtime depends on the selection of the pivot elements, which happens randomly.
\begin{recall}[]{QuickSort}
As covered in the Algorithms \& Data Structures lecture, here are some important facts
\begin{itemize}
\item Time complexity: \tcl{n \log(n)}, \tct{n \log(n)}, \tco{n^2}
\item Performance is dependent on the selection of the pivot (the closer to the middle the better, but not in relation to its current location, but rather to its value)
\item In the algorithm below, \textit{ordering} refers to the operation where all elements lower than the pivot element are moved to the left and all larger than it to the right of it.
\end{itemize}
\end{recall}
\begin{algorithm}
\caption{\textsc{QuickSort}}
\begin{algorithmic}[1]
\Procedure{QuickSort}{$A$, $l$, $r$}
\If{$l < r$}
\State $p \gets$ \textsc{Uniform}($\{l, l + 1, \ldots, r\}$) \Comment{Choose pivot element randomly}
\State $t \gets$ \textsc{Partition}($A$, $l$, $r$, $p$) \Comment{Return index of pivot element (after ordering)}
\State \Call{QuickSort}{$A$, $l$, $t - 1$} \Comment{Sort to the left of pivot}
\State \Call{QuickSort}{$A$, $t$, $r$} \Comment{Sort to the right of pivot}
\EndIf
\EndProcedure
\end{algorithmic}
\end{algorithm}
\newcommand{\qsv}{\mathcal{T}_{i, j}}
We call $\qsv$ the random variable describing the number of comparisons executed during the execution of \textsc{QuickSort}($A, l, r$).
To prove that the average case of time complexity in fact is \tct{n \log(n)}, we need to show that
\begin{align*}
\E[\qsv] \leq 2(n + 1) \ln(n) + \text{\tco{n}}
\end{align*}
which can be achieved using a the linearity of the expected value and an induction proof. (Script: p. 154)
\fhlc{Cyan}{Selection problem}
For this problem, we want to find the $k$-th smallest value in a sequence $A[1], \ldots, A[n]$. An easy option would be to simply sort the sequence and then return the $k$-th element of the sorted array. The only problem: \tco{n \log(n)} is the time complexity of sorting.
Now, the \textsc{QuickSelect} algorithm can solve that problem in \tco{n}
\begin{algorithm}
\caption{\textsc{QuickSelect}}
\begin{algorithmic}[1]
\Procedure{QuickSelect}{$A, l, r, k$}
\State $p \gets \textsc{Uniform}(\{l, l + 1, \ldots, r\})$ \Comment{Choose pivot element randomly}
\State $t \gets \textsc{Partition}(A, l, r, p)$
\If{$t = l + k - 1$}
\State \Return{$A[t]$} \Comment{Found element searched for}
\ElsIf{$t > l + k - 1$}
\State \Return{\Call{QuickSelect}{$A, l, t - 1, k$}} \Comment{Searched element is to the left}
\Else
\State \Return{\Call{QuickSelect}{$A, t + 1, r, k - t$}} \Comment{Searched element is to the right}
\EndIf
\EndProcedure
\end{algorithmic}
\end{algorithm}
\subsubsection{Primality test}
Deterministically testing for primality is very expensive if we use a simple algorithm, namely \tco{\sqrt{n}}. There are nowadays deterministic algorithms that can achieve this in polynomial time, but they are very complex.
Thus, randomized algorithms to the rescue, as they are much easier to implement and also much faster. With the right precautions, they can also be very accurate, see theorem 2.74 for example.
A simple randomized algorithm would be to randomly pick a number on the interval $[2, \sqrt{n}]$ and checking if that number is a divisor of $n$. The problem: The probability that we find a \textit{certificate} for the composition of $n$ is very low (\tco{\frac{1}{n}}). Looking back at modular arithmetic in Discrete Maths, we find a solution to the problem:
\begin{theorem}[]{Fermat's little theorem}
If $n \in \N$ is prime, for all numbers $0 < a < n$ we have
\begin{align*}
a^{n - 1} \equiv 1 \texttt{ mod } n
\end{align*}
\end{theorem}
Using exponentiation by squaring, we can calculate $a^{n - 1} \texttt{ mod } n$ in \tco{k^3}.
\begin{algorithm}
\caption{\textsc{Miller-Rabin-Primality-Test}}\label{alg:miller-rabin-primality-test}
\begin{algorithmic}[1]
\Procedure{Miller-Rabin-Primality-Test}{$n$}
\If{$n = 2$}
\State \Return \texttt{true}
\ElsIf{$n$ even or $n = 1$}
\State \Return \texttt{false}
\EndIf
\State Choose $a \in \{2, 3, \ldots, n - 1\}$ randomly
\State Calculate $k, d \in \Z$ with $n - 1 = d2^k$ and $d$ odd \Comment{See below for how to do that}
\State $x \gets a^d \texttt{ mod } n$
\If{$x = 1$ or $x = n - 1$}
\State \Return \texttt{true}
\EndIf
\While{not repeated more than $k - 1$ times} \Comment{Repeat $k - 1$ times}
\State $x \gets x^2 \texttt{ mod } n$
\If{$x = 1$}
\State \Return \texttt{false}
\EndIf
\If{$x = n - 1$}
\State \Return \texttt{true}
\EndIf
\EndWhile
\State \Return \texttt{false}
\EndProcedure
\end{algorithmic}
\end{algorithm}
This algorithm has time complexity \tco{\ln(n)}. If $n$ is prime, the algorithm always returns \texttt{true}. If $n$ is composed, the algorithm returns \texttt{false} with probability at least $\frac{3}{4}$.
\newpage
\fhlc{Cyan}{Notes} We can determine $k, d \in \Z$ with $n - 1 = d2^k$ and $d$ odd easily using the following algorithm
\begin{algorithm}
\caption{Get $d$ and $k$ easily}\label{alg:get-d-k}
\begin{algorithmic}[1]
\State $k \gets 1$
\State $d \gets n - 1$
\While{$d$ is even}
\State $d \gets \frac{d}{2}$
\State $k \gets k + 1$
\EndWhile
\end{algorithmic}
\end{algorithm}
What we are doing here is removing the all even divisors from $d$, to make it odd.
\subsubsection{Target-Shooting}
The Target-Shooting problem is the following: Given a set $U$ and a subset thereof $S \subseteq U$, whose cardinality is unknown, how large is the quotient $\frac{|S|}{|U|}$. We define an indicator variable for $S$ by $I_S : U \rightarrow \{0, 1\}$, where $I_S(u) = 1$ if and only if $u \in S$
The Target-Shooting algorithm approximates the above quotient:
\begin{algorithm}
\caption{Target-Shooting}\label{alg:target-shooting}
\begin{algorithmic}[1]
\Procedure{TargetShooting}{$N, U$}
\State Choose $u_1, \ldots, u_N \in U$ randomly, uniformly and independently
\State \Return $N^{-1} \cdot \sum_{i = 1}^{N} I_S(u_i)$
\EndProcedure
\end{algorithmic}
\end{algorithm}
For this algorithm, two assumptions have to be made:
\begin{itemize}
\item $I_S$ has to be efficiently computable
\item We need an efficient procedure to choose a uniformly random element from the set $U$.
\end{itemize}
\begin{theorem}[]{Target-Shooting}
Let $\delta, \varepsilon > 0$. If $N \geq 3 \frac{|U|}{|S|} \cdot \varepsilon^{-2} \cdot \ln\left(\frac{2}{\delta}\right)$, the output of \textsc{Target-Shooting} is, with probability \textit{at least} $1 - \delta$, on the Interval $\left[ (1 - \varepsilon) \frac{|S|}{|U|}, (1 + \varepsilon) \frac{|S|}{|U|} \right]$
\end{theorem}
\subsubsection{Finding dulicates}
Deterministically, this could be achieved using a \textsc{HashMap} or the like, iterating over all items, hashing them and checking if the hash is available in the map.
This though uses significant amounts of extra memory and is also computationally expensive.
A cheaper option (in terms of memory and time complexity) is to use a \textit{Bloomfilter}.
The randomized algorithm also use Hash-Functions, but are not relevant for the exam.
\begin{definition}[]{Hash Function}
If we have $\mathcal{S} \subseteq U$, i.e. our dataset $\mathcal{S}$ is subset of a \textit{universe} $U$, a \textit{hash function} is an image $h: U \rightarrow [m]$, whereas $[m] = \{1, \ldots, m\}$ and $m$ is the number of available memory cells. It is assumed that we can \textit{efficiently} calculate a hash for an element and that all elements are randomly distributed, i.e. we have for $u \in U$ $\Pr[h(u) = i] = \frac{1}{m}$ for all $i \in [m]$
\end{definition}

BIN
semester2/analysis-i/assets/limits.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 420 KiB

BIN
semester2/analysis-i/assets/series.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 163 KiB

BIN
semester2/analysis-i/assets/taylor-polynomials.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 310 KiB

BIN
semester2/analysis-i/cheat-sheet.pdf Normal file
View File

Binary file not shown.

32
semester2/analysis-i/cheat-sheet.tex Normal file
View File

@@ -0,0 +1,32 @@
\documentclass{article}
\newcommand{\dir}{~/projects/latex} % IMPORTANT: No trailing slashes!
\input{\dir/include.tex}
\load{recommended}
\setLang{de}
\setupCheatSheet{Analysis Cheat-Sheet}
\begin{document}
\maketitle
\usetcolorboxes
\setcounter{numberingConfig}{3}
\setcounter{numberSubsections}{1}
\tableofcontents
\input{parts/fields.tex}
\input{parts/sequences-and-series.tex}
\input{parts/continuous-functions.tex}
\input{parts/differentiable-functions.tex}
\input{parts/riemann-integral.tex}
\input{parts/table-of-derivatives.tex}
\end{document}

212
semester2/analysis-i/parts/continuous-functions.tex Normal file
View File

@@ -0,0 +1,212 @@
\newsection
\section{\tr{Continuous Functions}{Stetige Funktionen}}
\subsection{\tr{Real-Valued functions}{Reellwertige Funktionen}}
\begin{simplebox}[]{blue}
\compactdef{\tr{Bounds}{Beschränkung}} \trLet $f \in \R^D$, \tr{where $\R^D$ is the set of all functions}{wobei $\R^D$ die Menge aller Funktionen} $f: D \rightarrow \R$\tr{, which is a vector space}{ ist, d.h. $\R^D$ ist ein Vektorraum}
\begin{itemize}
\item $f$ \tr{is \bi{bounded from above} if}{ist \bi{nach oben beschränkt}, falls}
$f(D) \subseteq \R$ \tr{is bounded from above}{nach oben beschränkt ist}.
\item $f$ \tr{is \bi{bounded from below} if}{ist \bi{nach unten beschränkt}, falls}
$f(D) \subseteq \R$ \tr{is bounded from below}{nach unten beschränkt ist}.
\item $f$ \tr{is \bi{bounded} if}{ist \bi{beschränkt} falls}
$f(D) \subseteq \R$ \tr{is bounded}{beschränkt ist}.
\end{itemize}
\end{simplebox}
\begin{simplebox}[]{blue}
\compactdef{\tr{Monotonicity}{Monotonie}} \trIf $D \subseteq \R$ \tr{we have the following terms for monotonicity}{gibt es die folgenden Monotoniebegriffe}:
\begin{itemize}
\item \tr{\bi{monotonically increasing} if}
{\bi{monoton wachsend}}
$\forall x, y \in D$ $x \leq y \Rightarrow f(x) \leq f(y)$
\item \tr{\bi{strictly monotonically increasing} if}
{\bi{streng monoton wachsend}, falls}
$\forall x, y \in D$ $x < y \Rightarrow f(x) < f(y)$
\item \tr{\bi{monotonically decreasing} if}
{\bi{monoton fallend}, falls}
$\forall x, y \in D$ $x \leq y \Rightarrow f(x) \geq f(y)$
\item \tr{\bi{strictly monotonically decreasing} if}
{\bi{streng monoton fallend}, falls}
$\forall x, y \in D$ $x < y \Rightarrow f(x) > f(y)$
\item \tr{\bi{monotone} if $f$ is monotonically increasing or monotonically decreasing}
{\bi{monoton}, falls $f$ monoton wachsend oder monoton fallend ist}
\item \tr{\bi{strictly monotone} if $f$ is strictly monotonically increasing or strictly monotonically decreasing}
{\bi{streng monoton}, falls $f$ streng monoton machsend oder streng monoton fallend ist}
\end{itemize}
\end{simplebox}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Continuity}{Stetigkeit}}
\shade{teal}{Intuition:} \tr{we can draw a continuous function without lifting the pen}{Eine stetige Funktion kann ohne den Stift zu heben gezeichnet werden}.
\compactdef{\tr{Continuity of $f$ in $x_0$}{Stetigkeit von $f$ in $x_0$}} \tr{If for every}{Falls für jedes} $\varepsilon > 0$ \tr{exists a}{ein} $\delta$ \tr{s.t.}{existiert, s.d.} $|x - x_0| < \delta \Rightarrow |f(x) - f(x_0)| < \varepsilon$
\compactdef{\tr{Continuity}{Stetigkeit}} $f$ \tr{continuous if continuous in all points of $D$}{stetig falls $f$ in allen Punkten von $D$ stetig ist}
\stepcounter{all} \shorttheorem $f$ \tr{is continuous in}{ist stetig in} $x_0 \Longleftrightarrow$ \tr{for}{für} $\seq{a}$ $\limni a_n = x_0 \Rightarrow f(a_n) = f(x_0)$
\shortcorollary \trLets $f$, $g$ \tr{continuous in $x_0$, then}{stetig in $x_0$, dann gilt} $f + g$, $\lambda \cdot f$, $f \cdot g$, $f \circ g$ \tr{are continuous in $x_0$ and if}{sind stetig in $x_0$ und falls} $g(x_0) \neq 0$,
\tr{}{ist} $\frac{f}{g}$ \tr{is continuous in $x_0$ for}{stetig in $x_0$ für} $\frac{f}{g}: D \cap \{ x \in D : g(x) \neq 0 \} \rightarrow \R$
\compactdef{\tr{Polynomial function}{Polynomiale Funktion}} $P(x) = a_n x^n + \ldots + a_0$, \trif $a_n \neq 0$, $\deg(P) = n$ (\tr{degree of}{Grad von} $P$)
\shortcorollary \tr{They are continuous on all of}{Sie sind stetig auf ganz} $\R$
\shortcorollary $P, Q$ pol. \tr{func. on}{funk. auf} $\R$ \trwith $Q \neq 0$, \tr{where}{wobei} $x_1, \ldots, x_m$ \tr{are zeros of $Q$. Then}{die Nullstellen von $Q$ sind. Dann gilt}: $\frac{P}{Q} : \R \backslash \{x_1, \ldots, x_m\} \rightarrow \R$ \tr{is continuous}{ist stetig}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Intermediate value theorem}{Zwischenwertsatz}}
\shorttheorem \trLet $I \subseteq \R$ \tr{be an interval}{ein Intervall}, $f: I \rightarrow \R$ \tr{a continuous function and}{eine stetige Funktion und} $a, b \in I$.
\tr{For each $c$ between $f(a)$ and $f(b)$ exists a $z$ between $a$ and $b$ with}{Für jedes $c$ zwischen $f(a)$ und $f(b)$ existiert ein $z$ zwischen $a$ und $b$ mit} $f(z) = c$
\shortcorollary \tr{Let $P$ be a polynomial with}{Sei $P$ ein Polynom mit} $\deg(P) = n$, $n$ \tr{odd. Then,}{ungerade. Dann hat} $P$ \tr{has \textit{at least} one zero in}{\textit{mind.} eine Nullstelle in} $\R$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{Min-Max-\tr{Theorem}{Satz}}
\stepcounter{all}\compactdef{\tr{Compact interval}{Kompaktes Intervall}} \tr{if interval $I$ is of form}{falls das Intervall $I$ von der Form} $I = [a, b], \smallhspace a \leq b$ \tr{}{ist}
\shortlemma $f, g$ \tr{continuous in $x_0$. Then}{stetig in $x_0$. Dann gilt}: $|f|$, $\max(f, g)$ \tr{and}{und} $\min(f, g)$ \tr{are continuous in}{sind stetig in} $x_0$ ($\min(f, g)$ \tr{is the minimum of the two functions at each}{ist das Minimum der beiden Funktionen für jedes} $x$)
\shortlemma $\seq{x}$ \tr{converging series in $\R$ with}{konvergente Reihe in $\R$ mit} $\displaystyle \limni x_n \in \R$ \trand $a \leq b$.
\trIf $\{ x_n : n \geq 1 \} \subseteq [a, b]$ \tr{we have}{dann gilt} $\displaystyle \limni x_n \in [a, b]$
%
\shorttheorem \trLet $f$ \tr{continuous on compact interval $I$. Then}{stetig auf dem kompakten Intervall $I$. Dann gilt}
$\exists u \in I$ \trand $\exists v \in I$ \trwith $f(u) \leq f(x) \leq f(v) \smallhspace \forall x \in I$. $f$ \tr{is bounded}{ist beschränkt}.
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Inverse function theorem}{Satz über die Umkehrabbildung}}
\shorttheorem \trLets $D_1, D_2 \subseteq \R$, $f: D_1 \rightarrow D_2$, $g: D_2 \rightarrow \R$, $x_0 \in D_1$.
\trIf $f$ \tr{cont. in}{stetig in} $x_0$, $g$ \tr{in}{auf} $f(x_0)$ \tr{then}{dann} $f \circ g : D_1 \rightarrow \R$ \tr{is continuous in}{stetig in} $x_0$\\
%
\shortcorollary \tr{If in theorem 3.5.1 $f$ continuous on}{Falls in Satz 3.5.1 $f$ stetig auf} $D_1$ \trand $g$ \tr{on}{auf} $D_2$, \tr{then}{dann ist} $g \circ f$ \tr{is continuous on}{stetig auf} $D_1$\\
\compacttheorem{\tr{Inverse function theorem}{Satz über Umkehrabbildung}} \trLet $f: I \rightarrow \R$ \tr{continuous, strictly monotone and let}{stetig, streng monoton und sei} $I \subseteq \R$ \tr{be an interval. Then}{ein Intervall. Dann gilt}: $J: = f(I) \subseteq \R$ \tr{is an interval and}{ist ein Intervall und} $f^{-1}: J \rightarrow I$ \tr{continuous and strictly monotone}{ist stetig und streng monoton}.
% \mediumhspace This means that a function is invertible $\Leftrightarrow$ $f$ is continuous and strictly monotone
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Real-Valued exponential function}{Reellwertige Exponentialfunktion}}
\tr{The exponential function}{Die Exponentialfunktion} $\exp: \C \rightarrow \C$ \tr{is usually given by a power series converging on all $\C$}{wird normalerweise durch eine auf ganz $\C$ konvergente Potenzreihe definiert}:
$\displaystyle \exp(z) := \sum_{n = 0}^{\infty} \frac{z^n}{n!}$, \tr{here for}{hier für} $z \in \R$. $\exp$ \tr{is bijective, continuous, strictly monotonically increasing and smooth}{ist bijektiv, streng monoton wachsend, glatt und stetig}. $\exp^{-1}(x) = \ln(x)$
\vspace{-0.5pc}
\shorttheorem $\exp: \R \rightarrow ]0, +\infty[$ \tr{is strictly monotonically increasing, continuous and surjective}{ist streng monoton wachsend, stetig und surjektiv}
\shortcorollary $\exp(x) > 0 \smallhspace \forall x \in \R$\\
\shortcorollary $\exp(z) > \exp(y) \smallhspace \forall z > y$
\shortcorollary $\exp(x) \geq 1 + x \smallhspace \forall x \in \R$
%
\shortcorollary $\ln: ]0, +\infty[ \rightarrow \R$
\tr{is strictly monotonically increasing, continuous and bijective. We have}{ist streng monoton wachsend, stetig und bijektiv. Es gilt}
$\ln(a \cdot b) = \ln(a) + \ln(b) \smallhspace \forall a, b \in ]0, +\infty[$. \tr{It is the inverse function of}{Dies ist die Umkehrabbildung von} $\exp$
\shortcorollary
\begin{simplebox}[]{teal}
\begin{enumerate}
\item \trFor $a > 0 \smallhspace ]0, +\infty[ \smallhspace \rightarrow \smallhspace ]0, +\infty[$ \tr{}{ist} $x \mapsto x^a$ \tr{is a continuous, strictly monotonically increasing bijection}{eine stetige, streng monoton wachsende Bijektion}.
\item \trFor $a < 0 \smallhspace ]0, +\infty[ \smallhspace \rightarrow \smallhspace ]0, +\infty[$ \tr{}{ist} $x \mapsto x^a$ \tr{is a continuous strictly monotonically decreasing bijection}{eine stetige, streng monoton fallende Bijektion}.
\end{enumerate}
\vspace{-0.8pc}
\begin{multicols}{3}
\begin{enumerate}
\setcounter{enumi}{2}
\item $\ln(x^a) = a \ln(x) \smallhspace \forall a \in \R, \smallhspace \forall x > 0$
\item $x^a \cdot x^b = x^{a + b} \smallhspace \forall a, b \in \R, \smallhspace \forall x > 0$
\item $(x^a)^b = x^{a \cdot b} \smallhspace \forall a, b \in \R, \smallhspace \forall x > 0$
\end{enumerate}
\end{multicols}
\end{simplebox}
% ────────────────────────────────────────────────────────────────────
\newsection
\subsection{\tr{Convergence of sequences of functions}{Konvergenz von Funktionenfolgen}}
\compactdef{\tr{Pointwise convergence}{Punktweise Konvergenz}} $\seq{f}$ \tr{converges pointwise towards a function}{konvergiert punktweise gegen eine Funktion} $f: D \rightarrow \R$ \tr{if for all}{falls für alle} $x \in D \smallhspace f(x) = \limni f_n(x)$\\
%
\stepcounter{all} \compactdef{Weierstrass} \tr{Sequence $f_n$ converges uniformly in $D$ to $f$ if}{Folge $f_n$ konv. gleichmässig in $D$ gegen $f$ falls}
$\forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \tr{s.t.}{s.d.} $\forall n \geq N, \smallhspace \forall x \in D: |f_n(x) - f(x)| < \varepsilon$\\
%
\shorttheorem $f_n$ \tr{sequence of (in $D$) continuous functions converging to $f$ uniformly in $D$. Then, $f$ is continuous (in $D$)}
{ist eine Folge von (in $D$) stetigen Funktionen die in $D$ gleichmässig konvergieren. Dann ist $f$ (in $D$) stetig}\\
%
\compactdef{\tr{Uniform convergence of}{Gleichmässige Konvergenz von} $\seq{f}$)} $f_n$ \trif $\forall x \in D \smallhspace f(x) : = \limni f_n(x)$ \tr{exists and}{existiert und} $\seq{f}$ \tr{converges uniformly to $f$}{gleichmässig gegen $f$ konvergiert}\\
%
\shortcorollary $f_n$ \tr{converges uniformly in}{konvergiert gleichmässig in} $D \Longleftrightarrow \forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \tr{such that}{so dass} $\forall n, m \geq N, \smallhspace \forall x \in D \smallhspace |f_n(x) - f_m(x)| < \varepsilon$\\
%
\shortcorollary \trIf $f_n$ \tr{is a uniformly converging sequence of functions, then}{eine gleichmässig konvergierende Funktionenfolge ist, dann ist} $f(x) := \limni f_n(x)$ \tr{is continuous}{stetig}\\
%
\shortdef $\displaystyle \sum_{k = 0}^{\infty} f_k(x)$ \tr{converges uniformly if}{konvergiert gleichmässig, falls} $\displaystyle S_n(x) := \sum_{k = 0}^{n} f_k(x)$ \tr{does}{gleichmässig konvergiert}
\shorttheorem \tr{Assume}{Angenommen, dass} $|f_n(x)| \leq c_n \smallhspace \forall x \in D$ \tr{and that}{und dass} $\displaystyle \sum_{n = 0}^{\infty} c_n$ \tr{converges}{konvergiert}.
\tr{Then}{Dann konvergiert} $\sum_{n = 0}^{f_n(x)}$ \tr{converges uniformly in}{gleichmässig in} $D$ \trand $f(x) := \sum_{n = 0}^{\infty} f_n(x)$ \tr{is continuous in}{ist stetig in} $D$\\
\compactdef{\tr{Radius of convergence}{Konvergenzradius}} \tr{See}{Siehe} \shade{teal}{\tr{C}{K} 2.7.19}
\shorttheorem \tr{A power series converges uniformly on}{Eine Potenzreihe konvergiert gleichmässig auf} $]-r, r[$ \tr{where}{wobei} $0 \leq r < \rho$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Trigonometric Functions}{Trigonometrische Funktionen}}
\shorttheorem $\sin: \R \rightarrow \R$ \trand $\cos: \R \rightarrow \R$ \tr{are continuous functions}{sind stetige Funktionen}
\shorttheorem
\begin{simplebox}[]{ForestGreen}
\begin{multicols}{2}
\begin{enumerate}
\item $\exp iz = \cos(z) + i \sin(z) \smallhspace \forall z \in \C$
\item $\cos(z) = \cos(-z)$ and $\sin(-z) = - \sin(z) \smallhspace \forall z \in \C$
\item $\displaystyle \sin(z) = \frac{e^{iz} - e^{-iz}}{2i}; \smallhspace \cos(z) = \frac{e^{iz} + e^{iz}}{2}$
\item $\sin(z + w) = \sin(z) \cos(w) + \cos(z) \sin(w)$\\
$\cos(z + w) = \cos(z) \cos(w) - \sin(z) \sin(w)$
\item $\cos(z)^2 + \sin(z)^2 = 1 \smallhspace z \in \C$
\end{enumerate}
\end{multicols}
\end{simplebox}
\shortcorollary $\sin(2z) = 2 \sin(z) \cos(z)$ and $\cos(2z) = \cos(z)^2 - \sin(z)^2$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{Pie (delicious)}
\shorttheorem \tr{The sine function has at least one zero on}{Die Sinusfunktion hat mindestens eine Nullstelle auf} $]0, +\infty[$ \trand $\pi := \inf\{ t > 0 : \sin(t) = 0\}$.
\tr{Then}{Dann gilt} $\sin(\pi) = 0, \smallhspace \pi \in ]2, 4[$; $\forall x \in ]0, \pi[ : \sin(x) > 0$ and $e^{\frac{i\pi}{2}} = i$
\shortcorollary $x \geq \sin(x) \geq x - \frac{x^3}{3!} \smallhspace \forall 0 \leq 0 \leq \sqrt{6}$ \shortcorollary
\begin{simplebox}[]{teal}
\begin{multicols}{2}
\begin{enumerate}
\item $e^{i\pi} = -1, \smallhspace e^{2i \pi} = 1$
\item $\sin\left( x + \frac{\pi}{2} \right)$, $\cos \left(x + \frac{\pi}{2}\right) = -\sin(x) \smallhspace \forall x \in \R$
\item $\sin(x + \pi) = -\sin(x)$, $\sin(x + 2\pi) = \sin(x) \smallhspace \forall x \in \R$
\item $\cos(x + \pi) = -\cos(x)$, $\cos(x + 2\pi) = \cos(x) \smallhspace \forall x \in \R$
\end{enumerate}
\end{multicols}
\vspace{-1.8pc}
\begin{multicols}{2}
\begin{enumerate}
\setcounter{enumi}{4}
\item \tr{Zeros of sine}{Nullstellen von Sinus} = $\{ k \cdot \pi : k \in \Z \}$\\
$\sin(x) > 0 \smallhspace \forall x \in ]2k\pi, (2k + 1)\pi[, \smallhspace k \in \Z$
$\sin(x) > 0 \smallhspace \forall x \in ](2k + 1)\pi, (2k + 2)\pi[, \smallhspace k \in \Z$
\item \tr{Zeros of cosine}{NullStellen von Cosinus} = $\{ \frac{\pi}{2} \cdot k \cdot \pi : k \in \Z \}$\\
$\cos(x) > 0 \smallhspace \forall x \in ]-\frac{\pi}{2} + 2k\pi, -\frac{\pi}{2} + (2k + 1)\pi[, \smallhspace k \in \Z$
$\cos(x) > 0 \smallhspace \forall x \in ]-\frac{\pi}{2} + (2k + 1)\pi, -\frac{\pi}{2} + (2k + 2)\pi[, \smallhspace k \in \Z$
\end{enumerate}
\end{multicols}
\end{simplebox}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Limits of functions}{Grenzwerte von Funktionen}}
\compactdef{\tr{Cluster point}{Häufungspunkt}}\tr{DE: ``Häufungspunkt''}:
$x_0 \in \R$ \tr{of}{von} $D$ \trif $\forall \delta > 0 \smallhspace (]x_0 - \delta, x_0 + \delta[ \backslash \{ x_0 \}) \cap D \neq \emptyset$\\
%
\stepcounter{all}\shortdef $A \in \R$ \tr{is the limit of $f(x)$ for}{ist der Grenzwert von $f(x)$ für} $x \rightarrow x_0$ \tr{denoted}{bezeichnet} $\limit{x}{x_0} f(x) = A$, \tr{where $x_0$ is a cluster point, if}{wobei $x_0$ ein Häufungspunkt ist, falls}:
\vspace{-0.5pc}
\begin{align*}
\forall \varepsilon \smallhspace \exists \delta > 0 \text{ s.t. } \forall x \in D \cap (]x_0 - \delta, x_0 + \delta[ \backslash \{ x_0 \}) : |f(x) - A| < \varepsilon
\end{align*}
\vspace{-2.2pc}
\setcounter{all}{6}\shorttheorem \trLets $D, E \subseteq \R$, $x_r$ \tr{a cluster point of $D$ and}{ein Häufungspunkt von $D$ und} $f: D \rightarrow E$ \tr{a function. Assume that}{eine Funktion. Angenommen, dass} $y_0 := \limit{x}{x_0}$ \tr{exists and}{existiert und} $y_0 \in E$. \trIf $g: E \rightarrow \R$ \tr{is continuous in $y_0$, we have}{in $y_0$ stetig ist, dann gilt} $\limit{x}{x_0} g(f(x)) = g(y_0)$
\fhlc{Cyan}{\tr{Left / Right hand limit}{Links- / Rechtsseitige Grenzwerte}}
\tr{Used when we have functions with poles, we approach them from both sides to evaluate said pole. Differently from at Kanti, we note it}{Wird gebraucht, wenn Funktionen Polstellen haben. Wir nähhern uns der Polstelle von beiden Seiten an, um sie zu evaluieren. Anders als an der Kanti notieren wir sie mit} $x \rightarrow x_0^-$ \tr{instead of}{anstelle von mit} $x \uparrow x_0$

164
semester2/analysis-i/parts/differentiable-functions.tex Normal file
View File

@@ -0,0 +1,164 @@
% /*
% * eth - differentiable-functions.tex
% *
% * Created by Janis Hutz 07/14/2025, Licensed under the GPL V3 License
% * https://janishutz.com, development@janishutz.com
% *
% *
% */
\newsection
\section{\tr{Differentiable Functions}{Differenzierbare Funktionen}}
\subsection{\tr{Differentiation}{Ableiten}}
\compactdef{\tr{Differentiability}{Differenzierbarkeit}} $f$ \tr{is differentiable in}{ist differenzierbar in} $x_0$ \trif $\displaystyle f'(x_0) = \limit{x}{x_0} \frac{f(x) - f(x_0)}{x - x_0} = \limit{h}{0} \frac{f(x_0 + h) - f(x_0)}{h}$ \tr{exists}{existiert}.\\
%
\stepcounter{all}\shorttheorem \tr{}{Sei} $x_0$ \tr{cluster point of}{Häufungspunkt von} $D$: $f$ \tr{is differentiable in}{differenzierbar in} $x_0 \Longleftrightarrow \exists c\in \R$ \trand $r: D \rightarrow \R$ \trwith (\tr{if it applies}{falls es zutrifft, ist} $c= f'(x_0)$ \tr{is unique}{eindeutig bestimmt}):
\begin{center}
$f(x) = f(x_0) + c(x - x_0) + r(x)(x - x_0) \text{ \tr{as well as}{sowie auch} } r(x_0) = 0 \text{ \tr{and $r$ is continuous in}{und $r$ ist stetig in} } x_0$
\end{center}
%
\shorttheorem $f$ \tr{differentiable in}{differenzierbar in} $x_0 \Leftrightarrow \exists \phi: D \rightarrow \R$ \tr{continuous in}{stetig in} $x=$ \trand $f(x) = f(x_0) + \phi(x)(x - x_0) \smallhspace \forall x \in D$. \tr{Then}{Dann ist} $\phi(x_0) = f'(x_0)$
\shortcorollary $x_0 \in D$ \tr{cluster point of}{Häufungspunkt von} $D$. \trIf $f$ \tr{differentiable in}{differenzierbar in} $x_0$, $f$ \tr{continuous in}{stetig in} $x_0$
\stepcounter{all}\shortdef $f$ \tr{is differentiable on all $D$ if for each cluster point $x_0$ it is differentiable in $x_0$}{ist auf ganz $D$ differenzierbar, falls $f$ für jeden Häufungspunkt $x_0$ in $x_0$ differenzierbar ist}
\begin{simplebox}[]{ForestGreen}
\setcounter{all}{9}\compacttheorem{\tr{Basic Differentiation rules}{Grundregeln vom Ableiten}} Let $f, g$ be functions differentiable in $x_0$
\vspace{-0.8pc}
\begin{multicols}{2}
\begin{itemize}
\item $(f + g)'(x_0) = f'(x_0) + g'(x_0)$
\item $(f \cdot g)'(x_0) = f'(x_0)g(x_0) + f(x_0)g'(x_0)$
\item \trif $g(x_0) \neq 0$, $\displaystyle \left( \frac{f}{g} \right)' (x_0) = \frac{f'(x_0) g(x_0) - f(x_0) g'(x_0)}{g(x_0)^2}$
\end{itemize}
\end{multicols}
\end{simplebox}
%
\setcounter{all}{11}\compacttheorem{\tr{Chain rule}{Kettenregel}} $x_0 \in D$ \tr{cluster point}{Häufungspunkt}, $f: D \rightarrow E$ \tr{differentiable in}{differenzierbar in} $x_0$
\trst $y_0 := f(x_0) \in E$ \tr{cluster point of $E$ and let}{ein Häufungspunkt von $E$ ist und sei}
$g: E \rightarrow \R$ \tr{differentiable in}{differenzierbar in} $y_0$. \tr{Then}{Dann gilt} $g \circ f: D \rightarrow \R$ \tr{differentiable in}{differenzierbar in} $x_0$ \trand
$(g \circ f)'(x_0) = g'(f(x_0))\cdot f'(x_0)$\\
%
\shortcorollary \trLet $f: D \rightarrow E$ \tr{be a bijective function, differentiable in $x_0$ (cluster point) and}{eine in $x_0$ (Häufungspunkt) differenzierbare Bijektion} $f'(x_0) \neq 0$ \tr{as well as}{und zudem sei} $f^{-1}$ \tr{continuous in}{stetig in} $y_0 = f(x_0)$. \tr{Then}{Dann ist} $y_0$ \tr{cluster point of}{ein Häufungspunkt von} $E$, $f^{-1}$ \tr{differentiable in}{differenzierbar in} $y_0$ \trand $\displaystyle (f^{-1})'(y_0) = \frac{1}{f'(x_0)}$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{First derivative: Important Theorems}{Erste Ableitung: Wichtige Sätze}}
\shortdef \textbf{(1)} $f$ \tr{has maximum at}{hat ein Maximum in} $x_0$ \trif $\exists \delta > 0$ \trst $f(x) \leq f(x_0) \smallhspace \forall x \in ]x_0 - \delta, x_0 + \delta[ \smallhspace \cap \smallhspace D$
\textbf{(2)} $f$ \tr{has minimum at}{hat ein Minimum in} $x_0$ \trif $\exists \delta > 0$ \trst $f(x) \geq f(x_0) \smallhspace \forall x \in ]x_0 - \delta, x_0 + \delta[ \smallhspace \cap \smallhspace D$
\textbf{(3)} $f$ \tr{has extrema in}{hat ein lokales Extremum in} $x_0$ \tr{if it is either max or min}{fall es entweder ein max oder min ist}
\begin{simplebox}[]{ForestGreen}
\shorttheorem \tr{Assume $f$ differentiable in $x_0$. From the following we have that if}{Angenommen, $f$ is in $x_0$ differenzierbar} $f'(x_0) = 0$, \tr{there is an extrema at $x_0$}{dann existiert in $x_0$ ein Extremum}
\vspace{-0.8pc}
\begin{multicols}{2}
\begin{enumerate}
\item \trIf $f'(x_0) > 0 \smallhspace \exists \delta > 0$ \trst $f(x) > f(x_0) \smallhspace \forall x \in ]x_0, x_0 + \delta[$ \trand $f(x) < f(x_0) \smallhspace \forall x \in ]x_0 - \delta, x_0[$
\item \trIf $f'(x_0) < 0 \smallhspace \exists \delta > 0$ \trst $f(x) < f(x_0) \smallhspace \forall x \in ]x_0, x_0 + \delta[$ \trand $f(x) > f(x_0) \smallhspace \forall x \in ]x_0 - \delta, x_0[$
\end{enumerate}
\end{multicols}
\end{simplebox}
\shorttheorem \trLet $f: [a, b] \rightarrow \R$ \tr{continuous and differentiable in}{stetig und differenzierbar in} $]a, b[$. \trIf $f(a) = f(b)$, $\exists \xi \in ]a, b[$ \trwith $f'(\xi) = 0$\\
%
\shorttheorem \trLet $f$ \tr{as above, then}{wie oben, dann} $\exists \xi \in ]a, b[$ s.t. $f(b) - f(a) = f'(\xi)(b - a)$
%
\shortcorollary \trLet $f, g$ \tr{as above}{wie oben} ($I = [a, b]$), \tr{then}{dann gilt}:
\begin{simplebox}[]{teal}
\begin{multicols}{2}
\begin{enumerate}
\item $f'(\xi) = 0 \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow f$ \tr{constant}{konstant}
\item $f'(\xi) = g'(\xi) \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow \exists c\in \R$ \trwith $f(x) = g(x) + c \smallhspace \forall x \in [a, b]$
\item $f'(\xi) \geq 0 \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow f$ \tr{mon. increasing on}{monoton wachsend auf} $I$
\item $f'(\xi) > 0 \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow f$ \tr{strictly mon. inc. on}{strikt mon. wachsend auf} $I$
\item $f'(\xi) \leq 0 \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow f$ \tr{mon. decreasing on}{monoton fallend auf} $I$
\item $f'(\xi) < 0 \smallhspace \forall \xi \in ]a, b[ \smallhspace \Rightarrow f$ \tr{strictly mon. dec. on}{strikt mon. fallend auf} $I$
\item \trIf $\exists M \geq 0$ \trst $|f'(\xi)| \leq M \smallhspace \forall \xi \in ]a, b[$, \tr{then}{dann gilt} $\forall x_1, x_2 \in [a, b] \smallhspace |f(x_1) - f(x_2)| \leq M|x_1 - x_2|$
\end{enumerate}
\end{multicols}
\end{simplebox}
\setcounter{all}{9}
\shorttheorem $f,g,\xi$ \tr{as defined previously. Then}{Wie vorhin definiert. Dann gilt} $g'(\xi)(f(b) - f(a)) = f'(\xi)(g(b) - g(a))$.
If $g'(x) \neq 0 \smallhspace x \in ]a, b[$, $g(a) \neq g(b)$ and $\displaystyle \frac{f(b) - f(a)}{g(b) - g(a)} = \frac{f'(\xi)}{g'(\xi)}$
%
\compacttheorem{L'Hospital\tr{'s rule}{}} $f, g$ \tr{as before}{wie vorhin}, \trwith $\displaystyle g'(x) \neq 0 \smallhspace \forall x \in ]a, b[$.
\trIf $\displaystyle \limit{x}{b^-} f(x) = 0$, $\displaystyle \limit{x}{b^-}g(x) = 0$ \trand $\displaystyle \lambda := \limit{x}{b^-} \frac{f'(x)}{g'(x)}$ \tr{exists, we have}{existiert, folgt} $\displaystyle \limit{x}{b^-} \frac{f(x)}{g(x)} = \limit{x}{b^-} \frac{f'(x)}{g'(x)}$
%
\setcounter{all}{13}\shortdef $f$ \bi{\tr{convex}{konvex}} \tr{on}{auf} $I$ \trif $\forall x \leq y \in I$ \trand $\lambda \in [0, 1]$
$f(\lambda x + (1 - \lambda)y) \leq \lambda f(x) + (1 - \lambda)f(y)$.
\bi{\tr{Strictly convex}{Streng konvex}} \trif \tr{$<$ instead of $\leq$ in all occurences}{jedes $<$ durch $\leq$ ersetzt wird}
\setcounter{all}{16} \shorttheorem $f$ \tr{(as usual) (strictly) convex}{(wie immer) (streng) konvex} $\Longleftrightarrow f'$ \tr{(strictly) monotonically increasing}{(streng) monoton wachsend}.
\shortcorollary \trIf $f''$ \tr{exists, then $f$ (strictly) convex if}{existiert ist $f$ (streng) konvex falls} $f'' \geq 0$ (\tror $f'' > 0$) \tr{on}{auf} $]a, b[$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Higher derivatives}{Höhere Ableitungen}}
\begin{definition}[]{\tr{Higher derivatives}{Höhere Ableitungen}}
\begin{enumerate}
\item \trFor $n \geq 2$, $f$ \tr{differentiable $n$ times in $D$ if}{$n$-mal differenzierbar in $D$ falls}
$f^{(n - 1)}$ \tr{is differentiable in $D$}{in $D$ differenzierbar ist}.
$f^{(n)} := (f^{(n - 1)})'$, $n$-\tr{th derivative of}{te Ableitung von} $f$
\item $f$ \tr{is $n$-times continuously differentiable in $D$ if}{ist $n$-mal stetig differenzierbar in $D$ falls}
$f^{(n)}$ \tr{exists and is continuous in}{existiert und ist stetig in} $D$
\item $f$ \tr{is called smooth (de: glatt) in $D$ if}{ist \bi{glatt} in $D$ falls} $\forall n \geq 1$ $f^{(n)}$ \tr{exists}{existiert}.
\end{enumerate}
\end{definition}
%
\stepcounter{all} \shorttheorem \textbf{\textit{(1)}} $(f + g)^{(n)} = f^{(n)} + g^{(n)}$, \bi{(2)}
%
$(f \cdot g)^{(n)} = \sum_{k = 0}^{n} {n \choose k} f^{(k)} g^{(n - k)}$ (binomial expansion), \trfor $f, g$ \tr{differentiable $n$ times}{$n$-mal differenzierbar}
%
\stepcounter{all} \shorttheorem $f, g$ \tr{as above; If}{wie oben; Falls} $g(x) \neq 0 \smallhspace \forall x \in D$, \tr{then}{dann ist} $\displaystyle \frac{f}{g}$ \tr{differentiable $n$-times in $D$}{$n$-mal in $D$ differenzierbar}
%
\shorttheorem \trLets $E, D \subseteq \R$ \tr{for which each point is a cluster point and}{für die jeder Punkt ein Häufungspunkt ist und} $f: D \rightarrow E$ \trand $g: E \rightarrow D$, \tr{both differentiable $n$ times. Then}{beide $n$-mal differenzierbar. Dann ist}
$(g \circ f)^{(n)}(x) = \sum_{k = 1}^{n} A_{n, k}(x) (g^{(k)} \circ f)(x)$ \tr{where}{wobei} $A_{n, k}$ \tr{is a polynomial in the functions}{ein Polynom in den Funktionen} $f', f^{(2)}, \ldots, f^{(n + 1 - k)}$ \tr{}{ist}
% ────────────────────────────────────────────────────────────────────
\newsection
\subsection{\tr{Power series and Taylor approximation}{Potenzreihen und Taylor Approximation}}
\shorttheorem \tr{Assume that}{Angenommen, dass} $\seq{f}$ \tr{(for $f_n$ and $f'_n$ continuously differentiable) and}{(für $f_n$ und $f'_n$ stetig differenzierbar) und} $\seq{f'}$ \tr{converge uniformly on}{gleichmässig auf} $]a, b[$ \tr{for}{konvergieren für} $f: ]a, b[ \rightarrow \R$ \trwith $\displaystyle f := \limni f_n$ \trand $\displaystyle p := \limni f'_n$. \tr{Then $f$ is continuously differentiable and $f' = p$}{Dann ist $f$ stetig differenzierbar und $f' = p$}
\shorttheorem \tr{Power series}{Potenzreihe} $\displaystyle \sum_{k = 0}^{\infty} c_k x^k$ \trwith
$\rho > 0$, $f(x) = \displaystyle \sum_{k = 0}^{\infty} c_k(x - x_0)^k$ \tr{differentiable on}{auf} $]x_0 - \rho, x_0 + \rho[$ \tr{and}{differenzierbar und} $\displaystyle f'(x) = \sum_{k = 1}^{\infty} kc_k(x - x_0)^{k - 1}$
\shortcorollary \tr{As}{Wie} in 4.4.1, $f$ \tr{smooth on conv. interval and}{glatt auf einem konvexen Invervall und} $\displaystyle f^{(j)}(x) \sum_{k = j}^{\infty} c_k \frac{k!}{(k - j)!} (x - x_0)^{k - j}$. \tr{Specifically}{Insbesondere}, $\displaystyle c_j = \frac{f^{(j)}(x_0)}{j!}$
\stepcounter{all}\shorttheorem $f$ \tr{continuous}{stetig}, $\exists f^{(n + 1)}$. \tr{For each}{Für jedes} $a < x \leq b$ $\exists \xi \in ]a, x[$ \trwith
$\displaystyle f(x) \sum_{k = 0}^{n} \frac{f^{(k)}(a)}{k!} (x - a)^k + \frac{f^{(n + 1)}(\xi)}{(n + 1)!}(x - a)^{n + 1}$
\compactcorollary{Taylor Approximation} \tr{Same as above, but}{Gleich wie oben, aber} $f: [c, d] \rightarrow \R$ \tr{instead of}{anstelle von} $f: [a, b] \rightarrow \R$ \trand $c < a < d$ \trand $\xi$ \tr{between}{zwischen} $x$ \trand $a$.
\shortcorollary $a < x_0 < b$ \trand $f$ \tr{as before, assume that}{wie zuvor, angenommen, dass} $f'(x_0) = f^{(2)}(x_0) = \ldots = f^{(n)}(x_0) = 0$. \tr{Then}{Dann gilt}:
\vspace{-0.8pc}
\begin{multicols}{2}
\begin{enumerate}
\item \tr{If $n$ even and $x_0$ local extrema}{Falls $n$ gerade und $x_0$ lokales Extremum}, $f^{(n + 1)}(x_0) = 0$
\item \tr{If $n$ odd and}{Falls $n$ ungerade und} $f^{(n + 1)}(x_0) > 0$, $x_0$ \tr{strict local minimum}{strikte lokale Minimalstelle}
\item \tr{If $n$ odd and}{Falls $n$ ungerade und} $f^{(n + 1)}(x_0) < 0$, $x_0$ \tr{strict local maximum}{strikte lokale Maximalstelle}
\end{enumerate}
\end{multicols}
\vspace{-1pc}
\shortcorollary $f$ \tr{differentiable twice and}{2-mal differenzierbar und} $a < x_0 < b$, \tr{assume}{wir nehmen an, dass} $f'(x_0) = 0$
\vspace{-0.8pc}
\begin{multicols}{2}
\begin{enumerate}
\item $f^{(2)}(x_0) > 0$, $x_0$ \tr{strict local minimum}{strikte lokale Minimalstelle}
\item $f^{(2)}(x_0) < 0$, $x_0$ \tr{strict local maximum}{strikte lokale Maximalstelle}
\end{enumerate}
\end{multicols}
\vspace{-0.8pc}
\subsection{Exercise Help}
\includegraphics[width=0.4\linewidth]{./assets/series.png}
\begin{multicols}{2}
\shade{ForestGreen}{\tr{Common limits}{Häufige Grenzwerte}}
\includegraphics[height=0.5\textheight]{./assets/limits.png}
\shade{ForestGreen}{\tr{Common Taylor Polynomials}{Bekannte Taylorreihen}}
\includegraphics[width=0.8\linewidth]{./assets/taylor-polynomials.png}
\end{multicols}

124
semester2/analysis-i/parts/fields.tex Normal file
View File

@@ -0,0 +1,124 @@
\newsection
\section{\tr{Fields}{Räume}}
\subsection{\tr{Real numbers}{Reelle Zahlen}}
\compacttheorem{Lindemann}
\translate{There is no equation of form}{Es gibt keine Gleichung der Form} $x^n + a_{n - 1} x^{n - 1} + \ldots + a_0 = 0$ \translate{with}{mit} $a_i \in Q$ \translate{such that}{so dass} $x = \pi$ \translate{is a solution}{eine Lösung ist}
\setcounter{all}{7}
\compactcorollary{\tr{Archimedic Principle}{Archimedisches Prinzip}} \trLet $x \in \R$ \trwith $x > 0$ \trand $y \in \R$. \tr{Then exists}{Dann existiert} $n \in \N$ \trwith $y \leq n \cdot x$
\setcounter{all}{9}
\begin{definition}[]{\translate{Max, min, absolute value}{Max, Min, Betrag}}
\translate{Let}{Seien} $x, y \in \R$. \translate{Then}{Dann}:
\vspace{-0.8pc}
\begin{multicols}{3}
\begin{enumerate}[label=\textit{(\roman*)}]
\item $\max\{x, y\} = \begin{cases}
x & \text{\trif} y \leq x \\
y & \text{\trif} x \leq y
\end{cases}$
\item $\min\{x, y\} = \begin{cases}
y & \text{\trif} y \leq x \\
x & \text{\trif} x \leq y
\end{cases}$
\item \translate{The absolute value of}{Der Absolutbetrag von}\\ $x \in \R : |x| = \max{x, -x}$
\end{enumerate}
\end{multicols}
\end{definition}
\begin{theorem}[]{\translate{Absolute value properties}{Eigenschaften des Absolutbetrags}}
\vspace{-0.6pc}
\begin{multicols}{4}
\begin{enumerate}[label=\textit{(\roman*)}]
\item $|x| \geq 0 \smallhspace \forall x \in \R$
\item $|xy| = |x||y| \smallhspace \forall x, y \in \R$
\item $|x + y| \leq |x| + |y|$
\item $|x + y| \geq ||x| - |y||$
\end{enumerate}
\end{multicols}
\end{theorem}
\compacttheorem{\translate{Young's Inequality}{Young'sche Ungleichung}} $\forall \varepsilon > 0, \smallhspace \forall x, y \in \R$ \translate{we have}{gilt}: $2|xy| \leq \varepsilon x^2 + \frac{1}{\varepsilon}y^2$
\begin{definition}[]{\tr{Bounds}{Schranken}}
\begin{enumerate}[label=\textit{(\roman*)}]
\item $c \in \R$ \tr{upper bound of $A$ if}{obere Schranke von $A$ falls} $\forall a \in A : a \leq c$.
$A$ \tr{bounded from above if upper bound for $A$ exists}{nach oben beschränkt falls eine obere Schranke für $A$ existiert}
\item $c \in \R$ \tr{lower bound of $A$ if}{untere Schranke von $A$ falls} $\forall a \in A : a \leq c$.
$A$ \tr{bounded from below if lower bound for $A$ exists}{nach unten beschränkt falls eine untere Schranke für $A$ existiert}
\item \tr{Element $m \in \R$ \textbf{maximum} of $A$ if $m \in A$ and $m$ upper bound of $A$}
{Element $m \in \R$ \textbf{Maximum} von $A$ falls $m \in A$ und $m$ obere Schranke von $A$ ist}
\item \tr{Element $m \in \R$ \textbf{minimum} of $A$ if $m \in A$ and $m$ lower bound of $A$}
{Element $m \in \R$ \textbf{Minimum} von $A$ falls $m \in A$ und $m$ untere Schranke von $A$ ist}
\end{enumerate}
\end{definition}
\setcounter{all}{15}
\begin{theorem}[]{Supremum \& Infimum}
\begin{enumerate}[label=\textit{(\roman*)}]
\item \tr{The least upper bound of a set $A$ bounded from above is called the \textbf{\textit{Supremum}} and given by}
{Die kleinste obere Schranke von einer nach oben beschränkten Menge $A$, gennant das \textbf{\textit{Supremum}} von $A$, ist definiert als}
$c := \sup(A)$.
\tr{It only exists if the set is upper bounded.}
{Es existiert nur falls die Menge nach oben beschränkt ist.}
\item \tr{The greatest lower bound of a set $A$ bounded from below is called the \textbf{\textit{Infimum}} and given by}
{Die grösste untere Schranke von einer nach unten beschränkten Menge $A$, gennant das \textbf{\textit{Infimum}} von $A$, ist definiert als}
$c := \inf(A)$.
\tr{It only exists if the set is lower bounded.}
{Es existiert nur falls die Menge nach unten beschränkt ist.}
\end{enumerate}
\end{theorem}
\begin{corollary}[]{Supremum \& Infimum}
\trLet $A \subset B \subset \R$
\vspace{-0.9pc}
\begin{multicols}{2}
\begin{enumerate}[label=\textit{(\arabic*)}]
\item \tr{If $B$ is bounded from above, we have}{Falls $B$ nach oben beschränkt ist, gilt} $\sup(A) \leq \sup(B)$
\item \tr{If $B$ is bounded from below, we have}{Falls $B$ nach unten beschränkt ist, gilt} $\inf(B) \leq \inf(A)$
\end{enumerate}
\end{multicols}
\end{corollary}
% ────────────────────────────────────────────────────────────────────
\setcounter{subsection}{2}
\subsection{\tr{Complex numbers}{Komplexe Zahlen}}
\textbf{\tr{Operations}{Operationen}}: $i^2 = -1$ (\tr{NOT}{NICHT} $i = \sqrt{-1}$ \tr{bc. otherwise}{da sonst} $1 = -1$).
%
\tr{Complex number}{Komplexe Zahl} $z_j = a_j + b_ji$.
\textit{Addition, \tr{Subtraction}{Subtraktion}} $(a_1 \pm a_2) + (b_1 \pm b_2)i$.
\textit{\tr{Multiplication}{Multiplikation}} $(a_1 a_2 - b_1 b_2) + (a_1 b_2 + a_2 b_1)i$.
\textit{Division} $\displaystyle\frac{a_1 b_1 + a_2 b_2}{b_1^2 + b_2^2} + \frac{a_2 b_1 - a_1 b_2}{b_1^2 b_2^2}i$;
\textbf{\tr{Parts}{Teile}}: $\mathfrak{R}(a + bi) := a$ (\tr{Real part}{Realteil}), $\mathfrak{I}(a + bi) := b$ (\tr{imaginary part}{Imaginärteil}),
$|z| := \sqrt{a^2 + b^2}$ (modulus),
$\overline{a + bi} := a-bi$ (\tr{complex conjugate}{Komplexe Konjugation});
\textbf{\tr{Polar coordinates}{Polarkoordinaten}}: $a + bi$ (\tr{normal form}{Normalform}), $r \cdot e^{i \phi}$ (\tr{polar form}{Polarform}).
Transformation polar $\rightarrow$ normal: $r \cdot \cos(\phi) + r \cdot \sin(\phi)i$.
Transformation normal $\rightarrow$ polar: $|z| \cdot e^{i \cdot \arcsin(\frac{b}{|z|})}$;
\begin{theorem}[]{\tr{Fundamental Theorem of Algebra}{Fundamentalsatz der Algebra}}
\trLet $n \geq 1, n \in \N$ \tr{and let}{und sei}
\[
P(z) = z^n + a_{n - 1}z^{n - 1} + \ldots + a_0, \mediumhspace a_j \in \C
\]
\tr{Then there exist}{Dann gibt es} $z_1, \ldots, z_n \in \C$ \tr{such that}{so dass}
\[
P(z) = (z - z_1)(z - z_2) \dots (z - z_n)
\]
\tr{The set}{Die Menge} $\{z_1, \ldots, z_n\}$ \tr{and the multiplicity of the zeros $z_j$ are hereby uniquely determined}
{und die Vielfachheit der Nullstellen $z_j$ sind eindeutig bestimmt.}
\end{theorem}
\shade{Aquamarine}{\tr{Surjectivity}{Surjektivität}}
\tr{Given a function}{Eine Funktion} $f: X \rightarrow Y$, \tr{it is surjective, iff}{ist Surjektiv, g.d.w.} $\forall y \in Y, \exists x \in X : f(x) = y$ (\tr{continuous function}{stetige Funktion})
\shade{Aquamarine}{\tr{Injectivity}{Injektivität}}
$x_1 \neq x_2 \Rightarrow f(x_1) \neq f(x_2)$

206
semester2/analysis-i/parts/riemann-integral.tex Normal file
View File

@@ -0,0 +1,206 @@
\newsection
\section{\tr{Integrals}{Riemann Integral}}
\subsection{\tr{Definition and integrability}{Definition und Integrabilitätskriterien}}
\compactdef{Partition} \tr{finite subset}{endliche Teilmenge} $P \subset I$ \tr{where}{wo} $I = [a, b]$ \trand $\{a, b\} \subseteq P$\\
%
\tr{Lower sum}{Untersumme}: $\displaystyle s(f, P) := \sum_{i = 1}^{n} f_i \delta_i, \smallhspace f_i = \inf_{x_{i - 1} \leq x \leq x_i} f(x)$,
\tr{Upper sum}{Obersumme}: $\displaystyle S(f, P) := \sum_{i = 1}^{n} f_i \delta_i, \smallhspace f_i = \sup_{x_{i - 1} \leq x \leq x_i} f(x)$,
$\delta_i$ sub-interval\\
%
\shortlemma \trLet $P'$ \tr{be a specification of}{eine Verfeinerung von} $P$, \tr{then}{dann} $s(f, P) \leq s(f, P') \leq S(f, P') \leq S(f, P)$;
\tr{for arbitrary}{für beliebige} $P_1, P_2$, $s(f, P_1) \leq S(f, P_2)$\\
%
\shortdef $f$ \tr{bounded is integrable if}{beschränkt ist integrierbar falls} $s(f) = S(f)$ \tr{and the integral is}{und das Integral ist} $\displaystyle\int_{a}^{b} f(x) \dx x$\\
%
\shorttheorem $f$ \tr{bounded, integrable}{beschränkt, integrierbar} $\Longleftrightarrow \forall \varepsilon > 0 \smallhspace \exists P \in \mathcal{P}(I)$ \trwith $S(f, P) - s(f, P) \leq \varepsilon$ \tr{where}{wobei} $\mathcal{P}(I)$ \tr{is the set of all partitions of $I$}{alle Paritionen von $I$ ist}\\
%
\setcounter{all}{8} \shorttheorem $f$ \tr{integrable}{integrierbar} $\Longleftrightarrow \forall \varepsilon > 0 \smallhspace \exists \delta > 0$
\trst $\forall P \in \mathcal{P}_{\delta}(I), S(f, P) - s(f, P) < \varepsilon$, \tr{where}{wobei} $\mathcal{P}_{\delta}(I)$ \tr{is set of $P$ for which}{die Menge von $P$ wofür} $\displaystyle \max_{1 \leq i \leq n} \delta_i \leq \delta$\\
%
\shortcorollary $f$ \tr{integrable with}{integrierbar mit} $A := \int_{a}^{b} f(x) \dx x \Longleftrightarrow \forall \varepsilon >0
\smallhspace \exists \delta > 0$ \trst $\forall P \in \mathcal{P}(I)$ \trwith $\delta(P) < \delta$ \trand $\xi_1, \ldots, \xi_n$ \trwith $\xi_i \in [x{i - 1}, x_i]$ \trand $P = \{ x_0, \ldots, x_n \}$, $\displaystyle \left|A - \sum_{i = 1}^{n} f(\xi_i)(x_i - x_{i - 1})\right| < \varepsilon$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Integrable functions}{Integrierbare Funktionen}}
\shorttheorem $f, g$ \tr{bounded, integrable and}{beschränkt, integrierbar und} $\lambda \in \R$. \tr{Then}{Dann gilt} $f + g$, $\lambda \cdot f$, $f\cdot g$, $|f|$, $\max(f, g)$, $\min(f, g)$ \trand $\frac{f}{g} (\text{\trif } |g(x)| \geq \beta > 0 \smallhspace \forall x \in [a, b]$ \tr{are all integrable}{alle integrierbar}
%
\stepcounter{all}\shortcorollary \trLets $P, Q$ \tr{be polynomials and $Q$ has no zeros on $[a, b]$. Then}{Polynome, $Q$ keine Nullstellen auf $[a, b]$, dann}: $[a, b] \rightarrow \R$ \trand $x \mapsto \frac{P(x)}{Q(x)}$ \tr{integrable}{int.}\\
%
\compactdef{\tr{uniform continuity}{Gleichmässige Stetigkeit}} \trif
$\forall \varepsilon > 0 \smallhspace \exists \delta > 0 \smallhspace \forall x, y \in D : |x - y| < \delta \Longrightarrow |f(x) - f(y)| < \varepsilon$
\stepcounter{all}\shorttheorem $f$ \tr{continuous on compact interval}{stetig auf kompaktem Intervall} $I = [a, b] \Longrightarrow f$ \tr{is uniformly continuous on}{ist gleichmässig stetig auf} $I$
\shorttheorem $f$ \tr{continuous}{stetig} $\Longrightarrow f$ \tr{integrable}{integrierbar}
\shorttheorem $f$ \tr{monotone}{monoton} $\Longrightarrow f$ \tr{integrable}{integrierbar}
%
\stepcounter{all}\shorttheorem $I \subset \R$ \tr{compact interval with}{kompaktes Intervall mit} $I = [a, b]$ \trand $f_1, f_2$ \tr{bounded, integrable and}{beschränkt, integrierbar und} $\lambda_1, \lambda_2 \in \R$. \\
\tr{Then}{Dann gilt}: $\displaystyle\int_{a}^{b}(\lambda_1 f_1(x) + \lambda_2 + f_2(x)) \dx x = \lambda_1 \int_{a}^{b} f_1(x) \dx x + \lambda_2 \int_a^b f_2(x) \dx x$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Inequalities and Intermediate Value Theorem}{Ungleichungen und Mittelwertsatz}}
\shorttheorem $f, g$ \tr{bounded, integrable and}{beschränkt, integrierbar und} $f(x) \leq g(x) \forall x \in [a, b]$, \tr{then}{dann} $\displaystyle\int_{a}^{b} f(x) \dx x \leq \int_{a}^{b} g(x) \dx x$
%
\shortcorollary \tr{if $f$ bounded, integrable}{falls $f$ beschränkt, integrierbar}, $\left| \displaystyle\int_{a}^{b} f(x) \dx x\right| \leq \int_{a}^{b} |f(x)| \dx x$
\shorttheorem \trLet $f, g$ \tr{bounded, integrable, then}{beschränkt, integrierbar, dann} $\left| \displaystyle\int_{a}^{b} f(x) g(x) \dx x \right| \leq \sqrt{\int_{a}^{b} f^2(x) \dx x} \cdot \sqrt{\int_{a}^{b} g^2(x) \dx x}$\\
%
\compacttheorem{\tr{Intermediate Value Theorem}{Mittelwertsatz}} $f$ \tr{continuous. Then}{stetig. Dann gilt} $\exists \xi \in [a, b]$ \trst $\displaystyle\int_{a}^{b} \dx x = f(\xi)(b - a)$
\stepcounter{all}\shorttheorem \trLet $f$ \tr{continuous, $g$ bounded and integrable with}{stetig, $g$ beschränkt und integrierbar mit} $g(x) \geq 0 \smallhspace \forall x \in [a, b]$.
\tr{Then}{Dann gilt} $\exists \xi \in [a, b]$ \trst $\displaystyle\int_{a}^{b} f(x) g(x) \dx x = f(\xi) \displaystyle\int_{a}^{b} g(x) \dx$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Fundamental theorem of Calculus}{Fundamentalsatz der Differentialrechnung}}
\begin{theorem}[]{\tr{First Fundamental Theorem of Calculus}{Erster Fundamentalsatz}}
\trLet $a < b$ \trand $f: [a, b] \rightarrow \R$ \tr{continuous. The function}{stetig. Die Funktion}
\begin{align*}
F(x) = \int_{a}^{x} f(t) \dx t, \smallhspace a \leq x \leq b
\end{align*}
\tr{is differentiable in}{ist differenzierbar in} $[a, b]$ \trand $F'(x) = f(x) \smallhspace \forall x \in [a, b]$
\end{theorem}
\shortproof \tr{Split the integral}{Intervall aufteilen}: $\int_{a}^{x_0} f(t) \dx t + \int_{x_0}^{x} f(t) \dx t = \int_{a}^{x} f(t) \dx t$, \tr{so}{also} $F(x) - F(x_0) = \int_{x_0}^{x} f(t) \dx t$.
\tr{Using the Intermediate Value Theorem, we get}{Mithilfe des Mittelwertsatzes erhalten wir} $\int_{x_0}^{x} f(t) \dx t = f(\xi)(x - x_0)$
\tr{and for}{und für} $x \neq x_0$ \tr{we have}{ergibt sich} $\frac{F(x) - F(x_0)}{x - x_0} = f(\xi)$
\tr{and since}{und da} $\xi$ \tr{is between}{zwischen} $x_0$ \trand $x$ \tr{and since}{liegt und da} $f$ \tr{continuous}{stetig ist},
$\limit{x}{x_0} \frac{F(x) - F(x_0)}{x - x_0} = f(x_0)$ \hspace{10cm} $\square$
\compactdef{\tr{Anti-derivative}{Stammfunktion}} $F$ \trfor $f$ \trif $F$ \tr{is differentiable in}{differenzierbar in} $[a, b]$ \tr{and}{ist und} $F' = f$ in $[a, b]$
\begin{theorem}[]{\tr{Second Fundamental Theorem of Calculus}{Zweiter Fundamentalsatz}}
$f$ \tr{as in 5.4.1. Then there exists an anti-derivative $F$ of $f$ that is uniquely determined bar the constant of integration and}{wie in 5.4.1. Dann existiert eine Stammfunktion $F$ von $f$ die eindeutig bestimmt ist bist auf die Integrationskonstante und}
\begin{align*}
\int_{a}^{b} f(x) \dx x = F(a) - F(b)
\end{align*}
\end{theorem}
\shortproof \tr{Existence of $F$ given by 5.4.1. If $F_1$ and $F_2$ are anti-derivatives of $f$, then}{Existenz von $F$ gegeben dur 5.4.1. Falls $F_1$ und $F_2$ Stammfunktionen von $f$ sind, dann} $F'_1 - F'_2 = f - f = 0$, i.e. $(F_1 - F_2)' = 0$.
\tr{From 4.2.5 (1) we have that}{Mithilfe von 4.2.5 (1) erhalten wir, dass} $F_1 - F_2$ \tr{is constant}{konstant ist}.
\tr{We have}{Wir haben} $F(x) = C + \int_{a}^{x} f(t) \dx t$, \tr{where}{wobei} $C$ \tr{is an arbitrary constant}{eine beliebige Konstante ist}.
\tr{Especially}{Insbesondere}, $F(b) = C + \int_{a}^{b} f(t) \dx t, F(a) = C$ \tr{and thus}{und deshalb} $F(b) - F(a) = C + \int_{a}^{b} f(t) \dx t - C = \int_{a}^{b} f(t) \dx t$
\stepcounter{all}
\compacttheorem{\textbf{\tr{Integration by parts}{Partielle Integration}}} $\displaystyle \int_{a}^{b} f(x) g'(x) \dx x = \left[f(x) g(x)\right]^b_a - \int_{a}^{b} f'(x) g(x) \dx x$. \tr{Be wary of cycles}{Aufgepasst mit Zyklen}
\compacttheorem{\textbf{\tr{Integration by substitution}{Integration durch Substitution}}}
$\phi$ \tr{continuous and differentiable. Then}{stetig und differenzierbar. Dann gilt}
$\displaystyle \int_{a}^{b} f(\phi(t)) \phi'(t) \dx t = \int_{\phi(a)}^{\phi((b))} f(x) \dx x$
\tr{To use the above, in a function choose the inner function appropriately, differentiate it, substitute it back to get a more easily integrable function}
{Um das Obige zu Nutzen muss die innere Funktion passend gewählt, abgeleitet und rücksubstituiert werden um eine einfacher integrable Funktion zu erhalten}.
\setcounter{all}{8}\shortcorollary $I \subseteq \R$ \trand $f: I \rightarrow \R$ \tr{continuous}{stetig}
\begin{simplebox}[]{teal}
\vspace{-0.5pc}
\begin{multicols}{2}
\begin{enumerate}
\item \trLet $a, b, c \in \R$ \tr{s.t. the closed interval with endpoints}{s.d. das abgeschlossenes Intervall mit Endpunkten} $a + c, b + c$ \tr{is contained in $I$. Then}{in $I$ enthalten ist. Dann gilt}
\begin{align*}
\int_{a + c}^{b + c} f(x) \dx x = \int_{a}^{b} f(t + c) \dx t
\end{align*}
\item \trLet $a, b, c \in \R, c \neq 0$ \tr{s.t. the closed interval with endpoints}{s.d. das abgeschlossene Intervall mit Endpunkten} $ac, b$ \tr{is contained in $I$. Then}{in $I$ enthalten ist. Dann gilt}
\begin{align*}
\frac{1}{c} \int_{ac}^{bc} f(x) \dx x = \int_{a}^{b} f(ct) \dx t
\end{align*}
\end{enumerate}
\end{multicols}
\end{simplebox}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Integration of converging series}{Integration einer konvergierenden Reihe}}
\shorttheorem \trLet $f_n: [a, b] \rightarrow \R$ \tr{be a sequence of bounded, integrable functions converging uniformly to $f$. Then $f$ bounded, integrable and}
{eine Folge von beschränkten, integrierbaren Funktionen die gleichmässig gegen $f$ konvergieren. Dann ist $f$ beschränkt integrierbar und}
$\limni \int_{a}^{b} f_n(x) \dx x = \int_{a}^{b} f(x) \dx x$
%
\shortcorollary $f_n$ \tr{s.t. the series converges. Then}{s.d. die Reihe konvergiert. Dann ist}
$\sum_{n = 0}^{\infty} \int_{a}^{b} f_n(x) \dx x = \int_{a}^{b} \left( \sum_{n = 0}^{\infty} f_n(x) \right) \dx x$\\
\shortcorollary $f(x) = \sum_{n = 0}^{\infty} x_k x^k$ \trwith $\rho > 0$.
\tr{Then}{Dann ist} $\forall 0\leq r < \rho$, $f$ \tr{integrable on}{integrierbar auf} $[-r, r]$ \trand $\forall x \in ]- \rho, \rho[, \int_{0}^{x} f(t) \dx t = \sum_{n = 0}^{\infty} \frac{c_n}{n + 1}x^{n + 1}$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{Euler-McLaurin \tr{summation}{Summationsformel}}
\shortdef $\forall k \geq 0$, \tr{the $k$-th Bernoulli-Polynomial}{das $k$-te Bernoulli Polynom} $B_k(x) = k!P_k(x)$, \tr{where}{wobei} $P_k' = P_{k - 1} \smallhspace \forall k \geq 1$ \trand $\int_{0}^{1} P_k(x) \dx x = 0 \smallhspace\forall k \geq 1$
%
\shortdef \trLet $B_0 = 1$. $\forall k \geq 2$ $B_{k - 1}$ \tr{is given recursively by}{ist rekursiv durch} $\sum_{i = 0}^{k - 1} {k \choose i} B_i = 0$ \tr{}{definiert}
%
\compacttheorem{\tr{McLaurin Series}{McLaurin Reihe}} $B_k(x) = \sum_{i = 0}^{k} {k \choose i} B_i x^{k - i}$
%
\stepcounter{all} \shorttheorem $f$ $k$ \tr{times continuously differentiable}{mal stetig differenzierbar}, $k \geq 1$. \tr{Then for}{Dann gilt für} $\tilde{B_k}(x) = \begin{cases}
B_k(x) & \text{\trfor } 0 \leq x < 1\\
B_k(x - n ) & \text{\trfor } n \leq x \leq n + 1 \text{ \tr{where}{wobei} } n \geq 1
\end{cases}$ \tr{that}{dass}
\vspace{-0.3pc}
\begin{enumerate}
\item \trFor $k = 1$: $\sum_{i = 1}^{n} f(i) = \int_{0}^{n} f(x) \dx x + \frac{1}{2} (f(n) - f(0)) + \int_{0}^{n} \tilde{B_1}(x) f'(x) \dx x$ \mediumhspace
\tr{below}{unten}: $\tilde{R_k} = \frac{(-1)^{k - 1}}{k!} \int_0^n \tilde{B_k}(x) f^{(k)}(x) \dx x$
\item \trFor $k \geq 2$: $\displaystyle \sum_{i = 1}^{n} f(i) = \int_{0}^{n} f(x) \dx x + \frac{1}{2} (f(n) - f(0)) + \sum_{j = 2}^{k} \frac{(-1)^j B_j}{j!} (f^{(j - 1)}(n) - f^{(j - 1)}(0)) + \tilde{R_k}$,
$\displaystyle \tilde{R_k} = \sum_{(-1)^{(k - 1)}}^{k!}\int_{0}^{n} \tilde{B_1}(x) f^{(k)}(x) \dx x$
\end{enumerate}
\vspace{-0.2pc}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\vspace{-0.5pc}
\subsection{\tr{Stirling's Formula}{Stirling'sche Formel}}
\vspace{-0.2pc}
\shorttheorem $\displaystyle n! = \frac{\sqrt{2\pi n}n^n}{e^n} \cdot \exp \left( \frac{1}{12n} + R_3(n) \right)$, $|R_3(n)| \leq \frac{\sqrt{3}}{216}\cdot \frac{1}{n^2} \smallhspace \forall n \geq 1$
\shortlemma $\forall m \geq n + 1 \geq 1: |R_3(m,n)| \leq \frac{\sqrt{3}}{216} \left( \frac{1}{n^2} - \frac{1}{m^2} \right)$
\vspace{-0.2pc}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\vspace{-0.5pc}
\subsection{\tr{Improper Integrals}{Uneigentliche Integrale}}
\vspace{-0.8pc}
\shortdef $f$ \tr{bounded and integrable on}{beschränkt und integrierbar auf} $[a, b]$.
\trIf $\displaystyle \limit{b}{\infty}\int_{a}^{b} f(x) \dx x$ \tr{exists, we denote it}{existiert, wir notieren als}
$\int_{a}^{\infty} f(x) \dx x$ \tr{and call $f$ integrable on}{und sagen $f$ ist integrierbar auf} $[a, +\infty[$
%
\stepcounter{all}\shortlemma $f: [a, \infty[ \rightarrow \R$ \tr{bounded and integrable on}{beschränkt und integrierbar auf} $[a,b] \forall b > 0$.
\trIf $|f(x) \leq g(x) \smallhspace \forall x \geq a$ \trand $g(x)$ \tr{integrable on}{integrierbar auf} $[a, \infty[$, \tr{then}{dann ist}
$f$ \tr{is integrable on}{integrierbar auf} $[a, \infty[$.
\trIf $0 \leq g(x) \leq f(x)$ \trand $\int_{a}^{\infty} g(x) \dx x$ \tr{diverges, so does}{divergiert, wie auch} $\int_{a}^{\infty} f(x) \dx x$
%
\stepcounter{all}\shorttheorem $f:[1, \infty[ \rightarrow [0, \infty[$ \tr{monotonically decreasing}{monoton fallend}. $\sum_{n = 1}^{\infty} f(n)$ \tr{converges}{konvergiert} $\Leftrightarrow \int_{1}^{\infty} f(x) \dx x$ \tr{converges}{konvergiert}
%
\setcounter{all}{8} \shortdef \trIf $f: ]a, b]$ \tr{is bounded and integrable on}{ist beschränkt und integrierbar auf} $[a + \varepsilon, b], \varepsilon > 0$, \tr{but not necessarily on}{aber nicht zwingend auf} $]a, b]$, \tr{then}{dann ist} $f$ \tr{is integrable if}{integrierbar falls}
$\limit{\varepsilon}{0^+} \int_{a + \varepsilon}^{b} f(x) \dx x$ \tr{exists, then called}{existiert, dann gennant} $\int_{a}^{b} f(x) \dx x$ \\
%
\setcounter{all}{11} \compactdef{Gamma function} \trFor $s > 0$ \tr{we define}{definieren wir} $\Gamma (s) := \int_{0}^{\infty} e^{-x}x^{s - 1} \dx x$\\
%
\shorttheorem \textbf{\textit{(1)}} $\Gamma(s)$ \tr{fulfills}{erfüllt} $\Gamma(1) = 1$, $\Gamma(s + 1) = s \Gamma(s) \smallhspace \forall s > 0$ \trand $\Gamma(\lambda x + (1 - \lambda)y) \leq \Gamma(x)^\lambda \Gamma(y)^{1 - \lambda} \smallhspace \forall x, y > 0, \smallhspace \forall 0 \leq \lambda \leq 1$\\
%
\textbf{\textit{(2)}} $\Gamma(s)$ \tr{sole function}{einzige Funktion} $]0, \infty[ \smallhspace \rightarrow \smallhspace ]0, \infty[$ \tr{that fulfills the above conditions}{die obige Voraussetzungen erfüllt}.
\tr{Additionally:}{Ausserdem:} $\displaystyle \Gamma(x) = \limni \frac{n!n^x}{x(x + 1) \dots (x + n)} \forall x > 0$
%
\shorttheorem \trLet $p, q > 1$ \trwith $\frac{1}{p} + \frac{1}{q} = 1$, \tr{for all}{für alle} $f, g: [a, b] \rightarrow \R$ \tr{continuous, we have}{stetig, dann gilt} $\int_{a}^{b} |f(x) g(x)| \dx x \leq ||f||_p ||g||_q$
\newsectionNoPB
\subsection{\tr{Partial fraction decomposition}{Partialbruchzerlegung}}
\tr{Used for rational polynomial functions. Start by splitting the fraction into parts (usually factorized, so find zeros)}
{Wird für rationale Polynom-Funktionen genutzt. Man started mit Aufteilen des Bruchs into (meistens) faktorisierte Teile. Suche Nullstellen}.
%
\tr{Split denominator into the found parts, e.g.}{Nenner in gefundene Teile unterteilen, z.B.} $\frac{a}{x - 4} + \frac{b}{x + 2}$,
\tr{then expand to the same denominator on all fractions}{dann alle Brüche auf denselben Nenner bringen}.
%
\tr{Then $p(x)$ (the numerator) of the original fraction has to equal the new fraction's numerator, so use SLE to find coefficients}
{Dann muss $p(x)$ (der Zähler) des ursprünglichen Bruch gleich dem des neuen Bruchs entsprechen, also Lineares Gleichungssystem zum Finden der Koeffizienten nutzen}.
%
\tr{Get the numerator into the form of a polynomial, so e.g.}{Den Zähler in die Form von Polynomen bringen, also z.B.} $(a + b) \cdot x + (2a - 4b)$,
\tr{then SLE is}{dann ist das SLE}
\begin{align*}
\begin{vmatrix}
2 = a + b \\
-4 = 2a - b
\end{vmatrix}
\Leftrightarrow a = \frac{2}{3}, b = \frac{4}{3} \mediumhspace \text{\tr{for our rational polynomial}{für unser rationales Polynom} } \frac{2x - 4}{x^2 - 2x - 8}
\end{align*}
\tr{We can then insert our coefficients into the split fraction (here}{Wir können denn die Koeffizienten in den aufgeteilten Bruch einsetzen (hier}
$\frac{a}{x - 4}\ldots$) \tr{and we can integrate normally}{und wir können normal integrieren}

177
semester2/analysis-i/parts/sequences-and-series.tex Normal file
View File

@@ -0,0 +1,177 @@
\newsection
\section{\tr{Sequences And Series}{Folgen und Reihen}}
% ────────────────────────────────────────────────────────────────────
\subsection{\tr{Limits}{Grenzwerte}}
\setcounter{all}{4}
\shortdef \tr{A sequence $\seq{a}$ is \textbf{\textit{converging}} if}{Eine Folge $\seq{a}$ heisst \textbf{\textit{konvergent}} falls}
$\exists l \in \R$ \tr{s.t.}{s.d.} $\forall \varepsilon > 0$
\tr{the set}{die Menge} $\{n \in \N^* : a_n \notin ]l - \varepsilon, l + \varepsilon[\}$
\tr{is finite. Every convergent sequence is bounded.}{endlich ist. Jede konvergente Folge ist beschränkt.}
\stepcounter{all}\shortlemma $(a_n)_{n \geq 1}$ \tr{converges to}{konvergiert gegen} $l = \limit{n}{\infty} a_n \Leftrightarrow \forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \tr{such that}{s.d.} $|a_n - l| < \varepsilon \smallhspace \forall n \geq N$
\begin{simplebox}[]{ForestGreen}
\stepcounter{all}\shorttheorem $\seq{a}$ \tr{and}{und} $\seq{b}$ \tr{converging}{konvergent}, $a = \limni a_n, b = \limni b_n$. \tr{Then}{Dann gilt}:
\vspace{-0.7pc}
\begin{multicols}{2}
\begin{itemize}
\item \textbf{(1)} $(a_n + b_n)_{n \geq 1}$ \tr{converging and}{konvergent und} $\limni (a_n + b_n) = a + b$;
\item \textbf{(2)} $(a_n \cdot b_n)$ \tr{converging and}{konvergent} $\limni (a_n \cdot b_n) = a \cdot b$;
\item \textbf{(3)} \tr{If additionally}{Falls zudem} $b_n \neq 0 \smallhspace \forall n \geq 1$ \tr{and}{and} $b \neq 0$, \tr{then}{dann gilt} $\left( a_n \div b_n \right)_{n \geq 1}$ \tr{converging and}{konvergent und} $\limni (a_n \div b_n) = a \div b$;
\item \textbf{(4)} \trIf $\exists K \geq 1$ \trwith $a_n \leq b_n \smallhspace \forall n \geq K \Rightarrow a \leq b$
\end{itemize}
\end{multicols}
\end{simplebox}
\newsectionNoPB
\subsection{\tr{Weierstrass Theorem}{Der Satz von Weierstrass}}
\shortdef $\seq{a}$ \bi{\tr{monotonically increasing (decreasing)}{monoton wachsend (fallend)}} \tr{if}{falls} $a_n \leq a_{n + 1}$ ($a_n \geq a_{n + 1}$) $\forall n \geq 1$
\compacttheorem{Weierstrass} $\seq{a}$ \tr{monotonically increasing (decreasing) and bounded from above (below) converges to}
{monoton wachsend (sinkend) und nach oben (unten) beschränkt konvergiert gegen} $\limni a_n = \sup\{a_n : n \geq 1\}$ ($\limni a_n = \inf\{a_n : n \geq 1\}$),
\tr{called supremum and infimum respectively}{genannt das Supremum und Infimum}
\setcounter{all}{6}\shortex \smallhspace $\limni \left( 1 + \frac{1}{n} \right)^n = e$
\compactlemma{Bernoulli \tr{Inequality}{Ungleichung}} $(1 + x)^n \geq 1 + n \cdot x \smallhspace \forall n \in \N, x > -1$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Limit Superior and limit inferior}{Limes Superior und Limes Inferior}}
\tr{We define for $\seq{a}$ two monotone sequences}{Für $\seq{a}$ definieren wir zwei monotone Folgen} $b_n = \inf \{ a_k : k \geq n \}$ \tr{and}{und} $c_n = \sup \{ a_k : k \geq n \}$,
\tr{then}{dann ist} $b_n \leq b_{n + 1} \smallhspace \forall n \geq 1$ \tr{and}{und} $c_{n + 1} \leq c_n \smallhspace \forall n \geq 1$, \tr{our series are bounded and converge and we have}{und beide Folgen sind beschränkt. Zudem konvergieren beide und es gilt} $\liminfni a_n := \limni b_n$ \tr{and}{und} $\limsupni a_n := \limni c_n$.
\tr{We also have}{Ausserdem gilt:} $\liminfni a_n \leq \limsupni a_n$.
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Cauchy-Criteria (Convergence Tests)}{Cauchy Kriterium (Konvergenzkriterien)}}
\shortlemma $\seq{a}$ \tr{converges if and only if it is bounded and}{konvergiert genau dann, wenn sie beschränkt ist und} $\liminfni a_n = \limsupni a_n$\\
\compacttheorem{\tr{Cauchy-Criteria}{Cauchy Kriterium}} $\seq{a}$ \tr{converging}{konvergent} $\Leftrightarrow \smallhspace \forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \tr{such that}{so dass} $|a_n - a_m| \leq \varepsilon \smallhspace \forall n, m \geq N$
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Bolzano-Weierstrass Theorem}{Der Satz von Bolzano-Weierstrass}}
\compactdef{\tr{Closed interval}{Abgeschlossenes Intervall}} \tr{Subset}{Teilmenge} $I \subseteq \R$
\tr{of form as seen below, with length}{der Form wie unten zu sehen und der Länge} $\mathcal{L}(I) = b - a$ (\tr{for}{für} \textit{(1)}) \tr{or}{oder} $\mathcal{L}(I) = +\infty$:
\vspace{-0.8pc}
\begin{multicols}{4}
\begin{enumerate}[label=\textit{(\arabic*)}]
\item $[a, b]; \smallhspace a \leq b; \smallhspace a, b \in \R$
\item $[a, +\infty[; \smallhspace a \in \R$
\item $]-\infty, a]; \smallhspace a \in \R$
\item $]-\infty, +\infty[ = \R$
\end{enumerate}
\end{multicols}
\vspace{-1.3pc}
\tr{An interval $I$ is closed}{Ein Intervall $I$ ist abgeschlossen} $\Leftrightarrow$ \tr{for every converging sequence of elements of $I$ the limit is also in $I$}{Für jede konvergente Folge aus Elementen von $I$ auch deren Grenzwerte in $I$ enthalten sind}
\setcounter{all}{5} \compacttheorem{Cauchy-Cantor} \trLet $I_1 \supseteq \ldots \supseteq I_n \supseteq I_{n + 1} \supseteq \ldots$ \tr{a sequence of closed intervals with}{eine Folge abgeschlossener Intervalle mit} $\mathcal{L}(I_i) < +\infty$.
\tr{Then}{Dann ist} $\bigcap_{n \geq 1}^{\infty} I_n \neq \emptyset$.
\tr{If additionally}{Falls zudem} $\limni \mathcal{L}(I_n) = 0$, \tr{then the set contains exactly one point.}{dann enthält die Menge genau einen Punkt.}
\shorttheorem $\R$ \tr{is not countable}{ist nicht abzählbar}
\compactdef{\tr{Subsequence of}{Teilfolge von} $\seq{a}$} $\seq{b}$ \tr{where}{wobei} $b_n = a_{l(n)}$ \tr{and}{und} $l(n) \leq l(n + 1) \smallhspace \forall n \geq 1$
\shorttheorem \textit{(Bolzano-Weierstrass)} \tr{Every bounded sequence has a convergent subsequence. Also:}{Jede beschränkte Folge besitzt eine konvergente Teilfolge. Zudem:}
$\displaystyle \liminfni a_n \leq \limni b_n \leq \limsupni a_n$
\vspace{-1pc}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Sequences in other spaces than just real numbers}{Folgen in Räumen ausserhalb der Reellen Zahlen}}
\shortdef \tr{Sequences in}{Folgen in} $\R^d$ \tr{and}{und} $\C$ \tr{are noted the same as in $\R$}{werden gleich wie in $\R$ notiert}\\
\shortdef $\seq{a}$ in $\R^d$ \tr{is \textit{converging} if}{heisst \textit{konvergent} falls} $\exists a \in \R^d$ \tr{such that}{so dass} $\forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \trwith $||a_n - a|| \leq \varepsilon \smallhspace \forall n \geq N$\\
\shorttheorem \trLet $b = (b_1, \ldots, b_n)$ (\tr{coordinates of $b$, since $b$ is a vector}{Koordinaten von $b$, da $b$ ein vektor ist}).
\tr{Then}{Dann ist} $\limni a_n = b \Leftrightarrow \limni a_{n, j} = b_j \smallhspace \forall 1 \leq j \leq d$\\
\setcounter{all}{6}\shorttheorem $\seq{a}$ \tr{converges}{konvergiert} $\Leftrightarrow$ $\seq{a}$ \tr{is a Cauchy-Sequence; Every bounded sequence has a converging subsequence.}{ist eine Cauchy-Folge; Jede beschränkte Folge hat eine konvergierende Teilfolge}
% ────────────────────────────────────────────────────────────────────
\newsectionNoPB
\subsection{\tr{Series}{Reihen}}
\compactdef{\tr{Convergence of a series}{Konvergenz}} $\ser{a}{\infty}$ \tr{converges if}{konvergiert falls} $\seq{S}$ (\tr{sequence of partial sums}{Folge von Partialsummen}) \tr{converges, i.e.}{konvergiert, d.h.} $\ser{a}{\infty} := \limni S_n$\\
\compactex{\tr{Geometric Series}{Geometrische Reihe}} \tr{Converges with limit}{Konvergiert gegen} $\frac{1}{1 - q}$, \tr{and}{und} $s_n = a_1 \cdot \frac{1 - q^n}{1 - q}$
\compactex{\tr{Harmonic Series}{Harmonische Reihe}} $\sum_{n = 1}^{\infty}\frac{1}{n}$ \tr{diverges}{divergiert}\\
\begin{simplebox}[]{ForestGreen}
\shorttheorem \tr{Let}{Seien} $\ser{a}{\infty}$ \tr{and}{und} $\ser{b}{\infty}$ \tr{be converging}{konvergent}, $\alpha \in \C$. \tr{Then}{Dann ist}:
\begin{enumerate}
\item $\displaystyle \sum_{k = 1}^{\infty} (a_k + b_k)$ \tr{converging and}{konvergent und}
$\displaystyle \sum_{k = 1}^{\infty} (a_k + b_k) = \left( \sum_{k = 1}^{\infty} a_k \right) + \left( \sum_{k = 1}^{\infty} b_k \right)$
\item $\displaystyle \sum_{k = 1}^{\infty} (\alpha \cdot a_k)$ \tr{converging and}{konvergent und}
$\displaystyle \sum_{k = 1}^{\infty} (\alpha \cdot a_k) = \alpha \cdot \left( \sum_{k = 1}^{\infty} a_k \right)$
\end{enumerate}
\end{simplebox}
\compacttheorem{\tr{Cauchy-Criteria}{Cauchy Kriterium}} \tr{A series}{Eine Reihe} $\ser{a}{\infty}$ \tr{is converging}{ist konvergent} $\Leftrightarrow \forall \varepsilon > 0 \smallhspace \exists N \geq 1$ \trwith $\left| \sum_{k = n}^{m} a_k \right| \leq \varepsilon \smallhspace \forall m \geq n \geq N$\\
\shorttheorem $\ser{a}{\infty}$ \trwith $a_k \geq 0 \smallhspace \forall k \in \N^*$ \tr{converges}{konvergiert} $\Leftrightarrow \seq{S}, S_n = \ser{a}{n}$ \tr{is bounded from above}{ist nach oben beschränkt}\\
\compactcorollary{\tr{Comparison theorem}{Vergleichssatz}} $\ser{a}{\infty}$ \tr{and}{und} $\ser{a}{\infty}$ \trwith $0 \leq a_k \leq b_k \smallhspace \forall k \geq K$ (\tr{where}{wo} $K \geq 1$), \tr{then}{dann gelten}:\\
\vspace{-0.9pc}
\begin{center}
$\ser{b}{\infty}$ \tr{converging}{konvergent} $\Longrightarrow \ser{a}{\infty}$ \tr{converging}{konvergent} \largehspace $\ser{a}{\infty}$ \tr{diverging}{divergent} $\Longrightarrow \ser{b}{\infty}$ \tr{diverging}{divergent}
\end{center}
\stepcounter{all}
\compactdef{\tr{Absolute convergence}{Absolute Konvergent}} \tr{A series for which}{Eine Reihe für welche} $\sum_{k = 1}^{\infty} |a_k|$ \tr{converges. Using the Cauchy-Criteria we get}{konvergiert. Eine Anwendung des Cauchy Kriteriums liefert}:\\
\shorttheorem \tr{A series converging absolutely is also convergent and}{Eine absolut konvergente Reihe ist auch konvergent und} $\left| \ser{a}{\infty} \right| \leq \sum_{k = 1}^{\infty} |a_k|$
\stepcounter{all}
\fhlc{Cyan}{\tr{Convergence tests}{Konvergenzkriterien}} \largehspace $\sum_{a = 0}^{\infty} \frac{1}{a^p}$ \tr{converges for}{konvergiert für} $n > 1$
\compacttheorem{Leibniz} \trLet $\seq{a}$ \tr{monotonically decreasing with}{monoton fallend mit} $a_n \geq 0 \smallhspace \forall n \geq 1$ \tr{and}{und} $\limni a_n = 0$. \tr{Then}{Dann konvergiert} $S := \sum_{k = 1}^{\infty} (-1)^{k + 1} a_k$ \tr{converges and}{und} $a_1 - a_2 \leq S \leq a_1$\\
\shade{red}{Usage} \tr{To show convergence, prove that}{Um Konvergenz zu zeigen, beweise dass} $\seq{a}$ \tr{is monotonically decreasing}{monoton fallend ist}, $a_n \geq 0$ \tr{and that the limit is $0$}{und dass der Grenzwert $0$ ist}
\setcounter{all}{14}
\compactdef{\tr{Reordering}{Umordnung}} \tr{A series}{Eine Reihe} $\ser{a'}{\infty}$ \tr{for a}{für eine} $\ser{a}{\infty}$ \tr{if there is a bijection}{falls eine Bijektion gibt} $\phi$ \tr{such that}{so dass} $a'_n = a_{\phi(n)}$\\
\stepcounter{all}
\compacttheorem{Dirichlet} \trIf $\ser{a}{\infty}$ \tr{has absolute convergence, every reordering of the series converges to the same limit.}{absolut konvergiert, so konvergiert jede Umordnung der Reihe zum selben Grenzwert.}
\compacttheorem{\tr{Ratio test}{Quotientenkriterium}} \tr{Series $s$ with}{Reihe $s$ mit} $a_n \neq 0 \smallhspace \forall n \geq 1$, $s$ \tr{has absolute convergence if}{konvergiert absolut falls}
$\displaystyle \limsupni \frac{|a_{n + 1}|}{|a_n|} < 1$. \trIf $\displaystyle \liminfni \frac{|a_{n + 1}|}{|a_n|} > 1$ \tr{it diverges. If any of the two limits are $1$, the test was inconclusive}{divergiert sie. Falls einer der Grenzwerte gleich $1$ ist, dann war der Test nicht eindeutig.}
\compacttheorem{\tr{Root test}{Wurzelkriterium}} \trIf $\displaystyle \limsupni \sqrt[n]{|a_n|} < 1$ \tr{the series converges. If the limit is larger than one, it diverges}{konvergiert die Folge. Falls der Grenzwert grösser als eins ist, divergiert sie}
\compactcorollary{\tr{Radius of convergence}{Konvergenzradius}} \tr{A power series of form}{Eine Potenzreihe der Form} $\sum_{k = 0}^{\infty} c_k z^k$ \tr{has absolute convergence for all}{konvergiert absolut für alle} $|z| < \rho$ \tr{and diverges for all}{und divergiert für alle} $|z| > \rho$.
\trLet $l = \limsupni \sqrt[k]{|c_k|}$, \tr{then}{dann ist} $\rho = \begin{cases}
+\infty & \text{\trif } l = 0 \\
\frac{1}{l} & \text{\trif } l > 0
\end{cases}$.
\tr{The \textit{radius of convergence} is then given by}{Der \textit{Konvergenzradius} ist dann definiert durch} $\rho$ \trif $\rho \neq \infty$
% ────────────────────────────────────────────────────────────────────
\fhlc{Cyan}{\tr{Double series}{Doppelreihen}}
\setcounter{all}{22} \shortdef \tr{For a double series}{Für eine Doppelreihe} $\sum_{i, j \geq 0}^{\infty} a_{ij}$, $\sum_{k = 0}^{\infty} b_k$ \tr{is a \bi{linear arrangement} if there exists a bijection}{ist eine \bi{lineare Anordnung} falls eine Bijektion} $\sigma$ \tr{s.t.}{existiert s.d.} $b_k = a_{\sigma(k)}$
\begin{simplebox}[]{ForestGreen}
\compacttheorem{Cauchy} \tr{Assume}{Wir nehmen an,} $\exists B \geq 0$ \tr{s.t.}{s.d.} $\displaystyle \sum_{i = 0}^{m} \sum_{j = 0}^{m} |a_{ij}| \leq B \smallhspace \forall m \geq 0$.
\tr{Then}{Dann gilt}: $\displaystyle S_i := \sum_{j = 0}^{\infty}a_{ij} \smallhspace \forall i \geq 0$ \tr{and}{und} $\displaystyle U_j := \sum_{i = 0}^{\infty} a_{ij} \smallhspace j \geq 0$
\tr{have absolute convergence, as well as}{konvergieren absolute, sowie} $\displaystyle \sum_{i = 0}^{\infty} S_i$ \tr{and}{und} $\displaystyle\sum_{j = 0}^{\infty} U_j$
\tr{and we have}{und es gilt}: $\displaystyle \sum_{i = 0}^{\infty} S_i = \sum_{j = 0}^{\infty} U_j$.\\
\tr{Every linear double series has absolute convergence with same limit.}{Jede lineare Anordnung konvergiert absolut mit demselben Grenzwert.}
\end{simplebox}
\compactdef{\tr{Cauchy-Product}{Cauchy Produkt}} $\displaystyle \sum_{n = 0}^{\infty} \left( \sum_{j = 0}^{n} a_{n - j}b_j \right) = a_0 b_0 + (a_0 b_1 + a_1 b_0) + (a_0 b_2 + a_1 b_1 + a_2 b_0) + \dots$ \tr{for two series}{für zwei Folgen} $\displaystyle \sum_{i = 0}^{\infty} a_i, \smallhspace \sum_{j = 0}^{\infty} b_j$
\stepcounter{all}
\shorttheorem \tr{If two series have absolute convergence, their Cauchy-Product converges and it is the terms of the two series expanded.}
{Falls zwei Reihen absolut konvergieren, so knovergiert auch ihr Cauchy Produkt und es besteht aus den ausmultiplizierten Termen der zwei Reihen.}
\shorttheorem \tr{Let $f_n$ be a sequence. We assume that:}{Sei $f_n$ eine Folge. Wir nehmen an, dass:}
\vspace{-0.7pc}
\begin{multicols}{2}
\begin{itemize}
\item $f(j) := \limni f_n(j)$ \tr{exists}{existiert} $\forall j \in \N$
\item $\exists g$ \tr{s.t.}{s.d.} $|f_n(j)| \leq g(j) \smallhspace \forall j, n \geq 0$ \tr{and}{und} $\sum_{j = 0}^{\infty} g(j)$ \tr{converges}{konvergiert}
\end{itemize}
\tr{Then}{Dann folgt} $\displaystyle\sum_{j = 0}^{\infty} f(j) = \limni \sum_{j = 0}^{\infty} f_n(j)$
\end{multicols}
\vspace{-0.7pc}
\shortcorollary \tr{For every}{Für jedes} $z \in \C$ \tr{we have}{konvergiert die Folge und es gilt} $\displaystyle \limni \left( 1 + \frac{z}{n} \right)^n = \exp(z)$ \tr{and it converges, where}{wo} $\exp(z) := 1 + z + \frac{z^2}{2!} + \frac{z^3}{3!} + \dots$

126
semester2/analysis-i/parts/table-of-derivatives.tex Normal file
View File

@@ -0,0 +1,126 @@
\newsection
\section{\tr{Table of derivatives and Antiderivatives}{Tabelle von Ableitungen und Stammfunktionen}}
\begin{multicols}{2}
\begin{tables}{lll}{\tr{Antiderivative}{Stammfunktion} & \tr{Function}{Funktion} & \tr{Derivative}{Ableitung}}
$\displaystyle \frac{x^{n + 1}}{n + 1}$ & $x^n$ & $n \cdot x^{n - 1}$ \\
$\ln|x|$ & $\displaystyle \frac{1}{x} = x^{-1}$ & $\displaystyle -x^{-2} = -\frac{1}{x^2}$ \\[0.2cm]
$\frac{2}{3} x^{\frac{3}{2}}$ & $\displaystyle \sqrt{x} = x^{\frac{1}{2}}$ & $\displaystyle \frac{1}{2 \cdot \sqrt{x}}$ \\[0.3cm]
$\frac{n}{n + 1} x^{\frac{1}{n} + 1}$ & $\displaystyle \sqrt[n]{x} = x^{\frac{1}{n}}$ & $\frac{1}{n} x^{\frac{1}{n} - 1}$ \\[0.3cm]
\hline \\[-0.2cm]
$e^x$ & $e^x$ & $e^x$ \\
$\exp(x)$ & $\exp(x)$ & $\exp(x)$ \\
$\frac{1}{a \cdot (n + 1)}(ax + b)^{n + 1}$ & $(ax + b)^n$ & $n\cdot (ax + b)^{n - 1} \cdot a$ \\
$x \cdot (\ln|x| - 1)$ & $\ln(x)$ & $\frac{1}{x} = x^{-1}$ \\
$\displaystyle \frac{1}{\ln(a)}\cdot a^x$ & $a^x$ & $a^x \cdot \ln(a)$ \\
$\frac{x}{\ln(a)} \cdot (\ln|x| - 1)$ & $\log_a|x|$ & $\displaystyle \frac{1}{x \cdot \ln(a)}$ \\[0.3cm]
\hline \\[-0.2cm]
$-\cos(x)$ & $\sin(x)$ & $\cos(x)$ \\
$\sin(x)$ & $\cos(x)$ & $-\sin(x)$ \\
$-\ln|\cos(x)|$ & $\tan(x)$ & $\displaystyle \frac{1}{\cos^2(x)}$ \\[0.3cm]
$x \cdot \arcsin(x) + \sqrt{1 - x^2}$ & $\arcsin(x)$ & $\displaystyle\frac{1}{\sqrt{1 - x^2}}$ \\
$x \cdot \arccos(x) - \sqrt{1 - x^2}$ & $\arccos(x)$ & $\displaystyle -\frac{1}{\sqrt{1 - x^2}}$ \\
$\displaystyle x \cdot \arctan(x) - \frac{\ln(x^2 + 1)}{2}$ & $\arctan(x)$ & $\displaystyle \frac{1}{x^2 + 1}$ \\[0.2cm]
$\ln|\sin(x)|$ & $\cot(x)$ & $\displaystyle -\frac{1}{\sin^2(x)}$ \\
$\cosh(x)$ & $\sinh(x)$ & $\cosh(x)$ \\
$\sinh(x)$ & $\cosh(x)$ & $\sinh(x)$ \\
$\ln|\cosh(x)|$ & $\tanh(x)$ & $\displaystyle \frac{1}{\cosh^2(x)}$ \\
& $\arcsinh(x)$ & $\frac{1}{\sqrt{1 + x^2}}$ \\
& $\arccosh(x)$ & $\frac{1}{\sqrt{x^2 - 1}}$ \\
& $\arctanh(x)$ & $\frac{1}{1 - x^2}$ \\
\end{tables}
\shade{teal}{\tr{Logarithms}{Logarithmen}}\\
\textit{(\tr{Change of base}{Basiswechsel})} $\log_a(x) = \frac{\ln(x)}{\ln(a)}$
\textit{(\tr{Powers}{Potenzen})} $\log_a(x^y) = y\log_a(x)$
\textit{(Div, Mul)} $\log_a(x \cdot (\div) y) = \log_a(x) +(-) \log_a(y)$\\
$\log_a(1) = 0 \smallhspace \forall a \in \N$
\shade{teal}{\tr{Integration by parts}{Partielle Integration}}
\tr{Should we get unavoidable cycle, where we have to integrate the same thing again, we may simply add the integral to both sides, and we thus have $2$ times the integral on the left side and then finish the integration by parts on the right hand side and in the end divide by the factor up front to get the result}
{Sollte sich ein unvermeidbarer Zyklus, wo wir immer wieder denselben Integral erhalten, bilden, können wir einfach das Integral zu beiden Seiten addieren und erhalten so $2$ mal das Integral auf der linken Seite und können dann die partielle Integration auf der rechten Seite abschliessen und schliesslich durch den Faktor auf der linken Seite dividieren, um das Resultat zu erhalten}.
\shade{teal}{\tr{Inverse hyperbolic functions}{Umkehrfunktion der Hyperbelfunktionen}}
\vspace{-0.5pc}
\begin{itemize}
\item $\arcsinh(x) = \ln \left( x + \sqrt{x^2 + 1} \right)$
\item $\arccosh(x) = \ln \left( x + \sqrt{x^2 - 1} \right)$
\item $\arctanh(x) = \frac{1}{2} \ln \left( \frac{1 + x}{1 - x} \right)$
\end{itemize}
\shade{teal}{\tr{Complement trick}{Komplement-Trick}}
$\sqrt{ax + b} - \sqrt{cx + d} = \frac{ax + b - (cx + d)}{\sqrt{ax +b} + \sqrt{cx + d}}$
\shade{teal}{\tr{Values of trigonometric functions}{Werte der trigonometrischen Funktionen}}
\begin{tables}{ccccc}{° & rad & $\sin(\xi)$ & $\cos(\xi)$ & $\tan(\xi)$}
0° & $0$ & $0$ & $1$ & $1$ \\
\hline
30° & $\frac{\pi}{6}$ & $\frac{1}{2}$ & $\frac{\sqrt{3}}{2}$ & $\frac{\sqrt{3}}{2}$ \\
\hline
45° & $\frac{\pi}{4}$ & $\frac{\sqrt{2}}{2}$ & $\frac{\sqrt{2}}{2}$ & $1$ \\
\hline
60° & $\frac{\pi}{3}$ & $\frac{\sqrt{3}}{3}$ & $\frac{1}{2}$ & $\sqrt{3}$ \\
\hline
90° & $\frac{\pi}{2}$ & $1$ & $0$ & $\varnothing$ \\
\hline
120° & $\frac{2\pi}{3}$ & $\frac{\sqrt{3}}{2}$ & $-\frac{1}{2}$ & $-\sqrt{3}$ \\
\hline
135° & $\frac{3\pi}{4}$ & $\frac{\sqrt{2}}{2}$ & $-\frac{\sqrt{2}}{2}$ & $-1$ \\
\hline
150° & $\frac{5\pi}{6}$ & $\frac{1}{2}$ & $-\frac{\sqrt{3}}{2}$ & $-\frac{\sqrt{3}}{2}$ \\
\hline
180° & $\pi$ & $0$ & $-1$ & $0$ \\
\end{tables}
\end{multicols}
\vspace{3mm}
\hrule
\begin{multicols}{2}
\shade{teal}{\tr{Trigonometrie}{Trigonometrie}}
$\cot(\xi) = \displaystyle\frac{\cos(\xi)}{\sin(\xi)}, \tan(\xi) = \frac{\sin(\xi)}{\cos(\xi)}$
$\sinh(x) := \frac{e^x - e^{-x}}{2} : \R \rightarrow \R$,
$\cosh(x) := \frac{e^x + e^{-x}}{2} : \R \rightarrow [1, \infty]$,
$\cosh(x) := \frac{\sinh(x)}{\cosh(x)} = \frac{e^x - e^{-x}}{e^x + e^{-x}} : \R \rightarrow [-1, 1]$
\begin{enumerate}
\item $\cos(x) = \cos(-x)$ \trand $\sin(-x) = -\sin(x)$
\item $\cos(\pi - x) = -\cos(x)$ \trand $\sin(\pi - x) \sin(x)$
\item $\sin(x + w) = \sin(x) \cos(w) + \cos(x) \sin(w)$
\item $\cos(x + w) = \cos(x) \cos(w) - \sin(x) \sin(w)$
\item $\cos(x)^2 + \sin(x)^2 = 1$
\item $\sin(2x) = 2 \sin(x) \cos(x)$
\item $\cos(2x) = \cos(x)^2 - \sin(x)^2$
\end{enumerate}
\end{multicols}
\hrule
\shade{teal}{\tr{Further derivatives}{Weitere Ableitungen}}
\begin{multicols}{2}
\begin{tables}{cc}{$F(x)$ & $f(x)$}
$\frac{1}{a} \ln|ax + b|$ & $\frac{1}{ax + b}$ \\
$\frac{ax}{c} - \frac{ad - bc}{c^2} \ln|cx + d|$ & $\frac{a (cx + d) - c(ax + b)}{(cx + d)^2}$ \\
$\frac{x}{2} f(x) + \frac{a^2}{2} \ln|x + f(x)|$ & $\sqrt{a^2 + x^2}$ \\
$\frac{x}{2} f(x) - \frac{a^2}{2} \arcsin\left( \frac{x}{|a|} \right)$ & $\sqrt{a^2 - x^2}$ \\
$\frac{x}{2} f(x) - \frac{a^2}{2} \ln|x + f(x)|$ & $\sqrt{x^2 - a^2}$ \\
$\ln(x + \sqrt{x^2 \pm a^2})$ & $\frac{1}{\sqrt{x^2 \pm a^2}}$\\
$\arcsin \left( \frac{x}{|a|} \right)$ & $\frac{1}{\sqrt{x^2 - a^2}}$\\
$\frac{1}{a}\arctan \left( \frac{x}{|a|} \right)$ & $\frac{1}{a^2 - x^2}$\\
\end{tables}
\begin{tables}{cc}{$F(x)$ & $f(x)$}
$-\frac{1}{a} \cos(ax + b)$ & $\sin(ax + b)$\\
$\frac{1}{a} \sin(ax + b)$ & $\cos(ax + b)$\\[1mm]
\hline
$x^x$ & $x^x \cdot (1 + \ln|x|)$\\
$(x^x)^x$ & $(x^x)^x \cdot (x + 2x\ln|x|)$\\
$x^{(x^x)}$ & $x^{(x^x)} \cdot (x^{x - 1} + \ln|x| \cdot x^x (1 + \ln|x|))$\\
\hline\\[-3mm]
$\frac{1}{2}(x - \frac{1}{2} \sin(2x))$ & $\sin(x)^2$\\[1mm]
$\frac{1}{2}(x + \frac{1}{2} \sin(2x))$ & $\cos(x)^2$\\
\end{tables}
\end{multicols}

BIN
semester2/parallel-programming-compact/pprog-compact-summary.pdf Normal file
View File

Binary file not shown.

177
semester2/parallel-programming-compact/pprog-compact-summary.tex Normal file
View File

@@ -0,0 +1,177 @@
\documentclass{article}
\newcommand{\dir}{~/projects/latex} % IMPORTANT: No trailing slashes!
\input{\dir/include.tex}
\load{recommended}
\setup{Parallel Programming}
\begin{document}
\startDocument
\usetcolorboxes
\section{Formulas}
\begin{formula}[]{Amdahl's Law}
Let $W_{ser}$ be non-parallelizable work, $W_{par}$ parallelizable work,
$T_1 = W_{ser} + W_{par}$ the processing time on one processor and $T_p$ the time taken on $p$ processors.
From this we can get an upper bound for speed-up, given by $T_p \geq W_{ser} + \frac{W_{par}}{P}$.
Then $S_p$ denotes the speed-up for $p$ processors, given by
\begin{align*}
S_p \leq \frac{W_{ser} + W_{par}}{W_{ser} + \frac{W_{par}}{P}}
\end{align*}
If now $f$ is the fraction of work that is non-parallelizable, then $W_{ser} = fT_1$ and $W_{par} = (1 - f)T_1$ and we get
\begin{align*}
S_p \leq \frac{1}{f + \frac{1 - f}{P}}
\end{align*}
\end{formula}
On the other hand, Gustafson's law is an optimistic take on Amdahl's law.
So, where Amdahl's law asks for doing the same work faster, Gustafson's law asks for more work to be done in the same time
\begin{formula}[]{Gustafson's law}
Again, $f$ denotes the non-parallelizable work, $p$ denotes the number of processors and $T_1$ denotes time for one processor, where $T_L$ is the constant time assumed
\begin{align*}
W = p(1 - f)T_{L} + fT_{L} \\
S_p \leq \frac{T_1}{T_P} = f + p(1 - f)
\end{align*}
\end{formula}
\begin{itemize}
\item $t_{max} :=$ Time for longest stage in pipeline
\item \fancydef{Latency} $L$ Sum of all stages in pipeline
\item \fancydef{Balanced Pipeline} All stages take equally long
\item \fancydef{Throughput} $\frac{1}{t_{max}}$
\end{itemize}
Time for $i$ iterations $i \cdot t_{max} + (L - t_{max}) = (i - 1) t_{max} + L$
\newsection
\section{Java BS}
\verb|ExecutorService| is not suited to Divide \& Conquer (or any other non-flat structure), we can give it \verb|Callable| (need to implement \verb|call| method, which returns) or a \verb|Runnable| (need to implement \verb|run| method, doesn't return).
We can create an \verb|ExecutorService| using \verb|java.util.concurrent.Executors.newFixedThreadPool(int threads)| and we can add tasks using \verb|ex.submit(Task task)|
When submitting \verb|Callable| to \verb|ExecutorService|, a \verb|Future| is returned (= Promise).
For Divide \& Conquer, use \verb|java.util.concurrent.ForkJoinPool|, to which we submit \verb|RecursiveTask| (returns) or \verb|RecursiveAction| (doesn't return).
\verb|RecursiveTask| and \verb|RecursiveAction| support the following methods:
\begin{itemize}
\item \verb|compute()| runs the new task in the current thread
\item \verb|fork()| runs the task in a new thread
\item \verb|join()| waits for the task to finish (like the \verb|join()| of Threads)
\end{itemize}
To start execution, run \verb|pool.invoke(Task task)| on the \verb|ForkJoinPool|
\newsection
\section{Locking}
For synchronization, we also have \verb|volatile|, which does guarantee that all threads see the value immediately (as the variable is written back to memory immediately and not stored in each Thread's internal cache) and it enforces memory consistency, i.e. instructions are not reordered around such variables.
It however is not atomic and does not guarantee that two consequent accesses (e.g. read and increment) could be reordered / interfered with, just as if we used a normal variable
A sequentially consistent memory model enforces that actions of threads become visible in program order.
If we need atomic operations, they are available in \verb|java.util.concurrent.atomic|, e.g. \verb|AtomicInteger|.
These variables provide the following methods: \verb|get()|, \verb|set(E newValue)|, \verb|compareAndSet(E expect, E newValue)| (CAS operation, which sets if the value of the variable is equal to \verb|expect|) and \verb|getAndSet(E newValue)| (Updates \& returns old value).
We also have the TAS (Test And Set) operation, which is, like CAS, provided by Hardware and allows us to test if value is $0$ to set it to $1$ and get a return of \verb|true| if it was zero.
For TAS based locks, use exponential back-off (wait exponentially longer between accesses)
\subsection{Monitors}
Inside \verb|synchronized| blocks, we can use \verb|wait()| (releases lock and waits to be woken again), \verb|notify()| (wakes up \textit{random} Thread) and \verb|notifyAll()| (wakes up all threads). The \verb|wait()| should \textit{always} be inside a for loop, as the condition could be incorrect when the thread is woken.
If we however want to manually acquire locks with the Java Lock interface, we need to use \verb|Conditions|, which we can obtain using \verb|lock.newCondition()|.
They offer \verb|await()|, \verb|signal()| and \verb|signalAll()|, which all work similar to their \verb|synchronized| counterparts.
\shade{red}{IMPORTANT} Always use \verb|try-catch-finally| blocks around locked elements to ensure lock is released again to avoid deadlocks
Finally, some concepts for locking:
\begin{itemize}
\item Coarse grained locking: One lock for entire structure, very safe, but very slow
\item Fine grained locking: In lists, every element has lock, lock previous and current element, to move through list, lock next, then release previous, move to next. For a list, we then need to lock (\textit{number of elements}) $+ 1$ (for head) for traversal and for insert at the end a further one time for tail.
\item Optimistic synchronization: Traverse list without locking, then lock when updating / reading. Much faster, but need to traverse twice and not starvation free. We just need to lock the predecessor and tail to insert and for a contains operation, predecessor and current node.
\end{itemize}
Monitor locks are reentrant locks in Java.
A \verb|static synchronized| method locks the whole class, not just the instance of it.
\subsection{Lock-free programming}
\vspace{-0.7pc}
\begin{multicols}{2}
\begin{itemize}
\item Wait-Free $\Rightarrow$ Lock-free
\item Wait-Free $\Rightarrow$ Starvation-free
\item Lock-Free $\Rightarrow$ Deadlock-free
\item Starvation-Free $\Rightarrow$ Deadlock-free
\item Starvation-Free $\Rightarrow$ Livelock-free
\item Deadlock-Free AND fair $\Rightarrow$ Starvation-free
\end{itemize}
\end{multicols}
To program lock-free, use hardware concurrency features like TAS \& CAS
\subsection{ABA-Problem}
Occurs if a thread fails to recognize that a variable's value was \textit{temporarily} changed (and the changed back to the original), thus not noticing state change
\textbf{\textit{Solutions}}: DCAS (Double Compare And Set, not available on most platforms), GC (Garbage Collection, very slow), Pointer-Tagging (Only delays problem, but practical), Hazard Pointers (Before reading, pointer marked as hazard), Transactional Memory
\newsection
\section{Consistency / Linearisability}
Between Invocation and Response states, method in pending state.
\textbf{\textit{Linearization}} Each method should appear to take effect \textit{immediately}.
When deciding whether or not something is linearizable, decide if there is an order of commit such that the desired effects happen.
A commit can happen at any point during a function's life-cycle and the same applies to a read / dequeue, etc.
\textbf{\textit{History}} Complete sequence of invocations \& responses.
History linearisable if it can be extended to another one by adding $\geq 0$ responses that took effect, or discard $\geq 0$ pending invocations that have not taken effect (yet).
It is sequentially consistent, if we can add $\geq 0$ pending responses, or same as before (Linearizability implies Sequential Consistency)
We can check if a history is linearizable, if we can find linearization points such that the responses are correct.
It is sequentially consistent, if we can find a sequential execution order (non-interleaved calls) such that the history is valid.
We are allowed to move operations of other threads in between to make the result correct, but we are not allowed to change the order of operations in a thread.
We may only reorder if the operations are not overlapping.
\begin{multicols}{2}
The below history:
\verb|A: r.write(2)|\\
\verb|A: r:void|\\
\verb|A: r.write(1)|\\
\verb|A: r:void|\\
\verb|B: r.read()|\\
\verb|B: r:2|\\
Can be rewritten as:
\verb|A: r.write(2)|\\
\verb|A: r:void|\\
\verb|B: r.read()|\\
\verb|B: r:2|\\
\verb|A: r.write(1)|\\
\verb|A: r:void|\\
\end{multicols}
And is thus sequentially consistent. (The history is thus also sequential, as actions between multiple threads are not interleaved)
A more detailed explanation:
\begin{itemize}
\item For Linearisability, we need to respect the program order and the real-time ordering of the method calls (i.e. we can't move them around)
\item For sequential consistency, we only need operations done by a single thread to respect program order. Ordering across threads not needed.
\end{itemize}
\section{Consensus}
$n$ threads should agree on picking one element of e.g. a list. Consensus number is largest number of threads such that the consensus problem can be solved.
\shortex \smallhspace Atomic reg: 1; CompareAndSwap: $\infty$; wait-free FIFO queue: 2; TAS \& getAndSet: 2
\section{Transactional Memory}
Atomic by definition, programmer defines atomic code sections. Issue: Still not standardized / WIP
\section{Message Passing Interface}
Used to send messages between threads. Other threads can choose when to handle, if at all
\end{document}