eth-janishutz/eth-summaries
1
0
Fork 0
mirror of https://github.com/janishutz/eth-summaries.git synced 2026-09-10 19:15:25 +02:00
Code Issues Packages Projects Releases Wiki Activity

[AMR] Fix errors pointed out to me

Browse Source
This commit is contained in:
janishutz
2026-08-09 08:30:17 +02:00
parent cc5f111813
commit a8b3bd7b20
24 changed files with 50 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files
electives/amr/parts/05_planning-control/02_learning-to-act/02_reinforcement-learning.tex
+2 -2
View File
@@ -2,9 +2,9 @@
\bi{Passive} \textit{Direct Evaluation} Act according to policy $\vec{\pi}$, store sum of discounted rewards, average them. (But too simple)
\textit{Sample-Based} Use $V_{i + 1}^{\vec{\pi}}(\vec{x}) = Q(\vec{x}, \vec{\pi}(\vec{x}))$ w/ $V_0^\pi(\vec{x}) = 0$.
We need state trans. model, instead $\tilde{R}_j$ (approx. prob. w/ statistics) and thus
We need state transition model for that (we don't have that), instead $\tilde{R}_j$ (approx. probability w/ statistics) and thus
\[
V_{i + 1}^{\vec{x}}(\vec{x}) = \frac{1}{N} \sum_{j = 1}^{N} \tilde{R}_j(\vec{x}, \vec{\pi}(\vec{x}), \vec{x}_+) + \gamma V_i^{\vec{\pi}}(\vec{x}_+)
V_{i + 1}^{\vec{\pi}}(\vec{x}) = \frac{1}{N} \sum_{j = 1}^{N} \left( \tilde{R}_j(\vec{x}, \vec{\pi}(\vec{x}), \vec{x}_+) + \gamma V_i^{\vec{\pi}}(\vec{x}_+) \right)
\]
\bi{Active} Find optimal policy $\vec{\pi}$ instead of state values $V(\vec{x})$.