Vendôme

In reinforcement learning, an agent and environment interaction’s total components can all be described as functions— some to be mapped onto others. Imagine an agent making a decision in a virtual environment. Such a decision is made by an algorithm that falls into this form: and below are the steps to get to this form At each step t the agent:  Receives observation Executes action The environment:  Receives action Emits observation Where Gt stands for total return, summing over all reward Rt. Every reward at point t is a scalar feedback signal. The agent’s goal is to maximize cumulative reward.  This part was a little difficult to understand for me. The expected cumulative reward conditioned on state s, is given by the function v(s). That’s the first two equations. We then define Gt recursively such that the base case is Rt+1 and the recursive case is v(St+1). A mapping from states to actions is called a policy. This is when certain action At is selected to, for exa...

I would like to thank UCLA for finally granting me unrestricted MathType access, my matcha latte for granting me the energy to learn Overleaf this afternoon, and AGI safety fundamentals for allowing me into this course. Here are my notes for this week.  Given a linear regression equation written in the form:  is random error term, is y-intercept and is the slope, where we aim to find parameters will minimize error in model predictions.    The cost/loss function measures the degree of inaccuracy of model predictions.  Here, X, Y and n are given. The function is , with beta parameters. We begin computing gradient descent by computing partial derivatives.  Specifically, calculates change in total loss wrt to change in , . Since we want to minimize total loss, so if is negative, then we want to increase .  Bias is the amount of error generated from explaining real-world data with a simple model, whereas variance is how much change can be yielded from "to...

"Who is John Galt?" Despite popular sentiments about Ayn Rand--- my English teacher's relentless efforts to stop me from advocating the objectivist philosophy and my well-read counterparts advertising equally riveting schools of thoughts--- I enjoyed Anthem, The Fountainhead, Atlas Shrugged, and the Romantic Manifesto.  It is quite contrarian: the Russian-American novelist intended this in her characters.  Dagny Taggart's journey from D'anconia, to Rearden, to Galt can easily be summed up in this quote: "it contained her pride in herself and that it should be she whom he had chosen as his mirror." Earth-movers, labor generators, progress-drivers sit at the center of Atlas Shrugged's plot. They cherish ethical passivity of self-interest. It urges a maximization of means of production from every member of society.  It is easy to idolize the railroad operating president, when she is laced in a description as gorgeous as brocade--- "she threw her ca...

Technological temptations are instances of an actor (foreign powers, corporations, international policymakers, etc.) facing a decision about pursuing a particular technology, with substantial incentives to continue research and investment, to capture substantial value. We are particularly interested in cases in which the direct benefits substantially outweigh the direct costs, but the overall risks (or perceived risks) outweigh the overall benefits, or those in which a technology was very tempting, but an actor chose not to pursue it, whether there were external costs or not.  Chemical and biological weapons, human reproductive experimentation, nuclear power are all examples of technologies that we have tested yet left obsolete in the more recent decade. In this case study, we hope to unearth the underlying causes for these temptations, extraneous reasons (if present) for not adopting AI technology. We will particularly have a deeper-dives into AI Winters and temptations about neu...