AGI Fundamentals: Week 0

I would like to thank UCLA for finally granting me unrestricted MathType access, my matcha latte for granting me the energy to learn Overleaf this afternoon, and AGI safety fundamentals for allowing me into this course. Here are my notes for this week. 

Given a linear regression equation written in the form: 

is random error term, is y-intercept and is the slope, where we aim to find parameters will minimize error in model predictions. 

 

The cost/loss function measures the degree of inaccuracy of model predictions. 

Here, X, Y and n are given. The function is , with beta parameters. We begin computing gradient descent by computing partial derivatives. 

Specifically, calculates change in total loss wrt to change in , . Since we want to minimize total loss, so if is negative, then we want to increase . 

Bias is the amount of error generated from explaining real-world data with a simple model, whereas variance is how much change can be yielded from "too much" adherence to testing data (or so that I gathered). To reduce bias and to reduce variance, we use a regularization term. 

Regularization term: 

 

where it reduces the chance of one feature overriding the loss. For example, a large beta coefficient can be reduced by taking the sum of its squares and then multiplied by lambda. The lambda coefficient is a hyperparameter, which is often tuned by interns, to prevent model overfitting. 

Putting it all together: 

This is one concept of supervised learning. 

______________________________________________________

1. In a CNN, weights are not specified, instead, they are learned via optimization. Optimization uses either an objective function or a loss function, dependent on dataset characteristics. The most common optimization algorithm is gradient descent, updating step changes in the weight based on the direction of the gradient wrt the objective function.

2. Fully connected networks are not used as widely as convolutional networks, recurrent networks, and transformers.

3. Reinforcement learning requires designing a reward function in the training setup. Meeting the data-insufficiency bottleneck is imminent, so DL tends to generalize **AND** conduct transfer learning. Self-supervised learning is tasked with but not limited to: 1. Image generation 2. language modeling 3. behavioral cloning (a good read) Self-supervised learning techniques are: 1. Autoregression 2. GANs 3. Diffusion modeling Supervised learning is tasked with but not limited to: 1. Regression 2. Classification 3. Reward modeling Supervised learning techniques are: 1. Support vectors machines 2. Gaussian processes Reinforcement learning is tasked with not limited to: 1. Exploration 2. Temporal credit assignment 3. Multiagent credit assignment Reinforcement learning techniques are: 1. Q-learning 2. Policy gradients 3. Intrinsic motivation??? [what is this?]

-More to follow-

https://docs.google.com/document/d/19AIWZKPqRoKYul2Pj_rF7wnT_3zLpBTebSiR-1ogtYE/edit#heading=h.313oaiblxob8