Base optimizer and mesa-optimizer comparison

-

 


Here I will be documenting my understanding of what a base optimizer and a mesa-optimizer are, and putting them in comparison. 

The Github link is here: https://github.com/evhub/mesa-optimization/blob/master/post1.md

- The base optimizer is a gradient descent process to create a model, and the model is designed to accomplish some specific task
- The mesa optimizer produces a base optimizer that is itself good at optimizing systems 
- The mesa-optimizer is different from a subsystem because it is an optimization process, not an agent. 
- Mesa-optimization happens when the base optimizer can find a model that exists for the purpose of optimizing another 
- Within every optimizer, there are objectives
- Unlike the base objective, the mesa-objective is not specified directly by the programmers
- Mesa-optimization sometimes leads to mismatch of base and mesa-objectives. This is called misalignment. 
- We can call a model generated by the base optimizer as a learned algorithm

Now we will clarify with an example: 

Base optimizer is Evolution
Base objective is caring about certain favorable alleles' frequency in the population
Mesa-optimizers are human brains 
Mesa-objective is organisms’ behavior—behavior that is not necessarily aligned with evolution. For example, choosing to not have children and thereby decreasing the exchange of favorable alleles' frequency. 

Comments

Post a Comment

Popular posts from this blog

ArtificiaI General Intelligence Safety Fundamentals Notes

-
Image
AGISF - Week 1 - Artificial General Intelligence Reinforcement Learning Foundation models : models trained on broad data that can be adapted to a wide range of downstream tasks. Much of foundation models are enabled by transfer learning and scale. In which scale required:  1. GPU throughput and memory capacity, both increased 10x over the last four years.  2. Transformer architecture  3. Availability of training data  Homogenization: all the defects of the foundation model are inherited by all the adapted models downstream.  Four Background Claims by MIRI, 2015 CE  This post aims to illustrate MIRI's mission objective.  1. Interactions between disparate modules is the general intelligence that we can not replicate in code, yet. MIRI suggests to take heed.  2. Given that the Church-Turing thesis hold, then computers can replicate the functional input/output behavior of brains as physical systems--- they might supercede soon.  3. Intellectual advantage will create incentive to
Read >>

A year, introspectively summarized

-
Image
               A year, introspectively summarized      A primal and gravitational feeling is the feeling of belonging--- though the whole sense of "purpose" is artificial, defined deliberately by people who want others to emulate them. Emulation stunts creativity, just as the proclivity to belong stunts agency. I am delighted by the platform that I am given to practice my agentic self, now and continuously onward.       There will be a time when t he basis of your understanding, you learn, was fabricated---even some of your strongest beliefs need reinterpretation, the very framework under which you operate proves inflexible and restrictive. Perhaps the best remediation strategy then is desensitization of the action-oriented part of your mind. You must compromise scaling for precision.       In January, I wrangled with partial differential equations and AI Safety fundamentals course with BlueDot. In February, I wrapped up two of my internships. In March, I explored much of Los
Read >>

Vicissitude

-
Image
Vicissitude Did the heavens hang higher to aggrandize your vicissitude? From the primal vines you serpentine ushering empires and visions sublime   If thunder be your heartbeat and determination then let lightning linger in your mind from deprivation unbounded by ecclesiastical measures, you drape your golden body over boundless wonders  The dreamers drown in idolization and ideals but sharper is the blade of guilt the thirst for ravishing power and the departure from righteousness your celestial eyes close and halt On the descent, since the fall the leathery wings enshroud your undying faith In your shadow woke Wicked Mephisto  down is the plunder into ecstacy  but who can share your burden of villainy?
Read >>