Preference Model Open-Sources Karotte RL Framework Used in 1M Runs

AI13 days ago

The reinforcement learning software helps artificial intelligence laboratories construct training environments that are resistant to reward hacking. Preference Model developed the Karotte framework over the last year for use in frontier labs, where it was tested across more than 1M evaluation runs and subject to red-teaming to identify security vulnerabilities.

Venture capital firm a16z announced an investment in Preference Model this week to support the development of infrastructure for capable and aligned models. The company focuses on AI research and machine learning engineering by utilizing automation to find where models are weak and generating tasks specifically designed to target those performance gaps.

Mapped to your portfolio

Possible scenarios

42%Karotte Becomes Standard+0.2%
26%Open RL Fragments+0.0%
19%Commercial Packaging Accelerates+0.1%

Each figure is the modeled move of the S&P 500 if the outcome occurs. For information only. Not investment advice.

Looking for more context or analysis?

Tilt chat provides reliable, tailored insights, alerts, and automations.

Preference Model Open-Sources Karotte RL Framework Used in 1M Runs

In this story

Preference Model Andreessen Horowitz

Portfolio

SPY
State Street SPDR S&P 500 ETF Trust
100%

Portfolio

SPY
State Street SPDR S&P 500 ETF Trust
100%

Mapped to your portfolio

Possible scenarios

42%Karotte Becomes Standard+0.2%
26%Open RL Fragments+0.0%
19%Commercial Packaging Accelerates+0.1%

Each figure is the modeled move of the S&P 500 if the outcome occurs. For information only. Not investment advice.