AI300

Ilya Sutskever / Founder of SSI
1261.
The transformer is well named, as it transformed everything
LLM score 10 · over 3 years ago
Ilya Sutskever / Founder of SSI
1262.
The S of our collective LSTM is quite short
LLM score 70 · over 3 years ago
Ilya Sutskever / Founder of SSI
1263.
Would be cool to implement dasher using today’s LMs:›
LLM score 20 · over 3 years ago
Ilya Sutskever / Founder of SSI
1264.
I find it interesting that evolution applies to all “structures”, not just living things: ideologies, organizations, management structures, technologies, etc
LLM score 40 · over 3 years ago
Ilya Sutskever / Founder of SSI
1265.
Deep learning is based on the audacious conjecture that biological neurons and artificial neurons are not that different.›
LLM score 80 · over 3 years ago
Ilya Sutskever / Founder of SSI
1266.
I wonder which insights from developmental psychology will apply to our future NNs
LLM score 70 · over 3 years ago
Ilya Sutskever / Founder of SSI
1267.
“prompting” is a transitory term that’s relevant only thanks to flaws in our models
LLM score 80 · over 3 years ago
Ilya Sutskever / Founder of SSI
1268.
it's not the worst for an AGI effort to contribute to a future plurality of AGIs all of whom love humanity
LLM score 10 · over 3 years ago
Ilya Sutskever / Founder of SSI
1269.
Human culture is critical civilization-enabling infrastructure.›
LLM score 20 · over 3 years ago
Ilya Sutskever / Founder of SSI
1270.
I find it funny when people say that evolution is “stupid”, when you consider that it’s by far the most intelligent force that can exist in our universe
LLM score 10 · over 3 years ago
Ilya Sutskever / Founder of SSI
1271.
If the transition from pre to post deep learning is any guide, it is essentially impossible to convince almost anyone of the feasibility and impact of truly transformative tech, regardless of how correct ones arguments are
LLM score 75 · over 3 years ago
Ilya Sutskever / Founder of SSI
1272.
a near term effect of human level AGI could be not unlike that of massive scale very high skilled immigration ›
LLM score 85 · over 3 years ago
Ilya Sutskever / Founder of SSI
1273.
working towards AGI while not feeling the AGI is the real risk
LLM score 20 · over 3 years ago
Ilya Sutskever / Founder of SSI
1274.
People who understand math often think that simple logical reasoning applies to all areas of life.›
LLM score 30 · over 3 years ago
Ilya Sutskever / Founder of SSI
1275.
a big mistake in old school ML was the belief that the logarithm is bounded from above
LLM score 75 · over 3 years ago
Zihang Dai / xAI Cofounder
1276.
In NLP, the O(TD^2) linear projections in Transformer often cost more FLOPs than the O(T^2D) attention as commonly D > T.›
LLM score 95 · over 5 years ago
Alec Radford / OpenAI Researcher
1277.
By the way - I think a valid (if extreme) take on GPT-2 is "lol you need 10,000x the data, 1 billion parameters, and a supercomputer to get current DL models to generalize to Penn Treebank."
LLM score 82 · about 7 years ago
Alec Radford / OpenAI Researcher
1278.
The DL CV community is having a "oh wait, bags of local features are a really strong baseline for classification" moment with the BagNet paper.›
LLM score 92 · about 7 years ago
Alec Radford / OpenAI Researcher
1279.
Nice discussion of the progress in NLU that's happening with BERT, OpenAI GPT, ULMFiT, ELMo, and more covered by @CadeMetz in the @nytimes I'm super excited to see how far this line of research will be able to get in the next few years!›
LLM score 8 · over 7 years ago
Alec Radford / OpenAI Researcher
1280.
Been meaning to check this - thanks @Thom_Wolf ! Random speculation: the bit of weirdness going on in BERT's position embeddings compared to GPT is due to the sentence similarity task.›
LLM score 82 · over 7 years ago