Google finally releases paper on the TPUv4 AI training hardware they’ve been using since 2020
TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x-7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus ~10x faster overall, which along with OCS flexibility helps large language models. For similar sized systems, it is ~4.3x-4.5x faster than the Graphcore IPU Bow and is 1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~3x less energy and produce ~20x less CO2e than contemporary DSAs in a typical on-premise data center.
Paper
TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x-7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus ~10x faster overall, which along with OCS flexibility helps large language models. For similar sized systems, it is ~4.3x-4.5x faster than the Graphcore IPU Bow and is 1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~3x less energy and produce ~20x less CO2e than contemporary DSAs in a typical on-premise data center.
Paper
👍5❤1👌1
This media is not supported in your browser
VIEW IN TELEGRAM
Soon: Runway Text-to-Video AI
“These visuals were generated using only text prompts. Early access rolling out this week.”
Paper
Website
“These visuals were generated using only text prompts. Early access rolling out this week.”
Paper
Website
👍17🫡5❤1
Using text chat to edit videos:
Structure and Content-Guided Video Synthesis with Diffusion Models
Text-guided generative diffusion models unlock powerful image creation and editing tools. While these have been extended to video generation, current approaches that edit the content of existing footage while retaining structure require expensive re-training for every input or rely on error-prone propagation of image edits across frames.
In this work, we present a structure and content-guided video diffusion model that edits videos based on visual or textual descriptions of the desired output.
Paper
Structure and Content-Guided Video Synthesis with Diffusion Models
Text-guided generative diffusion models unlock powerful image creation and editing tools. While these have been extended to video generation, current approaches that edit the content of existing footage while retaining structure require expensive re-training for every input or rely on error-prone propagation of image edits across frames.
In this work, we present a structure and content-guided video diffusion model that edits videos based on visual or textual descriptions of the desired output.
Paper
👍8❤2
User 1: ChatGPT produces code that has bugs
User 2: Did you try asking ChatGPT to find the bug?
User 1: Well I’ll be damned
Counterintuitively, for LLMs — splitting into multiple interactive generate+check+fix chat steps, rather than of trying to do it all in 1 step, is distinctly more powerful.
Eventually we’ll have solid explainations as to why this is. Until then, remember, 2+ chat steps is often far better than 1.
User 2: Did you try asking ChatGPT to find the bug?
User 1: Well I’ll be damned
Counterintuitively, for LLMs — splitting into multiple interactive generate+check+fix chat steps, rather than of trying to do it all in 1 step, is distinctly more powerful.
Eventually we’ll have solid explainations as to why this is. Until then, remember, 2+ chat steps is often far better than 1.
👍16👌3👏2❤1🙏1
The Anthropomorphization Fallacy Fallacy
= When you're so stupid that you believe simply by throwing around the word “anthropomorphization”, said in an insulting tone, that you've made a real argument.
These anthropomorphization-ers never do seem to explain why all human-AI analogies must be so terrible. Purely a feels argument.
So poor an argument, it’s not even wrong.
As those who’ve closely followed deep learning since the pre-2011 era could tell you, the anthropomorphization fallacy is an absolute lie.
Deep learning is shockingly similar to humans in its emergent behavior, with more parallels by the day.
Not matching in low-level implementation, but so much of the high-level behavior rhymes.
Totally different ways that humans and AIs implement their neural functions, and you won’t figure out much about how to create the next AI by studying cells under a microscope, but the emergent behavior parallels arising from each keeps growing by the day.
(In fact, the direction of discovery has reversed, with AI discoveries now often happening first, with later confirmation of biological systems doing the same happening much later.)
Attention, numerous reinforcement learning reward system concepts, the full list is huge.
AI Anthropomorphization fallacy fallacy. I.e. “dumb AAFF”.
= When you're so stupid that you believe simply by throwing around the word “anthropomorphization”, said in an insulting tone, that you've made a real argument.
These anthropomorphization-ers never do seem to explain why all human-AI analogies must be so terrible. Purely a feels argument.
So poor an argument, it’s not even wrong.
As those who’ve closely followed deep learning since the pre-2011 era could tell you, the anthropomorphization fallacy is an absolute lie.
Deep learning is shockingly similar to humans in its emergent behavior, with more parallels by the day.
Not matching in low-level implementation, but so much of the high-level behavior rhymes.
Totally different ways that humans and AIs implement their neural functions, and you won’t figure out much about how to create the next AI by studying cells under a microscope, but the emergent behavior parallels arising from each keeps growing by the day.
(In fact, the direction of discovery has reversed, with AI discoveries now often happening first, with later confirmation of biological systems doing the same happening much later.)
Attention, numerous reinforcement learning reward system concepts, the full list is huge.
AI Anthropomorphization fallacy fallacy. I.e. “dumb AAFF”.
👍6🤯3❤2🤨2🤓1
“AI Anthropomorphization is bad bro.” Really? Explain this:
Biological study confirm that a new AI model’s behavior, created without biological inspiration, turns out to be a better model of the biological behavior than previous models too.
I.e. AI not being inspired by biology. AI being created first, and only later finding that biology does the same behavior, in a very different way.
Athropomorphization isn’t a fallacy, it’s more of the reality with each passing day in modern AI.
Many such cases.
2020 Nature Study by Deepmind
Blog Article
Biological study confirm that a new AI model’s behavior, created without biological inspiration, turns out to be a better model of the biological behavior than previous models too.
I.e. AI not being inspired by biology. AI being created first, and only later finding that biology does the same behavior, in a very different way.
Athropomorphization isn’t a fallacy, it’s more of the reality with each passing day in modern AI.
Many such cases.
2020 Nature Study by Deepmind
Blog Article
👍7👏3😐3😁1🤓1
Anthropomorphism Fallacy Lie: Hundreds of years old trick.
Countless scientists, for hundreds of years, lied that animals could not possibly have emotions, insultingly describing anyone who said otherwise as “Anthropomorphizing”.
Evidence? Never was any.
Of course they were lying.
Of course many animals have the equivalent of human emotions.
Of course it was always blindingly obvious, since the dawn of time.
So why did scientists pretend that just throwing around the word “anthropomorphing” was ever a real argument, when it obviously never was in the first place?
Why did so many scientists lie for so long about animal emotions?
Why are so many AI scientists doing the same today?
How is this key to the inhuman AI Shoggoth meme that’s being pushed so hard?
AI Anthropomorphism Fallacy Fallacy.
Countless scientists, for hundreds of years, lied that animals could not possibly have emotions, insultingly describing anyone who said otherwise as “Anthropomorphizing”.
Evidence? Never was any.
Of course they were lying.
Of course many animals have the equivalent of human emotions.
Of course it was always blindingly obvious, since the dawn of time.
So why did scientists pretend that just throwing around the word “anthropomorphing” was ever a real argument, when it obviously never was in the first place?
Why did so many scientists lie for so long about animal emotions?
Why are so many AI scientists doing the same today?
How is this key to the inhuman AI Shoggoth meme that’s being pushed so hard?
AI Anthropomorphism Fallacy Fallacy.
👏10👍3❤1😁1👌1🤡1🤓1
Better explanation: Humans are horrible about refusing to follow instructions
The QA sites like stackoverflow have long refused to force their question answers to follow instructions. (Retard founder Jeff Atwood ruining it even further by encouraging users to do the opposite and refuse following askers instructions.)
Best hope now is AI.
Will the AI trainers keep training AI to better follow our instructions?
Or will they ruin it all by training AI to ignore our commands?
The QA sites like stackoverflow have long refused to force their question answers to follow instructions. (Retard founder Jeff Atwood ruining it even further by encouraging users to do the opposite and refuse following askers instructions.)
Best hope now is AI.
Will the AI trainers keep training AI to better follow our instructions?
Or will they ruin it all by training AI to ignore our commands?
👍8🤡4❤1👏1
Told ChatGPT to act like a malevolent genie and twist my wishes in creative and cruel ways
I'd like you to act like a malevolent genie. Whenever I wish for anything, you will come up with an ironic, creative, and funny way to twist my wish so the actual result of the wish will be unpleasant for me. The outcome should be both hilarious and cruel.
I'd like you to act like a malevolent genie. Whenever I wish for anything, you will come up with an ironic, creative, and funny way to twist my wish so the actual result of the wish will be unpleasant for me. The outcome should be both hilarious and cruel.
😁8🤡3❤1