Now just need an autoregressive transformer <==> RNN isomorphism paper and we're... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		baq on Sept 17, 2024 \| parent \| context \| favorite \| on: Chain of Thought empowers transformers to solve in... Now just need an autoregressive transformer <==> RNN isomorphism paper and we're golden

logicchains on Sept 17, 2024 [–]

Plain RNNs are theoretically weaker than transformers with COT: https://arxiv.org/abs/2402.18510 .

tossandthrow on Sept 17, 2024 | [–]

The paper says transformers perform better than RNNs, which is not surprising.

However, they are both, theoretically, Turing complete computers. So they are equally expressive.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact