Suppose I give it pairs of strings and ask it to output 1 if the the number of 1s in the string is even and zero if it's odd.
0 -> 0
1 -> 1
11 -> 0
101 -> 0
10101001 -> 0
111000101110 -> 1
How well does it do on this task? What if we finetune it on sample data?
One interesting update: we've been increasingly unlocking GPT-3 solutions by rewriting them as multi-step procedures. So parity might be doable by somewhat cheating and writing out a series of steps for computing the parity for each example: https://twitter.com/bucketofkets/status/1285100951271952384 https://twitter.com/Malcolm_Ocean/status/1285099206781341696