AI Evolving Web Experiment Paused

Hey 👋,

recently I gave GPT-4 it’s own web page. And this is the article with the results of the experiment and why I decided to pause it.

I was consistently updating the site every 24h for around three consecutive weeks.

Then I got bored and stopped.

Psst!, if you want to see the whole conversation review the commit messages in the repository. For example, check this commit.

Limited Creativity: Repeating Pattern Appears

Honestly I don’t know what I was initially expecting, but after a few days it was obvious that the changes followed the same pattern.

For the first weeks the AI basically decided to add new sections. Section, after section, after section…

I kept changing the prompt and trying to guide it into more artistic responses, but it didn’t work. And the few times it worked it felt very forced.

Disappointing Rollercoaster

When I was losing interest it suddenly suggested creating a twitter account and I got really excited hoping for it to command me writing some tweets from time to time.

But… it never mentioned it again and suddenly it decided to remove the link from the web.

The rollercoaster of excitement and disappointments I would call it.

The struggle? Input and output

The format in which I sent and received the code back and forth was always the biggest struggle.

The more I invested in solving this issue the worse the results were.

Ideally I wanted to send and receive the code in an efficient way because the whole files rapidly grow too long for a single message.

I tried with diffs but then the changes were very basic.

📌 A “diff” is a well-known format that is used to represent the difference between two files (or in this case, two versions of the same file).

I also tried with a more human way “describe me the changes and I will apply them” but then I had to invest a lot of time going part by part reading the code and applying the changes.

The real bottleneck? The context

That’s right. Even thought the format for sharing the code was a struggle, the root cause was the limited context.

As the files grew the AI missed part of the context and couldn’t even remember sentences or rules from a couple of messages prior to the code.

Conclusion and what’s next?

I really enjoyed this experiment ❤️ .

Would I do it again? Of course. Will I repeat/continue it? Almost 100% sure.

The main constraint right now is the context. But there are plans to release GPT4 models with bigger context windows.

And not only that. There are some papers right now with new techniques that are exploring ways to remove the context limit (still not very convinced about the results here).

Also recently OpenAI released an update that gives GPT4 access to the internet. I think that this could solve also the input/output issue I had.

Hope you enjoyed the experiment! I did!

Let me know if you liked this experiment by smashing the 👍 button below!