I just got back from Australia, so I’m not dead but the time zone still makes me feel a bit like it.
I’ve used the Large Language Models (LLM), GPT3/4 a fair bit in a professional capacity. My work is more on the finance side, with quantitative models, but it’s all patterns in the end I guess.
The biggest question this stuff has raised for me in the last couple of years is that is this really that clever or is it more that people are just simpler than we thought? I know it sounds facetious (hello everyone, it’s been a while) but people are really good in ‘filling in gaps’, it is how are brains work. We do it if we see three rocks positioned and then swear it looks like a face, we do it when we see a poem copied with certain tokens replaced when given a subject - we attribute complex emergent behavior through our interpretation rather than it being ‘intelligent’.
I’m not trying to undersell it, as it is important, I’m just saying that this isn’t a breakthrough but just a steady march of deep learning layered convolutional nets progress. The training material and the front-end really show it off well. It’s crossing the chasm for sure and will be really popular and be the main contact point like google/browser is today.
As for what jobs will it impact? I think a lot, and it is no small irony for me that the people that were sniffingly predicting the end of blue-collar factory / trades work as dinosaurs are now probably the perfect first white collar replacement jobs. Programming, sales, consultants are all going to be impacted. It doesn’t mean those jobs go, it just means that plain ‘info workers’ will have to evolve. The way I see it is that if you were a 411 operator in the year 2000 and then Google became popular and phones got better and more connected then you probably saw your job go. Is that bad? If your job today isn’t really about creativity or decision making relying on loose information then you could be in trouble. You now have to create value not by just knowing things but in using that information better than a LLM can regurgitate.
So ‘AI’ is a bit of an overloaded term, and one thing I don’t see mentioned in the froth of what’s going on now is that the models are highly asymmetrical, in that they take massive resources to generate and very little resources to query and use. What this means is that something like GPT-3.5 is effectively frozen in time at about May 2021. GPT-4 is newer and incorporates web queries as part of the answer parser, but it is still relying on the fact that finding the training weights takes a commitment that’s nothing like using them. GPT 5 tries to fix some of these things, but the math is against it. This means it does not learn like we do - we are great at making good or bad decisions very quickly with new information that doesn’t have a current pattern. What jobs will do well with that?
The dystopian take on this is that shortly we’ll have GPT powered Office tools automatically creating sales proposal powerpoints to be emailed to GPT powered Office outlook that then reads the powerpoint and declines the sales proposal. Weeks of white collar hell shortened to seconds. Future models just get trained on content that previous models have generated and internet published, so everything becomes derivative and slightly duller. The era of ‘info punk’ might come back eventually and we grow tired of everything being boring and generated. We’ll see.
Anyway, back to bed!
PS I thought that poem was really nice. Anyone here can use a GPT-4 derivative today using the Bing Chat, I think it says there is a waiting list but if you register then it is instant access.
Chicks dig my Hotmail.
. 