So everyone is talking about AI, either panicking or fantazing about the rise of AI, once again the future is both arriving fast and very slowly at the same time. Here is a capture of my current thought on the subject of AI, actually more questions than answers. Personally I think this is another bubble like the crypto one, but with a bit more substance. However there is some inherent limitations.

1. Energy Training AI, especially with Nvidia GPU is power intensive. There is a rush to equipe data center with the latest GPU, which consumption keeps climbing (400W !!! ). The powering of all the promised AI will be limited by the amount of electricity we decide to contribute to it.

2. Training Data Soon we will run out of material to train AI, and even without the legal limitations of fair use. I do not think, we can train AI on data generated by AI, we will end up with the same problem we have with photocopier. On top of this, the current training data, is generated by human worker (not paid very well to say the least) which are used to label picture and videos. There is also the problem that the trained data might be stained by cultural bias (ie who is doing the training). Also can Ai expand beyond the training data, for example if we were to train an AI only with data generated before 1850, would AI be able to answer question about general relativity or quantum phenomenon. Actually can AI expand and imagine something "new" ? How will it react when faced with something never seen before ?

3. Diminishing Returns Even if we set aside the energy and data issues, it looks like we are going to hit the eternal law of diminishing returns. Idealist thinks we are about to reach an exponential return as we fed more data (which will be limited), a more realistic approach would suggest a linear relationship between amount of training and "intelligence" produced; but I think each percentage increment of "returned intelligence" will require always exponentiallly increasing amount of training data.

See on arxiv the Paper: No "Zero-Shot" Without Exponential Data and the Youtube video from Computerphile.

1. Most non-engineer believe AI will not be able to replace human in term of emotional interaction, I actually think the opposite. Emotional recognition might be easier to achieve than we think, and human are pretty bad at it ( we are not telepath ), so AI is likely to surpass humans.

2. We also have to recognise, we don't understand some serious things about AI and machine learning:

3. Why is the last LLM so stable ?
4. In particular why are they so resilient to overfitting ? In traditional machine learning, overfitting is always an issue, yet here the more parameters, the better the model is.... This is so counter-intuitive.

5. Hallucinations looks to be unavoidable.

6. Verification

7. As we trust more of our systems to AI systems, how will we verify them ? How will we fix them (they are a black box) ? How will we be able to detect hallucinations as they will become more difficult to detect.
8. In the end, will we be able to trust the AI system ? Or will we trust them too much ?

Having say all that, I also think AI could be really helpful and useful on sone problems.

1. Parsing through scientific paper We have (as a human species) generated a vast amount of scientific papers, actually in the last few decades an increasing number. So many that most scientist are not able to read of aware of all of them. AI could help read and parse all those paper, and look for few things, in particular:
2. detect fraudulent paper (data manipulation for example, already being used).

3. find common insights and unusual new ideas.

4. Parsing through legal documents Again the amount of legal paper, makes the use of AI ideal, to detect logical incoherence.

Finally, some have expressed hope that universal revenue might solve the massive loss of jobs caused by AI, but: - AI ( and robots ) seems to be a promise for people owning assets (ie rich people) to rely less and less on human workers, and not for human worker to be free (well free of having a job). - Who will pay the universal revenue ? Who are we going to tax to finance it, whith more and more tax reduction and tax evasion from the companies and richest people. - I can see that the price of labour will go down, as human worker will now compete with automated systems.