Engineering video
14:34 · 4 February 2025
A developer discusses struggling with a growing project of over 30 Python files where the AI forgets imports and breaks code, noting that Cursor rules do not solve the underlying organizational issues.
Hear it at 0:00 →Many software engineers let emotions control them by following exaggerated social media claims from tech executives rather than doing independent analysis.
Hear it at 1:26 →LLMs rely on predicting the next token through inference, and increasing model size often leads to worse suggestions and hallucinations rather than exponential quality improvements.
Hear it at 2:20 →Techniques like mixture of experts route questions to specific subsets, but AI trained on a codebase tends to generate more legacy code rather than understanding evolving business reality.
Hear it at 4:46 →Tools like TypeScript use static typing to provide fast, reliable autocomplete across huge codebases, making them more efficient than statistically approximating answers via LLMs.
Hear it at 6:52 →Developers often get paralyzed or distracted by AI and blockchain trends instead of improving their craft, creating a strong advantage for those who continue building their engineering skills.
Hear it at 11:06 →okay so my project became so big that cloud can properly understand it so I made a project in Python entirely using cruor with composer and CLA but it has gotten to a point where the whole code base it's over 30 python files code it's super disorganized might even have duplicated loops and Cloud keeps forgetting basic stuff like the Imports at this point when asking to optimize the code or to fix a bug it doesn't even recognize the main issue and just ends up deleting random lines or breaking everything completely yes I can relate I have zero knowledgeable python it's actually a miricle I got this far with
the project but now it's almost impossible to keep track of things what do I do already tried using csor rules but doesn't seem to work cursor rules are actually sets of goals you can give to carser to kind of follow your coding style uh you know if you're using python you can tell you to code python a certain way or Javas screen in this video we want to cover three points that we think are super important and number one is the current Madness of the crowds and the the AI Gold Rush that we are right now in why the feature it's actually a bug and we're going to talk about in fence on how that changes AI
coding and why this is the best absolute best time to get ahead as a developer in how to do it because we actually believe that behind the paranoia there is a lot of opportunities for ambitious developer to actually get ahead in these times and actually you know see yourself two years from now in a very good position so number one the current Madness of the crowds and the AI gold Dash so I think the the biggest problem is that a lot of people these days and a lot of software Engineers are getting their information strictly from social media and YouTube and whatever the the YouTubers are kind of you know masticating from whatever
you know the Sal Sam almans and uh you know the Nvidia ctOS are seeing right and the problem with this people is they have a vested interest into making you believe that AI will fundamentally change the game so people they just let their emotions control them and I think it's it's you know I don't blame the developers because a lot of them didn't have the time or the resources to do their own analysis and that's the first advice of this video you know do your own analysis play with the AI coding tools you have at hand and understand the architecture behind it and how they work and that's what B and I will also
do in this video and going back to you bden what is the the biggest issue that you see with AI in coding when it comes down to coding sure so as I mentioned um the big problem is how the the fundamental Foundation of AI which is inference which is basically the ability to predict the next token based on statistically analyzing whatever your data set is and the current hypothesis behind uh even things like Devin and all the AI coding tools is that if this is my size of the model so let's say we kind of plot here this would be the size size of the model or data set right and then at the top we have the quality
let's say so this would be like quality of the coding suggestions or the autonomy or the size of the code base but let's let's put all those metrix into a single one that it's going to be the overarching quality what everybody's trying to say is that we have kind of this curve where you know the more you increase the site the exponentially you increase the quality and that's along those lines there like a some sort of a singularity where you basically replace developers with AI because it became so smart that the quality is even higher than a human developer the reality and what they are hiding from us is that the
curve probably if you look at the evidence and and the AI research looks more like this where it goes kind of into a plate and nobody really knows where we are here but you could see like gpt3 right kind of came out here and then GPT 4 and everybody saying you know like GPT 5 or DPS or whatever will be here where boom We repl developers you know zuk B said that but the reality is probably GPT 45 will be probably here slightly better and then you're going to have the next models here and then open AI will need to somehow justify what they've been doing to the investors and the reason for that is that when you
increase the size we have our llm which is literally a network right a weighted Network and so the idea is that I give an input and I get an output right and so the the whole tesis beyond all the coding tools is that if this is my model and gets very big and then I give it my whole code base then the you know the suggestion will get like better right it's like actually quality code so that would be my code base and then I have changes the problem is that when you increase the size of the model the suggestions you get are not better they are actually worse because the bigger the model the bigger this network the
more likely is that the question you ask gets answered by a different section of the model because it thinks oh actually you know what you gave me it's similar to this so it canot differentiate between the different section it has so the more code you throw at it the more likely that the suggestion you get it will be more of a hallucination and there's different ways they try to combat this and one of them that we've seen right now with dips is what they call the CLE of X where you can imagine this as you have inside a model you don't actually have a single model like a single llm and in what you have inside
it it's different subsets of models so you basically have a subset for coding then you have one for mathematics and inside the coding one you know you might have one for JavaScript objectoriented and when you ask a question it actually goes you know and R your question to the actual export right and then gives you back the answer so that's how they try to cvent the fact that you know the bigger the model the more it will hallucinate the problem with the CLE of experts is that if AI will only be an expert on your own code so let's say you train it on your code base and this wakes a lot you say well I'm going to
train the the model on my code base so it answers the questions as I want and the problem there is that imagine you you know you will end up just generating a lot of Legacy code so a lot of the things what developers do every day is we look at the code base and we try to move it towards where the business has moved which doesn't exist on text and why the technology is moving around us that's why we always you know the thing we hate most is deing with Legacy but that's actually a lot of part of the job and the problem with an AI That's trained on your code base is that it will actually generate more Legacy and
if it's not trained on your code base just this huge AI where you have all the code in the world it will hallucinate and these two problems will not be solved by you know increasing data they will not be solved by increasing hardware and they will not be solved by increasing the context which is you know I can take in your H Cod and again that's the fundamental problem that AI has right now whatever um whatever solution it proposes it's kind of trained on this snapshot of reality that's reflected into the code base but that's not reality itself so until AI can have its own scientific process right it can internalize a problem
solving mechanism and it can interact and it can try things itself in the in reality U then we we won't get there and I honestly believe we are maybe I don't know centuries away from that who knows right but going back to the uh to inference uh you mention this as a feature like the the the feature becomes the bug what what do you exactly mean by that so whenever you go to a bigger code base right in the in the Reddit PST they say they had 30 python files this is probably this is a very small project this is not a realsize production Enterprise application like we're talking about applications that have
hundreds of files even if it's a microservice and the problem with AI is that with llms right now is that they will statistically try to infer the next token and so they will give you sort of a statistical statistical answer to your whatever your question is and the problem with statistical answers is that uh first of all they they are geometrically expensive so the more input we give uh for every literally for every single token that the llm answers with you will have to do kind of around in the network itself and kind of go through all those different weights to come up with the right answer and that's
very first of all it's extremely inefficient the only reason they can pull it off is because it's not from a compute point of view it's extremely extremely inefficient but we have better ways to actually give suggestions in huge projects which are things like typescript right we have a a type system and yes typescript and intelligent and autocomplete are not as powerful in giving your suggestions but they're extremely powerful when you talk about thousands of lines of code like you could literally use static typing as in typescript to work on code bases that are huge and you get great suggestions
in in seconds and in terms of the computer you're doing well on your local machine so there's this mentality that I'm going to approximate the answer right I look at all the code on the internet all the code on GitHub and based on that code in my codebase I will approximate the answer statistically that's a much more inefficient way to solve that problem than having Dev tools like like typescript uh you know static typing and intelligent combin and so of course they're not as powerful but they're much much faster so that's where we need to make a you know a distinctions between approximating an
answer and giving an answer and what humans are very good and developers are really good at giving you an answer and what AI is really good it's kind of you know suggesting some some approximation of what it thinks the answer but again this does not eliminate the human factor you still need a validator of the proposed solution and this is why people like Mark zberg and safe for CEO saying oh we're not hiring Engineers oh we don't need Engineers I bet right now with all this deep seek coming out I bet right now they are where do they went do they went to the saes people to ask them what's happening no they went back to
the engineers and this is why you can see behind the curtain and you can see that these people are bluffing yeah um the market you know the the the it Market needs more Engineers than ever uh than ever before and it will need more Engineers than ever before despite claiming otherwise exactly and I think they will see themselves as you mentioned you know they will uh they believe in this you know this kind of you know this will happen and then we don't need any more developers they sell that because they want to sell their a tools but the reality is more looking like this where they going to have a lot
of Legacy code that you know the people they laid off only the people they laid off knew about it and the projects will be very big very bloated it's going to be very hard to keep track on things and they'll have to rehire the people they fired if they really want to S sell working software because yes this maybe it's someone that did some side project but we're talking about Enterprise software things that people pay money for and what will happen I would is that companies that actually have a great engineering team and have great code will start gaining market share because they'll have an engineering advantage
over companies that are over investing in sales or in Ai and what one things you probably seen is that uh even if all the uh stocks plummeted when DPI came out apple is going up because Apple downplayed AI in their strategy and apple has a lot of capital that they didn't spend in training unnecessary models or hiring AI experts that have actually no no nothing to do if you think about this AI will replace developers I think this train of thought comes from actually Elon Musk getting into Twitter uh way before this AI thing was a big deal and getting rid of theirs and still being able to run the company
but what you see if you look at uh you know at x.com reviews on on the apps you will see people complain and they they do not complain you know some of them complain of course about the political position of Elon Musk but I want to keep it about the product and you will see that the product the ux everything got wor right and X started having issues which kind of proves the point that mik have been mistaken uh but it seems like this this big CEO are just modeling Elon at this point because they probably don't know any better and that brings him to a third point you know why is this the best time to get ahead as a
developer so from what I see everybody's kind of busy and Paralyzed by AI a lot of developers jumped into learning machine learning all of the sudden or thinking about going back to UNI I think a PhD or Master because they believe they that developer skills would be replaced and everybody's kind of paralyzed and instead of getting better at the craft which is it's it's a complex and it needs time and you need to invest time to get good everybody's either paralyzed or trying to jump on you know the gold rush and become an AI ml developer and something similar is not the same but something similar
happened back in the days with blockchain where there are if you remember I think in 2021 a lot of people we were coaching developers back then and a lot of people would say well I'm going to become a blockchain developer because they make half a million or something and then six months later the same person will be unemployed and going back to their original job that actually pays real money so people are getting distracted or they feel like they they have to act on this anxiety and they're actually not getting better at the craft and so they will wake up 6 months from now maybe one year from now in the same
point they left and it's the people that actually get better today that will have a huge advantage in one year two years from now and again this comes from this madness of the crows and what I would tell the people watching us developers watching us it's we got to stop you know expecting social media or the algorithm or Mark zberg or whoever make this decision for us right because whether you choose to stay in the sector or you choose to Branch out because there's no future in software that's a major life decision and you cannot leave this to the crowd right and the crowd can be your boss you know I see Tech CEOs and
CTO and I see people that they look very qualified on paper that know very little about the specifics of AI and they haven't done their analysis and they are you know saying things and then you have all this mid level Dev Junior devs that are like hey I cannot take it anymore I'm so confused I don't think there's a future in this I I you know I invested so much time into um you know branching into this career learning how to code and now they're basically telling me it's over right and I think that's a very dangerous message and my advice to the mid level to the junior Dev even even to the senior Dev who's questioning
right if you're questioning your career choice do not I invite you to dive deep to go down the rabbit hole and do your own analysis do not listen to these people and do not make the mistake of thinking that they know more than you sure the chief scientists the people that actually build those llms they know a lot more than you but you know they cannot tell much because it will damage the pr of the companies they work for right there's a lot of confusion there's a lot of noise and this is when your engineering you have to use whatever you learn as an engineer and do your own analysis right and you will see you know
as B and I see every day that this is a big it's a big Bluff and six months from now the people that are going to be ahead the developers that will you know get up and even in the last two years I mean you know CH GPT came out in October what was it 2022 ever since we saw dozens of developers you know getting pay races getting promoted doing amazing a their job right and you have to see behind the smoke that's the one advice I would give you do not follow the crowd these people you know one person says something the other people repeat it that's that's how social media works you know you have DPS coming out and then
you have every other YouTuber making a video about dips very fast to get a click but the the Nal they they all talk about the same and they'll talk about AGI when like freaking CLA cannot handle 30 python 5 we are far away from that and the only type of of real intelligence we have now is the human intelligence so go back to that trust yourself believe yourself do your own analysis and keep on your skills Forks as for us thank you bgan and we will see you folks in the next one
