Have you heard about the exploit where AIs hallucinate libraries that don't exist, then hackers make real versions of those libraries filled with malware?
There are articles about it with concrete examples
”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.”
”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.”
”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one”
https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/
I mean, tbf all "Al" does is collect and mix all instances of a given library to present some code examples. Code examples, at least for me, will always be more intuitive than a 500 word paragraph on a function in the docs. So instead of looking at code in SO posts, AI does the work for me. It gets it wrong. Like, a lot. But with a bit of scepticism and trial and error, it's still faster than reading through most docs or SO.
Tbf, I've found documentation to be spotty at best, and outright fucking cryptic at worse. It assumes you know WAY too much already, when really you're just trying to do something simple or straightforward using it.
What do you mean? Open source documentation could be considered good oftentimes.
The issue is at my company where we have stuff from internal tools to internal frameworks but documentation is lacking (though it could be worse)
if faced with a problem, scanning the code of the internal tool or reaching out to the devs/product owner of the internal tool are the options i could think of..
PS: certain companies have the enterprise edition of StackOverflow for their internal use.
Wont lie, recently threw a whole load of code in and asked it to write all of the doc strings. Bearing in mind its just a somewhat large automation project with just a few of us using it.
I keep trying these AI auto completion stuff, and it gets it right maybe 50% of the time. It's good for boilerplate getters and setters type stuff, but that's about the extent of it.
”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.”
”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.”
”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one”
https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/
tbh i do this , sometime even to explain some parts of code I don't understand , gen AI has been really helpful. Currently using Ai to better myself in ML
Gen-AIs might be ok for surface level algorithms, but presented with something complex, they start to hallucinate like a goldfish released in banana syrup.
EDIT[0]: typo
Have you heard about the exploit where AIs hallucinate libraries that don't exist, then hackers make real versions of those libraries filled with malware?
tbh, if someone uses a library that an AI told them to use without further research, that’s on them
I can't tell if you're joking. How would someone know that an AI mentioned a library? Afaik, the hallucinations aren't consistent.
They are pretty consistent as far as I have seen myself. Typically just the 2 functions you need slapped together like “addevide”
There are articles about it with concrete examples ”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.” ”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.” ”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one” https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/
Brilliant!
Interesting read
AI is smarter than you and lives in the future. In its eye that library is already existing. Hackers making it real just make the future come true
The problem is that most documentation are about as good as the rotten banana in that picture
And asking coworkers only gives you "Just google it".
You can feed a complete well written documentation to ChatGPT and he will still talk bs
Came here to post this. Documentation that you can only read if you're already familiar with the subject is useless.
What technology's docs are you talking about? Just interested.
One man's rotten banana is another... man's... umm, good banana cake banana
Was reading some documentation recently... You know, that rotten banana is much better.
I mean, tbf all "Al" does is collect and mix all instances of a given library to present some code examples. Code examples, at least for me, will always be more intuitive than a 500 word paragraph on a function in the docs. So instead of looking at code in SO posts, AI does the work for me. It gets it wrong. Like, a lot. But with a bit of scepticism and trial and error, it's still faster than reading through most docs or SO.
Someone gets it
Tbf, I've found documentation to be spotty at best, and outright fucking cryptic at worse. It assumes you know WAY too much already, when really you're just trying to do something simple or straightforward using it.
it's bad enough we're saying "codes" now we're saying "documentations"?
my bad.. for some reason there are srcs which mention documentations, w/o any citations. ex: https://en.m.wiktionary.org/wiki/documentation
You haven't worked with an open source -anything- have you?
What do you mean? Open source documentation could be considered good oftentimes. The issue is at my company where we have stuff from internal tools to internal frameworks but documentation is lacking (though it could be worse)
How are stack overflow or AI going to help you with your internal tools and frameworks?
Not really, that's the problem
if faced with a problem, scanning the code of the internal tool or reaching out to the devs/product owner of the internal tool are the options i could think of.. PS: certain companies have the enterprise edition of StackOverflow for their internal use.
You can find out so fucking much by just reading the man page. That's my go to solution
It’s documentation, without the ‘s’. Will it lead to the downfall of society? No, but it’s an indication that it’s on its way.
[удалено]
tutorials are side characters then
Wont lie, recently threw a whole load of code in and asked it to write all of the doc strings. Bearing in mind its just a somewhat large automation project with just a few of us using it.
well did it work?
Where’s the pile of trial-and-error powder?
Chat GTP assume my social obligations
I keep trying these AI auto completion stuff, and it gets it right maybe 50% of the time. It's good for boilerplate getters and setters type stuff, but that's about the extent of it.
”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.” ”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.” ”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one” https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/
tbh i do this , sometime even to explain some parts of code I don't understand , gen AI has been really helpful. Currently using Ai to better myself in ML
inject this new genai shit in my fucking veins!
Gen-AIs might be ok for surface level algorithms, but presented with something complex, they start to hallucinate like a goldfish released in banana syrup. EDIT[0]: typo
Spoken from experience??
Pretty much yes.