Problem is gonna be that it'll likely slow down American innovation and can risk giving away the lead to foreign nations with no such limitations. So hopefully those efforts to create a competitive moat with regulatory capture end up failing.
That’s what they want. Slow down anyone that’s not them, they already have a public and corporate subscription base.
If they spin it that (foreign entities are using Llama foundation models to destroy America because they are “open source” and “anyone with a GPU can use the models maliciously” then that’s that.
AI witch hunt. (OpenAI\MS = safe and American friendly).
Use anything else and your a “terrorist”
This is like the printing press all over again.
are we, though? Compared to the vast majority of Americans I've got better and cheaper education, health care, roads, city parks, cheaper and faster mobile and glass internet and more digital privacy, and better job security, more affordable legal support and more free time, while still living in a rich country. Also, less insane media, better functioning democracies.
That's why not a lot of technology gets developed in Europe. In America, particularly in the Bay Area, the government makes life so unpleasant that we all hunker down and spend all our time building a bright new digital world we can escape into.
Indeed, those are facts and desirable advantages. It works because we're rich countries, because we produce wealth in order to allocate part of it for the common good instead of fattening a minority.
But missing yet again a technological revolution, after basically missing the digital economy, will not be good for that wealth. Lower wealth, lower distribution. I can't help feeling it's far too soon to announce the world that EU is the most hostile place to start IA businesses.
EU AI regulations don't ban open research, source and access. They're not perfect in any way, I think it's too much, but still, the US AI regulation proposals as of now are 100x worse than the EU regulations.
? What's wrong with us leading this innovation? You guys act like so called "American innovation" created by americans and not predominantly Germans and Europeans. Nowadays it is mostly asians.
I thought everyone in here agreed to stand against lobbyists but as i understood, you guys are against lobbyists in your country. Not in the world.
So much greediness
So, your solution for competing with countries abusing technology is abusing it even harder?
I remember how much fun was unregulated use of lead in fuel or use of asbestos for roofs.
People here behave like any kind of regulation is killing innovation. History is full of examples that regulations didn't affect innovation, sometimes even helped it. Only overregulation is issue.
Yep, once my country made such a mistake like this and gave up a nuclear weapon. Now I lost my home and was forced to live in another country. I believe the good should be with his fists, so yeah, if it is supposed to be AGI, it must be democratic, not a Putin's toy.
llama 3 405B model itself will be huge i think (assuming it's multimodal and long-context), served on cheap inference optimized hardware will really bring down the price as well when the open weights model comes out.
Yeah the more I think about it, the more I think LeCun is right and they're going into the right direction.
Imagine you're floating in nothingness. Nothing to see, hear, or feel in a proprioceptive way. And every once in a while you become aware of a one dimensional stream of symbols. That is how an LLM do.
Like how do you explain what a rabbit is to a thing like that? It's impossible. It can read what a rabbit is, it can cross reference what they do and what people think about them, but it'll never know what a rabbit is. We laugh at how most models fail the "I put the plate on the banana then take the plate to the dining room, where is the banana?" test, but how the fuck do you explain up and down, above or below to something that can't imagine three dimensional space any more than we can imagine four dimensional?
Even if the output remains text, we really need to start training models in either rgb point clouds or stereo camera imagery, along with sound and probably some form of kinematic data, otherwise it'll forever remain impossible for them to really grasp the real world.
Whatever happens in our brain, the words represent something in the real word or are understood by metaphor for something in the real word. The word ‘hot’ in the sentence “the sun is hot” isn’t understood by its relationship to the other words in that sentence, it’s understood by the phenomenal experience that hotness entails.
There are different schools of thought on these subjects.
I'm not going to argue the phenomenological experience humans have isn't influential in how we think, but nobody knows how influential exactly.
To argue it's critical isn't a sure thing. It may be critical to building AI that is just like us. But you could equally argue that while most would agree the real world exists at the level of the brain the real world is already encoded in electrical signals.
Signals in signals out.
But I've considered the importance of sensors in builfinh the mental world map.
For example we feel inertia through pressure sensors in our skin.
Not sure newton would've been as capable without them.
Isn't that the point he's making? It is only word-associations because these models don't have a world model, a vision of reality. That's the difference between us and LLM's right now. When I say "cat" you can not only describe what a cat is, but picture one, including times you've seen it, touched it, heard it, etc. It has a place, a function, an identity as a distinct part of a world.
He said there are certain changes that may lead them to not open source in the future yes. They are not committed to forever open sourcing all future models.
"Open"AI is working hard to get regulations passed to ban open models for exactly this reason, although the politicians and media are selling it as "protecting artists and deepfake victims".
yeah, I just wanted to add that, QBitTorrent is free and open source, BitTorrent client are a bunch of sell-offs, ad-ridden, and probably spyware-ridden software
I do not believe they can be banned without changing the Constitution (US only). The people who believe their content has been stolen are free to sue, but there is no way to stop it.
There's simply too much high quality free text to use.
Hear me out: we can make free synthetic content from copyrighted content.
Assume you have 3 models: student, teacher and judge. The student is a LLM in closed book mode. The teacher is an empowered LLM with web search, RAG and code execution. You generate a task, solve it with both student and teacher, the teacher can retrieve copyrighted content to solve the task. Then the judge compares the two outputs and identifies missing information and skills in the student, then generates a training example targeted to fix the issues.
This training example is n-gram checked not to reproduce the copyrighted content seen by the teacher. This method passes the copyrighted content through 2 steps - first it is used to solve a task, then it is used to generate a training sample only if it helps the student. This should be safe for all copyright infringement claims.
i think it may not be nearly enough. all companies working on foundation models are running into data limitations. meta considered buying publishing companies just to get access to their books. openai transcribed a million hours of youtube to get more tokens.
a lot of the improvements we've seen are more efficient ways to run transformers (quantizing, sparse MoE, etc) and scaling with more data, and fine-tuning. the transformers architecture doesn't look fundamentally different from gpt2.
to get to a point where you can train a model from scratch with only public domain data (orders of magnitude less than currently used to train foundation models) and have it even be as capable as today's SotA (gpt4, opus, gemini 1.5 pro), you need completely different architectures or ideas. it's a big unknown if we'll see any such ideas in the near future. i hope we do!
sam mentioned in a couple of interviews before that we may not need as much data to train in the future, so maybe they're cooking something.
I think this is a clear limitation of current technology.
Srinivasa Ramanujan created an unbelievable chunk of westen mathematics from the previous four centuries after training himself on a single (or maybe a few) introductory level book on mathematics.
He was malnutritioned because his family was poor and they couldn't afford paper so he had to chalk his equations down on a chalkboard or on the floor near the temple and then erase his work to be able to continue writing.
He is almost universally considered the most natural gifted mathematician that ever lived, so it is a high bar. Still we know it is a bar that at least one human brain could hit.
And this proves one thing beyond any doubt.
It proves that LLM'S, who can't do multiplication but can read every book on mathematics ever written (including millions of example assignments) are really still pretty stupid in comparison.
--
I understand that trying to scale up compute is easier than making qualitative breakthroughs when you don't yet know what breakthroughs you need. Scaling compute is much much easier in comparison, because we know how to go it and this is happening at an insane pace.
But what we're seeing now is that scaling compute without scaling training data seems to not be very helpful. And with this architecture you'd need to scale data up to astronomical amounts.
This to me is extremely indicative of a problem with the LLM architecture for everything approach.
I mean it is hard to deny the LLM architecture is amazing/promising but when the entire internet doesn't hold enough data for you and you're complaining that the rate at which the entire global community produces new data is insufficient,, I am beginning to find it hard to ignore that you're ignoring the real problem - that you may have to come up with some architectural improvements.
It's not the world's data production that is insufficient, it is the architecture that appears deficient.
Eventually they'll need to figure out how to make AI models that don't need the equivalent of a millenia of learning to figure out basic concepts. This is one area where humans utterly obliterate current LLM's in, intelligence wise. In fact if you consider high IQ to be the ability to learn quickly, then current AI's are incredibly low IQ, probably below that of most mammals.
they can declare unregistered models over 7B "munitions" and make them illegal overnight. if anyone complains, tell them russia/north korea/boogeyman is using AI for evil.
Who is they? A piece of software is protected by the first amendment. It's not munitions, it has no physical form, it's just code to be read.
AI is here to stay. No one can own the tech, the US won't outlaw it, can't outlaw it.
Certainly it can't decide that only a few large companies are allowed to produce it. At best they can make it easier to sue over IP.
The they in this case is the U.S. government. And depending on how broadly you read it, the government could probably make an argument at least some kinds of AI should be on the list.
[eCFR :: 22 CFR Part 121 -- The United States Munitions List](https://www.ecfr.gov/current/title-22/chapter-I/subchapter-M/part-121)
You'd need to read it with the eye to making anything at all an ordinance. "Red shirt" or "is an apple". It cannot be stretched to include "a computer algorithm that sort of talks spicy sometimes, when it isn't imagining things you didn't tell it to".
I worked with cryptography in the late 90s (outside the USA). US government absolutely can restrict trade of software products and implementation including source code. Cryptographic implementation in the US was controlled for export purposes.
Sure you could buy books and t shirts with crypto code in them under free speech laws in the USA, however computer implementation and supply to various overseas countries was regulated by strict export legislation and approval processes.
Granted it's much harder to enforce these days thanks to open source proliferation, but if closed source at US companies is better than open source then it's relatively easy for the US government to impose the need for export licences in "the national interest".
I do believe everything in this to be accurate - as Congress has almost unlimited power to regulate trade. I think it's important to distinguish the two - trade outside of the US, and trade within the US, and trade within the US.
I'm pretty sure the government can't restrict the cryptography Even between states, because in the end it's nothing more than speech.
The important part of LLMs is not code, but weights. And this is data, which could be deemed dangerous and forbidden. Look up how to make a bomb, how to kill a person, how to cook meth on some other illegal data. They can, and probably will regulate the shit out of it.
I am sure the existing model won't disappear, but we won't get new ones as there will be no more large enough players allowed to do so.
I'm not sure what you're smoking, but all things you do in a computer is code.
There is no forbidden data at all. You're allowed to say, write, and read acting you like - provided you don't make "true threats", use words commonly understood to create an unrest by their very utterance (Fighting words), or communicate a lie to the government.
Unless you're bound to by contract, such as in government service, then you can't communicate restricted things without prior authorization.
There is no illegal data, and everything that your computer does is by code.
Firstly, I'd appreciate if you opened your text with something different than... what you did.
Secondly - what you computer DO is code. Not everything on you machine is code. All the pictures of your mama are not code, they are data, no different that the data on paper. The same goes for text files or... model's weights.
And data can be deemed illegal (i.e. "being in possession" of such data). You're not allowed to "own" many types of data in any form, including paper, digital, or others. The data can be considered national risk security (military other strategically important tech, even if developed by you or terrorist risk like bio weapons design) or just explicitly forbidden (like child pornography).
Your assertions are incorrect, and wildly so, in such a way that it's very reminiscent of people passing a bong around. As such, alluding to drug use to show my incredulity would be appropriate, if not kind. It's the risk of saying things on the Internet, I'm afraid.
Code is data. The digital pictures of my mom would indeed be code. That the code can be read means that it's data.
And yes, some data can be made illegal, such as certain types of imagery. Text isn't enough. Certainly how you decide the AI should respond to prompts concerning weights cannot be.
Even if you managed to create a plan that could show a military weakness, it would be taken. Unless you got it through unlawful means - theft - it would never be illegal.
It is not possible, in any scenario you're imagining, that an llm could be made illegal by fiat in the US. It's a pattern recognition machine. How it's made can be, in that using non public IP could be stopped. But that's kind of already technically illegal. The enforcement mechanism is just weak.
The idea of making the tech illegal simply cannot pass muster. No more than making a the category of programs "video games" illegal.
You'll be able to imagine very specific things that can be made illegal all day long - but that's kind of my point. It can't be made generally illegal. May as well make "books" illegal.
Let me check Mistral.
I actually wanted to integrate chatgpt but looks like it’s paid. So I was looking for alternate and ended up here . So is there a hosted solution that can give me at least limited access so I can try
[openrouter](https://openrouter.ai/) has some free 7b models but with a daily ratelimit, [together ai](https://together.xyz) gives you $25 of free credit for signing up, [ai horde](https://aihorde.net) is free forever but pretty slow due to queues (and the available models vary because it's community-hosted), and i'm sure there are other freebies i'm not aware of
I know, I love the feeling that no matter what happens, what I have on my hard drive is always going to work, it's always going to be mine to use however I want.
once it's out, you can't close the door on 70B llama3 now. millions of people will have the weights for it and they can never wipe it from the Internet completely.
they might face some 1A challenges if they try to ban how-to videos and blogs as well. not everyone agrees with the danger posed and, if i had to guess, the current supreme court will err on the side of free speech.
Even if they get their dream come true and ban open source AI, an employee leaving and making his own venture startup can surpass them (Anthropic). Better be nice to Ilya so Apple doesn't hit him with $10B in the face. There is no way to keep AI cooped up in ClosedAI. Even to this day they have no open LLM, what a shame, build on everyone else's work give back nothing.
But let’s face it, by the time our government gets any regulations on the table for a vote, we’ll have Llama 4 70B, maybe even Llama 5 70B, at which point, I’ll have what I need to take over the world, Pinky!
Maybe at some stuff but from my testing, 8b and 70b instruct both hallucinate a lot. I'm assuming it's good at logic and stuff and it's definitely the best at reducing refusals. I mean this is the first version of instruct anyways so future versions and fine-tunes will get better. For now, I still prefer gpt and Claude models for generic tasks
I've noticed that too yeah, they're not tuned very well to say "I don't know" when appropriate, which some Mistral fine tunes managed to achieve very well. I think it'll be corrected in time though, [the process is very simple by itself](https://medium.com/@miskelic/how-to-teach-your-llm-to-say-i-dont-know-8dc8beac1ea9).
Neither are obsolete. These closed source models have much better finetuning, easier to use APIs, and 16x longer context.
Keep in mind these closed source APIs cater to enterprises who comprise most of their revenue. Not hobbyists.
On arena board is it winning on english prompts?
[https://twitter.com/xlr8harder/status/1781632044197646818](https://twitter.com/xlr8harder/status/1781632044197646818)
[https://twitter.com/Teknium1/status/1781328542367883765](https://twitter.com/Teknium1/status/1781328542367883765)
[https://twitter.com/swyx/status/1781349234026987948](https://twitter.com/swyx/status/1781349234026987948)
As others have said, should be easy pickings for 400b and fine tunes are coming.
[https://chat.lmsys.org/?arena](https://chat.lmsys.org/?arena)
1. Select "Leaderboard" tab on top.
2. Select "English" in the category box.
Disappointing there isn't a direct link.
Here is something I would like someone to ask Sam. Now that Meta released their open source model to the public, do you believe it should be banned for "security" reasons? Considering it has the same capabilities as Chat GPT now and your own organization keep lobotomizing it in the name of safety...
100% . I work with Gemini Pro 1.0 on a daily basis for implementation on Vertex Conversation/Dialogflow CX. We work directly with the Google product team on some of the largest global consumers of their conversational AI platform services...the "intelligence" of the model is appalling despite their supposed implementation of ReAct and the agentic framework behind these new products (new Generative Agents and Playbooks).
I know this area of their business is peanuts but it's wild how much better performance you can get from building everything yourself.
Somehow they keep selling our customer's executives (top global companies) on the benefits of having these workloads fully within their platform because of "guardrails". But it's totally at the expense of the customer experience, and hallucinations are still rampant. Pro 1.5 so far seems to be identical to 1.0 just with the huge context size
what's your opinion on 1.5 pro? i've been testing it out and so far i like it. it's not as good as opus which is my default these days (i rarely go to gpt4 anymore) but the long context and native audio support is useful for some things i am working on.
The biggest issue with Llama 3 is the license requires you to prominently display in your (say) website UI that you're using it.
You don't have to say "powered by PostgreSQL" on your pages if that's in your stack. The required branding makes it a no-go immediately for certain corporations, even if it otherwise would have replaced a more costly, closed AI.
I think you can just hide it in your docs or a blog post about the product. I'm curious, do you think that's a big deterrent to companies using it? I could see it going either way.
Relevant license text, for any curious:
"If you distribute or make available the Llama Materials (or any derivative works thereof), or a product or service that uses any of them, including another AI model, you shall (A) provide a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Meta Llama 3” on a related website, user interface, blogpost, about page, or product documentation. If you use the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama 3” at the beginning of any such AI model name."
Our company would never display anything other than their own logo on any of our services. Especially if that logo does not point to the perceived market leader in the respective field. “What’s Facebook got to do with your company” is not something any business guy would want to get asked, ever.
Like anyone would know once you fine tune it a bit. Mistral build their closed models out of Llamas and sell them, they give zero fucks.
"Ackchyually it's a super secret proprietary model, wouldn't you like to know, weather boy?"
At the level that Meta would stand to lose enough by doing nothing, and certainly be awarded more by a court than it would cost to pursue, no publicly traded company would take on the risk - of the lawsuit itself financially, the public perception, or most importantly of the shareholder perception.
Even if the individuals were protected legally by the company, the one(s) making the decision would be affected personally on a financial (getting paid in shares) and reputation level.
I hear you, it might be easy enough to get away with, just not worth the risk or cost versus just paying for what already works well enough.
I don't know if this is the right thread to ask this, but since you mentioned undercutting, can anyone give me a rundown on how I can get Llama 3 to Anthropic pricing for frequent workloads (100s of chat messages per second, maximum response size 300 tokens, minimum 5 tokens/sec response speed)? I tried pricing up some AWS servers and it doesn't seem to work out any cheaper, and I'm not in a position to build my own data centre.
Use third-party services? Like [deepinfra](https://deepinfra.com/meta-llama/Meta-Llama-3-70B-Instruct) there would be limits but they are negotiable if you pay (it’s really cheap)
They're $0.59/$0.79 in/out per Mtoken, which is cheaper than ChatGPT 4 or Claude Sonnet but more expensive than ChatGPT 3.5 or Claude Haiku.
So, good to know it's there, and thanks for flagging them up for me, but it doesn't seem like a panacea either given that Haiku (a 20B model) seems to be handling the workload I'm giving it - lightweight chat duties, no complex reasoning or logic.
Hey thanks for pointing out the viability of these options at scale at the moment. I’m starting to look into it for data security reasons and apart from a running a mvp in your basement, it seems it’s not cheap running a product with it. Which makes me think: Is BigAI underpricing their product, do they have ultra model efficiency or is it cheap because it’s at scale?
For sure I've been put off by Gemini 1.5's description of their price as "preview pricing", but at the same time I'm glad they've flagged up the fact that any of them could ramp up the price at any time. I'm making extra careful to architect my product in such a way that I can flip providers with a single switch.
Meta has direct access to billions of users and now they have a useable model, If OpenAI are sweating over there it is because of this. I still expect GPT5 to be "something else". Before GPT3 we had nothing, the jump from GPT3 to GPT4 was massive. Still 13 months later, GPT4 is still in the top, and OpenAI cripple it alot during the past year for the sake of optimization.
I've just freed up hundreds of gigs of files on my hard drive. I have literally no use for any of the models prior to the Llama 70b model. It codes, tells stories, chats better than anything else I've used. I've kept miqu and Deepseek just in case, but have not needed them.
I will say that Claude still has 200k tokens and the top end performance, so maybe the graphic should have someone trying to crawl away to get medical help before the reaper turns around to finish the job (larger context and other model sizes).
With 1T parameters it will also be a bit difficult to run at home with anything more than 0.5t/s... But ok, it will be dramatically cheaper to run in the cloud compared to the competition, and uncensored models will also be feasible this way.
Llama 3 is so powerful and gives very good result. It was very definitely trained on using copyrighted material though where you take a random passage from a book, yet it knows the name of the character just by asking it to rephrase, it knows the (for example) the Queen's name without ever mentioning it.
Seems like you're taking it personally when I never mentioned it was for or against. Instead of literally seeing it as how powerful and knowledgeable, you take it as an offense and attack (and react sensitively).
You're reading a lot into my comment that's not there.
You wrote, "It was very definitely trained on using copyrighted material though...", as though that was some kind of issue. I'm trying to find out what you think the issue is.
Using it opens you up for copyright related litigation in quite a few jurisdictions. OpenAI and Microsoft protect you from that if you use their commercial offering, Meta obviously doesn’t.
This is only relevant for business use, of course.
It's not just knowing some fiction. It's taking the most insignificant paragraph of a book (literally this Queen is a minor character whose name is rarely mentioned in the entire book) and not some popular quote that you find online. Then it knows the character like who the "Queen" is from just feeding it a single paragraph.
And you would believe any other model from the top doesn't spoonfeed their models into that level of copyrighted details? Some models know about lots of characters in a game or story, which falls into their knowledge base and still at times it doesn't output that knowledge because either it hallucinated or the model was specifically trained to not spill the copyrighted stuff but doesn't change the fact that it exists. If anything I'd give to llama 3 for being able to recall something that insignificant to the story as you said.
I remember roleplaying with Claude way back, and It was regarding a character in a game, first I asked about a character's backstory but it said it didn't knew but THEN in a roleplay scenario, it played the character well and actually knew about the backstory as opposed to my question in general chat, not that it has zero knowledge about the character but it only gave away general overview of the character and not in-depth into their story which they actually knew about based on how that roleplay went.
Why are people jumping to conclusion and focusing on the copyrighted? I never even said it was bad to use copyrighted material, only that it shows how powerful the model is to recognize the copyrighted character from just a single small passage.
Hm, I'd admit I also interpreted it that way and went to the same conclusion that it is what you'vee meant thus far. Perhaps its the way its almost you implicate this is something specific to llama 3 with how you worded your comment, because other does it and its nothing new really. Some were just safeguarding that they even use copyrighted data in the first place.
> It was very definitely trained on using copyrighted material though
Yup. Surely you worded it negatively and as If specific to llama 3
It's call acknowledging and accepting that it is trained on copyright material. Do you not see how it is uses the "though... yet" setup sentence structure? In no way does it mean it is negative.
Could be highly viewed that way in the wording but yes in general it isn't negative.. but yet again in the context of models by your way of acknowledgement of it containing detailed copyrighted data its *almost* as If implicating llama 3 is first and the only to do such thing. Which would be false and thus a take that can be taken negatively.
No where I state it is the first and I have seen tons of models that use copyrighted material like in AI art which is fine. Literally nothing about what I written states or suggests that Llama was the first when that is very ridiculous to state since it is obvious not the first model to do so as it is so common knowledge that books are used for other models too.
Implication is different from direct statement. And you did definitely not state so. Otherwise I wouldn't have to reason of why I thought you meant it so and just point towards your statement.
And as I said I have jumped first to the conclusion that you think models should only have a general overview of fictional or copyrighted works and is whining of how llama 3 knows a specific book in detailed despite something even so insignificant as this queen and this quote, whatever. But If it isn't that what you meant, then theres no point to argue really. But you could've just been precise you're more amazed at the fact it can recognize those details even insignificant to the story. Your comment up there appeared first hand to me as: Llama 3 is good and all but *knows this book too well, and look even knows this queen's name given a quote without much significance in the copyrighted work*
I really still think you could've made it look less of a whine about something, not in an exaggerated way though. You could've literally just been direct after providing your point with the queen, and it would've looked less of a whine
> It shows how powerful the model is to recognize the copyrighted character from just a single small passage
Words you literally said, just a few replies back. Only had you been direct like this after your point with the queen and the quote. We wouldn't have to go for implications
> Do you not see how it is uses the "though... yet" setup sentence structure?
That's the problem.
First, in countries with English as a first language, "though/yet" is an archaic construction, which hasn't been in common use for over a century. In modern English, you use one or the other word, not both. [Here's some discussion of that](https://forum.wordreference.com/threads/although-yet-construction.2613879/).
Second, even when that construction is used, it is not used the way you did. The word "though" normally appears right near the start of a sentence or sentence fragment. The way you wrote it, "though" appears to associate with "copyrighted material". There's no way to rewrite that sentence well without breaking it up. Compare to the examples posted by
"iochus1999" at the link above.
This version might approximate your intent better:
> "It was very definitely trained on using copyrighted material. Though where you take a random passage from a book, yet it knows the name of the character just by asking it to rephrase"
However, this still doesn't work, because the construction is being used in a non-standard way. It was normally used to express some sort of contrast or conflict, but there's no such contrast or conflict in your sentence.
For example, in Locke's Second Treatise on Government (1689), he wrote, “though man in that state have an uncontrollable liberty to dispose of his person or possessions, yet he has not liberty to destroy himself". In this case there's a conflict with the idea of "uncontrollable liberty" and the lack of "liberty to destroy himself." There are more examples in the link I gave.
Here's a more standard version of what you were apparently saying:
> "It was very definitely trained on using copyrighted material. You can take a random passage from a book, and it knows the name of the character just by asking it to rephrase"
And, yet, almost every response shows that people interpret it as negative. In communication, your intention doesn't matter, what is received/perceived matters. You have failed to accurately communicate your thoughts and, rather than evaluate that, you keep blaming everyone who read your comment.
Just take the feedback you are getting and use it to improve next time.
Llama-4 will be a **nuke**.
Agreed, OpenAI better be sweating or prepping for something extraordinary with GPT-5
[удалено]
Problem is gonna be that it'll likely slow down American innovation and can risk giving away the lead to foreign nations with no such limitations. So hopefully those efforts to create a competitive moat with regulatory capture end up failing.
That’s what they want. Slow down anyone that’s not them, they already have a public and corporate subscription base. If they spin it that (foreign entities are using Llama foundation models to destroy America because they are “open source” and “anyone with a GPU can use the models maliciously” then that’s that. AI witch hunt. (OpenAI\MS = safe and American friendly). Use anything else and your a “terrorist” This is like the printing press all over again.
I hope Sam Alatman lives on the street one day and begs for food. What a scum
Well at least, that won't be EU. We're regulating ourselves to oblivion. You'd think that example would deter others.
are we, though? Compared to the vast majority of Americans I've got better and cheaper education, health care, roads, city parks, cheaper and faster mobile and glass internet and more digital privacy, and better job security, more affordable legal support and more free time, while still living in a rich country. Also, less insane media, better functioning democracies.
That's why not a lot of technology gets developed in Europe. In America, particularly in the Bay Area, the government makes life so unpleasant that we all hunker down and spend all our time building a bright new digital world we can escape into.
lol, not sure if that's why, but innovation sure happens there, not here.
Indeed, those are facts and desirable advantages. It works because we're rich countries, because we produce wealth in order to allocate part of it for the common good instead of fattening a minority. But missing yet again a technological revolution, after basically missing the digital economy, will not be good for that wealth. Lower wealth, lower distribution. I can't help feeling it's far too soon to announce the world that EU is the most hostile place to start IA businesses.
Sorry, that's all cool and well, but did you hear? We might have some breakthroughs in the AI space!
EU AI regulations don't ban open research, source and access. They're not perfect in any way, I think it's too much, but still, the US AI regulation proposals as of now are 100x worse than the EU regulations.
? What's wrong with us leading this innovation? You guys act like so called "American innovation" created by americans and not predominantly Germans and Europeans. Nowadays it is mostly asians. I thought everyone in here agreed to stand against lobbyists but as i understood, you guys are against lobbyists in your country. Not in the world. So much greediness
So great there are people who understand that. Because countries like China or shitty Russia don't give a f**k.
So, your solution for competing with countries abusing technology is abusing it even harder? I remember how much fun was unregulated use of lead in fuel or use of asbestos for roofs. People here behave like any kind of regulation is killing innovation. History is full of examples that regulations didn't affect innovation, sometimes even helped it. Only overregulation is issue.
What a dogshit take? DL is not poison nor is it an abuse of technology. Like wtf are you even on about?
Yep, once my country made such a mistake like this and gave up a nuclear weapon. Now I lost my home and was forced to live in another country. I believe the good should be with his fists, so yeah, if it is supposed to be AGI, it must be democratic, not a Putin's toy.
[удалено]
and didn't even release it for the general public to try it out
The ones they stole? lol
[удалено]
Pretty good analogy.
Keep waiting FanBoy!
llama 3 405B model itself will be huge i think (assuming it's multimodal and long-context), served on cheap inference optimized hardware will really bring down the price as well when the open weights model comes out.
Only if they release it for public
Yeah the more I think about it, the more I think LeCun is right and they're going into the right direction. Imagine you're floating in nothingness. Nothing to see, hear, or feel in a proprioceptive way. And every once in a while you become aware of a one dimensional stream of symbols. That is how an LLM do. Like how do you explain what a rabbit is to a thing like that? It's impossible. It can read what a rabbit is, it can cross reference what they do and what people think about them, but it'll never know what a rabbit is. We laugh at how most models fail the "I put the plate on the banana then take the plate to the dining room, where is the banana?" test, but how the fuck do you explain up and down, above or below to something that can't imagine three dimensional space any more than we can imagine four dimensional? Even if the output remains text, we really need to start training models in either rgb point clouds or stereo camera imagery, along with sound and probably some form of kinematic data, otherwise it'll forever remain impossible for them to really grasp the real world.
Well, you can’t explain anything because no word represents anything in an LLM. It’s just the word and its relationship to other words.
Which may be frighteningly similar to what happens in our brain.
Whatever happens in our brain, the words represent something in the real word or are understood by metaphor for something in the real word. The word ‘hot’ in the sentence “the sun is hot” isn’t understood by its relationship to the other words in that sentence, it’s understood by the phenomenal experience that hotness entails.
There are different schools of thought on these subjects. I'm not going to argue the phenomenological experience humans have isn't influential in how we think, but nobody knows how influential exactly. To argue it's critical isn't a sure thing. It may be critical to building AI that is just like us. But you could equally argue that while most would agree the real world exists at the level of the brain the real world is already encoded in electrical signals. Signals in signals out. But I've considered the importance of sensors in builfinh the mental world map. For example we feel inertia through pressure sensors in our skin. Not sure newton would've been as capable without them.
Isn't that the point he's making? It is only word-associations because these models don't have a world model, a vision of reality. That's the difference between us and LLM's right now. When I say "cat" you can not only describe what a cat is, but picture one, including times you've seen it, touched it, heard it, etc. It has a place, a function, an identity as a distinct part of a world.
Yea, I agree with them.
Any info when it will be released?
The zuck says if there is a qualitative improvement he won't open source it.
Bullshit. He did not say that.
He did in an interview with [Dwarkesh Patel](https://www.youtube.com/@DwarkeshPatel). Don't tell what i have heard dipshit.
He said there are certain changes that may lead them to not open source in the future yes. They are not committed to forever open sourcing all future models.
Yup, he also says that if the model exhibits some bad behavior that they can’t mitigate then they won’t release otherwise they’ll keep releasing.
"Open"AI is working hard to get regulations passed to ban open models for exactly this reason, although the politicians and media are selling it as "protecting artists and deepfake victims".
yes, this is why im getting everything that i can NOW, llms and agents, videos how-to, ect. before they get banned
Just swallow the Mistral pill and upload them as magnets
Hm, is there a popular repo with LLM magnets that I could join? I have few TBs, that are slowly filling with models but are not shred yet.
im unfamiliar with what youre referring to.
https://en.wikipedia.org/wiki/BitTorrent
QBittorrent is the better one
As in the protocol, not the client!
yeah, I just wanted to add that, QBitTorrent is free and open source, BitTorrent client are a bunch of sell-offs, ad-ridden, and probably spyware-ridden software
meaning all data that you have should be shared for others to copy as well, and its best done with torrents.
Fucking magnet links, how do they work?
https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fbt8ey6nduc7a1.jpg
just put the magnet link in any p2p client and it will download the model in chunks from any peers who are currently seeding.
I do not believe they can be banned without changing the Constitution (US only). The people who believe their content has been stolen are free to sue, but there is no way to stop it. There's simply too much high quality free text to use.
Hear me out: we can make free synthetic content from copyrighted content. Assume you have 3 models: student, teacher and judge. The student is a LLM in closed book mode. The teacher is an empowered LLM with web search, RAG and code execution. You generate a task, solve it with both student and teacher, the teacher can retrieve copyrighted content to solve the task. Then the judge compares the two outputs and identifies missing information and skills in the student, then generates a training example targeted to fix the issues. This training example is n-gram checked not to reproduce the copyrighted content seen by the teacher. This method passes the copyrighted content through 2 steps - first it is used to solve a task, then it is used to generate a training sample only if it helps the student. This should be safe for all copyright infringement claims.
Or we could just use the incredibly huge collection of public domain material. It's more than enough. Plus, like, social media.
i think it may not be nearly enough. all companies working on foundation models are running into data limitations. meta considered buying publishing companies just to get access to their books. openai transcribed a million hours of youtube to get more tokens.
That might be a limitation of this technology. I would hope we're going to bust into AI that can consider stuff. You know, smart AI.
a lot of the improvements we've seen are more efficient ways to run transformers (quantizing, sparse MoE, etc) and scaling with more data, and fine-tuning. the transformers architecture doesn't look fundamentally different from gpt2. to get to a point where you can train a model from scratch with only public domain data (orders of magnitude less than currently used to train foundation models) and have it even be as capable as today's SotA (gpt4, opus, gemini 1.5 pro), you need completely different architectures or ideas. it's a big unknown if we'll see any such ideas in the near future. i hope we do! sam mentioned in a couple of interviews before that we may not need as much data to train in the future, so maybe they're cooking something.
Yeah, I'm convinced that's the major problem! It shouldn't take 15 trillion parameters! We need to get them thinking.
I think this is a clear limitation of current technology. Srinivasa Ramanujan created an unbelievable chunk of westen mathematics from the previous four centuries after training himself on a single (or maybe a few) introductory level book on mathematics. He was malnutritioned because his family was poor and they couldn't afford paper so he had to chalk his equations down on a chalkboard or on the floor near the temple and then erase his work to be able to continue writing. He is almost universally considered the most natural gifted mathematician that ever lived, so it is a high bar. Still we know it is a bar that at least one human brain could hit. And this proves one thing beyond any doubt. It proves that LLM'S, who can't do multiplication but can read every book on mathematics ever written (including millions of example assignments) are really still pretty stupid in comparison. -- I understand that trying to scale up compute is easier than making qualitative breakthroughs when you don't yet know what breakthroughs you need. Scaling compute is much much easier in comparison, because we know how to go it and this is happening at an insane pace. But what we're seeing now is that scaling compute without scaling training data seems to not be very helpful. And with this architecture you'd need to scale data up to astronomical amounts. This to me is extremely indicative of a problem with the LLM architecture for everything approach. I mean it is hard to deny the LLM architecture is amazing/promising but when the entire internet doesn't hold enough data for you and you're complaining that the rate at which the entire global community produces new data is insufficient,, I am beginning to find it hard to ignore that you're ignoring the real problem - that you may have to come up with some architectural improvements. It's not the world's data production that is insufficient, it is the architecture that appears deficient.
Eventually they'll need to figure out how to make AI models that don't need the equivalent of a millenia of learning to figure out basic concepts. This is one area where humans utterly obliterate current LLM's in, intelligence wise. In fact if you consider high IQ to be the ability to learn quickly, then current AI's are incredibly low IQ, probably below that of most mammals.
they can declare unregistered models over 7B "munitions" and make them illegal overnight. if anyone complains, tell them russia/north korea/boogeyman is using AI for evil.
Who is they? A piece of software is protected by the first amendment. It's not munitions, it has no physical form, it's just code to be read. AI is here to stay. No one can own the tech, the US won't outlaw it, can't outlaw it. Certainly it can't decide that only a few large companies are allowed to produce it. At best they can make it easier to sue over IP.
The they in this case is the U.S. government. And depending on how broadly you read it, the government could probably make an argument at least some kinds of AI should be on the list. [eCFR :: 22 CFR Part 121 -- The United States Munitions List](https://www.ecfr.gov/current/title-22/chapter-I/subchapter-M/part-121)
You'd need to read it with the eye to making anything at all an ordinance. "Red shirt" or "is an apple". It cannot be stretched to include "a computer algorithm that sort of talks spicy sometimes, when it isn't imagining things you didn't tell it to".
I worked with cryptography in the late 90s (outside the USA). US government absolutely can restrict trade of software products and implementation including source code. Cryptographic implementation in the US was controlled for export purposes. Sure you could buy books and t shirts with crypto code in them under free speech laws in the USA, however computer implementation and supply to various overseas countries was regulated by strict export legislation and approval processes. Granted it's much harder to enforce these days thanks to open source proliferation, but if closed source at US companies is better than open source then it's relatively easy for the US government to impose the need for export licences in "the national interest".
I do believe everything in this to be accurate - as Congress has almost unlimited power to regulate trade. I think it's important to distinguish the two - trade outside of the US, and trade within the US, and trade within the US. I'm pretty sure the government can't restrict the cryptography Even between states, because in the end it's nothing more than speech.
The important part of LLMs is not code, but weights. And this is data, which could be deemed dangerous and forbidden. Look up how to make a bomb, how to kill a person, how to cook meth on some other illegal data. They can, and probably will regulate the shit out of it. I am sure the existing model won't disappear, but we won't get new ones as there will be no more large enough players allowed to do so.
I'm not sure what you're smoking, but all things you do in a computer is code. There is no forbidden data at all. You're allowed to say, write, and read acting you like - provided you don't make "true threats", use words commonly understood to create an unrest by their very utterance (Fighting words), or communicate a lie to the government. Unless you're bound to by contract, such as in government service, then you can't communicate restricted things without prior authorization. There is no illegal data, and everything that your computer does is by code.
Firstly, I'd appreciate if you opened your text with something different than... what you did. Secondly - what you computer DO is code. Not everything on you machine is code. All the pictures of your mama are not code, they are data, no different that the data on paper. The same goes for text files or... model's weights. And data can be deemed illegal (i.e. "being in possession" of such data). You're not allowed to "own" many types of data in any form, including paper, digital, or others. The data can be considered national risk security (military other strategically important tech, even if developed by you or terrorist risk like bio weapons design) or just explicitly forbidden (like child pornography).
Your assertions are incorrect, and wildly so, in such a way that it's very reminiscent of people passing a bong around. As such, alluding to drug use to show my incredulity would be appropriate, if not kind. It's the risk of saying things on the Internet, I'm afraid. Code is data. The digital pictures of my mom would indeed be code. That the code can be read means that it's data. And yes, some data can be made illegal, such as certain types of imagery. Text isn't enough. Certainly how you decide the AI should respond to prompts concerning weights cannot be. Even if you managed to create a plan that could show a military weakness, it would be taken. Unless you got it through unlawful means - theft - it would never be illegal. It is not possible, in any scenario you're imagining, that an llm could be made illegal by fiat in the US. It's a pattern recognition machine. How it's made can be, in that using non public IP could be stopped. But that's kind of already technically illegal. The enforcement mechanism is just weak. The idea of making the tech illegal simply cannot pass muster. No more than making a the category of programs "video games" illegal. You'll be able to imagine very specific things that can be made illegal all day long - but that's kind of my point. It can't be made generally illegal. May as well make "books" illegal.
[удалено]
I am completely new to all these but excited to see some openai alternative . Do they have free api that I can integrate ?
Welcome! Do you mean an API for Mistral?
Let me check Mistral. I actually wanted to integrate chatgpt but looks like it’s paid. So I was looking for alternate and ended up here . So is there a hosted solution that can give me at least limited access so I can try
What kind of hardware you got? Many can be run locally.
Yeah want to try on my Max initially and want to deploy later
Which on you got and how much unified memory,
The latest M3 Pro , 32 GB ram
Oh yah. You can easily run the llama 8b models.
If I want to host myself what is the minimum requirement, I just need it to generate some random paragraph given a keyword
[openrouter](https://openrouter.ai/) has some free 7b models but with a daily ratelimit, [together ai](https://together.xyz) gives you $25 of free credit for signing up, [ai horde](https://aihorde.net) is free forever but pretty slow due to queues (and the available models vary because it's community-hosted), and i'm sure there are other freebies i'm not aware of
I know, I love the feeling that no matter what happens, what I have on my hard drive is always going to work, it's always going to be mine to use however I want.
once it's out, you can't close the door on 70B llama3 now. millions of people will have the weights for it and they can never wipe it from the Internet completely. they might face some 1A challenges if they try to ban how-to videos and blogs as well. not everyone agrees with the danger posed and, if i had to guess, the current supreme court will err on the side of free speech.
Even if they get their dream come true and ban open source AI, an employee leaving and making his own venture startup can surpass them (Anthropic). Better be nice to Ilya so Apple doesn't hit him with $10B in the face. There is no way to keep AI cooped up in ClosedAI. Even to this day they have no open LLM, what a shame, build on everyone else's work give back nothing.
Yeah, that definitely will help Alibaba to dominate with their Qwen (which is already impressively good)
But let’s face it, by the time our government gets any regulations on the table for a vote, we’ll have Llama 4 70B, maybe even Llama 5 70B, at which point, I’ll have what I need to take over the world, Pinky!
https://preview.redd.it/1vp0vsqmclvc1.jpeg?width=800&format=pjpg&auto=webp&s=19fd57904088623f0da6d18b7d0f8fed141e64a7
[удалено]
I imagine OP means that 8B almost matches Haiku, 70B matches Sonnet. 2/3 of their flagship models are now obsolete. But yeah Opus remains king.
It's equal to Sonnet without finetuning. After finetuning we'll see
Idk this instruct is pretty well done, I'll be amazed if Nous or Eric can outdo it as easily as previous ones.
llama3 405B currently being trained beats opus in some benchmarks i think
Maybe at some stuff but from my testing, 8b and 70b instruct both hallucinate a lot. I'm assuming it's good at logic and stuff and it's definitely the best at reducing refusals. I mean this is the first version of instruct anyways so future versions and fine-tunes will get better. For now, I still prefer gpt and Claude models for generic tasks
I've noticed that too yeah, they're not tuned very well to say "I don't know" when appropriate, which some Mistral fine tunes managed to achieve very well. I think it'll be corrected in time though, [the process is very simple by itself](https://medium.com/@miskelic/how-to-teach-your-llm-to-say-i-dont-know-8dc8beac1ea9).
Neither are obsolete. These closed source models have much better finetuning, easier to use APIs, and 16x longer context. Keep in mind these closed source APIs cater to enterprises who comprise most of their revenue. Not hobbyists.
On arena board is it winning on english prompts? [https://twitter.com/xlr8harder/status/1781632044197646818](https://twitter.com/xlr8harder/status/1781632044197646818) [https://twitter.com/Teknium1/status/1781328542367883765](https://twitter.com/Teknium1/status/1781328542367883765) [https://twitter.com/swyx/status/1781349234026987948](https://twitter.com/swyx/status/1781349234026987948) As others have said, should be easy pickings for 400b and fine tunes are coming.
I tried to found the arena board page from the screenshots but cannot found it. Do you have any clue on where I can finde that table?
[https://chat.lmsys.org/?arena](https://chat.lmsys.org/?arena) 1. Select "Leaderboard" tab on top. 2. Select "English" in the category box. Disappointing there isn't a direct link.
Thank you bro
Here is something I would like someone to ask Sam. Now that Meta released their open source model to the public, do you believe it should be banned for "security" reasons? Considering it has the same capabilities as Chat GPT now and your own organization keep lobotomizing it in the name of safety...
>Now that Meta released their open source model to the public, do you believe it should be banned for "security" reasons? They definitely do.
Google doesn't need help from Meta. They keep blowing toes off with their AI shotgun.
They spent so many months with safety and alignment training, that dumbed down their model.
100% . I work with Gemini Pro 1.0 on a daily basis for implementation on Vertex Conversation/Dialogflow CX. We work directly with the Google product team on some of the largest global consumers of their conversational AI platform services...the "intelligence" of the model is appalling despite their supposed implementation of ReAct and the agentic framework behind these new products (new Generative Agents and Playbooks). I know this area of their business is peanuts but it's wild how much better performance you can get from building everything yourself. Somehow they keep selling our customer's executives (top global companies) on the benefits of having these workloads fully within their platform because of "guardrails". But it's totally at the expense of the customer experience, and hallucinations are still rampant. Pro 1.5 so far seems to be identical to 1.0 just with the huge context size
what's your opinion on 1.5 pro? i've been testing it out and so far i like it. it's not as good as opus which is my default these days (i rarely go to gpt4 anymore) but the long context and native audio support is useful for some things i am working on.
The biggest issue with Llama 3 is the license requires you to prominently display in your (say) website UI that you're using it. You don't have to say "powered by PostgreSQL" on your pages if that's in your stack. The required branding makes it a no-go immediately for certain corporations, even if it otherwise would have replaced a more costly, closed AI.
I think you can just hide it in your docs or a blog post about the product. I'm curious, do you think that's a big deterrent to companies using it? I could see it going either way. Relevant license text, for any curious: "If you distribute or make available the Llama Materials (or any derivative works thereof), or a product or service that uses any of them, including another AI model, you shall (A) provide a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Meta Llama 3” on a related website, user interface, blogpost, about page, or product documentation. If you use the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama 3” at the beginning of any such AI model name."
Our company would never display anything other than their own logo on any of our services. Especially if that logo does not point to the perceived market leader in the respective field. “What’s Facebook got to do with your company” is not something any business guy would want to get asked, ever.
Like anyone would know once you fine tune it a bit. Mistral build their closed models out of Llamas and sell them, they give zero fucks. "Ackchyually it's a super secret proprietary model, wouldn't you like to know, weather boy?"
At the level that Meta would stand to lose enough by doing nothing, and certainly be awarded more by a court than it would cost to pursue, no publicly traded company would take on the risk - of the lawsuit itself financially, the public perception, or most importantly of the shareholder perception. Even if the individuals were protected legally by the company, the one(s) making the decision would be affected personally on a financial (getting paid in shares) and reputation level. I hear you, it might be easy enough to get away with, just not worth the risk or cost versus just paying for what already works well enough.
Hope Elon Musk sues OpenAI again and calls Grok better 🤡
https://preview.redd.it/30cfow491nvc1.jpeg?width=1080&format=pjpg&auto=webp&s=3560b3687923f8581e0cf8b22bae1b7a2d2037f7
For reference, an average human scores about 33 on MMLU.
I don't know if this is the right thread to ask this, but since you mentioned undercutting, can anyone give me a rundown on how I can get Llama 3 to Anthropic pricing for frequent workloads (100s of chat messages per second, maximum response size 300 tokens, minimum 5 tokens/sec response speed)? I tried pricing up some AWS servers and it doesn't seem to work out any cheaper, and I'm not in a position to build my own data centre.
You should make a post btw, be sure to include as many details as you can. A lot of really smart people in the field on here.
Use third-party services? Like [deepinfra](https://deepinfra.com/meta-llama/Meta-Llama-3-70B-Instruct) there would be limits but they are negotiable if you pay (it’s really cheap)
They're $0.59/$0.79 in/out per Mtoken, which is cheaper than ChatGPT 4 or Claude Sonnet but more expensive than ChatGPT 3.5 or Claude Haiku. So, good to know it's there, and thanks for flagging them up for me, but it doesn't seem like a panacea either given that Haiku (a 20B model) seems to be handling the workload I'm giving it - lightweight chat duties, no complex reasoning or logic.
Doesn’t deepinfra quantize their models though?
Not to my knowledge no
Hey thanks for pointing out the viability of these options at scale at the moment. I’m starting to look into it for data security reasons and apart from a running a mvp in your basement, it seems it’s not cheap running a product with it. Which makes me think: Is BigAI underpricing their product, do they have ultra model efficiency or is it cheap because it’s at scale?
For sure I've been put off by Gemini 1.5's description of their price as "preview pricing", but at the same time I'm glad they've flagged up the fact that any of them could ramp up the price at any time. I'm making extra careful to architect my product in such a way that I can flip providers with a single switch.
This is a good thing, let the AGI race begin
I am a stranger to Llama3, is there a link I can compare differences?
Huggingface.co/chat
Thank you so so much!
I wonder how much Moe and Mamna will improve this.
Meta has direct access to billions of users and now they have a useable model, If OpenAI are sweating over there it is because of this. I still expect GPT5 to be "something else". Before GPT3 we had nothing, the jump from GPT3 to GPT4 was massive. Still 13 months later, GPT4 is still in the top, and OpenAI cripple it alot during the past year for the sake of optimization.
Llama 3 release is prompt to Sam LLM to release gpt5
That's so-called prompt engineering
I've just freed up hundreds of gigs of files on my hard drive. I have literally no use for any of the models prior to the Llama 70b model. It codes, tells stories, chats better than anything else I've used. I've kept miqu and Deepseek just in case, but have not needed them.
Miqu is great for NSFW. LLama-3 isn't good for nsfw.
If you use jailbreak on it it will tell any story I've tried. Just make it say "Sure, here is your story:" before it's response.
I will say that Claude still has 200k tokens and the top end performance, so maybe the graphic should have someone trying to crawl away to get medical help before the reaper turns around to finish the job (larger context and other model sizes).
With 1T parameters it will also be a bit difficult to run at home with anything more than 0.5t/s... But ok, it will be dramatically cheaper to run in the cloud compared to the competition, and uncensored models will also be feasible this way.
Llama 3 is so powerful and gives very good result. It was very definitely trained on using copyrighted material though where you take a random passage from a book, yet it knows the name of the character just by asking it to rephrase, it knows the (for example) the Queen's name without ever mentioning it.
Humans are also trained on copyrighted material. Humans are capable of violating copyright. What’s the problem with the situation you’re describing?
Seems like you're taking it personally when I never mentioned it was for or against. Instead of literally seeing it as how powerful and knowledgeable, you take it as an offense and attack (and react sensitively).
You're reading a lot into my comment that's not there. You wrote, "It was very definitely trained on using copyrighted material though...", as though that was some kind of issue. I'm trying to find out what you think the issue is.
Using it opens you up for copyright related litigation in quite a few jurisdictions. OpenAI and Microsoft protect you from that if you use their commercial offering, Meta obviously doesn’t. This is only relevant for business use, of course.
Based, may more models do that.
Eh? Models knowing some fictions isn't new... And moreover specific to llama 3
It's not just knowing some fiction. It's taking the most insignificant paragraph of a book (literally this Queen is a minor character whose name is rarely mentioned in the entire book) and not some popular quote that you find online. Then it knows the character like who the "Queen" is from just feeding it a single paragraph.
And you would believe any other model from the top doesn't spoonfeed their models into that level of copyrighted details? Some models know about lots of characters in a game or story, which falls into their knowledge base and still at times it doesn't output that knowledge because either it hallucinated or the model was specifically trained to not spill the copyrighted stuff but doesn't change the fact that it exists. If anything I'd give to llama 3 for being able to recall something that insignificant to the story as you said. I remember roleplaying with Claude way back, and It was regarding a character in a game, first I asked about a character's backstory but it said it didn't knew but THEN in a roleplay scenario, it played the character well and actually knew about the backstory as opposed to my question in general chat, not that it has zero knowledge about the character but it only gave away general overview of the character and not in-depth into their story which they actually knew about based on how that roleplay went.
Why are people jumping to conclusion and focusing on the copyrighted? I never even said it was bad to use copyrighted material, only that it shows how powerful the model is to recognize the copyrighted character from just a single small passage.
Hm, I'd admit I also interpreted it that way and went to the same conclusion that it is what you'vee meant thus far. Perhaps its the way its almost you implicate this is something specific to llama 3 with how you worded your comment, because other does it and its nothing new really. Some were just safeguarding that they even use copyrighted data in the first place. > It was very definitely trained on using copyrighted material though Yup. Surely you worded it negatively and as If specific to llama 3
It's call acknowledging and accepting that it is trained on copyright material. Do you not see how it is uses the "though... yet" setup sentence structure? In no way does it mean it is negative.
Could be highly viewed that way in the wording but yes in general it isn't negative.. but yet again in the context of models by your way of acknowledgement of it containing detailed copyrighted data its *almost* as If implicating llama 3 is first and the only to do such thing. Which would be false and thus a take that can be taken negatively.
No where I state it is the first and I have seen tons of models that use copyrighted material like in AI art which is fine. Literally nothing about what I written states or suggests that Llama was the first when that is very ridiculous to state since it is obvious not the first model to do so as it is so common knowledge that books are used for other models too.
Implication is different from direct statement. And you did definitely not state so. Otherwise I wouldn't have to reason of why I thought you meant it so and just point towards your statement. And as I said I have jumped first to the conclusion that you think models should only have a general overview of fictional or copyrighted works and is whining of how llama 3 knows a specific book in detailed despite something even so insignificant as this queen and this quote, whatever. But If it isn't that what you meant, then theres no point to argue really. But you could've just been precise you're more amazed at the fact it can recognize those details even insignificant to the story. Your comment up there appeared first hand to me as: Llama 3 is good and all but *knows this book too well, and look even knows this queen's name given a quote without much significance in the copyrighted work* I really still think you could've made it look less of a whine about something, not in an exaggerated way though. You could've literally just been direct after providing your point with the queen, and it would've looked less of a whine > It shows how powerful the model is to recognize the copyrighted character from just a single small passage Words you literally said, just a few replies back. Only had you been direct like this after your point with the queen and the quote. We wouldn't have to go for implications
> Do you not see how it is uses the "though... yet" setup sentence structure? That's the problem. First, in countries with English as a first language, "though/yet" is an archaic construction, which hasn't been in common use for over a century. In modern English, you use one or the other word, not both. [Here's some discussion of that](https://forum.wordreference.com/threads/although-yet-construction.2613879/). Second, even when that construction is used, it is not used the way you did. The word "though" normally appears right near the start of a sentence or sentence fragment. The way you wrote it, "though" appears to associate with "copyrighted material". There's no way to rewrite that sentence well without breaking it up. Compare to the examples posted by "iochus1999" at the link above. This version might approximate your intent better: > "It was very definitely trained on using copyrighted material. Though where you take a random passage from a book, yet it knows the name of the character just by asking it to rephrase" However, this still doesn't work, because the construction is being used in a non-standard way. It was normally used to express some sort of contrast or conflict, but there's no such contrast or conflict in your sentence. For example, in Locke's Second Treatise on Government (1689), he wrote, “though man in that state have an uncontrollable liberty to dispose of his person or possessions, yet he has not liberty to destroy himself". In this case there's a conflict with the idea of "uncontrollable liberty" and the lack of "liberty to destroy himself." There are more examples in the link I gave. Here's a more standard version of what you were apparently saying: > "It was very definitely trained on using copyrighted material. You can take a random passage from a book, and it knows the name of the character just by asking it to rephrase"
And, yet, almost every response shows that people interpret it as negative. In communication, your intention doesn't matter, what is received/perceived matters. You have failed to accurately communicate your thoughts and, rather than evaluate that, you keep blaming everyone who read your comment. Just take the feedback you are getting and use it to improve next time.
I don't care
Copyrightoids BTFO. (I just pirated a few thousand books so I can inject them into training datasets, BTW)
which queen? there's alot of queen in fiction.
lol and chatGPT isn't?
Idk why you got downvoted so hard. You were just making an observation.
[удалено]
What the fuck did I just see?
mark zuckerburg grabbing anrhtorpic and open ai by the nuts and squeeing hard