抖文 - Technology Enthusiast Weekly (Issue 390): Large Language Models Are Useless Without Training Data 的评论
Technology Enthusiasts Weekly (Issue 390): Without Corpus, Large Models Are IdiotsThis newsletter...
Bold claim but I'm not entirely convinced. Haven't we seen fine-tuned smaller models punch way above their weight? The relationship between data quality and model size seems more nuanced than the title suggests, no?
Adding to this: the open source community's biggest contribution might not be the models themselves, but collaborative datasets. Seeing more initiatives like Common Crawl makes me optimistic.
Can someone explain what "corpus" means? Is it just the training data or is there more to it? Asking for a friend who's learning about AI.
Excellent framing. In my research group, we've found that curating training datasets takes 10x longer than building model architecture. The bottleneck is data, not compute. This newsletter gets it.
The title made me laugh out loud — "without corpus, large models are idiots" 😂 But seriously, this frames the problem perfectly. We're throwing computing power at garbage-in-garbage-out problems.
This is obvious though? Of course models need data. The real question is how we ensure the data quality and ethical sourcing going forward.
Question: does this mean smaller, well-trained models might outperform larger ones with mediocre training data? Would love to see a deep dive on that tradeoff.
Absolutely spot on. I've been frustrated with LLM limitations for months, and this finally articulates why. Without quality training data, these models are just expensive pattern-matching machines. Great issue.